orizu 0.5.25 → 0.5.26

package/vendor/skills/orizu/references/building-judges.md

@@ -23,7 +23,7 @@ arrive with:
 - multiply labeled rows for measuring human–human agreement.
 
 If the labels are Likert scores or an “overall quality” bundle, return to the
-**J4 — Ground truth via annotation (conditional)** and rewrite the rubric as one binary question
+First win's conditional ground-truth annotation stage and rewrite the rubric as one binary question
 per failure mode before building a judge.
 
 - A **judge** is the evaluator, often an LLM instruction set plus a runner.
@@ -58,7 +58,7 @@ rows or overall agreement. Judge–human kappa should normally land within about
 0.05–0.10 of human–human kappa.
 
 When human–human kappa is itself low (below ~0.6), the rubric or labels are the
-bottleneck. Return to **J4 — Ground truth via annotation (conditional)**: clarify the
+bottleneck. Return to First win's conditional ground-truth annotation stage: clarify the
 failure-mode definition, add boundary examples, repair the labeler workflow, and
 relabel a shared sample. Recompute the human ceiling before changing judge
 instructions. Judge tuning against disputed labels only automates the dispute.
@@ -112,7 +112,7 @@ assertion, such as schema validation when schema validity is the entire failure
 definition, does not need prompt tuning or model-selection statistics. Record
 why the assertion is identical to the human-approved rule. If it is gating, it
 still needs an agreed judge trust bar and the registered, submitted evidence path from
-**J5 — Build & validate judges**. A regex, keyword, or other heuristic is not
+First win's judge-validation stage. A regex, keyword, or other heuristic is not
 tautological merely because it is code; validate it against the human labels
 like any other judge.
 
@@ -207,11 +207,10 @@ that as overfitting and return to a new development cycle rather than iterating
 on judge-test rows.
 
 The application final-held-out partition reserved for the single
-seed-versus-selected comparison in the dataset-design reference (J3) never
+seed-versus-selected comparison in the dataset-design reference never
 appears in judge train, judge dev, judge test, or any other judge-alignment
 material. A judge selected for agreement on those rows would bias the final
-comparison through the judge itself. Those rows need no human labels. At the
-**J6 — Optimize & promote**, the accepted judge scores the application
+comparison through the judge itself. Those rows need no human labels. During Promote's final comparison, the accepted judge scores the application
 final-held-out rows exactly once for that comparison; this application scoring
 is not judge-alignment measurement or tuning.
 
@@ -243,10 +242,10 @@ evals/<project>/
 In a local workflow under the user's token, push the executable artifacts and
 register both the row scorer used for per-example feedback and the set scorer
 used for kappa plus confusion diagnostics. Hosted sessions follow the artifact
-authority and commit rules in `SKILL.md`. Substitute IDs returned by each
+authority and commit rules in `authority-map.md`. Substitute IDs returned by each
 command in subsequent commands:
 
-For a custom deterministic code assertion that must complete the submitted J5
+For a custom deterministic code assertion that must complete the submitted judge-validation
 evidence path, use the executable `prompt_runner` control-plane shape. The
 runner implements the assertion without calling an LLM; the judge artifact is
 its versioned execution contract and output-schema identity. A `runner_only`
@@ -290,7 +289,7 @@ orizu scorers register --project <team/project> --name <kappa-scorer-name> --man
 
 For the LLM judge-test split, execute the row scorer, submit its row results,
 and then execute the dependent set scorer. `scorers exec` submits the aggregate
-score run by default, so omit `--no-submit` at the J5 — Build & validate judges
+score run by default, so omit `--no-submit` at the First win judge-validation
 boundary. See `prompt-control-plane.md` for the canonical submission behavior:
 
 ```bash
@@ -315,7 +314,7 @@ for acceptance:
 orizu scores accept <score-run-id> --project <team/project> --json
 ```
 
-J5 — Build & validate judges is complete only when every gating judge clears its
+First win's judge-validation stage is complete only when every gating judge clears its
 agreed judge trust bar, the scorer versions are registered, `runners exec`, `scores submit`,
 and the measured `scorers exec` run have all succeeded, and the human curator has
 accepted that score run. Gate decisions only on accepted evidence. A locally
@@ -341,14 +340,14 @@ decision with the human label and returns per-row feedback for reflection.
 
 Materialize those components in `orizu.instruction-set.json` before GEPA. Create
 the stable instruction set once, or push a revised seed for a later optimization
-cycle. `SKILL.md` is authoritative for who runs each mutation; hosted sessions
+cycle. `authority-map.md` is authoritative for who runs each mutation; hosted sessions
 prepare the manifest and exact human handoff:
 
 ```bash
-# Local surface (agent-allowed under the user's token): see the SKILL.md authority map.
+# Local surface (agent-allowed under the user's token): see `authority-map.md`.
 orizu instructions create ./evals/<project>/judges/<failure-mode>/orizu.instruction-set.json --project <team/project> --model-config <provider/model> --json
 
-# Local surface (agent-allowed under the user's token): see the SKILL.md authority map.
+# Local surface (agent-allowed under the user's token): see `authority-map.md`.
 orizu instructions push ./evals/<project>/judges/<failure-mode>/orizu.instruction-set.json --project <team/project> --set <judge-instruction-set> --json
 ```
 
@@ -430,7 +429,7 @@ Only after parity holds, publish fresh instruction and judge versions before
 touching judge-test. Under the user's token, the local sequence is:
 
 ```bash
-# Local surface (agent-allowed under the user's token): see the SKILL.md authority map.
+# Local surface (agent-allowed under the user's token): see `authority-map.md`.
 orizu instructions push ./evals/<project>/judges/<failure-mode>/orizu.instruction-set.json --project <team/project> --set <judge-instruction-set> --json
 
 orizu judges push ./evals/<project>/judges/<failure-mode>/judge --project <team/project> --runner-version <materialized-judge-runner-version-id> --json
@@ -464,7 +463,7 @@ submitted judge-test commands above deliberately use `<new-judge-version-id>`.
 Require human acceptance and gatekeeper-grade judge-test validation for any ship
 or promotion decision even when the judge used for reflection met only the
 optimization-signal judge trust bar. Hosted sessions commit the accepted component map and
-follow the handoff boundary in `SKILL.md`.
+follow the handoff boundary in `authority-map.md`.
 
 The default GEPA scorer contract supplies a GEPA-shaped row containing
 `source_row`, `candidate_id`, and `candidate_output`. If reusing a scorer runner
@@ -487,7 +486,7 @@ in `prompt-control-plane.md` before choosing either contract.
 - [ ] Development rows were used for iteration and judge-test was used only
       after the candidate was frozen.
 - [ ] The application final-held-out partition was excluded from all judge
-      material and is reserved for one accepted-judge scoring pass at J6.
+      material and is reserved for one accepted-judge scoring pass during Promote.
 - [ ] The versioned judge and runners are stored; row and set scorers are
       registered.
 - [ ] `runners exec` evidence was submitted with `scores submit`, and a measured

package/vendor/skills/orizu/references/cli-reference.md

@@ -80,7 +80,7 @@ orizu projects create --name "Quality Eval" --team my-team
 
 ### Instruction sets
 
-Follow the [Authority map](../SKILL.md#authority-map) for the executor on the current surface.
+Follow the [Authority map](authority-map.md) for the executor on the current surface.
 
 ```bash
 orizu instructions list --project my-team/quality-eval [--status active|archived|all] [--json]
@@ -105,7 +105,7 @@ should not teach it. A set can be addressed by its stable slug or exact name;
 the slug does not change when the display name changes. `list` returns each
 instruction set and its ordered shape. `show` returns the default and every
 model-config profile's production state. Use `--status archived` or `--status
-all` to include archived sets. The [Authority map](../SKILL.md#authority-map) selects the
+all` to include archived sets. The [Authority map](authority-map.md) selects the
 executor for each surface. Archive and restore change visibility only:
 archived sets continue to resolve and sync. `create` and `push` read a local
 manifest; JSON output is one document per line. `push --set` updates the named
@@ -115,7 +115,7 @@ or slug-addressed set independently of the manifest's display name.
 changing its production resolution. `profiles promote` moves that profile to a
 version, and `profiles rollback` creates a new version copied from `--to`,
 including that target version's settings version, before moving production.
-`SKILL.md` supplies the promotion-decision and execution hand-off. For a
+`authority-map.md` supplies the promotion-decision execution hand-off. For a
 single-component set that wraps a prompt, only default-profile promote and rollback also move the wrapped
 prompt's production label in the same transaction and print `default profile →
 prompt label`. This coupling is one-way: moving the prompt label directly does
@@ -262,7 +262,7 @@ Delete dataset:
 - There is no non-interactive confirmation flag.
 
 Edit rows requirements:
-- `--file` rows must include canonical `id` for each row being updated.
+- Every `--file` row must carry a non-empty string `id`.
 
 ### Tasks
 
@@ -369,6 +369,14 @@ Defaults:
 - format defaults to `jsonl`
 - output file defaults to `<taskId>.<format>`
 
+Complete:
+
+```bash
+orizu tasks complete --task <taskId>
+```
+
+Mark an active or paused task completed only after its required response-bearing export has been verified. Completion does not publish the task report; use the Report commands above after completion.
+
 ### Instruction Control Plane
 
 For instruction sets, judges, runners, run submission, optimizer artifacts,
@@ -483,6 +491,7 @@ Use these shortcuts only in TTY environments where prompts can run.
 
 ## Notes and Limits
 
+- `--json` may appear before or after a command. Non-streaming commands emit one JSON document, with long-running summaries on the final stdout line; streaming `orizu run tail --json` instead emits JSONL, one event object per line.
 - `tasks create` creates a draft by default, does not require `--assignees`, and pins the app's current version when the task is created.
 - `tasks create --publish --assignees <...>` intentionally creates and ships immediately.
 - `--assignees` and assignment-file rows accept member emails or user IDs. Email selectors resolve to canonical member IDs within the task's project team; ambiguous or unknown emails are rejected.
@@ -491,9 +500,9 @@ Use these shortcuts only in TTY environments where prompts can run.
 - Assignment completion payloads are validated against the pinned app-version `output_json_schema`.
 - `datasets delete-rows` requires `--row-ids`.
 - `datasets delete` requires interactive terminal confirmation and has no non-interactive confirmation flag.
-- `datasets edit-rows` requires row objects in `--file` to include canonical `id`.
+- `datasets edit-rows --file` requires every row to carry a non-empty string `id`.
 - `--row-ids` is the canonical row selection for delete operations.
-- Locked datasets reject append/edit/delete row mutations.
+- Locked datasets reject append/edit/delete row mutations, but whole-dataset deletion does not check the lock.
 - Row deletes are rejected when targeted rows are assignment-referenced.
 - Login callback binding defaults to `127.0.0.1:43123`. If that port is unavailable, set `ORIZU_AUTH_PORT` to an available port from 1024–65535.
 - New CLI logins use personal access tokens rather than short-lived Supabase session credentials.

package/vendor/skills/orizu/references/dataset-design.md

@@ -1,6 +1,6 @@
 # Dataset design
 
-Use this reference for **J3 — Build the dataset** after the improvement plan is ratified and before annotation or judge work. Treat the plan's approved source choice and named scenario classes as input constraints rather than reopening them while moving data.
+Use this reference during First win's dataset-curation stage after the improvement plan is ratified and before annotation or judge work. Treat the plan's approved source choice and named scenario classes as input constraints rather than reopening them while moving data.
 
 ## Carry the approved source choice into the dataset
 
@@ -8,6 +8,16 @@ Inventory each approved source and record its owner, row shape, time range, scen
 
 If there is no approved golden data, continue with representative production traces, a random sample, and structured synthetic cases for scenario classes that real traces do not cover. Keep provenance on every row so later versions can be audited, and have the human ratify the source choice and spot-check examples before the dataset is used.
 
+## Getting data in
+
+Use agent glue to move data from each ratified source into Orizu:
+
+1. **Pull from the source.** Use the source provider's own CLI or API to export the approved rows. Follow that source's authentication and pagination contract, and preserve its stable locator and provenance without teaching a provider-specific path here.
+2. **Normalize locally.** Transform the export into CSV, JSON, or JSONL with stable row IDs and the fields required by the ratified plan. Validate row shape and provenance before sending the file to Orizu.
+3. **Load into Orizu.** Use `orizu datasets upload` for an initial dataset, or use `orizu datasets push <path>` as the path-first, automation-friendly alternative for creating an initial dataset from a CSV, JSON, or JSONL file. Use `orizu datasets append` for approved additional rows. Then follow the version and split procedure below; loading rows does not waive its coverage or review gates.
+
+We deliberately do not build per-provider connectors while this agent-glue method covers the need. Revisit that decision only when an approved source cannot be reached by an agent-driven pull, such as streaming-only telemetry or push-based webhooks.
+
 ## Build a diverse sample and make coverage visible
 
 Create a coverage table from the ratified plan before sampling. Give every scenario class a planned row count, source, ground-truth status, and actual row count. Preserve the production distribution with a random sample, then deliberately add rare, costly, new, and previously failing cases. Keep successful cases as controls. Deduplicate repeated traces and keep stable row IDs; do not let one conversation or near-duplicate appear on both sides of the split.
@@ -18,7 +28,7 @@ Before upload, fail the coverage check if any scenario class from the plan has z
 
 Build one predefined split with exactly three application partitions for the primary dataset at dataset-build time: train, validation, and final-held-out. Auxiliary partitions, such as a migration-only parity corpus, may coexist in specialized workflows without changing this three-partition application doctrine. Validation is the valset that ranks candidates during optimization. Apply the Vocabulary contract to final-held-out: exclude its rows from candidate ranking, selection, reflection, and judge development.
 
-**Judge-development data (J5)** comes from the labeled export, and `references/building-judges.md` owns its split mechanics; exclude final-held-out rows from all judge-development material (judge-dev and judge-test/alignment) because judge selection on final-held-out labels biases the final comparison through the judge itself.
+**Judge-development data** comes from the labeled export, and `references/building-judges.md` owns its split mechanics; exclude final-held-out rows from all judge-development material (judge-dev and judge-test/alignment) because judge selection on final-held-out labels biases the final comparison through the judge itself.
 
 Final-held-out rows need no human labels; exclude their row IDs from annotation tasks.
 
@@ -30,7 +40,7 @@ Budget this together with the valset. With the skilled/custom proposer, the engi
 
 No budget dry-run exists, so calculate the resolved metric-call limit before launching a skilled/custom proposer. Explicit `--max-metric-calls n` resolves to `n`; `--max-full-evals f` resolves to `f * (len(trainset) + len(valset))`; and named/default `--budget` presets resolve through the formula in `packages/orizu-gepa-python/src/orizu_gepa/optimizer.py:361-405` using the chosen preset, component count, and valset size. Require `resolved_metric_calls - len(valset) >= b`; otherwise increase or change the budget, or record that the coverage guarantee does not hold. If the available rows or optimization budget cannot support the computed size, add rows, combine only genuinely equivalent scenario classes with human approval, or record that the plan is not ready for optimization.
 
-The canonical split-file schema and version/split command contract live in `references/prompt-control-plane.md` under “Dataset Versions And Splits.” J3 adds the coverage requirement: use a predefined three-way split when scenario-class coverage matters, and build its partitions from the independently reviewed coverage table rather than a random split that could omit a class. For example:
+The canonical split-file schema and version/split command contract live in `references/prompt-control-plane.md` under “Dataset Versions And Splits.” First win's dataset-curation stage adds the coverage requirement: use a predefined three-way split when scenario-class coverage matters, and build its partitions from the independently reviewed coverage table rather than a random split that could omit a class. For example:
 
 ```json
 {
@@ -67,7 +77,7 @@ The dataset version freezes its rows. Server-side split membership is mutable, b
 
 ## Guard live rows before annotation and version later data
 
-Use the dataset mutation contract in `SKILL.md` under “Execution facts” rather than inferring lock or row-edit semantics here.
+Use the dataset mutation contract in `cli-reference.md` rather than inferring lock or row-edit semantics here.
 
 After source approval and the split audit, the dataset must be locked before annotation starts because tasks pin live dataset rows rather than the immutable version:
 
@@ -90,11 +100,11 @@ Never rewrite a prior version to make later labels or corrections appear histori
 
 ## Exit criterion
 
-J3 exits only when the evidence records all of the following:
+Dataset curation exits only when the evidence records all of the following:
 
 - the human-ratified source choice and spot-check;
 - the plan's complete scenario-class list, with nonzero total, training, validation, and final-held-out counts for every scenario class;
 - an immutable dataset version ID plus the reviewed local split file and recorded split-set ID whose predefined train, validation, and final-held-out rows match that coverage table;
-- final-held-out row IDs excluded from all judge-development material (judge-dev and judge-test/alignment) and reserved for the final J6 comparison;
+- final-held-out row IDs excluded from all judge-development material (judge-dev and judge-test/alignment) and reserved for Promote's final comparison;
 - for the annotation path, proof that the dataset was locked before any task was created; and
 - either approved golden ground truth with annotation skipped or reduced, or rows ready for the eval-strategy and annotation path.

package/vendor/skills/orizu/references/eval-strategy.md

@@ -1,26 +1,34 @@
 # Eval strategy and labeler validation
 
-Use this reference only after **J3 — Build the dataset** has versioned the PRIMARY dataset and created its three-way split set. If approved golden data already supplies all required ground truth, record the source, approval, and scenario-class coverage, then skip annotation. If only unresolved rows need labels, clone or filter those rows into a separate dataset because automatic assignments cover every current dataset row and explicit assignment files require uniform whole-dataset coverage.
+Use this reference only after the current flow has versioned the PRIMARY dataset and created its three-way split set. If approved golden data already supplies all required ground truth, record the source, approval, and scenario-class coverage, then skip annotation. If only unresolved rows need labels, clone or filter those rows into a separate dataset because automatic assignments cover every current dataset row and explicit assignment files require uniform whole-dataset coverage.
 
 Final-held-out rows need no human labels; exclude their row IDs from annotation tasks and from the unresolved-rows clone/filter.
 
 A derived unresolved-rows dataset is a labeling vehicle, not an optimization dataset. Preserve each source row's canonical `id` field from `datasets download` in the derived row's `row_data`; this is the stable identifier used by split files. Before creating any task, create and record the derived dataset's immutable version ID and lock the live dataset; it does not need its own per-class three-way split. Task creation waits for both records.
 
+## Annotation artifact map
+
+When ground truth requires human labels, keep the three artifacts and their owners distinct:
+
+1. **Strategy:** ratify the eval strategy in this reference before building a labeler or task.
+2. **App:** author and validate the app through the [app reference](building-apps.md). An app renders one row and returns its structured response; it does not own assignments or the labeling round.
+3. **Task:** create the task draft only after the app feedback gate passes. A task binds the pinned app version to dataset rows and assignments. Follow the [CLI reference](cli-reference.md) for task create, draft-URL approval, publish, assignment, status, export, and completion commands; return here for response-completeness proof and the annotation exit criterion.
+
 ## Guide the eval-strategy conversation
 
-Start from the ratified improvement plan and J3 coverage table; do not ask the human to restate facts already recorded. Walk through the decisions below with concrete rows, propose a draft answer where the evidence supports one, and ask the human to correct or ratify it.
+Start from the ratified improvement plan and dataset coverage table; do not ask the human to restate facts already recorded. Walk through the decisions below with concrete rows, propose a draft answer where the evidence supports one, and ask the human to correct or ratify it.
 
 1. **Experience and decision.** Ask which exact user experience the eval represents and which change or release decision it must inform. Confirm the unit being judged: one response, conversation, agent step, or complete run.
 2. **Good and bad by scenario class.** For every scenario class, show representative rows and draft one binary question per observed failure mode. Ask what visible evidence makes each answer pass, fail, or unlabelable. Do not bundle independent judgments or replace them with a Likert question.
-3. **Eval-set composition.** Confirm which versioned rows need human labels, the intended counts per scenario class, and any rare cases that must be present even when production frequency is low. Keep J3 validation for candidate ranking and keep its final-held-out partition under the separate partition doctrine in `references/dataset-design.md`.
+3. **Eval-set composition.** Confirm which versioned rows need human labels, the intended counts per scenario class, and any rare cases that must be present even when production frequency is low. Keep First win's validation partition for candidate ranking and keep its final-held-out partition under the separate partition doctrine in `references/dataset-design.md`.
 4. **Ground truth.** Ask who is qualified to label each question, what context they need, how ambiguity or disagreement will be adjudicated, and whether user behavior or another approved golden-data field already answers it. Import approved golden data instead of relabeling it.
-5. **Downstream validation.** Explain that these labels will later validate automated judges. Ask which decision class each judge will power and record it for **J5 — Build & validate judges**, where the human sets the numerical judge trust bar from measured agreement, TPR, and TNR rather than accepting a default here.
+5. **Downstream validation.** Explain that these labels will later validate automated judges. Ask which decision class each judge will power and record it for First win's judge-validation stage, where the human sets the numerical judge trust bar from measured agreement, TPR, and TNR rather than accepting a default here.
 
 Persist the ratified eval strategy in project context. It must name the experience and unit, dataset version and split set, scenario classes and row counts, binary questions with pass/fail/unlabelable rules, ground-truth source, labeler qualifications, disagreement path, golden-data decision, and the future decision attached to each judge trust bar. Do not author the labeler until the human ratifies this record.
 
-`references/building-judges.md` owns the J5 input envelope and is being aligned there. This reference records the evidence handed to that contract. For a golden-data-only path, validate the field mapping against the binary questions, spot-check every scenario class with the human, and record the PRIMARY versioned dataset ID plus the approved ground-truth field per failure mode in the plan and report; then skip the remaining J4 annotation path.
+`references/building-judges.md` owns the judge-validation input envelope and is being aligned there. This reference records the evidence handed to that contract. For a golden-data-only path, validate the field mapping against the binary questions, spot-check every scenario class with the human, and record the PRIMARY versioned dataset ID plus the approved ground-truth field per failure mode in the plan and report; then skip the remaining annotation path.
 
-For a mixed path, after annotation export, group derived records by the canonical `id` at `dataset_row.row_data.id`, the same identifier emitted by `datasets download` and used by split files, and require each key to match one primary row. Preserve every per-rater record in each row group, including the rater identity from the export, then aggregate according to the eval strategy's agreement rules. Record J5's labeled input as the union of the golden field mapping for golden rows and the grouped task export for annotated rows, keyed by that canonical primary-row `id`. Continue below only for questions or rows whose ground truth still requires human labels.
+For a mixed path, after annotation export, group derived records by the canonical `id` at `dataset_row.row_data.id`, the same identifier emitted by `datasets download` and used by split files, and require each key to match one primary row. Preserve every per-rater record in each row group, including the rater identity from the export, then aggregate according to the eval strategy's agreement rules. Record the judge-validation stage's labeled input as the union of the golden field mapping for golden rows and the grouped task export for annotated rows, keyed by that canonical primary-row `id`. Continue below only for questions or rows whose ground truth still requires human labels.
 
 ## Validate the labeler before tasks or labels
 
@@ -30,7 +38,7 @@ Run the smoke test and preview flow from that reference with representative rows
 
 Record the rows previewed, the feedback, the revisions, and the approval. Do not create or publish a task, and do not start labeling, until this labeler-feedback gate is confirmed.
 
-Use the draft-first ordering, URL-approval gate, and assignee-selector contract in `SKILL.md` under “Execution facts”; use `references/cli-reference.md` for the command surface. This reference adds one versioning safeguard: the task pins the app version selected at creation.
+Use the draft-first ordering, URL-approval gate, assignee-selector contract, and command surface in `cli-reference.md`. This reference adds one versioning safeguard: the task pins the app version selected at creation.
 
 If draft testing reveals a problem, revise through `orizu apps update`, repeat the preview and pointed-feedback gate, capture the returned version number `n`, and explicitly repin the draft before testing its URL again:
 
@@ -38,7 +46,7 @@ If draft testing reveals a problem, revise through `orizu apps update`, repeat t
 orizu tasks update --task <taskId> --app <appId> --version <n>
 ```
 
-Then return to the approval and publish sequence in “Execution facts.” If the strategy requires an explicit row-to-labeler map, it must still cover every row in this task's separate annotation dataset uniformly.
+Then return to the task create, draft-URL test, approval, and publish sequence in `references/cli-reference.md` under “End-to-End Flows” → “New Team to Export”. If the strategy requires an explicit row-to-labeler map, it must still cover every row in this task's separate annotation dataset uniformly.
 
 After labeling, inspect the machine-readable status before task completion or export:
 
@@ -52,13 +60,8 @@ Verify that `task.counts.completed` equals `task.totalRequiredAssignments` and t
 orizu tasks export --task <task-id> --format jsonl --out ./labels.jsonl
 ```
 
-Parse every non-empty JSONL line. Require the export row count to equal `task.totalRequiredAssignments`, require each `dataset_row_id` to appear exactly `task.requiredAssignmentsPerRow` times, and require every record to carry a non-null `response_id` and `response_data`. Any mismatch means a planned row lacks a response and blocks completion. Only after this response-level proof may the task be completed and its report published:
-
-```bash
-orizu tasks complete --task <task-id>
-orizu tasks report set --task <task-id> --report-file ./task-report.md --json
-```
+Parse every non-empty JSONL line. Require the export row count to equal `task.totalRequiredAssignments`, require each `dataset_row_id` to appear exactly `task.requiredAssignmentsPerRow` times, and require every record to carry a non-null `response_id` and `response_data`. Any mismatch means a planned row lacks a response and blocks completion. Only after this response-level proof may the task be completed and its report published; follow the Completion and Report commands in the [CLI reference](cli-reference.md), in that order.
 
 ## Exit criterion
 
-The annotation path exits only when the ratified eval strategy, labeler feedback, app approval, complete assignment-status evidence, response-complete exported labels, and published task report are recorded together. The golden-data path exits with the PRIMARY versioned dataset ID and approved per-failure-mode field mapping recorded. The mixed path additionally records the canonical-row-`id`-keyed union of golden mappings and per-rater annotated exports. In every path, report ground-truth and disagreement counts per scenario class so J5 receives auditable evidence rather than only an aggregate.
+The annotation path exits only when the ratified eval strategy, labeler feedback, app approval, complete assignment-status evidence, response-complete exported labels, and published task report are recorded together. The golden-data path exits with the PRIMARY versioned dataset ID and approved per-failure-mode field mapping recorded. The mixed path additionally records the canonical-row-`id`-keyed union of golden mappings and per-rater annotated exports. In every path, report ground-truth and disagreement counts per scenario class so judge validation receives auditable evidence rather than only an aggregate.

package/vendor/skills/orizu/references/instructions-after-prompts.md

@@ -77,12 +77,12 @@ text, then prepare a manifest such as:
 ```
 
 Commit the complete manifest and its component files, then follow the
-[Authority map](../SKILL.md#authority-map) for each mutation.
+[Authority map](authority-map.md) for each mutation.
 
 ## Consolidation
 
 Consolidation changes ownership and visibility. Create the consolidated set
-through the surface selected by the [Authority map](../SKILL.md#authority-map) and inspect its complete
+through the surface selected by the [Authority map](authority-map.md) and inspect its complete
 profiles before changing any pointer:
 
 ```bash

package/vendor/skills/orizu/references/migrate-existing-gepa-setup.md

@@ -6,7 +6,7 @@ For a customer already running official GEPA from a plain script — inline
 become the set's fixed shape, and its values become that profile's components.
 Use `orizu instructions` as the only creation and update path for those
 customer-owned instructions. Prepare the complete manifest, then follow the
-[Authority map](../SKILL.md#authority-map) for its mutations.
+[Authority map](authority-map.md) for its mutations.
 
 The ordering rule: **nothing is optimized until parity is proven.** The last
 step before `run-gepa` is always `orizu scorers verify-parity`, which runs the
@@ -33,10 +33,10 @@ orizu setup --team <team-slug> --project <project-slug>
 One dataset holds every row; require four disjoint, nonempty partitions:
 `TRAINSET`, `VALSET`, `PARITYSET`, and `FINAL_HELD_OUT`. Mirror existing
 train/validation membership exactly. Here, `parity` is a migration-only
-auxiliary partition, not an application partition; the canonical J3
+auxiliary partition, not an application partition; the canonical dataset-design
 application doctrine remains train, validation, and final-held-out. Source and
 approve representative `PARITYSET` rows before reserving final-held-out, then
-reserve `FINAL_HELD_OUT` only from the remaining rows. Apply J3's coverage
+reserve `FINAL_HELD_OUT` only from the remaining rows. Apply First win's dataset-coverage
 discipline:
 every named scenario class must have nonzero train, validation, and
 final-held-out counts. Stop when that coverage, ordering, or four-way
@@ -104,7 +104,7 @@ orizu --local instructions create ./orizu.instruction-set.json \
   --model-config <provider/model> --json
 ```
 
-The agent-created runner version is unlabeled. No standalone runner-label command exists. Following the [Authority map](../SKILL.md#authority-map), if a human accepts its exact ID as default, the human curator runs `orizu --local runners pull gepa-candidate-runner --project <team>/<project> --version <candidate-runner-version-id> --out ./candidate-runner-default-handoff`, leaves that fresh exact-version directory unchanged, then runs `orizu --local runners push ./candidate-runner-default-handoff --project <team>/<project> --name gepa-candidate-runner --label default --json`. The manifest
+The agent-created runner version is unlabeled. No standalone runner-label command exists. Following the [Authority map](authority-map.md), if a human accepts its exact ID as default, the human curator runs `orizu --local runners pull gepa-candidate-runner --project <team>/<project> --version <candidate-runner-version-id> --out ./candidate-runner-default-handoff`, leaves that fresh exact-version directory unchanged, then runs `orizu --local runners push ./candidate-runner-default-handoff --project <team>/<project> --name gepa-candidate-runner --label default --json`. The manifest
 carries the set's `name`, human-readable `description`, fixed `shape`, and component values. Put each original dict
 value in a file without changing its bytes. This one-component example maps
 the old `system` key to `system.md`:
@@ -308,7 +308,7 @@ gate. Repeat the command with `--split train` and `outputs-train.jsonl`; the
 explicit `parity` command above covers the independently selected corpus.
 Require exit 0 on all three parity runs.
 
-Keep final-held-out untouched until J6. It is exercised exactly once, at the
+Keep final-held-out untouched until Promote. It is exercised exactly once, at the
 final seed-vs-selected comparison, by the already-parity-proven scorer and
 supplies no scorer-development feedback.
 
@@ -317,6 +317,10 @@ agreed. A limited run reports `parity: false` with
 `scope: {compared, total}` and the line "Smoke check passed on N of M rows"
 — it exits 0, but it is not the proof.
 
+## Router handoff after parity
+
+When the router sends an existing setup here, stop after all three scorer parity checks pass. Open `flows/first-win.md`; do not continue directly to section 7. First win owns the validated-judge optimization run, and `flows/promote.md` separately owns its report, validation, and human promotion decision. The remaining sections are reference detail only when the active flow explicitly reaches them.
+
 ## 7. Optimizer and run
 
 ```bash
@@ -361,7 +365,7 @@ reflected on each round.
 Pass `--scorer-candidate-field <field>` here too if you passed it to
 `verify-parity`: the payload the runner sees must be the one parity was proven
 under. Select one candidate on validation evidence, then reserve one final-held-out seed-versus-selected-candidate comparison for the promotion decision.
-Write and attach the optimization report, then obtain the human decision. Following the [Authority map](../SKILL.md#authority-map), in the simpler one-shot path a human curator runs `orizu --local optimizations promote <run-id> --candidate <candidate-id> --label production --project <team>/<project> --json` after acceptance. In the equivalent two-stage path for separate materialization, the coding agent materializes once with the unlabeled command below.
+Write and attach the optimization report, then obtain the human decision. Following the [Authority map](authority-map.md), in the simpler one-shot path a human curator runs `orizu --local optimizations promote <run-id> --candidate <candidate-id> --label production --project <team>/<project> --json` after acceptance. In the equivalent two-stage path for separate materialization, the coding agent materializes once with the unlabeled command below.
 
 ```bash
 orizu --local optimizations promote <run-id> --candidate <candidate-id> \

package/vendor/skills/orizu/references/optimization-reports.md

@@ -99,7 +99,7 @@ Server events redact row snapshots and reflection prompts by default, so an expo
 ## How To Build The Report
 
 1. Confirm the seed, all candidate ids, their parent/child relationships, and their validation results. Include rejected, failed, or incomplete candidates and mark unavailable comparisons plainly; a score from one small training batch is not a substitute for a full validation score. A failed proposal that produced no child candidate id is a run event, not another candidate.
-2. Map every evaluation row to a scenario class named in the improvement plan or dataset. If the logs lack a class, report that classification gap and return to dataset design; do not silently invent classes.
+2. Map every evaluation row to a scenario class named in the improvement plan or dataset. If the logs lack a class, record the classification gap and recommend **Gather more evidence**; do not silently invent a class or leave the in-progress Promote flow. Keep the gap in the decision record and complete Promote's human decision; after Promote exits, route the recorded review incident through `flows/triage.md`.
 3. Select the candidate on validation evidence. Explain why each other candidate was not selected, including any useful strength the selected candidate gives up.
 4. Diagnose optimizer health from the frontier size, proposal acceptance rate, budget state, and validation score-over-time trajectory. This is what distinguishes a plateau from a promising search stopped by its budget.
 5. Determine whether the current execution surfaces can reproduce the selected system before claiming a Final-held-out comparison. The optimization run does not execute the Final-held-out split: `run-gepa` and its logs contain only training and validation data.
@@ -127,7 +127,7 @@ Server events redact row snapshots and reflection prompts by default, so an expo
 
    Current `runners exec` cannot address an instruction-set profile version. Multi-component promotion returns a `profileVersionId` plus component statuses without the component prompt-version ids needed by that command, and one prompt-version execution cannot reproduce the complete profile. For a set-of-one instruction set, `instructions push` creates the profile version that carries the selected component value, but the optimization run does not supply that profile as its result and `runners exec` still cannot address its `profileVersionId`. Execute by component prompt-version id only when an independently confirmed association identifies that exact pushed profile and supplies its manifest, settings, and pins. Do not infer a profile through a component prompt-version id: implicit resolution can select a newer profile that shares that component. If the required exact profile context cannot be established, mark instruction-set Final-held-out execution unavailable, recommend **Gather more evidence**, and name exact profile-addressed runner execution as the required product capability. Never reuse validation rows or reopen candidate ranking with Final-held-out.
 6. Decide whether the improvement is larger than expected noise and whether any regression is unacceptable. **Run-to-run noise** means score movement caused by sampling or model variation rather than by the instruction change. The stored `confidence_interval_95` describes one system's single score run; it is not a confidence interval for the seed-to-selected change. Comparing two per-system intervals for overlap is not a significance test. Use `evaluations.jsonl` only for validation diagnostics; Final-held-out significance requires the separately generated Final-held-out comparison artifact. Sanctioned change evidence is repeated Final-held-out seed and selected runs or paired per-row differences from that Final-held-out artifact. Otherwise write “significance was not measured.” Statistical significance means the measured change is unlikely to be explained by measured chance variation alone; it does not make a tiny gain useful. The **practical threshold** is the smallest improvement the team agreed would matter; if none exists, state that no threshold was agreed and recommend agreeing one before the next run. A **scenario guardrail** is the largest decline the team agreed to accept for one scenario class.
-7. Write a recommendation and named next move for every scenario that did not improve or regressed. When the move changes examples, coverage, labels, or split design, return to **J3 — Build the dataset** rather than changing instructions without measurement.
+7. Write a recommendation and named next move for every scenario that did not improve or regressed. When the move changes examples, coverage, labels, or split design, keep it as a named next move until Promote records the human decision; after Promote exits, route the recorded review incident through `flows/triage.md` rather than changing instructions without measurement or jumping into an internal First win stage.
 
 For a rate, report the change in percentage points: 72% to 81% is +9 percentage points, not +9%. If judge alignment is reported, define **Cohen's kappa** as agreement after removing the amount expected by chance. Define a **confusion matrix** as the counts of correct and incorrect decisions for each class. Define any other statistical term in one plain sentence when it first appears.
 
@@ -187,15 +187,15 @@ Group outcomes by real scenario from the improvement plan or dataset, not only b
 
 | Scenario class | Why it matters | Coverage | Seed -> selected on validation | What this class teaches | Next move and owner |
 | --- | --- | --- | --- | --- | --- |
-| [class] | [user/system consequence] | [training row count / validation row count] | [fixed, unchanged failures, regressions, and row ids] | [plain-language lesson] | [named action, owner, and journey step] |
+| [class] | [user/system consequence] | [training row count / validation row count] | [fixed, unchanged failures, regressions, and row ids] | [plain-language lesson] | [named action, owner, and flow] |
 
-For each class that did not improve or regressed, choose and tailor a named move:
+For each class that did not improve or regressed, choose and tailor a named move in the Promote decision record without leaving the in-progress flow. Complete Promote's human decision first; after Promote exits, route through `flows/triage.md` before executing any move below. Triage owns the eval-gap decision and any admitted transition into a complete improvement flow.
 
-- **Add coverage — J3 — Build the dataset:** collect or create more examples for this class, create a new dataset version, revisit split and optimizer-batch coverage, then rerun optimization.
-- **Repair ground truth — J3 — Build the dataset, then conditional J4 — Ground truth via annotation:** send named contested rows for human re-review, correct rationales or labels, create a new dataset version, and revalidate the judge before optimizing again.
-- **Improve evaluator feedback — J5 — Build & validate judges:** make the scorer explain the concrete disagreement, revalidate it, then rerun.
-- **Change the application — J2 — Assess & plan:** when the failure needs retrieval, tools, state, or code rather than instructions, return to the ratified improvement plan and name the engineering change.
-- **Guard a regression — J3 — Build the dataset:** add the regressed rows and nearby counterexamples to a new dataset version, preserve them in validation, then rerun candidate selection.
+- **Add coverage — Triage:** establish the missing-coverage branch before collecting or creating more examples, creating a new dataset version, revisiting split and optimizer-batch coverage, and rerunning optimization.
+- **Repair ground truth — Triage:** establish the broken-eval branch before sending named contested rows for human re-review, correcting rationales or labels, creating a new dataset version, and revalidating the judge before optimizing again.
+- **Improve evaluator feedback — Triage:** establish the broken-eval branch before making the scorer explain the concrete disagreement, revalidating it, and rerunning.
+- **Change the application — Triage:** establish that the failure needs retrieval, tools, state, or code rather than instructions, amend the ratified improvement plan when required, and name the engineering change.
+- **Guard a regression — Triage:** establish the missing-coverage branch before adding the regressed rows and nearby counterexamples to a new dataset version, preserving them in validation, and rerunning candidate selection.
 
 ## What Changed In The Selected Version
 
@@ -224,7 +224,7 @@ This section contains exactly the seed and the candidate selected on validation;
 ## Recommendation And Named Next Moves
 
 1. **[Promote / Do not promote / Gather more evidence] — [owner]:** [Decision and reason.]
-2. **[Scenario class] — [owner]:** [Specific dataset, annotation, judge, or application action; artifact to create; journey step to resume.]
+2. **[Scenario class] — [owner]:** [Specific dataset, annotation, judge, or application action; artifact to create; admitted flow to resume.]
 3. **[Post-promotion monitoring, if promoting] — [owner]:** [Metric, scenario class, sample source, review point, and rollback trigger.]
 
 - **Extend the search —** budget was binding and validation was still improving; rerun with a larger budget.
@@ -261,6 +261,6 @@ Recommend **Do not promote** when the seed was selected, no valid candidate exis
 
 Recommend **Gather more evidence** when exact instruction-set Final-held-out execution is unavailable, either scorer contract lacks the normalization and per-row paired-score path needed for a Final-held-out decision, plain-prompt policy requires Final-held-out proof before production, a plain-prompt live-first decision lacks a confirmed external/manual rollback path, set-of-one runner/settings provenance cannot be proven, the Final-held-out sample is too small, run-to-run noise was not measured and could plausibly explain the gain, no practical threshold was agreed, candidate evaluations are incomplete, or redaction prevents scenario diagnosis. Name the cheapest next measurement and do not imply that uncertainty is improvement.
 
-When a class is absent or thin in training, the optimizer cannot learn it from reflection. Return to J3 — Build the dataset. When labels or rationales are contested, repair ground truth and create a new dataset version before rerunning. When the scorer gives vague feedback or disagrees with human judgment, return to J5 — Build & validate judges. When the system needs tools, state, retrieval, or code, return to J2 — Assess & plan rather than forcing the behavior into instructions.
+When a class is absent or thin in training, labels or rationales are contested, the scorer gives vague feedback or disagrees with human judgment, or the system needs tools, state, retrieval, or code, keep the finding in the Promote decision record and recommend Gather more evidence or Do not promote as the evidence supports. Complete the human decision without leaving Promote; after Promote exits, route through `flows/triage.md`. Triage decides whether coverage is missing, an eval is broken, or an application change is required; it amends the ratified plan when required and enters the complete admitted flow for any instruction optimization. Do not jump directly into a First win internal stage or force an application capability into instructions.
 
 If a run ends with budget exhausted, the coding agent may finish the run. A multi-component instruction-set non-seed finish records `--best-candidate` and the report but omits the prompt-only `--result-prompt-version`; its earlier unlabeled profile materialization is agent-safe and its later idempotent production-label move is human-only. For a plain-prompt non-seed run, the human performs the candidate's single production-labelled optimization promotion and the agent records the returned `promptVersionId` when finishing. For a set-of-one instruction-set non-seed run, the agent never calls optimization promotion, push, or a serving-pointer command under the current provenance gap; it recommends Gather more evidence and finishes without a result prompt version. Seed-selected and no-valid-candidate runs finish without promotion or a result version. Auto-promotion is human-only; coding agents must not enable it or configure a run to trigger it.

package/vendor/skills/orizu/references/optimization-with-gepa.md

@@ -1,6 +1,8 @@
 # Optimization With GEPA
 
-How to optimize an instruction-set profile against validated judges and scorers. Prepare the complete instruction-set manifest, follow the [Authority map](../SKILL.md#authority-map) for its mutation, and then run the bundled `orizu optimizations run-gepa` flow. Use this reference for GEPA mechanics, custom optimizer implementations, and optional DSPy context for customers already using DSPy.
+How to optimize an instruction-set profile against validated judges and scorers. Prepare the complete instruction-set manifest, follow the [Authority map](authority-map.md) for its mutation, and then run the bundled `orizu optimizations run-gepa` flow. Use this reference for GEPA mechanics, custom optimizer implementations, and optional DSPy context for customers already using DSPy.
+
+When First win loads this reference, stop after the optimization run records a selected candidate or no-valid-candidate outcome. Do not write the decision report or run a promotion command until `flows/promote.md` is active. Steps 8–9 below describe Promote-owned continuation mechanics, not First win work.
 
 Select the profile explicitly with `--instruction-set <slug-or-exact-name> --model-config <identity>`. Those two selectors replace the legacy `--candidate-version-id` path and cannot be combined with it. The connector resolves that model config's production profile, or the set default when the profile has no production version, and optimizes its complete component map. `--component-selector round-robin` is the default and updates one component per round; `--component-selector all` updates every component per round. The runner receives a multi-component candidate as a component map, while a one-component set keeps the existing single-body runner contract. Git-pinned components and malformed component maps are refused before a run starts. Automatic promotion is refused for multi-component sets; write the report, obtain the human promotion decision, and follow the Authority map. Manifest create/push is the pre-run seed path, not a second accepted-candidate materialization; after a run, use the human one-shot or equivalent idempotent two-stage path in step 9.
 
@@ -57,7 +59,7 @@ Use the control plane described in `prompt-control-plane.md` when you want runs,
 6. Use `orizu optimizations run-gepa --optimizer-version-id <optimizer-version-id>` with the set slug and model-config identity. It uses official GEPA by default; `--engine legacy` is a frozen safety hatch while migration parity is proven. Use `orizu optimizations start` plus event logging for a custom optimizer. Candidates are ranked on the partition passed as `--val-split`; use that partition for validation, not Final-held-out.
 7. Use set scorers for selection/tracked reporting when the meaningful metric is batch-level; execute builtin set scorers with `orizu scorers exec` or submit precomputed aggregates with `orizu scores submit --aggregate`.
 8. Write and attach the optimization report by following “How To Build The Report” step 5 in `references/optimization-reports.md`: “Determine whether the current execution surfaces can reproduce the selected system before claiming a Final-held-out comparison.” Its “Flat-row scorer contract,” “Default GEPA scorer contract,” and “Instruction-set execution limits” subsections own the executable boundaries and current gaps.
-9. After the human accepts the report, follow the [Authority map](../SKILL.md#authority-map). Simpler one-shot path: a human curator runs `orizu optimizations promote <run-id> --candidate <id> --label production --project <team/project>`, materializing and labeling once. In the two-stage path for multi-component or deliberately unlabeled materialization, an agent may materialize with `orizu optimizations promote <run-id> --candidate <id> --project <team/project>`. A human curator re-runs that same promotion as `orizu optimizations promote <run-id> --candidate <id> --label production --project <team/project>`. The idempotent finalizer reuses the existing materialized profile version, moves production to it, and creates no duplicate profile version or candidate-promoted event.
+9. After the human accepts the report, follow the [Authority map](authority-map.md). Simpler one-shot path: a human curator runs `orizu optimizations promote <run-id> --candidate <id> --label production --project <team/project>`, materializing and labeling once. In the two-stage path for multi-component or deliberately unlabeled materialization, an agent may materialize with `orizu optimizations promote <run-id> --candidate <id> --project <team/project>`. A human curator re-runs that same promotion as `orizu optimizations promote <run-id> --candidate <id> --label production --project <team/project>`. The idempotent finalizer reuses the existing materialized profile version, moves production to it, and creates no duplicate profile version or candidate-promoted event.
 
 The bundled Orizu GEPA-style optimizer supports configurable budget, minibatch size (default 3), candidate selection strategy, reflection model/template, reflection provider settings, evaluation caching, and optional auto-promotion. Use the report-first path: run without auto-promotion, write the report, obtain the human promotion decision, then route the manual promotion command through the Authority map. Only pass `--log-row-snapshots` when raw data in event logs is intentional.