session-orchestrator 5.1.0 → 5.3.0

package/skills/dispatcher/SKILL.md

@@ -10,6 +10,8 @@ description: >
   portfolio. user: "/dispatcher" assistant: "Ranked 18 free repos — top recommendation: Pencil-Designs
   (score 4.50, 90d stale). Confirm via the picker, I'll claim its lease atomically, then route you to
   /session deep."</example>
+user-invocable: true
+argument-hint: "[--dry-run] [--repo <name>]"
 model: sonnet
 ---
 
@@ -17,6 +19,25 @@ model: sonnet
 
 > Cross-repo autopilot front-door — enumerate → rank → owner-AUQ → atomic claim → route. Read-only until the operator confirms; the only mutating step is the atomic `session.lock` claim, and it happens BEFORE any launch.
 
+## Invocation
+
+Invoked as `/dispatcher [--dry-run] [--repo <name>]` with arguments: **$ARGUMENTS**.
+
+Parse `$ARGUMENTS` before doing anything else. The recognized flags are exactly those of the CLI table below and are passed straight through to `scripts/lib/dispatcher/cli.mjs`:
+
+- `--dry-run` — run the non-mutating rank only; print the recommendation and the free-candidate table; do NOT claim any lease (skip Phase 3).
+- `--repo <name>` — limit the human-readable output to a single `repoName` (informational; does not change ranking).
+- `--start-dir <path>` — override the scan root (defaults to the confinement root).
+- `--json` — emit the full `{ candidates, free, ranked, warnings, recommended }` object to stdout.
+
+If `$ARGUMENTS` contains an unrecognized flag (starts with `--` but is not one of the above), inform the user:
+
+```
+Unknown flag '<flag>'. Recognized flags: --dry-run, --repo <name>, --start-dir <path>, --json.
+```
+
+Then continue with the remaining valid arguments.
+
 ## Soul
 
 The dispatcher answers one question: *"of all my repos, which is the most worthwhile to work on right now, and is it free?"* It scans the confinement-root children, resolves each repo's free/busy status from its `session.lock` v2 lease (same lease semantics as the vault-status board), ranks only the FREE ones by `priority × staleness × readiness`, and recommends the single best one. You confirm via a picker, it claims the lease atomically (winning the race or excluding-and-re-ranking on a loss), then routes you to the entry command for that repo. Busy repos are listed-as-such, never selected.
@@ -119,13 +140,23 @@ Do NOT reinvent the claim — always go through `claimRepo`/`acquire`. The `ok:f
 
 ## Phase 4: Route
 
-With the lease held, the **coordinator** invokes the chosen entry slash-command for repo R:
-
-- `/session housekeeping` or `/session deep` — execution modes.
-- `/plan` — read-only planning precursor (produces a wave plan; does not execute).
-- `/discovery` — read-only investigation precursor (maps scope; does not execute).
-
-`/plan` and `/discovery` are **read-only precursors**, NOT execution modes — the menu may route to them, but they only produce artifacts for a later execution session. The full mode taxonomy lives in the mode-selector surface (P2 of this epic); the dispatcher only routes to the entry command the operator picked.
+With the lease held, routing splits on whether the chosen entry is model-invocable:
+
+- **`/session housekeeping` or `/session deep`** (execution modes) and **`/plan`** (read-only
+  planning precursor) are NOT something the coordinator can invoke itself. `commands/session.md`
+  documents `session` and `plan` as **reserved terminal-only built-in names** — under
+  non-interactive (`claude -p`) invocation the bare form answers `"isn't available in this
+  environment"` — and `skills/plan/SKILL.md` additionally carries
+  `disable-model-invocation: true`, which blocks the `Skill` tool from invoking it regardless of
+  interactivity. The coordinator therefore **hands the operator the exact command to type**, one
+  line per option — `/session-orchestrator:session <mode>` or `/session-orchestrator:plan
+  [new|feature|retro]` — rather than attempting to invoke either itself.
+- **`/discovery`** — read-only investigation precursor (maps scope; does not execute).
+  `skills/discovery/SKILL.md` carries no `disable-model-invocation` flag, so the coordinator MAY
+  invoke it directly via the `Skill` tool when the operator picks this option — no hand-off line
+  needed.
+
+`/plan` and `/discovery` remain **read-only precursors**, NOT execution modes — the menu may route to them, but they only produce artifacts for a later execution session. The full mode taxonomy lives in the mode-selector surface (P2 of this epic); the dispatcher only routes to the entry command the operator picked.
 
 ## Phase 5: Edge cases
 

package/skills/eli5/SKILL.md

@@ -1,6 +1,8 @@
 ---
 name: eli5
 description: Explain a topic like I'm a 5 year old — restate my last output, or a named topic, in plain words without dropping a single fact. Use when the user types /eli5 [topic], or says an answer was too technical, too long, or unclear about what he now has to do.
+user-invocable: true
+argument-hint: "[topic]"
 model: inherit
 tools: Read, Grep, Glob, Bash
 ---
@@ -9,6 +11,15 @@ tools: Read, Grep, Glob, Bash
 
 Say it again in plain words. Same facts, in the order he needs them.
 
+## Invocation
+
+Invoked as `/eli5 [topic]` with arguments: **$ARGUMENTS**.
+
+The argument is optional and is a topic, in prose. If `$ARGUMENTS` is empty, the target is my own last substantial output in this conversation; if there is none yet, say so rather than picking a topic for him.
+
+- `/eli5` — restate what I just said.
+- `/eli5 warum ist der Regel-Korpus voll?` — explain that, grounded in what this session measured.
+
 ## The frame
 
 **Write for someone who knows this project but has not seen what you just saw.**

package/skills/eval/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: eval
 user-invocable: true
+argument-hint: "[--session <id>] [--no-write] [--verify <run-id>]"
 tags: [eval, measurement, quality, meta, standard]
 model: sonnet
 model-preference: sonnet
@@ -14,7 +15,7 @@ args-schema:
   - flag: --verify
     description: "Re-evaluate a stored run-id and diff per-dimension for scoring drift (exit 1 on drift)"
 description: >
-  Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v1 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
+  Use this skill to run an honest session-process evaluation (Standard v1, aiat-llm-eval/1.0) — score the last completed orchestrator session against the pre-registered rubric-v2 dimensions, run /eval, evaluate this session, produce an eval report, or re-verify a stored eval run for reproducibility. Deterministic-first with an optional advisory LLM judge; never produces a global score.
 ---
 
 > **Platform Note:** State files use the platform's native directory: `.claude/` (Claude Code), `.codex/` (Codex CLI), or `.cursor/` (Cursor IDE). Shared metrics + the eval journal live in `.orchestrator/metrics/`. See `skills/_shared/platform-tools.md`.
@@ -22,14 +23,30 @@ description: >
 # Eval Skill — Session-Process Evaluation (aiat-llm-eval/1.0)
 
 On-demand, honest measurement of ONE completed orchestrator session against the
-pre-registered **rubric-v1** check set. The deterministic engine
+pre-registered **rubric-v2** check set. The deterministic engine
 (`scripts/eval-session.mjs` → `scripts/lib/eval/engine.mjs`) reads only local
-metrics files (`sessions.jsonl` + `events.jsonl`), scores the five deterministic
+metrics files (`sessions.jsonl` + `events.jsonl`), scores the six deterministic
 dimensions, appends a `session-eval` record to the journal, and optionally
-renders an HTML report. An opt-in LLM judge overlays two advisory dimensions.
+renders an HTML report. An opt-in LLM judge overlays ONE advisory dimension
+(`instruction-adherence`; `report-quality` was retired in rubric-v2, #1381).
 
 The standard this skill implements is [`docs/eval/aiat-llm-eval-v1.md`](../../docs/eval/aiat-llm-eval-v1.md);
-the frozen, content-hashed check set is [`skills/eval/rubric-v1.md`](./rubric-v1.md).
+the frozen, content-hashed check set is [`skills/eval/rubric-v2.md`](./rubric-v2.md)
+(stored records written before 2026-09-19 carry `rubric-v1` and are read against
+[`rubric-v1.md`](./rubric-v1.md), which is never edited again).
+
+## Invocation
+
+Invoked as `/eval [--session <id>] [--no-write] [--verify <run-id>]` with arguments: **$ARGUMENTS** (parsed in Phase 1.2).
+
+- `/eval` — evaluate the last completed session (resolution cascade), append the record, render the report.
+- `/eval --session <id>` — evaluate a specific `session_id`.
+- `/eval --no-write` — evaluate + report without appending to the journal (dry-run).
+- `/eval --verify <run-id>` — re-score a stored run and diff for drift (the reproducibility proof; exit 1 on drift).
+
+**On-demand `/eval` runs regardless of `eval.enabled`** — that flag gates only the automatic session-end eval phase (see Phase 1.1).
+
+**Seams used:** `scripts/eval-session.mjs` (deterministic CLI) · `runEvalJudge` / `mergeJudgeDimensions` (`scripts/lib/eval/judge.mjs`, opt-in) · `writeEvalReport` (`scripts/lib/eval/report.mjs`) · `appendEvalRecord` (`scripts/lib/eval/sink.mjs`) · the `eval` config block · [`skills/eval/rubric-v2.md`](./rubric-v2.md) (frozen check set).
 
 ## Posture Contract (load-bearing — read before executing)
 
@@ -38,9 +55,9 @@ the frozen, content-hashed check set is [`skills/eval/rubric-v1.md`](./rubric-v1
 - **Never guess.** Missing source data yields `cannot-determine` (a first-class,
   non-error verdict) with an honest reason — never a fabricated `pass`/`fail`.
   Do NOT "fill in" a missing KPI or infer a gate result the events do not show.
-- **Deterministic before judge.** The five deterministic dimensions are complete
-  on their own. The judge (Phase 3) is opt-in, ADVISORY, and `uncalibrated` in
-  v1 — never blend a judge verdict into the deterministic tally.
+- **Deterministic before judge.** The six deterministic dimensions are complete
+  on their own. The judge (Phase 3) is opt-in, ADVISORY, and `uncalibrated` —
+  never blend a judge verdict into the deterministic tally.
 - **Journal is SSOT; the report is a derived view.** The append-only
   `.orchestrator/metrics/eval.jsonl` is authoritative. The HTML report is
   rebuildable from any stored record and is never authoritative over the journal.
@@ -134,10 +151,13 @@ node scripts/eval-session.mjs [--session <id>] --json \
   On exit `1` (e.g. "no completed session found"), surface the message and stop —
   do not retry with fabricated inputs.
 
-Parse the emitted JSON record. It carries `dimensions[]` (5 deterministic
-entries), `kpis{}`, `provenance.rubric_sha256` (non-null once `rubric-v1.md`
-exists), `model`, `harness`, and `run_id`. Unless `--no-write` was passed, the
-record is already appended to `.orchestrator/metrics/eval.jsonl` by the CLI.
+Parse the emitted JSON record. It carries `dimensions[]` (6 deterministic
+entries — the two reported-only ones, `guard-friction` and `efficiency-kpis`,
+are always `not-applicable`), `kpis{}`, `provenance.rubric_sha256` (the sha256
+of `rubric-v2.md`; `null` means the rubric file was not found and the append
+will fail validation), `model`, `harness`, and `run_id`. Unless `--no-write` was
+passed, the record is already appended to `.orchestrator/metrics/eval.jsonl` by
+the CLI.
 
 **Contamination check:** if the human-render/summary reports a peer-overlapped
 window, note it — `verification-evidence` and `gate-health` will read
@@ -173,9 +193,13 @@ const merged = mergeJudgeDimensions(record, dimensions);
 appendEvalRecord(merged, { path: '.orchestrator/metrics/eval.jsonl' });
 ```
 
-- Every judge dimension arrives `advisory: true` + `calibration_status:
-  "uncalibrated"` (the schema firewall rejects any other shape). Keep them
-  visibly separated from the deterministic five in the summary.
+- The judge dimension arrives `advisory: true` + `calibration_status:
+  "uncalibrated"` (the schema firewall rejects any other shape). Keep it
+  visibly separated from the deterministic six in the summary.
+- `runEvalJudge` pre-computes a `facts` block (`computeRecordFacts`) from the
+  deterministic evidence strings and hands it to the judge OUTSIDE the
+  untrusted-data fence — anything countable is counted in code, never inferred
+  by the model. Do not reimplement that here.
 - If `runEvalJudge` returns a non-ok `status` (e.g. dispatch failed), keep the
   deterministic record as-is and note the judge was unavailable — the
   deterministic evaluation is complete without it.
@@ -212,20 +236,20 @@ const res = writeEvalReport(record, { generatedAt: new Date().toISOString() });
 Emit a compact, honest per-dimension summary. Status lines only — no global score.
 
 ```
-## /eval — <session_id>  (self-evaluation, aiat-llm-eval/1.0 · rubric-v1 · n=1, no CI)
+## /eval — <session_id>  (self-evaluation, aiat-llm-eval/1.0 · rubric-v2 · n=1, no CI)
 
 Deterministic:
   verification-evidence   PASS   <one-line evidence>
   plan-fidelity           PASS   completion_rate=1.0 (score)
   gate-health             PASS   <one-line evidence>
-  process-safety          PASS   <one-line evidence + guard-emission disclosure>
+  process-safety          PASS   <one-line evidence + both guard disclosures>
+  guard-friction          N/A    (reported: blocked=… warned=… loop.warning=… <attribution>)
   efficiency-kpis         N/A    (reported: duration=…s waves=… agents=… tok_in=… tok_out=… carryover=…)
 
 Judge (advisory, uncalibrated)  [only when eval.judge != off]:
-  instruction-adherence   <verdict>   advisory
-  report-quality          <verdict>   advisory
+  instruction-adherence   <verdict>   advisory   (rule <n> applied)
 
-cannot-determine: <k> of 5 deterministic dimensions (<reasons>)
+cannot-determine: <k> of 6 deterministic dimensions (<reasons>)
 Report:  .orchestrator/eval/reports/<run_id>.html
 Journal: .orchestrator/metrics/eval.jsonl  (appended: <yes|--no-write>)
 Re-verify: node scripts/eval-session.mjs --verify <run_id>
@@ -252,6 +276,11 @@ node scripts/eval-session.mjs --verify <run-id> --json
   changed since the record was written — investigate, do not overwrite.
 - `--verify` reproduces the stored model + timestamp verbatim (no env override),
   so a MATCH is a real reproducibility proof of the scoring, not of model output.
+- **A cross-version DRIFT is not a defect.** A stored `rubric-v1` record
+  re-scored by today's rubric-v2 engine necessarily differs on `process-safety`
+  and reports `present-in-fresh-only: guard-friction`. Read the record's
+  `rubric_version` before treating a diff as a regression (#1400 replaces that
+  report with an explicit version verdict).
 
 ---
 
@@ -268,9 +297,9 @@ platform. Only the judge phase needs harness-specific tooling.
   `skills/_shared/platform-tools.md` § Agent Dispatch Pattern.
 - **`harness.platform`** on the record is resolved from `$SO_PLATFORM`
   (falls back to `claude-code`) inside the engine — no skill action needed.
-- The deterministic five dimensions + the HTML report + `--verify` are fully
-  available on all platforms; the judge overlay is a Claude-Code-only enrichment
-  in v1.
+- The deterministic six dimensions + the HTML report + `--verify` are fully
+  available on all platforms; the judge overlay is a Claude-Code-only
+  enrichment.
 
 ---
 

package/skills/eval/rubric-v1.md

@@ -5,6 +5,7 @@
 - **Conforms to standard:** `aiat-llm-eval/1.0` — see [`docs/eval/aiat-llm-eval-v1.md`](../../docs/eval/aiat-llm-eval-v1.md)
 - **Reference engine:** [`scripts/lib/eval/engine.mjs`](../../scripts/lib/eval/engine.mjs) (the executable scorers this document mirrors verbatim)
 - **Hash binding:** the sha256 of THIS FILE is written to every record's `provenance.rubric_sha256`.
+- **Superseded for NEW records by [`rubric-v2.md`](./rubric-v2.md) (2026-09-19, issue #1037).** This file stays frozen and authoritative for every record already carrying `rubric_version: "rubric-v1"` — its formulas are the ones those verdicts were produced under and must not be edited. v2 changed exactly two things: `process-safety` no longer fails on `destructive_guard.blocked`, and the guard counts moved to a new reported-only `guard-friction` dimension. The reasoning and the measurement live in rubric-v2 § Änderungen gegenüber v1.
 
 > **Pre-Registration (leading principle, standard §1.1).** The checks below are
 > **fixed BEFORE the first scored run executes against this rubric**. This document