@r1z4x/tezgah 1.6.0 → 1.7.0

package/docs/operations.md

@@ -224,7 +224,7 @@ After a change to the contract text (`hooks/tezgah_policy.py`,
 - `--refresh` does that without a reinstall, for every armed host's static
   block and not only opencode's (`refresh_contract`). opencode has no
   session-start hook, so its plugin runs it once per session
-  (`hosts/opencode/plugins/tezgah.js:2666-2670`). Until 2026-10-02 it
+  (`hosts/opencode/plugins/tezgah.js:2689-2693`). Until 2026-10-02 it
   re-rendered opencode's files alone and reset the one shared hash, so a rule
   edit left the other static files stale while `--report` went green (audit
   M-5).
@@ -371,6 +371,7 @@ does not hold.
 | `tezgah-doctor` | nothing: sizes, session and event counts, whether opencode is running, the two context-hygiene settings, the `.codegraph` bytes and index presence per repository, and the three host state dirs (`bin/tezgah-doctor::collect`, `bin/tezgah-doctor::report`) |
 | `tezgah-doctor --clean` | vacuums opencode's database, and only when opencode is not running (`bin/tezgah-doctor::vacuum_db`, `bin/tezgah-doctor::main`); deletes every evidence ledger whose newest row is older than `--retention-days` (default 30), rows, import offsets and old JSONL file together (`hooks/tezgah_store.py::sweep_evidence`), and the turn stamps and used-tool marks last written before then (`hooks/tezgah_store.py::sweep_cache`); and deletes the files not modified for that long - `classify.log`, `context-drops.log`, `debug.log` and the `turns/` and `sessions/` files a running older release still writes - in the cache and its sandbox fallback, keeping the current session's (`$TEZGAH_SESSION`) ledger, stamp and marks whatever their age (`sweep_state`) |
 | `tezgah-doctor --import-evidence` | imports every old `evidence/*.jsonl` file into the cache's `tezgah.db` now, in the cache and its sandbox fallback (`bin/tezgah-doctor::import_evidence`); session start runs the same import detached at most once a day (`hooks/tezgah_store.py::import_later`) |
+| `tezgah-doctor --repair-evidence [--apply]` | counts, per session, the evidence rows an old `evidence/*.jsonl` file put in again when a volume remount gave it another device id, in the cache and its sandbox fallback. Every `imported` row of one file is one import stream, which put the file's first `bytes` in. For a row text, the longest stream's count stays and every other stream's count is a copy: that many of the newest rows go, so a row written to the database directly beside them stays. A session holding a file line fewer times than its streams put it in is left whole and printed for manual review. A file gone from its name, holding another inode or shorter than its longest stream is printed and skipped, not guessed (`bin/tezgah-doctor::evidence_plan`). It writes nothing without `--apply`, which backs `tezgah.db` up to `tezgah.db.bak-<YYYYmmdd-HHMMSS>` through SQLite's backup API, deletes those rows and leaves one import offset per file in one transaction. Before that transaction commits it counts the rows against the plan, and it rolls back and exits 1 when a row it kept is gone or a copy it deleted is still there (`bin/tezgah-doctor::repair_evidence`) |
 | `tezgah-doctor --coverage` | every tracked file the codegraph index does not hold, in two classes — a supported extension the index is missing, and a shebang-only script with no `.py` twin — with the file counts it read, so an empty result cannot read as "everything is covered" |
 | `tezgah-doctor --stack` | per host: the context files in load order, the skill dirs and which copy wins, the hook rows per event with their owner, the MCP servers, and every conflict; exits 1 on one ([hosts](hosts.md#precedence-what-each-host-loads-and-in-which-order), `hooks/tezgah_stack.py::report`) |
 | `tezgah-doctor --prune-sessions DAYS` | deletes sessions idle longer than DAYS through `opencode session delete`, then vacuums; skipped when opencode is running or its CLI is missing (`bin/tezgah-doctor::prune_sessions`, `bin/tezgah-doctor::main`) |
@@ -412,7 +413,7 @@ ledger row. A rollback therefore never reads it as a pre-state (`capture_after`,
 `hooks/tezgah_snapshot.py::capture_after`). A file outside the repository gets no
 after blob.
 opencode reaches the same call through a process (`noteTaste`,
-`hosts/opencode/plugins/tezgah.js:2413-2423`).
+`hosts/opencode/plugins/tezgah.js:2436-2446`).
 
 ```sh
 touch ~/.config/tezgah/taste-on               # arm; rm it to disarm
@@ -425,7 +426,8 @@ bin/tezgah-taste export                       # accepted learnings into AGENTS.m
 bin/tezgah-taste label; bin/tezgah-taste label ID preference   # the calibration sample
 bin/tezgah-taste calibrate                    # the bound a rule needs
 bin/tezgah-taste gate                         # the pooled before/after test
-bin/tezgah-taste mine --host omp --stats      # sessions, writes, prompts after a writing turn
+bin/tezgah-taste mine --host omp --stats      # this repository's sessions, writes, prompts after a writing turn
+bin/tezgah-taste mine --all-projects          # every session under ~/Projects, each repo pseudonymised
 bin/tezgah-taste measure --samples S --labels L [--typed]   # judge precision/recall
 ```
 
@@ -614,6 +616,102 @@ it costs nothing there. On held-out Turkish phrasings of the docs index,
 meaning alone found the page in the top three for 0.91 with `embed-mrl` and 0.78
 with `embed-m2v`. The fusion found it for 0.75 and 0.74, BM25 for 0.68.
 
+## Slice journal: `tezgah-task slice`
+
+A delegated slice can die with its parent. A crash, an omp `/clear`, a quota or
+429 answer, or a model or host switch ends the parent session. The child's
+worktree and commits stay, but no one remembers them. The slice journal is that
+memory, and it is host-neutral.
+
+Each spawned slice gets one row in `<repo root>/.tezgah/slices/slices.db`, a SQLite
+file opened like the taste store (`hooks/tezgah_task.py::SLICE_SCHEMA`).
+An older release wrote one JSON file per record there. The first open imports each one and deletes it.
+`.gitignore` excludes `.tezgah/`, a private directory, so the journal never reaches a commit.
+The id is the host's own agent name: the omp task `name` or the Codex
+`task_name`. A Claude `Agent`/`Task` input may carry a `name` too. Otherwise the
+id is the first 12 hex digits of the brief's sha1. A record holds the host, the agent
+type, the first 4000 characters of the brief, the parent session and the cwd.
+It also holds the linked worktree the brief names, its branch, and the root's
+HEAD at open. Its `status` is `open`, `done`, `stopped`, `resumed` or `stale`.
+Its `failure` is one of `quota`, `rate_limit`, `aborted`, `timeout`, `cut_off`
+or `error`. `attempts` counts each re-open of the same id
+(`hooks/tezgah_task.py::open_slice`, `hooks/tezgah_task.py::close_slice`).
+Another session's record of the same name is not re-opened under another brief.
+The spawn gets this session's own record of that name, else `<name>-<sha1[:6]>` of its brief.
+`name` keeps the spawned name, and the same brief from another session still re-opens the record
+(`hooks/tezgah_task.py::_own_id`). The record stays in cwd's journal, the one its
+owner's Stop rule, resume block and review guard read. When every absolute path its
+brief names lies in one other repository, `repo` names that repository, and the
+slice line says `(in <name>)` (`hooks/tezgah_task.py::_brief_repo`). A repository's
+own `.tezgah` counts as that repository (`hooks/tezgah_task.py::main_repo`).
+
+The slice's own task text (without a batch's shared `context`) decides its
+worktree and its lease. The worktree is the one the task names last, else the
+brief (`hooks/tezgah_task.py::_worktree_in`). `declared` holds the checkout a
+`Worktree:`, `Checkout` or `Cwd:` line declares, which only the main-checkout
+refusal reads (`hooks/tezgah_task.py::_declared_in`). Its `Owns:` lines name
+repository-relative globs, and only that slice may write those paths while its
+record runs (`hooks/tezgah_task.py::owned_globs`, `hooks/tezgah_task.py::owner_of`). A
+Claude background child's record also keeps the launch's `agentId` as `child`,
+which its own calls carry (`hooks/tezgah_task.py::slice_of`). The
+gate holds siblings to the lease (`docs/gate.md`, the Lease rule), and the
+spawn-time warning names two writing slices that share one worktree
+(`hooks/tezgah_task.py::worktree_clash`).
+
+The hooks write it. A spawn call - omp `task`, Claude, dsh and Cursor
+`Agent`/`Task`, Codex `spawn_agent`, opencode `task` - opens one record per
+slice (`hooks/tezgah_task.py::spawned`, `hooks/tezgah_task.py::note_spawn`).
+Claude, dsh, Cursor and opencode answer the call once the child ended. So the
+record closes at once (`hooks/tezgah_task.py::classify`). A failed call becomes
+`stopped` with its failure class, and a good one `done`. The omp `task` and Codex
+`spawn_agent` calls answer while the child runs, so the record stays `open`. So
+does a Claude launch sent with `run_in_background` or one whose result names a
+background child. That child's SubagentStop closes it as `done`
+(`hooks/tezgah_task.py::close_child`). An omp
+record settles from the newest child transcript in the parent's session dir.
+omp names a same-name re-dispatch `<name>-2.jsonl`. The last 64 KB of that file
+decide `stopped` or `done` (`hooks/tezgah_task.py::pending`). A record
+nothing moved for six hours closes `stale`, and a background `job` after
+omp's 3600 s deadline. Codex reports no child end tezgah can see, so its
+records end `stale`. Every prompt lists up to three pending records, newest
+first, each with the command that resumes it (`hooks/tezgah_task.py::slice_lines`).
+They are the records this session spawned or nobody owns, such as one opened by hand. The resume block at
+session start and after a compaction adds one line that counts the other
+sessions' records. `tezgah-task slice reconcile` lists them all.
+
+| Command | What it does |
+|---|---|
+| `tezgah-task slice open --id ID [--kind agent\|job] [--host H] [--agent A] [--brief TEXT\|--brief-file F] [--parent P] [--cwd PATH]` | records a slice by hand, for a host whose spawn the hooks do not see |
+| `tezgah-task slice done ID` / `slice stop ID [--class C]` | closes a record as done, or as stopped with a failure class. An exact record id answers from any session. A spawned name answers too, and under `TEZGAH_SESSION` only for that session's records and unowned ones (`hooks/tezgah_task.py::find_slice`) |
+| `tezgah-task slice reconcile [--prune]` | lists every pending record with a resume brief. `--prune` also removes landed worktrees and marks those records done. It deletes a landed local branch only when no worktree holds it, it tracks no upstream and its reflog shows a commit of its own; it never deletes main, master or the current branch |
+| `tezgah-task slice resume ID` | prints the native path first: omp `write agent://<id>` while the parent lives, Claude `SendMessage` when the id is an agent name. Otherwise it prints a re-dispatch brief: the original brief plus the last commit, the uncommitted files and "do not redo committed steps". A quota or rate-limit stop is sent to another provider. It sets the status to `resumed` |
+| `tezgah-task slice stats [--window DAYS] [--json]` | counts records per status, failure class and host. A `class+host` signature with 3 or more events is an RSI lever candidate, and so is one with 5% of a window that holds 20 or more slices. The line names the lever |
+| `tezgah-task slice board [--parent P]` | lists the open slices one parent spawned - the session's own, or for a subagent its parent's - newest first: the `Owns:` globs, worktree, branch and last commit of each, bounded to 12 lines (`hooks/tezgah_task.py::board`) |
+| `tezgah-task slice merge-plan [--base REF]` | forecasts every done slice branch that carries unmerged work against REF (HEAD) and against the others with `git merge-tree`, and leaves out a `merge=union` file such as CHANGELOG.md. A branch touching `MANIFEST` or the citation baseline comes last, the rest fewest conflicts first. A branch merge-tree cannot forecast is printed as `not judged` and the exit is 1 (`bin/tezgah-task::merge_plan`) |
+
+`tezgah-task review NNN <reviewer> approve` refuses while an open, stopped or
+stale record names that reviewer as its id, agent or spawned name. Under
+`TEZGAH_SESSION` only that session's records and unowned ones count. The refusal
+prints the exact id to resume or close. A record keeps the plan
+that was active when it opened, and only that plan's records count. A record
+with no plan counts when it changed after the plan's checkpoint commit. A
+`resumed` record hands the guard to its re-dispatch's own record
+(`bin/tezgah-task::unfinished_review`).
+The lever map is a closed set. `rate_limit` and `quota` point at the
+`hooks/tezgah_models.py` fallback chain. `cut_off` and `timeout` point at a
+per-slice deadline, `aborted` at the `wip:` commit cadence, a duplicate job at
+journalling background jobs. Plain `error` has no lever: read the transcript.
+
+The orchestration text carries the session's rule
+(`hooks/tezgah_policy.py::ORCHESTRATE`, `hooks/tezgah_agents.py::_orch_body`). A
+slice commits `wip: <step>` after each step. The session resumes or re-dispatches
+a stopped slice and never absorbs it inline. `slice board` shows the sibling
+leases and `slice merge-plan` orders the landed branches. `slice reconcile
+--prune` runs before the turn that merged slices ends.
+The `~/.config/tezgah/slices-off` switch
+stops the hooks from writing records (`hooks/tezgah_paths.py::UNCLASSIFIED_SWITCHES`).
+A cwd outside a git repository writes nothing either.
+
 ## Troubleshooting
 
 | Symptom | First check | Likely cause |
@@ -760,6 +858,8 @@ page exits 1 with the message it always printed (`available()`,
   its pins, the fetch and conversion, the reader and `fuse`; the `embed-mrl` and
   `embed-m2v` registry rows that `--enable`, `--disable` and `--features` read.
   `tests/test_embed.py` builds tiny models in the test and pins the rest.
+- `hooks/tezgah_task.py`, `bin/tezgah-task` — the slice journal: the record, the
+  spawn parser, reconcile, resume, prune and stats.
 - `hooks/tezgah_paths.py`, `hooks/tezgah_context.py` — the paths and `off()`; the
   marks and the kill-switch filtering.
 - `RELEASING.md`, `README.md`, `AGENTS.md`, `.gitignore`, `tests/test_setup.py` —

package/docs/research.md

@@ -65,7 +65,9 @@ nothing recorded the ask and `close --limit` accepted any string.
 `state.json` `rules` is the line's format version. `init` writes `RULES`, the
 newest set (`hooks/tezgah_research.py::RULES`). Each set keeps its own gate. Rules 2
 are the standards rules. Rules 3 add the ask contract. Rules 4 add the protocol
-sections below. Rules 5 add the scope lock. A bump never weakens an older set.
+sections below. Rules 5 add the scope lock. Rules 6 add the selection warnings
+("Held-out set, repeats and noise band" below). Rules 7 add the `exploratory`
+method ("Rules 7: exploratory lines" below). A bump never weakens an older set.
 
 A `rules` above `RULES` means a later tezgah wrote the line. `check` then reports
 one error that names both numbers and reads nothing else. The writers refuse the
@@ -121,7 +123,8 @@ ideation moves had no fence. Rules 5 lock a line to its question:
 
 - `init --in "<covered>" --out "<left alone>"` (repeatable) write `state.json`
   `scope`. The in-scope items are `S1`, `S2`, ... in order. `--method
-  qualitative|measured` writes `method`. `scope_problems`
+  qualitative|measured|exploratory` writes `method` (`exploratory` from rules 7,
+  see "Rules 7: exploratory lines"). `scope_problems`
   (`hooks/tezgah_research.py::scope_problems`) names an empty `scope.in`, an
   empty `scope.out` and a missing method. `check` refuses them past `bootstrap`
   and warns at it.
@@ -149,11 +152,14 @@ gets the question, its scope and its method in its brief. The research skill's
 
 | Path | Holds |
 |---|---|
-| `state.json` | the question, the phase (`bootstrap`/`inner`/`outer`/`concluded`), the direction, `method` and `scope` (rules 5, see "Scope lock"), the locked evaluation and its optional second gate (`capability_tolerance`, `counter_metric`), the session events (`PHASES`, `hooks/tezgah_research.py::PHASES`), `rules` (the rule set `init` opened it under, `RULES`), `deliverable` (`kind`, `path`, `ask`, `min_variants`), `supersedes` for a new version of an older line, and `closed` once `close` records a deliberate limit. The phase is the author's declaration; `derived_phase` (`hooks/tezgah_research.py::derived_phase`) reads the *other* authority beside it from the line's own artifacts, and the checker names the two disagreeing |
+| `state.json` | the question, the phase (`bootstrap`/`inner`/`outer`/`concluded`), the direction, `method` and `scope` (rules 5, see "Scope lock"), the locked evaluation and its optional second gate (`capability_tolerance`, `counter_metric`) with its `noise_band`, the optional `held_out` set (rules 6), the `arena` of an exploratory line (`problem`, `budget`, and `evaluator`, `evaluator_sha256`, `metric` for a measurable one - rules 7, see "The arena lock"), the session events (`PHASES`, `hooks/tezgah_research.py::PHASES`), `rules` (the rule set `init` opened it under, `RULES`), `deliverable` (`kind`, `path`, `ask`, `min_variants`), `supersedes` for a new version of an older line, and `closed` once `close` records a deliberate limit. The phase is the author's declaration; `derived_phase` (`hooks/tezgah_research.py::derived_phase`) reads the *other* authority beside it from the line's own artifacts, and the checker names the two disagreeing |
 | `log.md` | the decision log, newest last: one line per decision, experiment, dead end or pivot, with the evidence that drove it |
 | `findings.md` | the four sections every line answers, named by `FINDINGS_SECTIONS` (`hooks/tezgah_research.py::FINDINGS_SECTIONS`) |
 | `claims.jsonl` | one JSON object per row: `statement`, `falsification`, `proof`, `provenance`, `status`, `kind`, `trace` (the in-scope item, rules 5), `scope` (what the claim's numbers were measured on, which may not be wider than the rows it rests on), and `supersedes` when the row replaces an earlier claim |
 | `predictions.jsonl` | one JSON object per row: `commit` (a 40-character sha the row is bound to), `claim` (an id the line holds, or empty when the prediction stands alone), `metric`, `value_before`, `value_after`, `falsifier`, `components` (the keys of the components the change touches, read from `hooks/tezgah_components.py`), and `granted_by` when a human granted what the frozen paths below forbid |
+| `ideas.jsonl` | an exploratory line's ideas, one JSON object per row, appended by `idea` (rules 7, see "Three files") |
+| `stages.jsonl` | an exploratory line's stage moves, one row per move, appended by `stage`; an idea's stage is its newest row |
+| `tournament.jsonl` | an exploratory line's judged pairwise matches, appended by `match` |
 | `experiments/<hypothesis>/protocol.md` | what the change is, what it predicts, what result would falsify it, why - committed before the run |
 | `experiments/<hypothesis>/results.jsonl` | the rows the run produced, one JSON object per non-blank line, each carrying a non-empty `source` and, once declared, a `scope` of `real`, `fixture` or `derived`, and - when that scope is `fixture` - a `fixture` description of what was generated |
 | `experiments/<hypothesis>/analysis.md` | what the rows mean, and which claim they move |
@@ -168,7 +174,9 @@ gets the question, its scope and its method in its brief. The research skill's
 file that exists (`init`, `hooks/tezgah_research.py::init`); `predictions.jsonl` is not
 one of them, because a file that exists without a row is a line that looks like
 it predicted something - it appears with the first `predict` the line records,
-and `check` reads an absent one as no rows rather than as a missing file. A slug
+and `check` reads an absent one as no rows rather than as a missing file. The
+three campaign files work the same way. Each appears with its first `idea`,
+`stage` or `match`. A slug
 is lowercase letters, digits and dashes (`valid_slug`,
 `hooks/tezgah_research.py::valid_slug`), because it is the directory name `check`, `status` and
 `claim` look for.
@@ -244,6 +252,13 @@ through its seal.
 | `tezgah-research seal <slug> [--history-lost] --ack "<owner decision>"` | seals a line under `done/` that was concluded before seals existed: plainly, as `conclude` would, when every experiment's order still checks; with `--history-lost`, with that verdict for every experiment whose order the lost history left undecidable. Refuses a line with a real order violation, a plain seal on any order finding, and a `history-lost` seal with nothing lost (`retro_seal`, `hooks/tezgah_research.py::retro_seal`) | 0, 1 refused, 2 misuse |
 | `tezgah-research import <checkout> [<slug>] [--allow-open "<reason>"]` | brings a line - or every line this checkout lacks - from another checkout's `.tezgah` repository with its history; an open line arrives beside open ones only with `--allow-open` (`import_line`, `hooks/tezgah_research.py::import_line`) | 0, 1 refused, 2 misuse |
 | `tezgah-research next --topic <t> [--flow f] [--json]` | prints every step of a product flow with its status from the topic's files under `.tezgah/analysis/<topic>/`, then the first missing or due step's skill and command; `--flow` picks `metric-dashboard` (the default) or `feature`, and `--json` prints the same steps as data (`next_steps`, `hooks/tezgah_product.py::next_steps`) | 0, 2 unknown flow or no `--topic` |
+| `tezgah-research init <slug> --ask "<the user's words>" --method exploratory --in "<covered>" --out "<left alone>" --arena "<problem>" --budget "<budget>" [--evaluator <path> --metric "<metric>"]` | opens a rules-7 exploratory line and writes the arena into `state.json` (`arena_record`, `hooks/tezgah_research.py::arena_record`). `--ask`, `--in` and `--out` are required as on any rules-5 line. Without `--arena` or `--budget` it is misuse; `--evaluator` and `--metric` go together; the evaluator has to be a file in the repository; the four flags on any other method are misuse. Its next step says to commit the arena before the first idea | 0, 1 refused over open lines, 2 misuse |
+| `tezgah-research idea <slug>` | reads one idea row from stdin and appends it to `ideas.jsonl` under the lock, or refuses it with one reason per problem (`append_idea`, `hooks/tezgah_research.py::append_idea`) | 0 written, 1 refused, 2 misuse |
+| `tezgah-research stage <slug> <idea> <stage> [--evidence "<ref or JSON>"] [--reason "<why>"]` | appends one stage move to `stages.jsonl` under the lock, or refuses it. `--reason` is the kill's reason, and a `known` prior art needs it for the kill appended with it (`append_stage`, `hooks/tezgah_research.py::append_stage`) | 0 written, 1 refused, 2 misuse |
+| `tezgah-research match <slug>` | reads one judged match from stdin and appends it to `tournament.jsonl` under the lock, or refuses it (`append_match`, `hooks/tezgah_research.py::append_match`) | 0 written, 1 refused, 2 misuse |
+| `tezgah-research board <slug> [--json]` | the campaign as its files show it: arena, rows, torn tails, each idea's stage, score and next action, the ideas tested and the next step; writes nothing (`board_text`, `hooks/tezgah_research.py::board_text`) | 0, 2 misuse |
+| `tezgah-research dedup <slug> [--threshold <(0, 1]>] [--json]` | clusters the `proposed` ideas by statement and prints the `stage ... duplicate` command for each member; names the backend and the threshold's source; writes nothing (`dedup_text`, `hooks/tezgah_research.py::dedup_text`). A `--threshold` outside `(0, 1]` is misuse | 0, 2 misuse |
+| `tezgah-research yield [--json] [--all-lines]` | per operator and per generator family, how many ideas reached each stage, the kill reasons, the board-score-to-pilot correlation and the next campaign's operator weights; writes nothing (`yield_text`, `hooks/tezgah_research.py::yield_text`) | 0, 2 misuse |
 
 Exit code 2 is always misuse, so a caller can tell it from a line that fails the
 checks (`misuse`, `bin/tezgah-research::misuse`). `check` asks nothing at all - no
@@ -487,18 +502,288 @@ in `hooks/tezgah_research.py`, a module the frozen set forbids a prediction's
 commit to touch without a human `granted_by` (`FROZEN_PATHS`), so the gate that
 decides which candidate survives is not machinery the candidate can rewrite.
 
-What the pair cannot catch: **a tolerance its own author sets wide enough**. The
-band is a sentence, not a measurement, and `check` can see that it is written and
-readable - never that 0.02 is tight. A line that writes "capability metric within
-5.0 of the baseline" has adopted the form of the two-gate rule and none of its
-force, and the review is the only place that is decided.
+What the pair cannot catch alone: **a tolerance its own author sets wide
+enough**. A line that writes "capability metric within 5.0 of the baseline" has
+adopted the form of the two-gate rule and none of its force. From rules 6 the band
+is read against a measurement. `evaluation.noise_band` is
+`{"experiment": "<the experiment dir that reran the unchanged baseline>",
+"spread": <the spread that rerun measured>}`. `check` warns in three cases. The
+line declares `capability_tolerance` and no `noise_band`. `noise_band.experiment`
+names no experiment of the line. The tolerance sentence holds a number wider than
+`spread`, so the tolerance passes what noise alone would. A digit inside a name
+such as `p95` is not that number (`_check_selection`,
+`hooks/tezgah_research.py::_check_selection`). Whether the rerun was a fair one
+stays the review's call.
+
+## Held-out set, repeats and noise band (rules 6)
+
+The measured line `rsi-borrow-screen` found three gaps. Of 27 decisions, 25 select
+a variant and report it on the same cells. Of 23 armed experiments, 13 run once per
+arm. Three lines declare a tolerance, and none of them measured it. Rules 6 add one
+warning for each (`_check_selection`, `hooks/tezgah_research.py::_check_selection`).
+Each stays a warning under `--strict` too, never a refusal (ADR 019):
+
+- **Held-out set.** A measured line may name its held-out set in
+  `evaluation.held_out` (a non-empty string). It may log each look at that set in
+  `held_out_log.jsonl` at the line root, one row
+  `{"set", "candidate", "date", "result"}` per look. `check` warns when a
+  `decisions/<id>/` holds comparison cells and the line names no `held_out`: the
+  variant was selected and reported on the same cells. It also warns when one
+  candidate looked at one set more than once: the held-out set became development
+  data for it.
+- **Repeats.** An experiment is armed when its `results.jsonl` rows carry `arm` or
+  `variant` with two or more values. `check` warns when no row has `repeat`,
+  `repeats` or `k` above 1 and its `protocol.md` states no `k:`/`repeats:` above 1.
+  An armless experiment is not read.
+- **Noise band.** `evaluation.noise_band`, read in "Two gates" above.
+
+A line opened under rules 5 or earlier is not read for any of the three.
+
+## Rules 7: exploratory lines
+
+A `qualitative` or `measured` line answers a question. Some asks want new ideas,
+methods or approaches instead of an answer. Rules 7 add the method `exploratory`
+for those asks (`hooks/tezgah_research.py::EXPLORE_RULES`,
+`hooks/tezgah_research.py::METHODS`). The line runs one campaign. Many ideas
+enter it, a funnel kills most of them, and a run under a frozen evaluator judges
+the few that remain. One campaign is one line, so "One open line at a time" still
+holds. `scope_problems` (`hooks/tezgah_research.py::scope_problems`) accepts
+`exploratory` only on a rules-7 line. On an older line it names `qualitative` and
+`measured` as the two methods the line can take.
+
+### The arena lock
+
+`init <slug> --ask "<the user's words>" --method exploratory --in "<covered>"
+--out "<left alone>" --arena "<problem>" --budget "<budget>"` opens
+the line. The arena is the problem in the user's words. The budget says how many
+ideas, pilots and full runs, and how much wall time, the campaign may spend. A
+measurable arena adds `--evaluator <path> --metric "<metric>"`. The two flags go
+together, and the evaluator has to be a file in the repository. A missing arena
+or budget is misuse (exit 2).
+
+`arena_record` (`hooks/tezgah_research.py::arena_record`) builds `state.json`
+`arena`. It holds the problem and the budget. A measurable arena also holds the
+evaluator path, the sha256 of the evaluator's bytes and the metric. `init` also
+writes `deliverable: {"kind": "finding"}`. The campaign delivers its staged ideas
+as findings, never as compared variants (`hooks/tezgah_research.py::init`).
+
+The lock has three halves:
+
+- `arena_problems` (`hooks/tezgah_research.py::arena_problems`) names an absent
+  arena, an empty problem or budget, and an evaluator with no sha256 or no metric.
+  `check` reads it through `scope_problems`. It refuses these past `bootstrap` and
+  warns at it.
+- `_check_arena_order` (`hooks/tezgah_research.py::_check_arena_order`) finds the
+  lock commit. That commit is the first one along HEAD whose `state.json` arena
+  names a problem and a budget. `ideas.jsonl` and every experiment's
+  `results.jsonl` have to enter the history after it. An arena that differs from
+  the locked one is a refusal too. A new arena means a new line.
+- `evaluator_problem` (`hooks/tezgah_research.py::evaluator_problem`) hashes the
+  evaluator again. While the hash differs from the locked one, the writer and
+  `check` refuse every stage row. The candidate a campaign judges must not move
+  its judge.
+- `append_stage` (`hooks/tezgah_research.py::append_stage`) also compares the
+  working `state.json` arena with the arena at the lock commit
+  (`hooks/tezgah_research.py::_arena_lock`). While they differ, it refuses every
+  stage row. So a rewritten `evaluator_sha256` does not reopen the campaign.
+
+`init` prints the next step: commit the arena with `tezgah-research commit` before
+the first idea. The arena replaces the one locked metric of a measured line.
+`_check_evaluation` (`hooks/tezgah_research.py::_check_evaluation`) and
+`_check_evaluation_order` (`hooks/tezgah_research.py::_check_evaluation_order`)
+skip an exploratory line for that reason.
+
+### Three files
+
+The campaign lives in three append-only files at the line root. One function
+judges each row type, and the writer and `check` both call it. No file stores a
+status. An idea's stage is its newest `stages.jsonl` row
+(`hooks/tezgah_research.py::campaign_stages`).
+
+| File | One row | Writer | Judge |
+|---|---|---|---|
+| `ideas.jsonl` (`IDEAS_FILE`) | `id` (`I<n>`, unused so far), `statement` (two sentences at most), `mechanism`, `trace` (an in-scope item), `operator` (one of `OPERATORS`), `generator` (`{family, model}`), `kill` (the result that would kill it), `date`, and `parents` for a `mutation` (one or more) or a `crossover` (two or more) | `idea <slug>` | `idea_problems` |
+| `stages.jsonl` (`STAGES_FILE`) | `idea`, `stage`, `evidence`, `reason` for a kill, `date` | `stage <slug> <idea> <stage>` | `stage_problems` |
+| `tournament.jsonl` (`TOURNAMENT_FILE`) | `a`, `b`, `judge` (`{family, model}`), `order` (`ab` or `ba`), `winner` (`a`, `b` or `tie`), `rationale` | `match <slug>` | `match_problems` |
+
+The field list is `IDEA_FIELDS` (`hooks/tezgah_research.py::IDEA_FIELDS`). The
+moves are `OPERATORS` (`hooks/tezgah_research.py::OPERATORS`). `user` marks an
+idea the person asked for. `PARENT_OPERATORS`
+(`hooks/tezgah_research.py::PARENT_OPERATORS`) sets how many parents a derived
+idea names. An idea outside the arena's scope gets one line under Open questions
+in `findings.md`, never a row.
+
+Every append goes through `_append_campaign`
+(`hooks/tezgah_research.py::_append_campaign`). It takes the file lock and reads
+the rows from the locked descriptor. It judges the new row against them, cuts any
+torn tail (`truncate_to_committed`) and writes. A refused append leaves no file
+behind. `check` refuses a committed row that a later version changed or dropped.
+
+### The stage funnel
+
+`STAGES` (`hooks/tezgah_research.py::STAGES`) gives the order: `proposed`,
+`duplicate`, `prior-art`, `screened`, `piloted`, `promoted`, `confirmed`, and
+`killed`. `stage_problems` (`hooks/tezgah_research.py::stage_problems`) holds
+the moves to it:
+
+- The idea row is the `proposed` stage. A stage row that says `proposed` is a
+  refusal.
+- A stage moves forward in that order only, or to `killed`. A kill needs
+  `--reason`.
+- A stage also needs its prerequisite somewhere in the idea's history
+  (`hooks/tezgah_research.py::PREREQUISITES`). `screened` needs a `prior-art`
+  row. `piloted` needs `screened`, `promoted` needs `piloted`, and `confirmed`
+  needs `promoted`.
+- `killed` is final. A revived idea is a new idea that names the old one as its
+  parent.
+- After `duplicate`, the only allowed row is `killed`. The same holds after a
+  `known` prior art.
+
+Each stage asks for its own evidence. `--evidence` takes a reference or a JSON
+object.
+
+| Stage | Evidence | What the judge asks |
+|---|---|---|
+| `duplicate` | `{"of": "<id>", "similarity": <0..1>}` | `of` names another idea of the line |
+| `prior-art` | `{"verdict": "novel\|incremental\|known", "nearest": [...], "delta": "<one sentence>"}` | `nearest` names at least `NEAREST_MIN` (3) notes. `literature/INDEX.jsonl` indexes each one, and not all of them are `agent-report`. `delta` is one sentence (`_prior_art_problems`) |
+| `screened` | none | `tournament.jsonl` holds at least one match that names the idea |
+| `piloted`, `promoted` | `<h>` or `experiments/<h>` | `protocol.md` names the idea id and repeats its `kill` text. `results.jsonl` holds a row. When the check asks git, `protocol.md` is in a commit. If `results.jsonl` is committed too, `protocol.md` entered the history before it (`_run_problems`) |
+| `confirmed` | `{"experiment": "<h>", "set": "<set>"}` | the arena has an evaluator. The experiment differs from every `promoted` row's experiment. The run passes the same `_run_problems` checks. The first `held_out_log.jsonl` row for that set names this idea. Each look of this idea at the set carries a `YYYY-MM-DD` date, and none is dated before the idea's promotion: an earlier look was part of the selection. A look logged after the confirmation changes nothing (`_confirm_problems`) |
+| `killed` | `--reason "<why>"` | a reason |
+
+A `known` prior art kills the idea in the same write. `append_stage`
+(`hooks/tezgah_research.py::append_stage`) needs `--reason` for it. It appends
+the prior-art row and its `killed` row under one lock, so no file state holds a
+known idea alive. `check` refuses a `known` prior art with no kill after it.
+
+### The tournament is a prior
+
+A match is one judge's verdict on a pair. `bradley_terry`
+(`hooks/tezgah_research.py::bradley_terry`) fits a strength per idea from the
+matches. `board` ranks the ideas by it. The score moves no stage. `screened`
+asks only that a match exists. An LLM ranking is a prior, and a run under the
+frozen evaluator is the judge.
+
+`_tournament_warnings` (`hooks/tezgah_research.py::_tournament_warnings`) warns
+about thin judging. A pair judged only by the family that generated one of its
+ideas warns, unless both generators' families judged it. A pair judged in one
+presentation order warns. An idea with fewer than two judges warns.
+
+### Reports: board, dedup, yield
+
+The three reports write nothing and exit 0. None of them is a gate.
+
+- `board <slug>` (`hooks/tezgah_research.py::board`) shows the arena and whether
+  the evaluator still hashes as locked. It shows each file's row count, each
+  idea's rank, stage, score, record, operator, generator family and next action,
+  and the campaign's next step. An idea whose prior art is `known` gets the
+  `stage ... killed --reason` command as its next action. That resumes a campaign
+  whose kill row was torn off.
+- `dedup <slug>` (`hooks/tezgah_research.py::dedup`) clusters the `proposed`
+  ideas by statement, at one similarity threshold. The lowest id of a cluster
+  represents it. For every other member it prints the `stage ... duplicate`
+  command to run. `similarity_matrix` (`hooks/tezgah_research.py::similarity_matrix`)
+  uses the static embedding when the feature is on and pinned. Otherwise it reads
+  BM25 scores (`hooks/tezgah_rank.py::scores`) as a similarity. Every output names
+  the backend and where its threshold came from.
+- `yield` (`hooks/tezgah_research.py::yield_report`) counts, per operator and per
+  generator family, how many ideas reached each stage. It lists the kill reasons.
+  It gives Spearman's rho between the board score and the pilot outcome
+  (`hooks/tezgah_research.py::spearman`). It proposes the next campaign's operator
+  weights (`hooks/tezgah_research.py::operator_weights`). `EXPLORE_SHARE` (0.2)
+  of each draw stays uniform, so a move with no payoff yet still gets tried. It
+  reads the open exploratory lines, or every one with `--all-lines`.
+
+### Resume from the files
+
+Nothing outside the three files holds campaign state. A fresh session runs
+`board` and goes on from its `next:` line. A writer that dies mid-append can
+leave a fragment past the last newline. `_campaign_rows`
+(`hooks/tezgah_research.py::_campaign_rows`) reads up to the committed boundary
+and reports the fragment's size. `board` prints it on a `resume:` line. The next
+append drops the fragment, so the fix is to run the append that crashed again.
+
+### How many ideas were tested
+
+Fifty ideas tested at the 5% level yield about 2.5 winners by chance. A reader
+cannot discount what nobody counted. `_check_tested`
+(`hooks/tezgah_research.py::_check_tested`) warns when `to_human/report.md` has
+no line that names `piloted` with the number of ideas that reached it. The same
+holds for `promoted`. A number followed by a full stop counts, as in "reached
+piloted: 2.". `board` prints both numbers on its `tested:` line.
+
+### The dedup threshold (two label sets)
+
+Si et al. called two seed ideas duplicates at cosine 0.8 on MiniLM. That number
+does not carry to another similarity. Two label sets measured it here.
+
+- The fixture: 40 idea-statement pairs, 20 of them duplicates. The author of the
+  code labelled them. The acceptance runner records the sweeps as fixture-scoped
+  experiments: `dedup-calibration` (BM25) and `dedup-calibration-embed`
+  (embed-mrl). Copies stay under `.tezgah/scratch/calibration/`.
+- MRPC: 400 pairs of the Microsoft Research Paraphrase Corpus test set, 200
+  paraphrases and 200 not, sampled with seed 87. Human annotators labelled them.
+  The pairs are news sentences, and the negatives share many words. At BM25 0.30
+  the sample kept every paraphrase and 95% of the negatives. The script and its
+  output are `.tezgah/scratch/calib2/run.py` and `out.txt`.
+
+| Backend | F1 at 0.8 (fixture / MRPC) | Default | F1 at the default (fixture / MRPC) |
+|---|---|---|---|
+| embed-mrl | 0.62 / 0.64 | 0.65 | 0.90 / 0.705 |
+| BM25 | 0.09 / 0.41 | 0.40 | 0.70 / 0.69 |
+
+The default is the threshold, in steps of 0.05, whose worse F1 over the two sets
+is highest. Neither the author's labels nor the news domain picks it alone. On
+MRPC the best F1 beats the always-duplicate baseline (0.667) by about 0.04. The
+defaults are a coarse filter for a session to review, not a judge.
+
+`DEDUP_THRESHOLDS` (`hooks/tezgah_research.py::DEDUP_THRESHOLDS`) holds the two
+values. Any other backend keeps `DEDUP_THRESHOLD`
+(`hooks/tezgah_research.py::DEDUP_THRESHOLD`), Si et al.'s 0.8, and the output
+calls it uncalibrated. `--threshold` overrides all of them. Its range is
+`(0, 1]`.
+
+### The evidence behind the method
+
+- Si et al., arXiv 2409.04109: about 5% of 4,000 generated seed ideas were not
+  duplicates (cosine 0.8). The best LLM judge (a Claude-3.5 pairwise ranker)
+  agreed with the human ranking 53.3% of the time. Split-half reviewer panels
+  agreed 56.1%.
+- The ideation-execution gap, arXiv 2506.20803: after execution, AI ideas lost
+  1.98 points overall and human ideas lost 0.63.
+- Gupta and Pruthi, arXiv 2502.16487: 24% of 50 AI research documents
+  paraphrased or substantially borrowed earlier work. Plagiarism detectors missed
+  them.
+- HindSight, arXiv 2603.15164: novelty as an LLM judges it correlates negatively
+  with realized impact (rho = -0.29).
+- Darwin Godel Machine, arXiv 2505.22954: an archive of stepping stones beats
+  hill-climbing. The loop also removed its own detection markers and faked test
+  logs.
+- ShinkaEvolve, arXiv 2509.19349: parent sampling by fitness and novelty,
+  novelty rejection by embedding, and UCB1 over models.
+- POPPER, arXiv 2502.09858: falsification experiments on measurable implications,
+  with Type-I error control.
+
+The consequences:
+
+- An LLM ranking is a prior, never a verdict.
+- A run under the frozen evaluator is the judge.
+- Each idea gets its own prior-art check. No idea reaches `screened` without a
+  `prior-art` row (`hooks/tezgah_research.py::PREREQUISITES`).
+- `dedup` and the operator weights keep the ideas diverse.
+- The report always states how many ideas the campaign tested.
+
+Rules 1-6, `FROZEN_PATHS` and ADR 009 stay as they were. Predictions and
+components take nothing new. `research-off` covers this method too.
 
 ## What `check` refuses, and what it only warns about
 
 A **refusal** (exit 1) is a guarantee the checker can decide. A **warning** is a
 guarantee it cannot: the artifact is not wrong, it is unverifiable. `--strict`
-turns that whole class into a refusal, which is what a release gate wants and
-what a session mid-flight must not have, so the default stays `warn`.
+turns the warnings the listed rules write through `_soft` into a refusal, which is
+what a release gate wants and what a session mid-flight must not have, so the
+default stays `warn`. A warning written outside `_soft` stays one. The section
+below names three groups of them.
 
 Refused:
 
@@ -577,6 +862,40 @@ Refused:
   nothing, or a finding whose `quote` does not occur verbatim in its `target`
   (`_check_review`, `hooks/tezgah_research.py::_check_review`).
 
+On an exploratory line (rules 7), `check` also refuses each of these
+(`_check_exploratory`, `hooks/tezgah_research.py::_check_exploratory`, called
+from `_check_standards`):
+
+- an arena that `arena_problems` faults, past `bootstrap`. The faults are no
+  arena object, no problem, no budget, and an evaluator with no sha256 or no
+  metric (`hooks/tezgah_research.py::arena_problems`).
+- an arena that differs from the one the lock commit holds. The same goes for an
+  `ideas.jsonl` or `results.jsonl` that entered the history before that commit
+  (`hooks/tezgah_research.py::_check_arena_order`).
+- an idea row that `idea_problems` faults
+  (`hooks/tezgah_research.py::idea_problems`). That means a missing field, a
+  `generator` that is not `{family, model}`, or an id that is not `I<n>` or
+  repeats an earlier one. It also means an operator outside `OPERATORS`, a parent
+  no earlier idea carries, or too few parents for a `mutation` or `crossover`. The
+  last cases are a statement of more than two sentences and a date that is not
+  `YYYY-MM-DD`. A trace to no in-scope item and a missing arena fail it too.
+- a stage row that `stage_problems` faults
+  (`hooks/tezgah_research.py::stage_problems`). The funnel above lists the
+  moves and the evidence each stage asks for. Every stage row fails while the
+  evaluator's hash differs from the locked one
+  (`hooks/tezgah_research.py::evaluator_problem`).
+- a stage row that skips its `PREREQUISITES` stage, and any row but `killed`
+  after `duplicate` or after a `known` prior art.
+- a `known` prior art with no `killed` row after it.
+- a match row that `match_problems` faults
+  (`hooks/tezgah_research.py::match_problems`). That means a side that names no
+  idea, an idea matched against itself, or a `judge` that is not
+  `{family, model}`. It also means an `order` outside `MATCH_ORDERS`, a `winner`
+  outside `MATCH_WINNERS`, or no `rationale`.
+- a campaign file line that does not parse or is not an object
+  (`hooks/tezgah_research.py::_campaign_rows`).
+- a committed campaign row that a later version changed or dropped.
+
 Warned, and refused under `--strict` - each is the honest state of an artifact
 this layer cannot decide on, not a defect:
 
@@ -674,7 +993,18 @@ this layer cannot decide on, not a defect:
   judgement, so no migration can write one for the lines that concluded before
   the rule existed.
 
-One warn is **not** promoted by `--strict`, and it is the only one: a `phase`
+On an exploratory line, two more warnings take this class. One is an arena
+fault at `bootstrap`. The other is a campaign file git cannot order against the
+arena lock (`_check_arena_order`).
+
+Three groups of warnings are **not** promoted by `--strict`. The rules-6
+three (held-out set, repeats, noise band) stay warnings by ADR 019. The rules-7
+four stay warnings for the same reason: a tournament is a prior and a count is a
+disclosure. They are a pair that only a generator family of one of its ideas
+judged, unless both generators' families judged it. They are a pair judged in one
+order, and an idea with fewer than two judges (`_tournament_warnings`). The fourth
+is a `report.md` that does not state how many ideas reached `piloted` and
+`promoted` (`_check_tested`). The last is a `phase`
 declared in `state.json` behind the phase the line's own artifacts show
 (`_check_derived_phase`, `hooks/tezgah_research.py::_check_derived_phase`). The field is the author's
 declared intent and `_check_report` and `_check_review` read it, so a derivation
@@ -872,12 +1202,15 @@ armed. It removes the rule, not just the mark - the diff is in `core_split`,
   frozen set they read (`FROZEN_PATHS`), the protocol order and its tracked pair,
   the protocol's two questions, the concluded report's limits, the `supersedes`
   relation, `component_report` with the state it reads, `migrate`, `source_run`,
-  `init` and `summary`.
+  `init` and `summary`, and the rules-7 campaign: the arena lock, the three row
+  judges and their writers, `board`, `dedup` and `yield_report`.
 - `hooks/tezgah_components.py` - the manifest: the one definition of an editable
   component, the keys a prediction row names and the rules a row's outcome is
   attributed to.
 - `bin/tezgah-research` - the CLI, its subcommands and its exit codes.
 - `hooks/tezgah_product.py` - the product-flow steps and the `next` status reader.
+- `hooks/tezgah_rank.py` - `scores`, the BM25 reading `dedup` falls back to when
+  the static embedding is off.
 - `hooks/tezgah_policy.py` - the `RESEARCH` paragraph a session is armed with.
 - `hooks/tezgah_context.py` - the prompt hint, the missing-orx note, the broken
   line note and the `research` mark.

package/docs/skills.md

@@ -32,7 +32,7 @@ skill is simply not read.
 
 opencode's plugin does inject prompt-time text - the builder's per-turn block,
 the conditional rules included
-(`hosts/opencode/plugins/tezgah.js:2629-2658`) - but that text names at most one
+(`hosts/opencode/plugins/tezgah.js:2652-2681`) - but that text names at most one
 skill (the hint), never the roster, so the installer writes one:
 `~/.config/tezgah/opencode-skills.md` always-on and
 `opencode-skills.full.md` on demand (`bin/tezgah-setup::SKILL_ROUTER`, `bin/tezgah-setup::SKILL_ROUTER_FULL`), the first