@r1z4x/tezgah 1.5.1 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (81) hide show
  1. package/CHANGELOG.md +126 -1
  2. package/MANIFEST +11 -0
  3. package/README.md +6 -3
  4. package/bin/consult +13 -10
  5. package/bin/tezgah-docs +18 -16
  6. package/bin/tezgah-doctor +152 -0
  7. package/bin/tezgah-research +286 -12
  8. package/bin/tezgah-route +48 -1
  9. package/bin/tezgah-setup +27 -14
  10. package/bin/tezgah-task +771 -46
  11. package/bin/tezgah-taste +48 -10
  12. package/docs/architecture.md +7 -4
  13. package/docs/citations-baseline.json +1 -1
  14. package/docs/clarity-baseline.json +5 -5
  15. package/docs/contract.md +112 -25
  16. package/docs/evidence.md +201 -28
  17. package/docs/feature-audit.md +25 -4
  18. package/docs/gate.md +70 -33
  19. package/docs/glossary.md +4 -4
  20. package/docs/hosts.md +8 -8
  21. package/docs/index.json +30 -10
  22. package/docs/judge.md +9 -3
  23. package/docs/models.md +35 -2
  24. package/docs/operations.md +103 -3
  25. package/docs/research.md +392 -13
  26. package/docs/skills.md +24 -6
  27. package/docs/status-line.md +5 -5
  28. package/docs/testing.md +2 -2
  29. package/docs/vision.md +12 -4
  30. package/hooks/hooks.json +2 -2
  31. package/hooks/lang/en.json +323 -0
  32. package/hooks/lang/tr.json +344 -0
  33. package/hooks/projects-posttooluse.py +24 -5
  34. package/hooks/projects-pretooluse.py +9 -4
  35. package/hooks/projects-stop.py +28 -16
  36. package/hooks/tezgah_agents.py +29 -9
  37. package/hooks/tezgah_clarity.py +7 -3
  38. package/hooks/tezgah_components.py +2 -1
  39. package/hooks/tezgah_context.py +460 -390
  40. package/hooks/tezgah_gate.py +640 -104
  41. package/hooks/tezgah_integrity.py +1480 -197
  42. package/hooks/tezgah_models.py +337 -15
  43. package/hooks/tezgah_paths.py +103 -10
  44. package/hooks/tezgah_policy.py +286 -130
  45. package/hooks/tezgah_product.py +107 -0
  46. package/hooks/tezgah_rank.py +29 -27
  47. package/hooks/tezgah_replay.py +200 -30
  48. package/hooks/tezgah_research.py +1297 -12
  49. package/hooks/tezgah_skill_pick.py +58 -17
  50. package/hooks/tezgah_stopspec.py +7 -0
  51. package/hooks/tezgah_store.py +74 -27
  52. package/hooks/tezgah_task.py +917 -8
  53. package/hosts/codex/hook.py +9 -10
  54. package/hosts/cursor/hook.py +21 -14
  55. package/hosts/dsh/hooks.json +2 -2
  56. package/hosts/omp/hook.py +44 -23
  57. package/hosts/omp/tezgah-hook.ts.in +93 -7
  58. package/hosts/opencode/plugins/tezgah.js +119 -39
  59. package/output-styles/tezgah.md +76 -42
  60. package/package.json +1 -1
  61. package/skills/analyze-app/SKILL.md +57 -0
  62. package/skills/design-contract/EVALS.md +24 -1
  63. package/skills/design-contract/SKILL.md +17 -1
  64. package/skills/design-contract/data-views.md +86 -0
  65. package/skills/feature-audit/SKILL.md +32 -3
  66. package/skills/i-have-adhd/EVALS.md +22 -2
  67. package/skills/i-have-adhd/SKILL.md +12 -5
  68. package/skills/plan-add/EVALS.md +8 -8
  69. package/skills/plan-add/SKILL.md +22 -21
  70. package/skills/plan-sync/EVALS.md +2 -2
  71. package/skills/plan-sync/SKILL.md +14 -8
  72. package/skills/pm-frameworks/SKILL.md +6 -0
  73. package/skills/ponytail/EVALS.md +21 -2
  74. package/skills/ponytail/SKILL.md +4 -4
  75. package/skills/product-analysis/EVALS.md +23 -1
  76. package/skills/product-analysis/SKILL.md +50 -11
  77. package/skills/product-flow/EVALS.md +81 -0
  78. package/skills/product-flow/SKILL.md +91 -0
  79. package/skills/research/SKILL.md +99 -2
  80. package/skills/tezgah-contract/EVALS.md +52 -7
  81. package/skills/tezgah-contract/SKILL.md +204 -85
package/docs/research.md CHANGED
@@ -65,7 +65,9 @@ nothing recorded the ask and `close --limit` accepted any string.
65
65
  `state.json` `rules` is the line's format version. `init` writes `RULES`, the
66
66
  newest set (`hooks/tezgah_research.py::RULES`). Each set keeps its own gate. Rules 2
67
67
  are the standards rules. Rules 3 add the ask contract. Rules 4 add the protocol
68
- sections below. Rules 5 add the scope lock. A bump never weakens an older set.
68
+ sections below. Rules 5 add the scope lock. Rules 6 add the selection warnings
69
+ ("Held-out set, repeats and noise band" below). Rules 7 add the `exploratory`
70
+ method ("Rules 7: exploratory lines" below). A bump never weakens an older set.
69
71
 
70
72
  A `rules` above `RULES` means a later tezgah wrote the line. `check` then reports
71
73
  one error that names both numbers and reads nothing else. The writers refuse the
@@ -121,7 +123,8 @@ ideation moves had no fence. Rules 5 lock a line to its question:
121
123
 
122
124
  - `init --in "<covered>" --out "<left alone>"` (repeatable) write `state.json`
123
125
  `scope`. The in-scope items are `S1`, `S2`, ... in order. `--method
124
- qualitative|measured` writes `method`. `scope_problems`
126
+ qualitative|measured|exploratory` writes `method` (`exploratory` from rules 7,
127
+ see "Rules 7: exploratory lines"). `scope_problems`
125
128
  (`hooks/tezgah_research.py::scope_problems`) names an empty `scope.in`, an
126
129
  empty `scope.out` and a missing method. `check` refuses them past `bootstrap`
127
130
  and warns at it.
@@ -149,11 +152,14 @@ gets the question, its scope and its method in its brief. The research skill's
149
152
 
150
153
  | Path | Holds |
151
154
  |---|---|
152
- | `state.json` | the question, the phase (`bootstrap`/`inner`/`outer`/`concluded`), the direction, `method` and `scope` (rules 5, see "Scope lock"), the locked evaluation and its optional second gate (`capability_tolerance`, `counter_metric`), the session events (`PHASES`, `hooks/tezgah_research.py::PHASES`), `rules` (the rule set `init` opened it under, `RULES`), `deliverable` (`kind`, `path`, `ask`, `min_variants`), `supersedes` for a new version of an older line, and `closed` once `close` records a deliberate limit. The phase is the author's declaration; `derived_phase` (`hooks/tezgah_research.py::derived_phase`) reads the *other* authority beside it from the line's own artifacts, and the checker names the two disagreeing |
155
+ | `state.json` | the question, the phase (`bootstrap`/`inner`/`outer`/`concluded`), the direction, `method` and `scope` (rules 5, see "Scope lock"), the locked evaluation and its optional second gate (`capability_tolerance`, `counter_metric`) with its `noise_band`, the optional `held_out` set (rules 6), the `arena` of an exploratory line (`problem`, `budget`, and `evaluator`, `evaluator_sha256`, `metric` for a measurable one - rules 7, see "The arena lock"), the session events (`PHASES`, `hooks/tezgah_research.py::PHASES`), `rules` (the rule set `init` opened it under, `RULES`), `deliverable` (`kind`, `path`, `ask`, `min_variants`), `supersedes` for a new version of an older line, and `closed` once `close` records a deliberate limit. The phase is the author's declaration; `derived_phase` (`hooks/tezgah_research.py::derived_phase`) reads the *other* authority beside it from the line's own artifacts, and the checker names the two disagreeing |
153
156
  | `log.md` | the decision log, newest last: one line per decision, experiment, dead end or pivot, with the evidence that drove it |
154
157
  | `findings.md` | the four sections every line answers, named by `FINDINGS_SECTIONS` (`hooks/tezgah_research.py::FINDINGS_SECTIONS`) |
155
158
  | `claims.jsonl` | one JSON object per row: `statement`, `falsification`, `proof`, `provenance`, `status`, `kind`, `trace` (the in-scope item, rules 5), `scope` (what the claim's numbers were measured on, which may not be wider than the rows it rests on), and `supersedes` when the row replaces an earlier claim |
156
159
  | `predictions.jsonl` | one JSON object per row: `commit` (a 40-character sha the row is bound to), `claim` (an id the line holds, or empty when the prediction stands alone), `metric`, `value_before`, `value_after`, `falsifier`, `components` (the keys of the components the change touches, read from `hooks/tezgah_components.py`), and `granted_by` when a human granted what the frozen paths below forbid |
160
+ | `ideas.jsonl` | an exploratory line's ideas, one JSON object per row, appended by `idea` (rules 7, see "Three files") |
161
+ | `stages.jsonl` | an exploratory line's stage moves, one row per move, appended by `stage`; an idea's stage is its newest row |
162
+ | `tournament.jsonl` | an exploratory line's judged pairwise matches, appended by `match` |
157
163
  | `experiments/<hypothesis>/protocol.md` | what the change is, what it predicts, what result would falsify it, why - committed before the run |
158
164
  | `experiments/<hypothesis>/results.jsonl` | the rows the run produced, one JSON object per non-blank line, each carrying a non-empty `source` and, once declared, a `scope` of `real`, `fixture` or `derived`, and - when that scope is `fixture` - a `fixture` description of what was generated |
159
165
  | `experiments/<hypothesis>/analysis.md` | what the rows mean, and which claim they move |
@@ -168,7 +174,9 @@ gets the question, its scope and its method in its brief. The research skill's
168
174
  file that exists (`init`, `hooks/tezgah_research.py::init`); `predictions.jsonl` is not
169
175
  one of them, because a file that exists without a row is a line that looks like
170
176
  it predicted something - it appears with the first `predict` the line records,
171
- and `check` reads an absent one as no rows rather than as a missing file. A slug
177
+ and `check` reads an absent one as no rows rather than as a missing file. The
178
+ three campaign files work the same way. Each appears with its first `idea`,
179
+ `stage` or `match`. A slug
172
180
  is lowercase letters, digits and dashes (`valid_slug`,
173
181
  `hooks/tezgah_research.py::valid_slug`), because it is the directory name `check`, `status` and
174
182
  `claim` look for.
@@ -238,14 +246,66 @@ through its seal.
238
246
  | `tezgah-research source <slug> <hypothesis> --run <orxRunId> [--command "..."] [--scope real\|fixture\|derived] [--fixture "<what was generated>"]` | keeps the receipt: runs `orx logs <runId>`, writes `raw/<runId>.log`, appends the results row `{"source": "orx:<runId>", ...}` and the scope and fixture description the filer states (`source_run`, `hooks/tezgah_research.py::source_run`). A `--scope fixture` filed without `--fixture` still writes the row and prints the field it still owes, so the tool is never the thing that makes its own checker warn silently; a given `--fixture` that is empty is misuse | 0, 1 nothing filed, 2 without orx |
239
247
  | `tezgah-research compare <slug> <decision>` | reads one variants x criteria cell from stdin and appends it to `decisions/<decision>/comparison.jsonl` under the lock, or refuses it by the rule `check` applies (`append_comparison` and `comparison_problems`, `hooks/tezgah_research.py`); it notes when `criteria.json` is not committed yet | 0, 1 refused, 2 misuse |
240
248
  | `tezgah-research close <slug> --limit "<reason>"` | concludes the line as a deliberate limit, writing the reasons it was still open into `state.json` `closed` and `log.md`, and seals it (`close_line`) | 0, 1 unreadable state, 2 misuse |
249
+ | `tezgah-research verdict <slug> <id> met\|not-met\|unanswerable --evidence "<ref>"` | records one success criterion's verdict and the evidence it rests on in the line's `state.json`, or refuses it with one `FAIL verdict:` line per problem (`verdict_problems`, `hooks/tezgah_research.py::verdict_problems`) | 0, 1 refused or unreadable state, 2 misuse |
250
+ | `tezgah-research conclude <slug>` | concludes a line once every success criterion carries a verdict, seals it and moves it to `done/` (`conclude_line`, `hooks/tezgah_research.py::conclude_line`) | 0, 1 refused, 2 misuse |
251
+ | `tezgah-research migrate-layout` | moves flat `research/<slug>` directories into `open/` or `done/` by each line's state, and is idempotent (`migrate_layout`, `hooks/tezgah_research.py::migrate_layout`) | 0, 2 misuse |
241
252
  | `tezgah-research seal <slug> [--history-lost] --ack "<owner decision>"` | seals a line under `done/` that was concluded before seals existed: plainly, as `conclude` would, when every experiment's order still checks; with `--history-lost`, with that verdict for every experiment whose order the lost history left undecidable. Refuses a line with a real order violation, a plain seal on any order finding, and a `history-lost` seal with nothing lost (`retro_seal`, `hooks/tezgah_research.py::retro_seal`) | 0, 1 refused, 2 misuse |
242
253
  | `tezgah-research import <checkout> [<slug>] [--allow-open "<reason>"]` | brings a line - or every line this checkout lacks - from another checkout's `.tezgah` repository with its history; an open line arrives beside open ones only with `--allow-open` (`import_line`, `hooks/tezgah_research.py::import_line`) | 0, 1 refused, 2 misuse |
254
+ | `tezgah-research next --topic <t> [--flow f] [--json]` | prints every step of a product flow with its status from the topic's files under `.tezgah/analysis/<topic>/`, then the first missing or due step's skill and command; `--flow` picks `metric-dashboard` (the default) or `feature`, and `--json` prints the same steps as data (`next_steps`, `hooks/tezgah_product.py::next_steps`) | 0, 2 unknown flow or no `--topic` |
255
+ | `tezgah-research init <slug> --ask "<the user's words>" --method exploratory --in "<covered>" --out "<left alone>" --arena "<problem>" --budget "<budget>" [--evaluator <path> --metric "<metric>"]` | opens a rules-7 exploratory line and writes the arena into `state.json` (`arena_record`, `hooks/tezgah_research.py::arena_record`). `--ask`, `--in` and `--out` are required as on any rules-5 line. Without `--arena` or `--budget` it is misuse; `--evaluator` and `--metric` go together; the evaluator has to be a file in the repository; the four flags on any other method are misuse. Its next step says to commit the arena before the first idea | 0, 1 refused over open lines, 2 misuse |
256
+ | `tezgah-research idea <slug>` | reads one idea row from stdin and appends it to `ideas.jsonl` under the lock, or refuses it with one reason per problem (`append_idea`, `hooks/tezgah_research.py::append_idea`) | 0 written, 1 refused, 2 misuse |
257
+ | `tezgah-research stage <slug> <idea> <stage> [--evidence "<ref or JSON>"] [--reason "<why>"]` | appends one stage move to `stages.jsonl` under the lock, or refuses it. `--reason` is the kill's reason, and a `known` prior art needs it for the kill appended with it (`append_stage`, `hooks/tezgah_research.py::append_stage`) | 0 written, 1 refused, 2 misuse |
258
+ | `tezgah-research match <slug>` | reads one judged match from stdin and appends it to `tournament.jsonl` under the lock, or refuses it (`append_match`, `hooks/tezgah_research.py::append_match`) | 0 written, 1 refused, 2 misuse |
259
+ | `tezgah-research board <slug> [--json]` | the campaign as its files show it: arena, rows, torn tails, each idea's stage, score and next action, the ideas tested and the next step; writes nothing (`board_text`, `hooks/tezgah_research.py::board_text`) | 0, 2 misuse |
260
+ | `tezgah-research dedup <slug> [--threshold <(0, 1]>] [--json]` | clusters the `proposed` ideas by statement and prints the `stage ... duplicate` command for each member; names the backend and the threshold's source; writes nothing (`dedup_text`, `hooks/tezgah_research.py::dedup_text`). A `--threshold` outside `(0, 1]` is misuse | 0, 2 misuse |
261
+ | `tezgah-research yield [--json] [--all-lines]` | per operator and per generator family, how many ideas reached each stage, the kill reasons, the board-score-to-pilot correlation and the next campaign's operator weights; writes nothing (`yield_text`, `hooks/tezgah_research.py::yield_text`) | 0, 2 misuse |
243
262
 
244
263
  Exit code 2 is always misuse, so a caller can tell it from a line that fails the
245
264
  checks (`misuse`, `bin/tezgah-research::misuse`). `check` asks nothing at all - no
246
265
  network, no model - so its answer is reproducible on a machine with no
247
266
  credential; only `--orx` asks the one question that leaves the machine.
248
267
 
268
+ ## The next step of a product flow
269
+
270
+ `tezgah-research next --topic <t>` tells the session which step of a product flow
271
+ comes next. Its step table is `FLOWS` (`hooks/tezgah_product.py::FLOWS`).
272
+ Each step names its `id`, the `skill` that runs it, what it `leaves` behind and
273
+ the `command` that writes it. The `product-flow` skill lists the same nine steps
274
+ in prose and names `next` as their status source.
275
+
276
+ Two flows exist. `metric-dashboard` is the default and has nine steps. `feature`
277
+ drops the metrics and instrumentation steps, unless the topic's `flow.md` already
278
+ holds `## metrics`. `--flow <f>` picks one.
279
+
280
+ `next_steps` (`hooks/tezgah_product.py::next_steps`) computes each status from the
281
+ topic's folder, `.tezgah/analysis/<topic>/`, and reads nothing else. Nothing
282
+ stores a status. A step is `done`, `missing` or `due`:
283
+
284
+ | Step | Done when |
285
+ |---|---|
286
+ | `forward-spec` | `.tezgah/analysis/<topic>/spec.md` exists |
287
+ | `re-measure` | `flow.md` holds `## re-measure`. Before that it reads `due` once the `observe after: YYYY-MM-DD` date under `## decision` has passed |
288
+ | every other step | `flow.md` holds its `## <step>` heading |
289
+
290
+ The `design` and `build` commands start with `tezgah-design check`. The design
291
+ check and the Stop UI rule judge those two steps. The flow only asks for their
292
+ note. A topic that is not kebab-case (at most 64 characters) reads no file. A
293
+ topic with no folder yet lists every step as `missing`.
294
+
295
+ The text output prints one line per step, then one `next:` line:
296
+
297
+ ```
298
+ <n>. <id> [<status>] <skill> - <detail>
299
+ next: <skill> - <command>
300
+ ```
301
+
302
+ The `next:` line names the first `missing` or `due` step's skill and command.
303
+ With no such step it reads `next: nothing left`. `--json` prints the same steps
304
+ as a list of objects with `id`, `skill`, `status`, `detail` and `command`.
305
+
306
+ `next` exits 0 when it prints the steps. It exits 2 for an unknown flow or a
307
+ missing `--topic`.
308
+
249
309
  ## The pair the order rule compares
250
310
 
251
311
  The order rule is decided on the commit graph, so it needs both files committable
@@ -442,18 +502,288 @@ in `hooks/tezgah_research.py`, a module the frozen set forbids a prediction's
442
502
  commit to touch without a human `granted_by` (`FROZEN_PATHS`), so the gate that
443
503
  decides which candidate survives is not machinery the candidate can rewrite.
444
504
 
445
- What the pair cannot catch: **a tolerance its own author sets wide enough**. The
446
- band is a sentence, not a measurement, and `check` can see that it is written and
447
- readable - never that 0.02 is tight. A line that writes "capability metric within
448
- 5.0 of the baseline" has adopted the form of the two-gate rule and none of its
449
- force, and the review is the only place that is decided.
505
+ What the pair cannot catch alone: **a tolerance its own author sets wide
506
+ enough**. A line that writes "capability metric within 5.0 of the baseline" has
507
+ adopted the form of the two-gate rule and none of its force. From rules 6 the band
508
+ is read against a measurement. `evaluation.noise_band` is
509
+ `{"experiment": "<the experiment dir that reran the unchanged baseline>",
510
+ "spread": <the spread that rerun measured>}`. `check` warns in three cases. The
511
+ line declares `capability_tolerance` and no `noise_band`. `noise_band.experiment`
512
+ names no experiment of the line. The tolerance sentence holds a number wider than
513
+ `spread`, so the tolerance passes what noise alone would. A digit inside a name
514
+ such as `p95` is not that number (`_check_selection`,
515
+ `hooks/tezgah_research.py::_check_selection`). Whether the rerun was a fair one
516
+ stays the review's call.
517
+
518
+ ## Held-out set, repeats and noise band (rules 6)
519
+
520
+ The measured line `rsi-borrow-screen` found three gaps. Of 27 decisions, 25 select
521
+ a variant and report it on the same cells. Of 23 armed experiments, 13 run once per
522
+ arm. Three lines declare a tolerance, and none of them measured it. Rules 6 add one
523
+ warning for each (`_check_selection`, `hooks/tezgah_research.py::_check_selection`).
524
+ Each stays a warning under `--strict` too, never a refusal (ADR 019):
525
+
526
+ - **Held-out set.** A measured line may name its held-out set in
527
+ `evaluation.held_out` (a non-empty string). It may log each look at that set in
528
+ `held_out_log.jsonl` at the line root, one row
529
+ `{"set", "candidate", "date", "result"}` per look. `check` warns when a
530
+ `decisions/<id>/` holds comparison cells and the line names no `held_out`: the
531
+ variant was selected and reported on the same cells. It also warns when one
532
+ candidate looked at one set more than once: the held-out set became development
533
+ data for it.
534
+ - **Repeats.** An experiment is armed when its `results.jsonl` rows carry `arm` or
535
+ `variant` with two or more values. `check` warns when no row has `repeat`,
536
+ `repeats` or `k` above 1 and its `protocol.md` states no `k:`/`repeats:` above 1.
537
+ An armless experiment is not read.
538
+ - **Noise band.** `evaluation.noise_band`, read in "Two gates" above.
539
+
540
+ A line opened under rules 5 or earlier is not read for any of the three.
541
+
542
+ ## Rules 7: exploratory lines
543
+
544
+ A `qualitative` or `measured` line answers a question. Some asks want new ideas,
545
+ methods or approaches instead of an answer. Rules 7 add the method `exploratory`
546
+ for those asks (`hooks/tezgah_research.py::EXPLORE_RULES`,
547
+ `hooks/tezgah_research.py::METHODS`). The line runs one campaign. Many ideas
548
+ enter it, a funnel kills most of them, and a run under a frozen evaluator judges
549
+ the few that remain. One campaign is one line, so "One open line at a time" still
550
+ holds. `scope_problems` (`hooks/tezgah_research.py::scope_problems`) accepts
551
+ `exploratory` only on a rules-7 line. On an older line it names `qualitative` and
552
+ `measured` as the two methods the line can take.
553
+
554
+ ### The arena lock
555
+
556
+ `init <slug> --ask "<the user's words>" --method exploratory --in "<covered>"
557
+ --out "<left alone>" --arena "<problem>" --budget "<budget>"` opens
558
+ the line. The arena is the problem in the user's words. The budget says how many
559
+ ideas, pilots and full runs, and how much wall time, the campaign may spend. A
560
+ measurable arena adds `--evaluator <path> --metric "<metric>"`. The two flags go
561
+ together, and the evaluator has to be a file in the repository. A missing arena
562
+ or budget is misuse (exit 2).
563
+
564
+ `arena_record` (`hooks/tezgah_research.py::arena_record`) builds `state.json`
565
+ `arena`. It holds the problem and the budget. A measurable arena also holds the
566
+ evaluator path, the sha256 of the evaluator's bytes and the metric. `init` also
567
+ writes `deliverable: {"kind": "finding"}`. The campaign delivers its staged ideas
568
+ as findings, never as compared variants (`hooks/tezgah_research.py::init`).
569
+
570
+ The lock has three halves:
571
+
572
+ - `arena_problems` (`hooks/tezgah_research.py::arena_problems`) names an absent
573
+ arena, an empty problem or budget, and an evaluator with no sha256 or no metric.
574
+ `check` reads it through `scope_problems`. It refuses these past `bootstrap` and
575
+ warns at it.
576
+ - `_check_arena_order` (`hooks/tezgah_research.py::_check_arena_order`) finds the
577
+ lock commit. That commit is the first one along HEAD whose `state.json` arena
578
+ names a problem and a budget. `ideas.jsonl` and every experiment's
579
+ `results.jsonl` have to enter the history after it. An arena that differs from
580
+ the locked one is a refusal too. A new arena means a new line.
581
+ - `evaluator_problem` (`hooks/tezgah_research.py::evaluator_problem`) hashes the
582
+ evaluator again. While the hash differs from the locked one, the writer and
583
+ `check` refuse every stage row. The candidate a campaign judges must not move
584
+ its judge.
585
+ - `append_stage` (`hooks/tezgah_research.py::append_stage`) also compares the
586
+ working `state.json` arena with the arena at the lock commit
587
+ (`hooks/tezgah_research.py::_arena_lock`). While they differ, it refuses every
588
+ stage row. So a rewritten `evaluator_sha256` does not reopen the campaign.
589
+
590
+ `init` prints the next step: commit the arena with `tezgah-research commit` before
591
+ the first idea. The arena replaces the one locked metric of a measured line.
592
+ `_check_evaluation` (`hooks/tezgah_research.py::_check_evaluation`) and
593
+ `_check_evaluation_order` (`hooks/tezgah_research.py::_check_evaluation_order`)
594
+ skip an exploratory line for that reason.
595
+
596
+ ### Three files
597
+
598
+ The campaign lives in three append-only files at the line root. One function
599
+ judges each row type, and the writer and `check` both call it. No file stores a
600
+ status. An idea's stage is its newest `stages.jsonl` row
601
+ (`hooks/tezgah_research.py::campaign_stages`).
602
+
603
+ | File | One row | Writer | Judge |
604
+ |---|---|---|---|
605
+ | `ideas.jsonl` (`IDEAS_FILE`) | `id` (`I<n>`, unused so far), `statement` (two sentences at most), `mechanism`, `trace` (an in-scope item), `operator` (one of `OPERATORS`), `generator` (`{family, model}`), `kill` (the result that would kill it), `date`, and `parents` for a `mutation` (one or more) or a `crossover` (two or more) | `idea <slug>` | `idea_problems` |
606
+ | `stages.jsonl` (`STAGES_FILE`) | `idea`, `stage`, `evidence`, `reason` for a kill, `date` | `stage <slug> <idea> <stage>` | `stage_problems` |
607
+ | `tournament.jsonl` (`TOURNAMENT_FILE`) | `a`, `b`, `judge` (`{family, model}`), `order` (`ab` or `ba`), `winner` (`a`, `b` or `tie`), `rationale` | `match <slug>` | `match_problems` |
608
+
609
+ The field list is `IDEA_FIELDS` (`hooks/tezgah_research.py::IDEA_FIELDS`). The
610
+ moves are `OPERATORS` (`hooks/tezgah_research.py::OPERATORS`). `user` marks an
611
+ idea the person asked for. `PARENT_OPERATORS`
612
+ (`hooks/tezgah_research.py::PARENT_OPERATORS`) sets how many parents a derived
613
+ idea names. An idea outside the arena's scope gets one line under Open questions
614
+ in `findings.md`, never a row.
615
+
616
+ Every append goes through `_append_campaign`
617
+ (`hooks/tezgah_research.py::_append_campaign`). It takes the file lock and reads
618
+ the rows from the locked descriptor. It judges the new row against them, cuts any
619
+ torn tail (`truncate_to_committed`) and writes. A refused append leaves no file
620
+ behind. `check` refuses a committed row that a later version changed or dropped.
621
+
622
+ ### The stage funnel
623
+
624
+ `STAGES` (`hooks/tezgah_research.py::STAGES`) gives the order: `proposed`,
625
+ `duplicate`, `prior-art`, `screened`, `piloted`, `promoted`, `confirmed`, and
626
+ `killed`. `stage_problems` (`hooks/tezgah_research.py::stage_problems`) holds
627
+ the moves to it:
628
+
629
+ - The idea row is the `proposed` stage. A stage row that says `proposed` is a
630
+ refusal.
631
+ - A stage moves forward in that order only, or to `killed`. A kill needs
632
+ `--reason`.
633
+ - A stage also needs its prerequisite somewhere in the idea's history
634
+ (`hooks/tezgah_research.py::PREREQUISITES`). `screened` needs a `prior-art`
635
+ row. `piloted` needs `screened`, `promoted` needs `piloted`, and `confirmed`
636
+ needs `promoted`.
637
+ - `killed` is final. A revived idea is a new idea that names the old one as its
638
+ parent.
639
+ - After `duplicate`, the only allowed row is `killed`. The same holds after a
640
+ `known` prior art.
641
+
642
+ Each stage asks for its own evidence. `--evidence` takes a reference or a JSON
643
+ object.
644
+
645
+ | Stage | Evidence | What the judge asks |
646
+ |---|---|---|
647
+ | `duplicate` | `{"of": "<id>", "similarity": <0..1>}` | `of` names another idea of the line |
648
+ | `prior-art` | `{"verdict": "novel\|incremental\|known", "nearest": [...], "delta": "<one sentence>"}` | `nearest` names at least `NEAREST_MIN` (3) notes. `literature/INDEX.jsonl` indexes each one, and not all of them are `agent-report`. `delta` is one sentence (`_prior_art_problems`) |
649
+ | `screened` | none | `tournament.jsonl` holds at least one match that names the idea |
650
+ | `piloted`, `promoted` | `<h>` or `experiments/<h>` | `protocol.md` names the idea id and repeats its `kill` text. `results.jsonl` holds a row. When the check asks git, `protocol.md` is in a commit. If `results.jsonl` is committed too, `protocol.md` entered the history before it (`_run_problems`) |
651
+ | `confirmed` | `{"experiment": "<h>", "set": "<set>"}` | the arena has an evaluator. The experiment differs from every `promoted` row's experiment. The run passes the same `_run_problems` checks. The first `held_out_log.jsonl` row for that set names this idea. Each look of this idea at the set carries a `YYYY-MM-DD` date, and none is dated before the idea's promotion: an earlier look was part of the selection. A look logged after the confirmation changes nothing (`_confirm_problems`) |
652
+ | `killed` | `--reason "<why>"` | a reason |
653
+
654
+ A `known` prior art kills the idea in the same write. `append_stage`
655
+ (`hooks/tezgah_research.py::append_stage`) needs `--reason` for it. It appends
656
+ the prior-art row and its `killed` row under one lock, so no file state holds a
657
+ known idea alive. `check` refuses a `known` prior art with no kill after it.
658
+
659
+ ### The tournament is a prior
660
+
661
+ A match is one judge's verdict on a pair. `bradley_terry`
662
+ (`hooks/tezgah_research.py::bradley_terry`) fits a strength per idea from the
663
+ matches. `board` ranks the ideas by it. The score moves no stage. `screened`
664
+ asks only that a match exists. An LLM ranking is a prior, and a run under the
665
+ frozen evaluator is the judge.
666
+
667
+ `_tournament_warnings` (`hooks/tezgah_research.py::_tournament_warnings`) warns
668
+ about thin judging. A pair judged only by the family that generated one of its
669
+ ideas warns, unless both generators' families judged it. A pair judged in one
670
+ presentation order warns. An idea with fewer than two judges warns.
671
+
672
+ ### Reports: board, dedup, yield
673
+
674
+ The three reports write nothing and exit 0. None of them is a gate.
675
+
676
+ - `board <slug>` (`hooks/tezgah_research.py::board`) shows the arena and whether
677
+ the evaluator still hashes as locked. It shows each file's row count, each
678
+ idea's rank, stage, score, record, operator, generator family and next action,
679
+ and the campaign's next step. An idea whose prior art is `known` gets the
680
+ `stage ... killed --reason` command as its next action. That resumes a campaign
681
+ whose kill row was torn off.
682
+ - `dedup <slug>` (`hooks/tezgah_research.py::dedup`) clusters the `proposed`
683
+ ideas by statement, at one similarity threshold. The lowest id of a cluster
684
+ represents it. For every other member it prints the `stage ... duplicate`
685
+ command to run. `similarity_matrix` (`hooks/tezgah_research.py::similarity_matrix`)
686
+ uses the static embedding when the feature is on and pinned. Otherwise it reads
687
+ BM25 scores (`hooks/tezgah_rank.py::scores`) as a similarity. Every output names
688
+ the backend and where its threshold came from.
689
+ - `yield` (`hooks/tezgah_research.py::yield_report`) counts, per operator and per
690
+ generator family, how many ideas reached each stage. It lists the kill reasons.
691
+ It gives Spearman's rho between the board score and the pilot outcome
692
+ (`hooks/tezgah_research.py::spearman`). It proposes the next campaign's operator
693
+ weights (`hooks/tezgah_research.py::operator_weights`). `EXPLORE_SHARE` (0.2)
694
+ of each draw stays uniform, so a move with no payoff yet still gets tried. It
695
+ reads the open exploratory lines, or every one with `--all-lines`.
696
+
697
+ ### Resume from the files
698
+
699
+ Nothing outside the three files holds campaign state. A fresh session runs
700
+ `board` and goes on from its `next:` line. A writer that dies mid-append can
701
+ leave a fragment past the last newline. `_campaign_rows`
702
+ (`hooks/tezgah_research.py::_campaign_rows`) reads up to the committed boundary
703
+ and reports the fragment's size. `board` prints it on a `resume:` line. The next
704
+ append drops the fragment, so the fix is to run the append that crashed again.
705
+
706
+ ### How many ideas were tested
707
+
708
+ Fifty ideas tested at the 5% level yield about 2.5 winners by chance. A reader
709
+ cannot discount what nobody counted. `_check_tested`
710
+ (`hooks/tezgah_research.py::_check_tested`) warns when `to_human/report.md` has
711
+ no line that names `piloted` with the number of ideas that reached it. The same
712
+ holds for `promoted`. A number followed by a full stop counts, as in "reached
713
+ piloted: 2.". `board` prints both numbers on its `tested:` line.
714
+
715
+ ### The dedup threshold (two label sets)
716
+
717
+ Si et al. called two seed ideas duplicates at cosine 0.8 on MiniLM. That number
718
+ does not carry to another similarity. Two label sets measured it here.
719
+
720
+ - The fixture: 40 idea-statement pairs, 20 of them duplicates. The author of the
721
+ code labelled them. The acceptance runner records the sweeps as fixture-scoped
722
+ experiments: `dedup-calibration` (BM25) and `dedup-calibration-embed`
723
+ (embed-mrl). Copies stay under `.tezgah/scratch/calibration/`.
724
+ - MRPC: 400 pairs of the Microsoft Research Paraphrase Corpus test set, 200
725
+ paraphrases and 200 not, sampled with seed 87. Human annotators labelled them.
726
+ The pairs are news sentences, and the negatives share many words. At BM25 0.30
727
+ the sample kept every paraphrase and 95% of the negatives. The script and its
728
+ output are `.tezgah/scratch/calib2/run.py` and `out.txt`.
729
+
730
+ | Backend | F1 at 0.8 (fixture / MRPC) | Default | F1 at the default (fixture / MRPC) |
731
+ |---|---|---|---|
732
+ | embed-mrl | 0.62 / 0.64 | 0.65 | 0.90 / 0.705 |
733
+ | BM25 | 0.09 / 0.41 | 0.40 | 0.70 / 0.69 |
734
+
735
+ The default is the threshold, in steps of 0.05, whose worse F1 over the two sets
736
+ is highest. Neither the author's labels nor the news domain picks it alone. On
737
+ MRPC the best F1 beats the always-duplicate baseline (0.667) by about 0.04. The
738
+ defaults are a coarse filter for a session to review, not a judge.
739
+
740
+ `DEDUP_THRESHOLDS` (`hooks/tezgah_research.py::DEDUP_THRESHOLDS`) holds the two
741
+ values. Any other backend keeps `DEDUP_THRESHOLD`
742
+ (`hooks/tezgah_research.py::DEDUP_THRESHOLD`), Si et al.'s 0.8, and the output
743
+ calls it uncalibrated. `--threshold` overrides all of them. Its range is
744
+ `(0, 1]`.
745
+
746
+ ### The evidence behind the method
747
+
748
+ - Si et al., arXiv 2409.04109: about 5% of 4,000 generated seed ideas were not
749
+ duplicates (cosine 0.8). The best LLM judge (a Claude-3.5 pairwise ranker)
750
+ agreed with the human ranking 53.3% of the time. Split-half reviewer panels
751
+ agreed 56.1%.
752
+ - The ideation-execution gap, arXiv 2506.20803: after execution, AI ideas lost
753
+ 1.98 points overall and human ideas lost 0.63.
754
+ - Gupta and Pruthi, arXiv 2502.16487: 24% of 50 AI research documents
755
+ paraphrased or substantially borrowed earlier work. Plagiarism detectors missed
756
+ them.
757
+ - HindSight, arXiv 2603.15164: novelty as an LLM judges it correlates negatively
758
+ with realized impact (rho = -0.29).
759
+ - Darwin Godel Machine, arXiv 2505.22954: an archive of stepping stones beats
760
+ hill-climbing. The loop also removed its own detection markers and faked test
761
+ logs.
762
+ - ShinkaEvolve, arXiv 2509.19349: parent sampling by fitness and novelty,
763
+ novelty rejection by embedding, and UCB1 over models.
764
+ - POPPER, arXiv 2502.09858: falsification experiments on measurable implications,
765
+ with Type-I error control.
766
+
767
+ The consequences:
768
+
769
+ - An LLM ranking is a prior, never a verdict.
770
+ - A run under the frozen evaluator is the judge.
771
+ - Each idea gets its own prior-art check. No idea reaches `screened` without a
772
+ `prior-art` row (`hooks/tezgah_research.py::PREREQUISITES`).
773
+ - `dedup` and the operator weights keep the ideas diverse.
774
+ - The report always states how many ideas the campaign tested.
775
+
776
+ Rules 1-6, `FROZEN_PATHS` and ADR 009 stay as they were. Predictions and
777
+ components take nothing new. `research-off` covers this method too.
450
778
 
451
779
  ## What `check` refuses, and what it only warns about
452
780
 
453
781
  A **refusal** (exit 1) is a guarantee the checker can decide. A **warning** is a
454
782
  guarantee it cannot: the artifact is not wrong, it is unverifiable. `--strict`
455
- turns that whole class into a refusal, which is what a release gate wants and
456
- what a session mid-flight must not have, so the default stays `warn`.
783
+ turns the warnings the listed rules write through `_soft` into a refusal, which is
784
+ what a release gate wants and what a session mid-flight must not have, so the
785
+ default stays `warn`. A warning written outside `_soft` stays one. The section
786
+ below names three groups of them.
457
787
 
458
788
  Refused:
459
789
 
@@ -532,6 +862,40 @@ Refused:
532
862
  nothing, or a finding whose `quote` does not occur verbatim in its `target`
533
863
  (`_check_review`, `hooks/tezgah_research.py::_check_review`).
534
864
 
865
+ On an exploratory line (rules 7), `check` also refuses each of these
866
+ (`_check_exploratory`, `hooks/tezgah_research.py::_check_exploratory`, called
867
+ from `_check_standards`):
868
+
869
+ - an arena that `arena_problems` faults, past `bootstrap`. The faults are no
870
+ arena object, no problem, no budget, and an evaluator with no sha256 or no
871
+ metric (`hooks/tezgah_research.py::arena_problems`).
872
+ - an arena that differs from the one the lock commit holds. The same goes for an
873
+ `ideas.jsonl` or `results.jsonl` that entered the history before that commit
874
+ (`hooks/tezgah_research.py::_check_arena_order`).
875
+ - an idea row that `idea_problems` faults
876
+ (`hooks/tezgah_research.py::idea_problems`). That means a missing field, a
877
+ `generator` that is not `{family, model}`, or an id that is not `I<n>` or
878
+ repeats an earlier one. It also means an operator outside `OPERATORS`, a parent
879
+ no earlier idea carries, or too few parents for a `mutation` or `crossover`. The
880
+ last cases are a statement of more than two sentences and a date that is not
881
+ `YYYY-MM-DD`. A trace to no in-scope item and a missing arena fail it too.
882
+ - a stage row that `stage_problems` faults
883
+ (`hooks/tezgah_research.py::stage_problems`). The funnel above lists the
884
+ moves and the evidence each stage asks for. Every stage row fails while the
885
+ evaluator's hash differs from the locked one
886
+ (`hooks/tezgah_research.py::evaluator_problem`).
887
+ - a stage row that skips its `PREREQUISITES` stage, and any row but `killed`
888
+ after `duplicate` or after a `known` prior art.
889
+ - a `known` prior art with no `killed` row after it.
890
+ - a match row that `match_problems` faults
891
+ (`hooks/tezgah_research.py::match_problems`). That means a side that names no
892
+ idea, an idea matched against itself, or a `judge` that is not
893
+ `{family, model}`. It also means an `order` outside `MATCH_ORDERS`, a `winner`
894
+ outside `MATCH_WINNERS`, or no `rationale`.
895
+ - a campaign file line that does not parse or is not an object
896
+ (`hooks/tezgah_research.py::_campaign_rows`).
897
+ - a committed campaign row that a later version changed or dropped.
898
+
535
899
  Warned, and refused under `--strict` - each is the honest state of an artifact
536
900
  this layer cannot decide on, not a defect:
537
901
 
@@ -629,7 +993,18 @@ this layer cannot decide on, not a defect:
629
993
  judgement, so no migration can write one for the lines that concluded before
630
994
  the rule existed.
631
995
 
632
- One warn is **not** promoted by `--strict`, and it is the only one: a `phase`
996
+ On an exploratory line, two more warnings take this class. One is an arena
997
+ fault at `bootstrap`. The other is a campaign file git cannot order against the
998
+ arena lock (`_check_arena_order`).
999
+
1000
+ Three groups of warnings are **not** promoted by `--strict`. The rules-6
1001
+ three (held-out set, repeats, noise band) stay warnings by ADR 019. The rules-7
1002
+ four stay warnings for the same reason: a tournament is a prior and a count is a
1003
+ disclosure. They are a pair that only a generator family of one of its ideas
1004
+ judged, unless both generators' families judged it. They are a pair judged in one
1005
+ order, and an idea with fewer than two judges (`_tournament_warnings`). The fourth
1006
+ is a `report.md` that does not state how many ideas reached `piloted` and
1007
+ `promoted` (`_check_tested`). The last is a `phase`
633
1008
  declared in `state.json` behind the phase the line's own artifacts show
634
1009
  (`_check_derived_phase`, `hooks/tezgah_research.py::_check_derived_phase`). The field is the author's
635
1010
  declared intent and `_check_report` and `_check_review` read it, so a derivation
@@ -827,11 +1202,15 @@ armed. It removes the rule, not just the mark - the diff is in `core_split`,
827
1202
  frozen set they read (`FROZEN_PATHS`), the protocol order and its tracked pair,
828
1203
  the protocol's two questions, the concluded report's limits, the `supersedes`
829
1204
  relation, `component_report` with the state it reads, `migrate`, `source_run`,
830
- `init` and `summary`.
1205
+ `init` and `summary`, and the rules-7 campaign: the arena lock, the three row
1206
+ judges and their writers, `board`, `dedup` and `yield_report`.
831
1207
  - `hooks/tezgah_components.py` - the manifest: the one definition of an editable
832
1208
  component, the keys a prediction row names and the rules a row's outcome is
833
1209
  attributed to.
834
1210
  - `bin/tezgah-research` - the CLI, its subcommands and its exit codes.
1211
+ - `hooks/tezgah_product.py` - the product-flow steps and the `next` status reader.
1212
+ - `hooks/tezgah_rank.py` - `scores`, the BM25 reading `dedup` falls back to when
1213
+ the static embedding is off.
835
1214
  - `hooks/tezgah_policy.py` - the `RESEARCH` paragraph a session is armed with.
836
1215
  - `hooks/tezgah_context.py` - the prompt hint, the missing-orx note, the broken
837
1216
  line note and the `research` mark.
package/docs/skills.md CHANGED
@@ -32,7 +32,7 @@ skill is simply not read.
32
32
 
33
33
  opencode's plugin does inject prompt-time text - the builder's per-turn block,
34
34
  the conditional rules included
35
- (`hosts/opencode/plugins/tezgah.js:2629-2658`) - but that text names at most one
35
+ (`hosts/opencode/plugins/tezgah.js:2652-2681`) - but that text names at most one
36
36
  skill (the hint), never the roster, so the installer writes one:
37
37
  `~/.config/tezgah/opencode-skills.md` always-on and
38
38
  `opencode-skills.full.md` on demand (`bin/tezgah-setup::SKILL_ROUTER`, `bin/tezgah-setup::SKILL_ROUTER_FULL`), the first
@@ -128,6 +128,12 @@ rebuilds that index only when a SKILL.md's mtime or size moves
128
128
  (`hooks/tezgah_skill_pick.py::_open_index`). A prompt reads its own terms'
129
129
  postings and the length column, never the corpus.
130
130
 
131
+ Some skills route through their own `INDEX.md`, like design-library's 39 entries.
132
+ The search indexes each routed entry as one row for the whole file, addressed
133
+ `skill://<skill>/<path>` (`hooks/tezgah_skill_pick.py::_entries`). The vendored
134
+ sections are short, and per section BM25 let them outrank whole skills on two
135
+ common words.
136
+
131
137
  The hook then appends one line, `A local skill search matched this request:
132
138
  skill://<name>:<start>-<end>`, with the section title. Every host but omp also
133
139
  gets the absolute path. The top section must carry 60% of the prompt's
@@ -174,15 +180,16 @@ it (`tests/test_skills.py::SkillStandards`).
174
180
  | `ponytail` | Forces the laziest solution that works, in levels lite/full/ultra; fires on any coding task, or "be lazy", "yagni", "shortest path". |
175
181
  | `i-have-adhd` | Shapes output for a reader who acts on it - action first, numbered steps, errors as location/cause/fix; fires on any answer the reader must act on, not on code, commits or docs. |
176
182
  | `no-ai-slop` | Edits a draft sharper and more human without losing the voice, or flags slop; fires on a draft review, on a draft in a language other than English, on text that must keep terms, identifiers or unit symbols verbatim, and before publishing English prose tezgah writes itself. |
177
- | `analyze-app` | Inspects and drives a running web or mobile app from its accessibility / DOM / view tree, its rendered screen (screenshots, blur and grayscale tests, breakpoints) and measurements taken in the page; fires when a task must verify behaviour or read the UI, not just read source. |
183
+ | `analyze-app` | Inspects and drives a running web or mobile app from its accessibility / DOM / view tree, its rendered screen (screenshots, blur and grayscale tests, breakpoints) and measurements taken in the page, and writes the `.tezgah/design/measurement.json` that `design-contract`'s check reads; fires when a task must verify behaviour or read the UI, not just read source. |
178
184
  | `plan-add` | Creates a plan under `.tezgah/plans/open/` with its own `allowed_paths:` scope, bootstraps the `.tezgah/plans/` layer if missing, commits it in the private `.tezgah` repository (`git -C .tezgah`, never the project's git); fires on `/tezgah:plan-add`, "track this as a plan". |
179
185
  | `plan-status` | Reads every open plan and `tezgah-task status` (each plan's Acceptance boxes ticked, the plans its `after:` waits on, the ready plans that can run at once), enriches rows with live PR state from `gh`, rewrites the README table (committed in the private `.tezgah` repository) and recommends the next plan; fires on `/tezgah:plan-status` or "what plans are open". |
180
- | `plan-sync` | Closes finished plans whose PR merged or closed through `tezgah-task close`, which moves a plan to `.tezgah/plans/done/` as done only when it records an approving `review:`; commits one message per plan in the private `.tezgah` repository; fires on `/tezgah:plan-sync`, "sync plans". |
186
+ | `plan-sync` | Closes finished plans whose PR merged or closed through `tezgah-task close`, which moves a plan to `.tezgah/plans/done/` as done only when it records an approving `review:`, clears an active task's phase and removes every merged worktree that holds nothing else; commits one message per plan in the private `.tezgah` repository; fires on `/tezgah:plan-sync`, "sync plans". |
181
187
  | `research` | Runs an auditable research line - two loops, protocol before run, six-dimension review, provenance; fires when the deliverable is evidence: a lit review, a hypothesis, a benchmark or ablation. |
182
- | `product-analysis` | Runs a product analysis across five axes - value (HEART), usability (heuristics, the cognitive walkthrough, WCAG 2.2, axe-core, the running app read through `analyze-app` down to component x state, tree and image both), feasibility (a cited `path:line`), competition (a dated teardown) and triage (keep/fix/cut/bet) - with one named evidence class per finding, executed as a research line; fires on a product, feature, retention or roadmap question rather than a code change. |
188
+ | `product-analysis` | Runs a product analysis across five axes - value (HEART), usability (heuristics, the cognitive walkthrough, WCAG 2.2, axe-core, the running app read through `analyze-app` down to component x state, tree and image both), feasibility (a cited `path:line`), competition (a dated teardown) and triage (keep/fix/cut/bet) - with one named evidence class per finding and a metric record per metric (North Star and inputs; goal, definition, window, baseline, logged or not), executed as a research line; fires on a product, feature, retention or roadmap question rather than a code change. |
189
+ | `product-flow` | Carries a new product capability - a metric dashboard, a metric, a feature - from research to re-measure by running `tezgah-research next --topic <t>` and doing the first missing step with the skill it names; the skill lists the nine steps in prose and `FLOWS` in `hooks/tezgah_product.py` holds their done rules; fires on "bu veriye uygun metric dashboard tasarla" or a new-feature ask, not on an audit of an existing feature. |
183
190
  | `pm-frameworks` | The two vendored product methods the analysis skill defers to - intended-vs-implemented and the Opportunity Solution Tree - kept offline so an offline session reads the method instead of citing its name. |
184
- | `feature-audit` | Audits ONE feature across every layer it is spread over - the surfaces a person uses (a control, a list or table, a detail view, a filter, a picker, a step flow, a notification), the route, the authorization, the service and the data - by filling the capability, field-contract, flow/step and interaction-dependency matrices, where a disagreement between layers is a finding and a fix needing an absent capability becomes a capability-change proposal; fires on an admin screen, CRUD, form, wizard, data table, filter or permission question about an existing feature. |
185
- | `design-contract` | The artifact shape of the per-repo `.tezgah/design-contract.md` - colour roles, type scale, spacing unit and rhythm, the component inventory, the state set every interactive control and data view owes - the derivation rule that reads it out of the repository's own tokens, and the rules `bin/tezgah-design check` fails on; fires on a UI or component turn that needs a floor, a token, a state set, or the design check itself. |
191
+ | `feature-audit` | Audits ONE feature across every layer it is spread over - the surfaces a person uses (a control, a list or table, a detail view, a filter, a picker, a step flow, a notification), the route, the authorization, the service and the data - by filling the capability, field-contract, flow/step and interaction-dependency matrices, where a disagreement between layers is a finding and a fix needing an absent capability becomes a capability-change proposal; fires on an admin screen, CRUD, form, wizard, data table, filter or permission question about an existing feature, and in forward mode specifies a feature not built yet so the post-build audit diffs against it. |
192
+ | `design-contract` | The artifact shape of the per-repo `.tezgah/design-contract.md` - colour roles, type scale, spacing unit and rhythm, the component inventory, the state set every interactive control and data view owes - the derivation rule that reads it out of the repository's own tokens, the rules `bin/tezgah-design check` fails on, and the data-view rules for dashboards (`skills/design-contract/data-views.md`); fires on a UI or component turn that needs a floor, a token, a state set, or the design check itself. |
186
193
  | `design-library` | The vendored MC Dean design/UX subset - 39 skills in the `cognitive-accessibility`, `adaptive-interfaces`, `inclusive-interaction`, `visual-critique` and `design-research` plugins - read one entry at a time through its own `INDEX.md`, never the tree; fires when a UI or product turn needs a cognitive-load, inclusive-interaction or visual-critique method tezgah has no counterpart for. |
187
194
  | `ai-research` | The vendored 98-skill library for AI/ML machinery - training and serving a model, benchmarks, interpretability, retrieval pipelines; read one entry, never the tree. |
188
195
  | `rl-env` | The vendored FineEnvs skill set for authoring an RL environment from a description - OpenEnv, OpenReward/ORS, Verifiers and NeMo Gym, plus the orchestrator that ports one description across all four; read one entry, never the tree. |
@@ -198,6 +205,17 @@ The generated specialists load skills first (`hooks/tezgah_agents.py::SPECIALIST
198
205
  | `tezgah-security` | the host's `wstg-*`, `attack-*`, `ci-assessment` and `llm-security` skills, which tezgah does not ship |
199
206
  | `tezgah-tester` | none |
200
207
 
208
+ ### The product chain
209
+
210
+ A new product capability runs as one chain: research -> `product-analysis` ->
211
+ decision -> forward spec -> `design-contract` -> build -> re-measure. Each link
212
+ leaves a file under `.tezgah/analysis/<topic>/` that names the one before it.
213
+ `flow.md` holds one `## <step>` note per step. The forward spec
214
+ (`feature-audit`'s forward mode) is `spec.md`. `FLOWS`
215
+ (`hooks/tezgah_product.py::FLOWS`) holds the steps and their done rules.
216
+ `tezgah-research next` prints them with their status (docs/research.md, "The
217
+ next step of a product flow").
218
+
201
219
  ## Vendored material
202
220
 
203
221
  `NOTICE` records every third-party skill and its terms: it opens by saying the
@@ -96,7 +96,7 @@ measure the surface cannot see, then armed-and-used or armed-not-used.
96
96
  | `graph` | a code-graph tool call (an `mcp__…codegraph…` tool) | `.no-graph` — a missing binary is `idx`'s report, not this mark's |
97
97
  | `orch` | a delegate call (`Task`/`Agent`/`spawn_agent`) or a subagent start | `orchestrate-off` |
98
98
  | `judge` | a shell command really ran `tezgah-triage` or `tezgah-docs` | `judge-off`, or no credential: no `TYPESAFE_API_KEY` and no non-empty `~/.config/typesafe/key`, and neither the fallback's `OPENROUTER_API_KEY` nor its `~/.config/openrouter/key` |
99
- | `gate` | never `on`: shown only as `gate✗`, right after the version, in two cases. One: the prompt hook found at least 3 gated calls in this session's transcript since its first turn: write and shell calls, every one of which the gate answers with a `began` or a `deny` row (Bash/PowerShell/Edit/Write/MultiEdit/NotebookEdit on Claude, `exec_command`/`shell`/`apply_patch` on Codex). Over the same span the ledger holds no row from the gate. A PostToolUse row is no proof: that hook keeps writing while a broken PreToolUse lets every call through. The same prompt carries the line "tezgah gate inactive on this host: ...". Two: a kill switch appeared after the session's first prompt; the prompt hook writes one `disarm` row naming it (`disarmed`, baseline a `switches` row of `<cache>/tezgah.db`). The mark is a `gate_inactive` row of the same database (`gate_inactive`, audit Phase 1.3) | a row from the gate appears and no switch armed mid-session is still armed |
99
+ | `gate` | never `on`: shown only as `gate✗`, right after the version, in two cases. One: the prompt hook found at least 3 gated calls in this session's transcript since its first turn: write and shell calls, every one of which the gate answers with a `began` or a `deny` row (Bash/PowerShell/Edit/Write/MultiEdit/NotebookEdit on Claude, `bash`/`edit`/`write` `toolCall` blocks on omp, `exec_command`/`shell`/`apply_patch` on Codex). Over the same span the ledger holds no row from the gate. A PostToolUse row is no proof: that hook keeps writing while a broken PreToolUse lets every call through. The same prompt carries the line "tezgah gate inactive on this host: ...". Two: a kill switch appeared after the session's first prompt; the prompt hook writes one `disarm` row naming it (`disarmed`, baseline a `switches` row of `<cache>/tezgah.db`). The mark is a `gate_inactive` row of the same database (`gate_inactive`, audit Phase 1.3) | a row from the gate appears and no switch armed mid-session is still armed |
100
100
  | `drift` | never `on`: shown as `drift✗` when a session start found tezgah's own hook entries changed since install (`hooks/tezgah_attest.py::run`). One mark per session and host, a `harness_drift` row of `<cache>/tezgah.db` keyed `<session hash>.<host>`; the session's `attest` row names what drifted | the next clean start of that host in that session |
101
101
  | `crash` | never `on`: shown as `crash✗` when a hook of this session could not import its core and ran without tezgah (`hooks/tezgah_guard.py::import_failed`, mark an `import_crash` row of `~/.cache/tezgah/tezgah.db` keyed `<session hash>`). When `tezgah_context` itself is the module that failed, `statusline.py` prints `tezgah ✗ core import failed` instead of the line | never by itself: the mark stays for the session |
102
102
 
@@ -160,11 +160,11 @@ the command really ran it (34-38,84) and is the one writer that can earn `judge`
160
160
  (a run of `bin/tezgah-triage` or `bin/tezgah-docs`, `shell_kind`, `hooks/tezgah_context.py::shell_kind`)
161
161
  — plus `hooks/projects-auto-init.py::main`
162
162
  (`orch` on subagent start), `hosts/codex/hook.py::main`,
163
- `hosts/cursor/hook.py::dispatch`, hosts/opencode/plugins/tezgah.js:2144-2151,
164
- hosts/opencode/plugins/tezgah.js:2702, `hosts/omp/hook.py::handle`. Both Claude (hooks/hooks.json) and dsh
163
+ `hosts/cursor/hook.py::dispatch`, hosts/opencode/plugins/tezgah.js:2167-2174,
164
+ hosts/opencode/plugins/tezgah.js:2725, `hosts/omp/hook.py::handle`. Both Claude (hooks/hooks.json) and dsh
165
165
  (hosts/dsh/hooks.json) wire PostToolUse to that one shared hook, which is how dsh
166
166
  gets any used mark at all: with no local transcript, the store is the only channel
167
- its line has (hooks/projects-posttooluse.py:33-37).
167
+ its line has (hooks/projects-posttooluse.py:45-49).
168
168
 
169
169
  Claude's line never reads the store: it parses `tool_use` blocks out of
170
170
  `payload["transcript_path"]` plus the subagent transcripts under it, which is how
@@ -185,7 +185,7 @@ Every other surface reads the store.
185
185
  `setWidget`, and it is plain because omp strips ANSI (12-29). `status_line()`
186
186
  builds the line and carries `idx_override`, so a per-tool redraw forks no git
187
187
  (`hosts/omp/hook.py::status_line`); it redraws on session start, session switch, turn end
188
- and each watched tool result (hosts/omp/tezgah-hook.ts.in:541-569,607-612,653-678).
188
+ and each watched tool result (hosts/omp/tezgah-hook.ts.in:596-636,674-679,720-745).
189
189
  **Motion:** while the agent runs (`agent_start` until an `agent_end` whose
190
190
  `isTerminal` is not `false`) the widget leads with a braille spinner, the tool
191
191
  in flight (`tool_execution_start`/`_end`, "thinking" between tools) and the
package/docs/testing.md CHANGED
@@ -69,11 +69,11 @@ No matrix cancels its other legs when one fails (`fail-fast: false`).
69
69
  pull request's base. A change only under `docs/`, `assets/`, a root `*.md`
70
70
  page or the GitHub templates is prose: it runs `docs` and nothing else. An
71
71
  empty diff, a base it cannot read, a release or a manual run runs everything.
72
- - `docs`, prose only: the citations audit and
72
+ - `docs`, prose only: the citations audit, the clarity audit and
73
73
  `tests/impacted.py --ref <base>`, which maps a docs page to the docs modules.
74
74
  - `test` (`ci.yml:100`) on Python 3.10 and 3.14, the floor and the ceiling.
75
75
  It byte-compiles, lints with ruff from `requirements-dev.txt` (pip cached),
76
- audits the docs citations and runs `TEZGAH_E2E_STRICT=1 python3
76
+ audits the docs citations and clarity and runs `TEZGAH_E2E_STRICT=1 python3
77
77
  tests/e2e_plan_flow.py`. On `main` it also runs the serial suite with `-v`.
78
78
  A pull request skips that step, and so does a release, which passes the
79
79
  `release` input and runs one 3.10 leg.