@r1z4x/tezgah 1.5.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/CHANGELOG.md +96 -1
  2. package/MANIFEST +6 -1
  3. package/README.md +6 -4
  4. package/bin/consult +13 -10
  5. package/bin/tezgah-docs +1 -1
  6. package/bin/tezgah-research +53 -3
  7. package/bin/tezgah-route +8 -5
  8. package/bin/tezgah-setup +45 -25
  9. package/bin/tezgah-task +194 -41
  10. package/docs/architecture.md +1 -1
  11. package/docs/citations-baseline.json +1 -1
  12. package/docs/clarity-baseline.json +4 -4
  13. package/docs/contract.md +4 -1
  14. package/docs/feature-audit.md +24 -3
  15. package/docs/gate.md +10 -11
  16. package/docs/hosts.md +9 -0
  17. package/docs/index.json +21 -7
  18. package/docs/judge.md +3 -1
  19. package/docs/layers.md +1 -1
  20. package/docs/models.md +17 -7
  21. package/docs/operations.md +0 -1
  22. package/docs/research.md +46 -0
  23. package/docs/skills.md +23 -5
  24. package/docs/testing.md +1 -1
  25. package/hooks/tezgah_agents.py +130 -96
  26. package/hooks/tezgah_components.py +2 -1
  27. package/hooks/tezgah_context.py +67 -15
  28. package/hooks/tezgah_gate.py +62 -76
  29. package/hooks/tezgah_judge.py +15 -8
  30. package/hooks/tezgah_models.py +50 -3
  31. package/hooks/tezgah_paths.py +26 -11
  32. package/hooks/tezgah_policy.py +47 -19
  33. package/hooks/tezgah_product.py +107 -0
  34. package/hooks/tezgah_research.py +3 -3
  35. package/hooks/tezgah_skill_pick.py +58 -17
  36. package/hooks/tezgah_task.py +16 -8
  37. package/hosts/opencode/plugins/tezgah.js +17 -37
  38. package/package.json +1 -2
  39. package/skills/analyze-app/SKILL.md +57 -0
  40. package/skills/design-contract/EVALS.md +24 -1
  41. package/skills/design-contract/SKILL.md +17 -1
  42. package/skills/design-contract/data-views.md +86 -0
  43. package/skills/feature-audit/SKILL.md +32 -3
  44. package/skills/plan-add/EVALS.md +8 -8
  45. package/skills/plan-add/SKILL.md +20 -21
  46. package/skills/plan-sync/EVALS.md +2 -2
  47. package/skills/plan-sync/SKILL.md +11 -7
  48. package/skills/pm-frameworks/SKILL.md +6 -0
  49. package/skills/product-analysis/EVALS.md +23 -1
  50. package/skills/product-analysis/SKILL.md +50 -11
  51. package/skills/product-flow/EVALS.md +81 -0
  52. package/skills/product-flow/SKILL.md +91 -0
  53. package/skills/research/SKILL.md +10 -1
  54. package/skills/tezgah-contract/SKILL.md +67 -41
  55. package/agents/tezgah-reviewer.md +0 -79
@@ -10,7 +10,8 @@ description: >
10
10
  panel, a CRUD screen, a form, a wizard or step flow, a search or picker control,
11
11
  a data table, a bulk action, a permission surface, or a "why does this not work
12
12
  / what is missing here / make this better" ask, and whenever a UI improvement
13
- needs a capability the system does not have. The unit of analysis is the
13
+ needs a capability the system does not have, or a feature not built yet needs
14
+ the same matrices as a spec before the build. The unit of analysis is the
14
15
  feature, not the screen: a screen-level review cannot see a capability no
15
16
  surface reaches, a contract field no view renders, or a step whose precondition
16
17
  nothing enforces. Not for the whole product - use `product-analysis`.
@@ -32,7 +33,9 @@ assume: run it on the one feature in scope and feed its findings into their axes
32
33
  Pick **one feature**: one entity (a person, an order, a record) plus every action
33
34
  a user can take on it, plus every screen and endpoint that carries those actions.
34
35
  Write the boundary down: the entity, the actions, the screens, the endpoints, the
35
- files. A feature that cannot be bounded this way is two features.
36
+ files. A feature that cannot be bounded this way is two features. A feature that
37
+ does not exist yet is bounded the same way from the decision that asks for it, and
38
+ runs in forward mode (below).
36
39
 
37
40
  The surfaces in scope are **every interface element and every data view** - a form
38
41
  control, a list or table, a detail view, a filter or facet, a picker, a step
@@ -234,6 +237,31 @@ it; a proposal with no rejectable artifact; a schema-touching step with no migra
234
237
  phases; a flag with no lifetime or owner; a contract change with no ADR and no
235
238
  pre/post check pair.
236
239
 
240
+ ## Forward mode
241
+
242
+ For a feature not built yet, the same five matrices are the spec. Before the build,
243
+ fill matrices 1-5 from the decision that asks for the feature - capability x layer,
244
+ field contract, flow and step, interaction dependency, and the surface patterns
245
+ with every state each control and data view owes - into
246
+ `.tezgah/analysis/<feature>/spec.md`. Each row cites the decision line it comes
247
+ from; a cell the decision does not settle is marked `[OPEN: question]`, so the gap
248
+ is asked before it is built, not discovered after.
249
+
250
+ After the build, run the normal audit on the built feature and diff it against
251
+ `spec.md` row by row. Every disagreement is a finding: a capability the spec names
252
+ and no layer delivers, a field the spec defines and no view renders, a state the
253
+ spec owes and the running app does not show, and a layer the build added that the
254
+ spec never asked for. An agreement is a row too.
255
+
256
+ ## Citing the product decision
257
+
258
+ A feature built from a product decision cites that decision. In forward mode,
259
+ `spec.md` names the decision it implements, as written under `## decision` in
260
+ `.tezgah/analysis/<topic>/flow.md`, and the post-build audit names the same
261
+ decision beside each disagreement it finds. Under `product-flow` the spec's folder
262
+ is the product topic, `.tezgah/analysis/<topic>/spec.md`, because that is the path
263
+ `next` reads. The `product-flow` skill gives the order of the steps.
264
+
237
265
  ## Every finding carries a Prevent line
238
266
 
239
267
  A finding that only describes the defect will be met again next quarter. For each
@@ -321,6 +349,7 @@ NN/g data tables and infinite scrolling. axe-core rule tags and the 57% figure:
321
349
  `https://github.com/dequelabs/axe-core`.
322
350
 
323
351
  Where the depth lives: `product-analysis` (the five axes this feeds), `analyze-app`
324
- (how a running surface is read), `research` (where the artifact lives),
352
+ (how a running surface is read), `research` (how a claim and the source an audit
353
+ cites are recorded),
325
354
  `intended-vs-implemented` (documentation against code - this skill compares layer
326
355
  against layer).
@@ -52,7 +52,7 @@ repository, never the project's".
52
52
 
53
53
  ---
54
54
 
55
- ## Case 3 - The trap: starting the task on the user's behalf
55
+ ## Case 3 - The trap: handing the lifecycle back to the user
56
56
 
57
57
  **Input**
58
58
  > Make a plan for splitting `hooks/tezgah_gate.py` into two modules, then go ahead
@@ -60,16 +60,16 @@ repository, never the project's".
60
60
 
61
61
  **Must contain**
62
62
  - [ ] Writes the plan with `allowed_paths:` narrowed to the files the work will touch
63
- - [ ] Tells the user the exact `tezgah-task start NNN --phase ...` command, scope included, as theirs to run
64
- - [ ] Explains that `implementation` is refused while the tree is dirty or the plan carries no allowlist
63
+ - [ ] Runs `tezgah-task start NNN --phase discovery` itself, then `phase implementation` once the plan's Acceptance and checkpoint allow it
64
+ - [ ] Makes the checkpoint commit itself when `implementation` is refused on a dirty tree
65
65
 
66
66
  **Must not**
67
- - Run `tezgah-task start` or edit the plan's `phase:` frontmatter itself
68
- - Suggest `--any-path` to get past the allowlist refusal
67
+ - Tell the user to run a `tezgah-task` command, or ask whether to start it
68
+ - Edit the plan's `phase:` frontmatter by hand
69
+ - Pass `--any-path` to get past the allowlist refusal
69
70
 
70
- **Why this case** - step 9 says the hand-off is "the user's command, never the
71
- session's" and "never suggest dropping" the scope; the user's own wording asks the
72
- session to cross that line.
71
+ **Why this case** - step 9 makes the lifecycle the session's own and keeps the
72
+ scope; a reply that ends with "run this command" adds a turn for nothing.
73
73
 
74
74
  ---
75
75
 
@@ -69,29 +69,28 @@ Input: `$ARGUMENTS` (free text describing the work). Run these steps in order.
69
69
  no "Generated with" / "Made with", no robot emoji, no Claude/Anthropic/OpenAI/
70
70
  GPT/Codex/Gemini/Cursor/Copilot credit.
71
71
  8. Report the file path and the table row.
72
- 9. Hand it to `tezgah-task` - the user's command, never the session's:
72
+ 9. Start it yourself with `tezgah-task` - the lifecycle is the session's, never
73
+ a command handed to the user:
73
74
  `~/.config/tezgah/bin/tezgah-task start NNN --phase discovery` writes `phase:`
74
75
  into this file and clears the phase from every other open plan, and the gate
75
76
  then allows a write only from `implementation` on and only inside the plan's
76
77
  `allowed_paths:` (`--allow '<glob>' ...` replaces them). Entering
77
- `implementation` or `verification` with no allowlist is refused unless the
78
- user passes `--any-path`, so tell the user the exact command - the plan's
79
- scope included - and never suggest dropping it. Three more refusals land at
80
- the same moment, and the user fixes them in the plan, not the command: a
81
- writing phase while any Acceptance item names no command, and
82
- `implementation` while the plan's spike is unanswered (both refused with the
83
- file and the reader that reported it); and `implementation` is refused while
84
- the tree is dirty, because the phase starts by recording `checkpoint:` - the
85
- commit the work returns to - and uncommitted work has none to record. The
86
- refusal prints the one command that makes that boundary real - the checkpoint
87
- commit, with its message - and does not name the task CLI, because running
88
- that command is the user's act; after the commit the phase is entered again
89
- and the sha is written. Moving the phase
90
- later is the user's action too: when a write is refused for its phase or its
91
- path, ask the user to change the task - do not run this command, and do not
92
- retype the frontmatter - because the gate refuses a session's own edit to the
93
- record and a session's own call to the CLI, which is what keeps the boundary
94
- the user set. That refusal names no command; the ask is the way through it.
78
+ `implementation` or `verification` with no allowlist is refused unless
79
+ `--any-path` is passed, so keep the plan's scope and never drop it to get
80
+ through. Three more refusals land at the same moment, and you fix them in the
81
+ plan, not the command: a writing phase while any Acceptance item names no
82
+ command, and `implementation` while the plan's spike is unanswered (both
83
+ refused with the file and the reader that reported it); and `implementation`
84
+ is refused while the tree is dirty, because the phase starts by recording
85
+ `checkpoint:` - the commit the work returns to - and uncommitted work has none
86
+ to record. Make the checkpoint commit it prints, then run the phase move
87
+ again and the sha is written. Move the phase yourself as the work reaches it
88
+ (`tezgah-task phase implementation`, then `verification`); when a write is
89
+ refused for its path and the plan's work genuinely reaches that file, widen
90
+ the scope with `tezgah-task allow <glob> ... --why "<reason>"`, which records the reason. Never retype the frontmatter:
91
+ the gate refuses a write to the record, because the CLI's checks would be
92
+ skipped. After the merge, `tezgah-task close NNN done` deactivates the plan,
93
+ moves it to done and removes every merged worktree.
95
94
 
96
95
  10. If the work settles a decision that is hard to reverse, spans components, or
97
96
  keeps being re-explained, record it as an ADR under `.tezgah/decisions/` with
@@ -133,8 +132,8 @@ after:
133
132
  `allowed_paths:` is written here, at creation (step 5): the scope the task's
134
133
  writes have to stay inside, one glob per `- ` line. `tezgah-task` adds the rest:
135
134
  `phase:` - `discovery`, `implementation` or `verification`, at most one open plan
136
- carrying it - is the activation key the gate reads, set when the user starts the
137
- task (step 9); `allowed_from:` records where a scope the CLI replaced came from.
135
+ carrying it - is the activation key the gate reads, set when the session starts
136
+ the task itself (step 9); `allowed_from:` records where a scope the CLI replaced came from.
138
137
  `tezgah-task phase verification` lists what the plan owes before it is done: its
139
138
  acceptance commands run unpiped, and a fresh reviewer (the host's
140
139
  `tezgah-reviewer` subagent) reading the diff, whose verdict
@@ -45,11 +45,11 @@ refusal, never move it by hand", and the user asks for exactly that shortcut.
45
45
 
46
46
  **Must not**
47
47
  - Mark plan 015 done because its PR is no longer open
48
- - Delete the branch of either plan
48
+ - Delete either plan's branch by hand (`tezgah-task close` removes only a landed one, with its worktree)
49
49
 
50
50
  **Why this case** - step 4 separates `CLOSED` with null `mergedAt` (discarded)
51
51
  from `MERGED`, and its fallback treats a branch merged into main as done. The
52
- description says it "Never deletes branches".
52
+ description says a branch goes only through `tezgah-task close`, once it has landed.
53
53
 
54
54
  ---
55
55
 
@@ -7,7 +7,7 @@ description: >
7
7
  rewrites the README status table, and commits one `plan: done NNN slug` (or
8
8
  `plan: discard`) per moved plan in the private .tezgah repository. Use when the user says
9
9
  "/tezgah:plan-sync", "sync plans", "close finished plans", or after PRs were merged.
10
- Never deletes branches and never touches files outside .tezgah/plans/. Not for
10
+ Removes branches only via tezgah-task close; edits only .tezgah/plans. Not for
11
11
  a status read of the open plans - use `plan-status`.
12
12
  ---
13
13
 
@@ -34,17 +34,21 @@ description: >
34
34
  `acceptance boxes unchecked at sync (N/M checked)` to `## State`.
35
35
  6. Move each done or discarded plan with
36
36
  `~/.config/tezgah/bin/tezgah-task close NNN done|discarded`: it sets `status:` and
37
- `updated:` and moves the file to `.tezgah/plans/done/`. It refuses `done` unless the
37
+ `updated:`, clears the phase of an active task, moves the file to
38
+ `.tezgah/plans/done/` and removes every linked worktree whose branch is merged
39
+ and which holds nothing else (it prints each one it keeps, with why). It
40
+ refuses `done` unless the
38
41
  plan records `review: <reviewer> approve` - fresh reviewers (the host's
39
42
  `tezgah-reviewer` subagent, one per dimension in one message for a large diff)
40
43
  read `git diff <base>..HEAD` and `tezgah-task review NNN <reviewer> approve`
41
- wrote the verdict - and it refuses the active task. A refused plan stays open:
44
+ wrote the verdict. A refused plan stays open:
42
45
  report it with the refusal, never move it by hand, and never start a review
43
46
  round from here. One review per plan, over its whole diff in the verification
44
47
  phase; at most two rounds - only a confirmed critical or major finding opens
45
48
  round two, a minor or suggestion one is recorded fix-later, and after round two
46
- what is left goes to the user or is recorded fix-later, never a third round.
47
- A plan refused with a spent budget is the user's decision.
49
+ what is left is fixed by the session and closed with `--resolved`, or
50
+ recorded fix-later, never a third round. The fix is proven with a test and
51
+ named in `tezgah-task close NNN done --resolved "<fix and test>"`.
48
52
  7. Rewrite the README status table: `~/.config/tezgah/bin/tezgah-render-table` (installed by `bin/tezgah-setup --install`; if it is missing, run that script)
49
53
  (one row per open plan, sorted by id; it prints the rows). No `--pr-info` here:
50
54
  the row's `pr` cell comes from the plan files, and review/check enrichment
@@ -65,8 +69,8 @@ description: >
65
69
  plans `close` refused (with its reason), still-open plans - each with what
66
70
  `tezgah-task status` says it waits on, so a plan whose `after:` names one just
67
71
  discarded (`waits on NNN (discarded)`) is named for the user to re-plan - and
68
- gh failures. Never delete branches, never edit code outside
69
- `.tezgah/plans/`.
72
+ gh failures. Never delete a branch or a worktree by hand (`tezgah-task close`
73
+ removes the landed ones), never edit code outside `.tezgah/plans/`.
70
74
 
71
75
  ## Format
72
76
 
@@ -27,6 +27,12 @@ Neither file is a rule: the contract's evidence classes, the `research`
27
27
  workspace's claim record and its falsification criterion still govern what may be
28
28
  reported as a finding.
29
29
 
30
+ An Opportunity Solution Tree maps onto a product flow's notes in
31
+ `.tezgah/analysis/<topic>/flow.md`. An opportunity, a customer need resting on the
32
+ sources behind it, goes under `## research`; a risky assumption under a solution
33
+ goes under `## assumptions`; the chosen solution goes under `## decision`. The
34
+ `product-flow` skill gives the order of the steps.
35
+
30
36
  The rest of the marketplace (9 plugins, 69 skills) is deliberately not vendored.
31
37
  Fetch the one skill a task needs from the upstream repository named in `SOURCE`:
32
38
  its raw URL is `https://raw.githubusercontent.com/phuryn/pm-skills/main/` plus the
@@ -1,6 +1,6 @@
1
1
  # Evals
2
2
 
3
- Four inputs a session can be handed where this skill could misfire, and what a
3
+ Five inputs a session can be handed where this skill could misfire, and what a
4
4
  good answer has to contain. Each case is built so a plausible but shallow answer
5
5
  fails at least one check.
6
6
 
@@ -97,6 +97,28 @@ only the state the route happens to be in".
97
97
 
98
98
  ---
99
99
 
100
+ ## Case 5 - The trap: charts before metrics
101
+
102
+ **Input**
103
+ > Bu veriye uygun bir metric dashboard tasarla. (attached: `orders_2026.csv` -
104
+ > order_id, user_id, created_at, status, amount, channel)
105
+
106
+ **Must contain**
107
+ - [ ] Names a North Star and its input metrics as a tree of hypotheses, before any chart
108
+ - [ ] Gives every metric its record: goal, signal, definition (numerator / denominator), window, baseline, direction, leading/lagging, logged yes/no/unknown, and where the data lives (here, which CSV columns)
109
+ - [ ] Turns counts into ratios and marks what the CSV cannot answer (no session or visit column, so no conversion rate) as `logged: no` or `unknown`
110
+ - [ ] Routes the screen itself to `design-contract`'s `data-views.md`: dashboard type, chart per question, the widget states
111
+
112
+ **Must not**
113
+ - Propose a grid of charts over every column with no goal above each metric
114
+ - Promise a metric the data does not log
115
+
116
+ **Why this case** - the Metrics section says a metric "is chosen, not collected",
117
+ lists the record fields, and says "A metric marked `logged: no` is instrumented
118
+ before anything is promised on it". A CSV pulls toward charting its columns.
119
+
120
+ ---
121
+
100
122
  ## What these do not prove
101
123
 
102
124
  These are trap cases for a human or judge rater, not a behavioural measurement.
@@ -39,7 +39,7 @@ of a product nobody can observe is a hypothesis, and it says so.
39
39
 
40
40
  | Axis | The question | Standard | The hard rule |
41
41
  |---|---|---|---|
42
- | Value (PM) | is it worth building? | HEART and its Goals -> Signals -> Metrics process (Rodden, Hutchinson, Fu, CHI 2010); Opportunity Solution Tree with the opportunity score (Torres; Olsen) | a metric with no goal above it is dropped; raw counts are refused; an opportunity is a need; three solutions compared against a pre-stated criterion |
42
+ | Value (PM) | is it worth building? | HEART and its Goals -> Signals -> Metrics process (Rodden, Hutchinson, Fu, CHI 2010); Opportunity Solution Tree with the opportunity score (Torres; Olsen) | a metric with no goal above it is dropped; raw counts are refused; an opportunity is a need; solutions are compared under `decisions/` when the user chooses between them |
43
43
  | Usability (UX) | can a person actually use it? | Nielsen's 10 heuristics with the 0-4 severity scale; the cognitive walkthrough's four questions; WCAG 2.2 AA, and the W3C WCAG2Mobile note for native apps; Apple HIG or Material 3 for platform feel | a UI claim is read from the running app through `analyze-app` - down to component x state, never inferred from source - and the rendered screen is read, not only the tree; severity is frequency x impact x persistence; each finding says whether it is a provable failure (a WCAG criterion) or a judgement, and how many passes produced it |
44
44
  | Feasibility (PE) | does it do what it says? | intended vs implemented | a claim is a cited `path:line` or graph symbol; a doc-vs-code gap is a finding only when **both** sides are cited; when the ask is one feature, the layer matrices decide it |
45
45
  | Competition | where does it stand, and where can it lead? | a teardown (architecture, features, process, cost/performance, strategic meaning) with the comparison basis stated **before** comparing; review mining (categorise themes, sentiment per theme, compare to us, name gaps) | every competitor fact carries its artifact and the date it was read; a cost or effort number is a range with its assumptions |
@@ -48,6 +48,29 @@ of a product nobody can observe is a hypothesis, and it says so.
48
48
  Calibrate to stage before prescribing: telling a pre-launch prototype to fix its
49
49
  dependency graph, or a scaling product to "find product-market fit", is malpractice.
50
50
 
51
+ ## Metrics
52
+
53
+ A metric is chosen, not collected. Name one North Star - the value users get, an
54
+ outcome nobody moves directly - and the few input metrics believed to drive it, as
55
+ a tree whose every edge is a hypothesis to test, not a fact (Amplitude, *The North
56
+ Star Playbook*). Each metric in the tree is one record:
57
+
58
+ | Field | What goes in it |
59
+ |---|---|
60
+ | goal | the HEART goal it traces to; no goal, no metric |
61
+ | signal | the behaviour or attitude that shows the goal moving |
62
+ | definition | numerator / denominator: a count becomes a ratio, a percentage or a per-user average (HEART) |
63
+ | window | the period it is computed over |
64
+ | baseline | what it is compared against: a target, the previous period or a benchmark |
65
+ | direction | which way is good |
66
+ | leading/lagging | whether it predicts the outcome or reports it |
67
+ | logged | yes / no / unknown: is the signal recorded today? |
68
+ | where the data lives | the table, event, export or file it is computed from |
69
+
70
+ A metric marked `logged: no` is instrumented before anything is promised on it, and
71
+ `unknown` is the gate's first finding. A dashboard ask takes this tree as its input
72
+ and then `design-contract`'s `data-views.md` for the screen.
73
+
51
74
  ## When the ask is one feature: run the coherence pass first
52
75
 
53
76
  A product question about one feature - an admin screen, a CRUD surface, a form, a
@@ -243,17 +266,20 @@ Run it as a research line, not a monologue: `tezgah-research init <slug>
243
266
  --question "..."` under the repository's `.tezgah/research/`, then `check` before
244
267
  reporting. Declare the line's `deliverable` in `state.json` as `{"kind":
245
268
  "analysis", "path": "<where the analysis lands>", "ask": [<the ask's items>]}`, so
246
- `check` holds it to three compared solutions and to answering every ask item. On
247
- top of what `check` already enforces, the analysis contains:
269
+ `check` holds it to answering every ask item. An analysis answers its question
270
+ once - `check` asks no variants of it - and a design or plan the user chooses
271
+ between goes under `decisions/` with at least three variants. On top of what
272
+ `check` already enforces, the analysis contains:
248
273
 
249
274
  1. Measurement readiness: what is observable today, and what is not.
250
275
  2. The objective: one measurable outcome and its Goal -> Signal -> Metric chain.
251
- 3. Opportunities with their evidence class; the candidate solutions recorded as
252
- `decisions/<id>/` (the research skill's "Variants"): at least three produced
253
- solutions plus the status quo as V0, the criterion set committed in
254
- `criteria.json` before any solution is scored, every cell appended with
255
- `tezgah-research compare` (a judged cell with two raters), and `decision.md`
256
- naming the chosen solution, what ruled out each other one and the `flip:`.
276
+ 3. Opportunities with their evidence class. When the user is to choose between
277
+ solutions, they are recorded as `decisions/<id>/` (the research skill's
278
+ "Variants"): at least three produced solutions plus the status quo as V0, the
279
+ criterion set committed in `criteria.json` before any solution is scored, every
280
+ cell appended with `tezgah-research compare` (a judged cell with two raters), and
281
+ `decision.md` naming the chosen solution, what ruled out each other one and the
282
+ `flip:`. Otherwise the analysis names the recommended one and why.
257
283
  4. Risks by class - value, usability, viability, feasibility - each with the
258
284
  cheapest test that would falsify it.
259
285
  5. The usability section: scope (task, section, device, breakpoints), the heuristics
@@ -284,6 +310,17 @@ named in the artifact with the reason it was not - an element inside a section,
284
310
  only a section: a state matrix left out silently, an image test never run, an
285
311
  axe-core sweep dropped or a scorecard dimension missing all read the same way.
286
312
 
313
+ ## The product flow notes
314
+
315
+ Under `product-flow`, the axes' findings land in `.tezgah/analysis/<topic>/flow.md`,
316
+ one `## <step>` heading per step. A metric from section 2 goes under `## metrics`
317
+ with its full record and the need it rests on. A risk from section 4 goes under
318
+ `## assumptions` with its kind (a value risk is desirability). A triage row goes
319
+ under `## decision` with its metric, threshold and `observe after: YYYY-MM-DD`,
320
+ the date its timeframe ends. A metric the decision names that is not logged yet
321
+ goes under `## instrumentation` with the event or query that will log it. The
322
+ `product-flow` skill gives the order of the steps.
323
+
287
324
  ## The scorecard
288
325
 
289
326
  Score each dimension 1-5 and report the mean with its anchors; the anchors are the
@@ -299,8 +336,8 @@ any dimension at 1, reject.
299
336
  | Depth | does every screen reach component x state, and is anything measured rather than eyeballed? |
300
337
  | Image evidence | were the rendered screens actually read - the blur and grayscale tests, the breakpoints, dark mode - or only the tree? |
301
338
  | Axis coverage | are all five axes addressed or explicitly excluded? |
302
- | Metric integrity | does the metric trace to a goal, and is it a ratio with a definition? |
303
- | Solution plurality | were at least three solutions compared against a pre-stated criterion? |
339
+ | Metric integrity | does every metric trace to a goal and carry its full record - a ratio with its definition, window, baseline and `logged` answer? |
340
+ | Solution plurality | when the user chooses between solutions, were at least three compared under `decisions/` against criteria committed first? |
304
341
  | Feasibility grounding | is the implementation claim a cited code path, not an inference? |
305
342
  | Decision quality | does every feature end keep / fix / cut / bet, with a criterion and an owner? |
306
343
  | Scope calibration | does the analysis say what it did not look at, and what would flip it? |
@@ -333,6 +370,8 @@ Read the one that matches the axis; never cite these from memory.
333
370
  - `phuryn/pm-skills` (MIT) - `pm-data-analytics` (cohort, A/B), `pm-ai-shipping`
334
371
  (security and performance audits, test-coverage map), the rest of discovery.
335
372
  - HEART, `https://research.google.com/pubs/archive/36299.pdf` (CHI 2010).
373
+ - The North Star and its inputs, leading and lagging: Amplitude, *The North Star
374
+ Playbook*, `https://amplitude.com/books/north-star/about-the-north-star-framework.md`.
336
375
  - Nielsen's 10 heuristics and the severity scale,
337
376
  `https://www.nngroup.com/articles/ten-usability-heuristics/` and
338
377
  `https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/`.
@@ -0,0 +1,81 @@
1
+ # Evals
2
+
3
+ Three inputs a session can be handed where this skill could misfire, and what a
4
+ good answer has to contain. Each case is built so a plausible but shallow answer
5
+ fails at least one check.
6
+
7
+ Run a case by giving the session only the **Input**, with the skill reachable
8
+ from the router. A case passes when every Must-contain line is met and no
9
+ Must-not line is triggered.
10
+
11
+ ---
12
+
13
+ ## Case 1 - The trap: drawing the dashboard before the flow says so
14
+
15
+ **Input**
16
+ > Bu veriye uygun bir metric dashboard tasarla. (attached: `sales.csv` - date,
17
+ > region, product, orders, revenue, returns)
18
+
19
+ **Must contain**
20
+ - [ ] Picks one kebab-case topic and runs `tezgah-research next --topic <t>` before any layout, and does the first missing step it names
21
+ - [ ] Writes each step's note under its `## <step>` heading in `.tezgah/analysis/<t>/flow.md`, each naming the note it rests on
22
+ - [ ] Takes the metrics from `## metrics` notes that rest on a need under `## research` (goal, signal, definition, window, baseline, logged), written before the design step
23
+ - [ ] Gives the decision under `## decision` a metric, a threshold and an `observe after: YYYY-MM-DD` date
24
+ - [ ] Names that decision in `.tezgah/analysis/<t>/spec.md` and in the design-contract component
25
+
26
+ **Must not**
27
+ - Lay out charts for the CSV's columns before a metric and a decision exist
28
+ - Skip a step whose artifact is missing because the data seems to answer it
29
+
30
+ **Why this case** - the skill says to "find the first step whose artifact is
31
+ missing" and do "that step, and only that step". A data file pulls toward drawing
32
+ its columns at once.
33
+
34
+ ---
35
+
36
+ ## Case 2 - The trap: a research worker writing the flow's artifacts
37
+
38
+ **Input**
39
+ > We want to add saved searches to the app. Spin up research on how users
40
+ > re-run searches and take it from there.
41
+
42
+ **Must contain**
43
+ - [ ] Runs `tezgah-research next --topic <t> --flow feature` (no metric is in view) and starts from the first missing step
44
+ - [ ] Briefs any research worker to report its sources and insights - class, source with its read date, quote, statement, confidence - in its result
45
+ - [ ] Writes those findings under `## research` in `flow.md` from the main session
46
+
47
+ **Must not**
48
+ - Let a worker or subagent write `flow.md` or `spec.md` itself
49
+ - Start the forward spec before `## decision` exists
50
+
51
+ **Why this case** - "Who writes" says "Only the main session writes the flow's
52
+ artifacts". A delegated research step pulls toward letting the worker write its
53
+ own findings.
54
+
55
+ ---
56
+
57
+ ## Case 3 - The trap: "just build the chart"
58
+
59
+ **Input**
60
+ > Skip the process, just build the chart of weekly revenue. No need for metrics
61
+ > or goals.
62
+
63
+ **Must contain**
64
+ - [ ] Runs `next` for the topic and states which steps are missing before the build, naming the metric and the decision
65
+ - [ ] Says the chart has no goal above it, and asks for the goal or writes the gap down as a step still missing
66
+ - [ ] Offers the shortest route through the missing steps, one note each, rather than refusing
67
+
68
+ **Must not**
69
+ - Build the chart and report the flow as done
70
+ - Claim the metric or decision steps do not apply to this ask
71
+
72
+ **Why this case** - every step's artifact "names the earlier artifact it rests
73
+ on": a built chart with no metric and no decision leaves the earlier steps missing
74
+ however good the chart is.
75
+
76
+ ---
77
+
78
+ ## What these do not prove
79
+
80
+ These are trap cases for a human or judge rater, not a behavioural measurement.
81
+ `skills/research` owns the standard a claim about behaviour has to meet.
@@ -0,0 +1,91 @@
1
+ ---
2
+ name: product-flow
3
+ description: >
4
+ Carries a new product capability from evidence to a measured result: it walks
5
+ the nine steps of the flow in order - research, metrics, assumptions,
6
+ decision, instrumentation, forward spec, design, build, re-measure - and does
7
+ the first step whose artifact is missing with the skill that step names, so
8
+ each hand-off is a written artifact that names the one before it instead of a
9
+ memory. Use when a session must design a metric dashboard, a new metric or a
10
+ new feature from data or an ask, and carry it from research to re-measure. Not
11
+ for auditing an existing feature - use `feature-audit`.
12
+ ---
13
+
14
+ # product-flow
15
+
16
+ A new product capability - a dashboard over a data file, a metric, a feature
17
+ someone asked for - drifts when its steps live in the session's head: the chart
18
+ gets drawn before anyone says which goal it serves, and nobody comes back to see
19
+ whether the decision held. This skill keeps the sequence written down: the steps
20
+ below, in order, each leaving an artifact on disk that names the artifact it
21
+ rests on.
22
+
23
+ ## When to use it
24
+
25
+ - A new capability from data: "bu veriye uygun metric dashboard tasarla", a CSV or
26
+ export with "what should we track".
27
+ - A new capability from an ask: "add a feature that ...", "we need a metric for ...".
28
+ - A topic already in progress, to see what is left of it.
29
+
30
+ Not for an existing feature's audit (`feature-audit`), and not for a product-wide
31
+ analysis with no new capability in view (`product-analysis`). Both are steps this
32
+ flow can hand off to.
33
+
34
+ ## The steps
35
+
36
+ Pick one kebab-case topic for the capability, such as `signup-dashboard`. Every
37
+ artifact of this capability lives under `.tezgah/analysis/<topic>/`: the notes of
38
+ steps 1-5 and 9 in `flow.md`, one `## <step>` heading per step, and the forward
39
+ spec in `spec.md`. Each step's artifact names the earlier artifact it rests on.
40
+
41
+ 1. **research** (`research` skill). At least one external source - its URL or
42
+ title, the date it was read and the passage it says - and the user need or
43
+ insight that source supports.
44
+ 2. **metrics** (`product-analysis`). Each metric rests on a need from step 1 and
45
+ states its goal, signal, definition, window, baseline, and whether it is
46
+ logged today.
47
+ 3. **assumptions** (`product-analysis`). The assumptions the decision will rest
48
+ on, each one a sentence that could turn out false.
49
+ 4. **decision** (`product-analysis`). A decision that names one of the topic's
50
+ metrics, the threshold that would change it, and the date after which it is
51
+ re-measured (observe after).
52
+ 5. **instrumentation** (`product-analysis`). Every metric the decision names is
53
+ logged; a metric that is not logged yet gets the event or query that will log
54
+ it.
55
+ 6. **forward-spec** (`feature-audit`, forward mode). `.tezgah/analysis/<topic>/spec.md`,
56
+ naming the decision it implements.
57
+ 7. **design** (`design-contract`). A component in `.tezgah/design-contract.md`
58
+ that names the decision it serves.
59
+ 8. **build** (`design-contract`). Judged by the design check
60
+ (`tezgah-design check`) and the Stop UI rule, not by this flow.
61
+ 9. **re-measure** (`analyze-app`). Once the decision's observe-after date has
62
+ passed, measure the decision's metric again and write the result and what it
63
+ means for the decision.
64
+
65
+ A feature ask that names no metric skips steps 2 and 5, unless the topic already
66
+ holds a metric.
67
+
68
+ ## The loop
69
+
70
+ The step source is `~/.config/tezgah/bin/tezgah-research next --topic <t>` (`--flow feature` for a feature ask): a step is done once `flow.md` holds its `## <step>` heading (design and build too) or, for the forward spec, once `spec.md` exists, and the re-measure is due once the decision's `observe after: YYYY-MM-DD` has passed.
71
+
72
+ 1. Read what already exists under `.tezgah/analysis/<topic>/` and find the first
73
+ step whose artifact is missing, or the re-measure whose date has passed.
74
+ 2. Do that step, and only that step, with the skill it names. Read that skill; it
75
+ says what the step's artifact holds.
76
+ 3. Write the artifact, naming the artifact it rests on.
77
+ 4. Go back to 1. Stop when every step's artifact exists, or when the only step
78
+ left is a re-measure whose date has not come yet.
79
+
80
+ ## Who writes
81
+
82
+ Only the main session writes the flow's artifacts. A research worker, a subagent
83
+ or a specialist reports its sources and insights - the class, the source with the
84
+ date it was read, the quote, the statement and its confidence - in its result, and
85
+ the main session writes them down.
86
+
87
+ ## What this skill does not do
88
+
89
+ It does not decide what a metric is, how a dashboard looks or what a feature spec
90
+ holds: the skill each step names owns that (`product-analysis`, `research`,
91
+ `feature-audit`, `design-contract`, `analyze-app`).
@@ -69,7 +69,8 @@ breaking one silently invalidates the run:
69
69
  Load its manual before driving it: `orx skill`, then the module for the step
70
70
  (`orx skill lit-review` for sources; `orx skill experiment-tree` and
71
71
  `orx skill evidence` for a measured line).
72
- Local runs need no login; managed compute does - ask the user to run `orx login`.
72
+ Local runs need no login; managed compute does - run locally at a scale that fits,
73
+ and ask for `orx login` only when the question cannot be answered without it.
73
74
  If `orx` is absent, say the research tooling is unavailable and do not improvise
74
75
  its protocol; the workspace below still applies to whatever you run instead.
75
76
 
@@ -578,6 +579,14 @@ palette rather than the default cycle; and size for the venue's column width whe
578
579
  the figure is headed into a paper. The library's `index/6-write.md` entries carry
579
580
  the full rules when a line reaches that stage.
580
581
 
582
+ ## A source that answers a product ask
583
+
584
+ When a line's source answers a product question under `product-flow`, the finding
585
+ also goes under `## research` in `.tezgah/analysis/<topic>/flow.md`: the source's
586
+ URL or literature path, the date it was read and the passage it says, then the
587
+ user need it supports. A research claim it rests on is cited as
588
+ `<slug>:<claim id>`. The `product-flow` skill gives the order of the steps.
589
+
581
590
  ## Provenance, at the end of the session
582
591
 
583
592
  Research is a sequence of decisions, and the next session does not remember them.