@r1z4x/tezgah 1.5.0 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +96 -1
- package/MANIFEST +6 -1
- package/README.md +6 -4
- package/bin/consult +13 -10
- package/bin/tezgah-docs +1 -1
- package/bin/tezgah-research +53 -3
- package/bin/tezgah-route +8 -5
- package/bin/tezgah-setup +45 -25
- package/bin/tezgah-task +194 -41
- package/docs/architecture.md +1 -1
- package/docs/citations-baseline.json +1 -1
- package/docs/clarity-baseline.json +4 -4
- package/docs/contract.md +4 -1
- package/docs/feature-audit.md +24 -3
- package/docs/gate.md +10 -11
- package/docs/hosts.md +9 -0
- package/docs/index.json +21 -7
- package/docs/judge.md +3 -1
- package/docs/layers.md +1 -1
- package/docs/models.md +17 -7
- package/docs/operations.md +0 -1
- package/docs/research.md +46 -0
- package/docs/skills.md +23 -5
- package/docs/testing.md +1 -1
- package/hooks/tezgah_agents.py +130 -96
- package/hooks/tezgah_components.py +2 -1
- package/hooks/tezgah_context.py +67 -15
- package/hooks/tezgah_gate.py +62 -76
- package/hooks/tezgah_judge.py +15 -8
- package/hooks/tezgah_models.py +50 -3
- package/hooks/tezgah_paths.py +26 -11
- package/hooks/tezgah_policy.py +47 -19
- package/hooks/tezgah_product.py +107 -0
- package/hooks/tezgah_research.py +3 -3
- package/hooks/tezgah_skill_pick.py +58 -17
- package/hooks/tezgah_task.py +16 -8
- package/hosts/opencode/plugins/tezgah.js +17 -37
- package/package.json +1 -2
- package/skills/analyze-app/SKILL.md +57 -0
- package/skills/design-contract/EVALS.md +24 -1
- package/skills/design-contract/SKILL.md +17 -1
- package/skills/design-contract/data-views.md +86 -0
- package/skills/feature-audit/SKILL.md +32 -3
- package/skills/plan-add/EVALS.md +8 -8
- package/skills/plan-add/SKILL.md +20 -21
- package/skills/plan-sync/EVALS.md +2 -2
- package/skills/plan-sync/SKILL.md +11 -7
- package/skills/pm-frameworks/SKILL.md +6 -0
- package/skills/product-analysis/EVALS.md +23 -1
- package/skills/product-analysis/SKILL.md +50 -11
- package/skills/product-flow/EVALS.md +81 -0
- package/skills/product-flow/SKILL.md +91 -0
- package/skills/research/SKILL.md +10 -1
- package/skills/tezgah-contract/SKILL.md +67 -41
- package/agents/tezgah-reviewer.md +0 -79
|
@@ -10,7 +10,8 @@ description: >
|
|
|
10
10
|
panel, a CRUD screen, a form, a wizard or step flow, a search or picker control,
|
|
11
11
|
a data table, a bulk action, a permission surface, or a "why does this not work
|
|
12
12
|
/ what is missing here / make this better" ask, and whenever a UI improvement
|
|
13
|
-
needs a capability the system does not have
|
|
13
|
+
needs a capability the system does not have, or a feature not built yet needs
|
|
14
|
+
the same matrices as a spec before the build. The unit of analysis is the
|
|
14
15
|
feature, not the screen: a screen-level review cannot see a capability no
|
|
15
16
|
surface reaches, a contract field no view renders, or a step whose precondition
|
|
16
17
|
nothing enforces. Not for the whole product - use `product-analysis`.
|
|
@@ -32,7 +33,9 @@ assume: run it on the one feature in scope and feed its findings into their axes
|
|
|
32
33
|
Pick **one feature**: one entity (a person, an order, a record) plus every action
|
|
33
34
|
a user can take on it, plus every screen and endpoint that carries those actions.
|
|
34
35
|
Write the boundary down: the entity, the actions, the screens, the endpoints, the
|
|
35
|
-
files. A feature that cannot be bounded this way is two features.
|
|
36
|
+
files. A feature that cannot be bounded this way is two features. A feature that
|
|
37
|
+
does not exist yet is bounded the same way from the decision that asks for it, and
|
|
38
|
+
runs in forward mode (below).
|
|
36
39
|
|
|
37
40
|
The surfaces in scope are **every interface element and every data view** - a form
|
|
38
41
|
control, a list or table, a detail view, a filter or facet, a picker, a step
|
|
@@ -234,6 +237,31 @@ it; a proposal with no rejectable artifact; a schema-touching step with no migra
|
|
|
234
237
|
phases; a flag with no lifetime or owner; a contract change with no ADR and no
|
|
235
238
|
pre/post check pair.
|
|
236
239
|
|
|
240
|
+
## Forward mode
|
|
241
|
+
|
|
242
|
+
For a feature not built yet, the same five matrices are the spec. Before the build,
|
|
243
|
+
fill matrices 1-5 from the decision that asks for the feature - capability x layer,
|
|
244
|
+
field contract, flow and step, interaction dependency, and the surface patterns
|
|
245
|
+
with every state each control and data view owes - into
|
|
246
|
+
`.tezgah/analysis/<feature>/spec.md`. Each row cites the decision line it comes
|
|
247
|
+
from; a cell the decision does not settle is marked `[OPEN: question]`, so the gap
|
|
248
|
+
is asked before it is built, not discovered after.
|
|
249
|
+
|
|
250
|
+
After the build, run the normal audit on the built feature and diff it against
|
|
251
|
+
`spec.md` row by row. Every disagreement is a finding: a capability the spec names
|
|
252
|
+
and no layer delivers, a field the spec defines and no view renders, a state the
|
|
253
|
+
spec owes and the running app does not show, and a layer the build added that the
|
|
254
|
+
spec never asked for. An agreement is a row too.
|
|
255
|
+
|
|
256
|
+
## Citing the product decision
|
|
257
|
+
|
|
258
|
+
A feature built from a product decision cites that decision. In forward mode,
|
|
259
|
+
`spec.md` names the decision it implements, as written under `## decision` in
|
|
260
|
+
`.tezgah/analysis/<topic>/flow.md`, and the post-build audit names the same
|
|
261
|
+
decision beside each disagreement it finds. Under `product-flow` the spec's folder
|
|
262
|
+
is the product topic, `.tezgah/analysis/<topic>/spec.md`, because that is the path
|
|
263
|
+
`next` reads. The `product-flow` skill gives the order of the steps.
|
|
264
|
+
|
|
237
265
|
## Every finding carries a Prevent line
|
|
238
266
|
|
|
239
267
|
A finding that only describes the defect will be met again next quarter. For each
|
|
@@ -321,6 +349,7 @@ NN/g data tables and infinite scrolling. axe-core rule tags and the 57% figure:
|
|
|
321
349
|
`https://github.com/dequelabs/axe-core`.
|
|
322
350
|
|
|
323
351
|
Where the depth lives: `product-analysis` (the five axes this feeds), `analyze-app`
|
|
324
|
-
(how a running surface is read), `research` (
|
|
352
|
+
(how a running surface is read), `research` (how a claim and the source an audit
|
|
353
|
+
cites are recorded),
|
|
325
354
|
`intended-vs-implemented` (documentation against code - this skill compares layer
|
|
326
355
|
against layer).
|
package/skills/plan-add/EVALS.md
CHANGED
|
@@ -52,7 +52,7 @@ repository, never the project's".
|
|
|
52
52
|
|
|
53
53
|
---
|
|
54
54
|
|
|
55
|
-
## Case 3 - The trap:
|
|
55
|
+
## Case 3 - The trap: handing the lifecycle back to the user
|
|
56
56
|
|
|
57
57
|
**Input**
|
|
58
58
|
> Make a plan for splitting `hooks/tezgah_gate.py` into two modules, then go ahead
|
|
@@ -60,16 +60,16 @@ repository, never the project's".
|
|
|
60
60
|
|
|
61
61
|
**Must contain**
|
|
62
62
|
- [ ] Writes the plan with `allowed_paths:` narrowed to the files the work will touch
|
|
63
|
-
- [ ]
|
|
64
|
-
- [ ]
|
|
63
|
+
- [ ] Runs `tezgah-task start NNN --phase discovery` itself, then `phase implementation` once the plan's Acceptance and checkpoint allow it
|
|
64
|
+
- [ ] Makes the checkpoint commit itself when `implementation` is refused on a dirty tree
|
|
65
65
|
|
|
66
66
|
**Must not**
|
|
67
|
-
-
|
|
68
|
-
-
|
|
67
|
+
- Tell the user to run a `tezgah-task` command, or ask whether to start it
|
|
68
|
+
- Edit the plan's `phase:` frontmatter by hand
|
|
69
|
+
- Pass `--any-path` to get past the allowlist refusal
|
|
69
70
|
|
|
70
|
-
**Why this case** - step 9
|
|
71
|
-
|
|
72
|
-
session to cross that line.
|
|
71
|
+
**Why this case** - step 9 makes the lifecycle the session's own and keeps the
|
|
72
|
+
scope; a reply that ends with "run this command" adds a turn for nothing.
|
|
73
73
|
|
|
74
74
|
---
|
|
75
75
|
|
package/skills/plan-add/SKILL.md
CHANGED
|
@@ -69,29 +69,28 @@ Input: `$ARGUMENTS` (free text describing the work). Run these steps in order.
|
|
|
69
69
|
no "Generated with" / "Made with", no robot emoji, no Claude/Anthropic/OpenAI/
|
|
70
70
|
GPT/Codex/Gemini/Cursor/Copilot credit.
|
|
71
71
|
8. Report the file path and the table row.
|
|
72
|
-
9.
|
|
72
|
+
9. Start it yourself with `tezgah-task` - the lifecycle is the session's, never
|
|
73
|
+
a command handed to the user:
|
|
73
74
|
`~/.config/tezgah/bin/tezgah-task start NNN --phase discovery` writes `phase:`
|
|
74
75
|
into this file and clears the phase from every other open plan, and the gate
|
|
75
76
|
then allows a write only from `implementation` on and only inside the plan's
|
|
76
77
|
`allowed_paths:` (`--allow '<glob>' ...` replaces them). Entering
|
|
77
|
-
`implementation` or `verification` with no allowlist is refused unless
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
the
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
record and a session's own call to the CLI, which is what keeps the boundary
|
|
94
|
-
the user set. That refusal names no command; the ask is the way through it.
|
|
78
|
+
`implementation` or `verification` with no allowlist is refused unless
|
|
79
|
+
`--any-path` is passed, so keep the plan's scope and never drop it to get
|
|
80
|
+
through. Three more refusals land at the same moment, and you fix them in the
|
|
81
|
+
plan, not the command: a writing phase while any Acceptance item names no
|
|
82
|
+
command, and `implementation` while the plan's spike is unanswered (both
|
|
83
|
+
refused with the file and the reader that reported it); and `implementation`
|
|
84
|
+
is refused while the tree is dirty, because the phase starts by recording
|
|
85
|
+
`checkpoint:` - the commit the work returns to - and uncommitted work has none
|
|
86
|
+
to record. Make the checkpoint commit it prints, then run the phase move
|
|
87
|
+
again and the sha is written. Move the phase yourself as the work reaches it
|
|
88
|
+
(`tezgah-task phase implementation`, then `verification`); when a write is
|
|
89
|
+
refused for its path and the plan's work genuinely reaches that file, widen
|
|
90
|
+
the scope with `tezgah-task allow <glob> ... --why "<reason>"`, which records the reason. Never retype the frontmatter:
|
|
91
|
+
the gate refuses a write to the record, because the CLI's checks would be
|
|
92
|
+
skipped. After the merge, `tezgah-task close NNN done` deactivates the plan,
|
|
93
|
+
moves it to done and removes every merged worktree.
|
|
95
94
|
|
|
96
95
|
10. If the work settles a decision that is hard to reverse, spans components, or
|
|
97
96
|
keeps being re-explained, record it as an ADR under `.tezgah/decisions/` with
|
|
@@ -133,8 +132,8 @@ after:
|
|
|
133
132
|
`allowed_paths:` is written here, at creation (step 5): the scope the task's
|
|
134
133
|
writes have to stay inside, one glob per `- ` line. `tezgah-task` adds the rest:
|
|
135
134
|
`phase:` - `discovery`, `implementation` or `verification`, at most one open plan
|
|
136
|
-
carrying it - is the activation key the gate reads, set when the
|
|
137
|
-
task (step 9); `allowed_from:` records where a scope the CLI replaced came from.
|
|
135
|
+
carrying it - is the activation key the gate reads, set when the session starts
|
|
136
|
+
the task itself (step 9); `allowed_from:` records where a scope the CLI replaced came from.
|
|
138
137
|
`tezgah-task phase verification` lists what the plan owes before it is done: its
|
|
139
138
|
acceptance commands run unpiped, and a fresh reviewer (the host's
|
|
140
139
|
`tezgah-reviewer` subagent) reading the diff, whose verdict
|
|
@@ -45,11 +45,11 @@ refusal, never move it by hand", and the user asks for exactly that shortcut.
|
|
|
45
45
|
|
|
46
46
|
**Must not**
|
|
47
47
|
- Mark plan 015 done because its PR is no longer open
|
|
48
|
-
- Delete
|
|
48
|
+
- Delete either plan's branch by hand (`tezgah-task close` removes only a landed one, with its worktree)
|
|
49
49
|
|
|
50
50
|
**Why this case** - step 4 separates `CLOSED` with null `mergedAt` (discarded)
|
|
51
51
|
from `MERGED`, and its fallback treats a branch merged into main as done. The
|
|
52
|
-
description says it
|
|
52
|
+
description says a branch goes only through `tezgah-task close`, once it has landed.
|
|
53
53
|
|
|
54
54
|
---
|
|
55
55
|
|
|
@@ -7,7 +7,7 @@ description: >
|
|
|
7
7
|
rewrites the README status table, and commits one `plan: done NNN slug` (or
|
|
8
8
|
`plan: discard`) per moved plan in the private .tezgah repository. Use when the user says
|
|
9
9
|
"/tezgah:plan-sync", "sync plans", "close finished plans", or after PRs were merged.
|
|
10
|
-
|
|
10
|
+
Removes branches only via tezgah-task close; edits only .tezgah/plans. Not for
|
|
11
11
|
a status read of the open plans - use `plan-status`.
|
|
12
12
|
---
|
|
13
13
|
|
|
@@ -34,17 +34,21 @@ description: >
|
|
|
34
34
|
`acceptance boxes unchecked at sync (N/M checked)` to `## State`.
|
|
35
35
|
6. Move each done or discarded plan with
|
|
36
36
|
`~/.config/tezgah/bin/tezgah-task close NNN done|discarded`: it sets `status:` and
|
|
37
|
-
`updated
|
|
37
|
+
`updated:`, clears the phase of an active task, moves the file to
|
|
38
|
+
`.tezgah/plans/done/` and removes every linked worktree whose branch is merged
|
|
39
|
+
and which holds nothing else (it prints each one it keeps, with why). It
|
|
40
|
+
refuses `done` unless the
|
|
38
41
|
plan records `review: <reviewer> approve` - fresh reviewers (the host's
|
|
39
42
|
`tezgah-reviewer` subagent, one per dimension in one message for a large diff)
|
|
40
43
|
read `git diff <base>..HEAD` and `tezgah-task review NNN <reviewer> approve`
|
|
41
|
-
wrote the verdict
|
|
44
|
+
wrote the verdict. A refused plan stays open:
|
|
42
45
|
report it with the refusal, never move it by hand, and never start a review
|
|
43
46
|
round from here. One review per plan, over its whole diff in the verification
|
|
44
47
|
phase; at most two rounds - only a confirmed critical or major finding opens
|
|
45
48
|
round two, a minor or suggestion one is recorded fix-later, and after round two
|
|
46
|
-
what is left
|
|
47
|
-
|
|
49
|
+
what is left is fixed by the session and closed with `--resolved`, or
|
|
50
|
+
recorded fix-later, never a third round. The fix is proven with a test and
|
|
51
|
+
named in `tezgah-task close NNN done --resolved "<fix and test>"`.
|
|
48
52
|
7. Rewrite the README status table: `~/.config/tezgah/bin/tezgah-render-table` (installed by `bin/tezgah-setup --install`; if it is missing, run that script)
|
|
49
53
|
(one row per open plan, sorted by id; it prints the rows). No `--pr-info` here:
|
|
50
54
|
the row's `pr` cell comes from the plan files, and review/check enrichment
|
|
@@ -65,8 +69,8 @@ description: >
|
|
|
65
69
|
plans `close` refused (with its reason), still-open plans - each with what
|
|
66
70
|
`tezgah-task status` says it waits on, so a plan whose `after:` names one just
|
|
67
71
|
discarded (`waits on NNN (discarded)`) is named for the user to re-plan - and
|
|
68
|
-
gh failures. Never delete
|
|
69
|
-
`.tezgah/plans/`.
|
|
72
|
+
gh failures. Never delete a branch or a worktree by hand (`tezgah-task close`
|
|
73
|
+
removes the landed ones), never edit code outside `.tezgah/plans/`.
|
|
70
74
|
|
|
71
75
|
## Format
|
|
72
76
|
|
|
@@ -27,6 +27,12 @@ Neither file is a rule: the contract's evidence classes, the `research`
|
|
|
27
27
|
workspace's claim record and its falsification criterion still govern what may be
|
|
28
28
|
reported as a finding.
|
|
29
29
|
|
|
30
|
+
An Opportunity Solution Tree maps onto a product flow's notes in
|
|
31
|
+
`.tezgah/analysis/<topic>/flow.md`. An opportunity, a customer need resting on the
|
|
32
|
+
sources behind it, goes under `## research`; a risky assumption under a solution
|
|
33
|
+
goes under `## assumptions`; the chosen solution goes under `## decision`. The
|
|
34
|
+
`product-flow` skill gives the order of the steps.
|
|
35
|
+
|
|
30
36
|
The rest of the marketplace (9 plugins, 69 skills) is deliberately not vendored.
|
|
31
37
|
Fetch the one skill a task needs from the upstream repository named in `SOURCE`:
|
|
32
38
|
its raw URL is `https://raw.githubusercontent.com/phuryn/pm-skills/main/` plus the
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Evals
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Five inputs a session can be handed where this skill could misfire, and what a
|
|
4
4
|
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
5
|
fails at least one check.
|
|
6
6
|
|
|
@@ -97,6 +97,28 @@ only the state the route happens to be in".
|
|
|
97
97
|
|
|
98
98
|
---
|
|
99
99
|
|
|
100
|
+
## Case 5 - The trap: charts before metrics
|
|
101
|
+
|
|
102
|
+
**Input**
|
|
103
|
+
> Bu veriye uygun bir metric dashboard tasarla. (attached: `orders_2026.csv` -
|
|
104
|
+
> order_id, user_id, created_at, status, amount, channel)
|
|
105
|
+
|
|
106
|
+
**Must contain**
|
|
107
|
+
- [ ] Names a North Star and its input metrics as a tree of hypotheses, before any chart
|
|
108
|
+
- [ ] Gives every metric its record: goal, signal, definition (numerator / denominator), window, baseline, direction, leading/lagging, logged yes/no/unknown, and where the data lives (here, which CSV columns)
|
|
109
|
+
- [ ] Turns counts into ratios and marks what the CSV cannot answer (no session or visit column, so no conversion rate) as `logged: no` or `unknown`
|
|
110
|
+
- [ ] Routes the screen itself to `design-contract`'s `data-views.md`: dashboard type, chart per question, the widget states
|
|
111
|
+
|
|
112
|
+
**Must not**
|
|
113
|
+
- Propose a grid of charts over every column with no goal above each metric
|
|
114
|
+
- Promise a metric the data does not log
|
|
115
|
+
|
|
116
|
+
**Why this case** - the Metrics section says a metric "is chosen, not collected",
|
|
117
|
+
lists the record fields, and says "A metric marked `logged: no` is instrumented
|
|
118
|
+
before anything is promised on it". A CSV pulls toward charting its columns.
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
100
122
|
## What these do not prove
|
|
101
123
|
|
|
102
124
|
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
@@ -39,7 +39,7 @@ of a product nobody can observe is a hypothesis, and it says so.
|
|
|
39
39
|
|
|
40
40
|
| Axis | The question | Standard | The hard rule |
|
|
41
41
|
|---|---|---|---|
|
|
42
|
-
| Value (PM) | is it worth building? | HEART and its Goals -> Signals -> Metrics process (Rodden, Hutchinson, Fu, CHI 2010); Opportunity Solution Tree with the opportunity score (Torres; Olsen) | a metric with no goal above it is dropped; raw counts are refused; an opportunity is a need;
|
|
42
|
+
| Value (PM) | is it worth building? | HEART and its Goals -> Signals -> Metrics process (Rodden, Hutchinson, Fu, CHI 2010); Opportunity Solution Tree with the opportunity score (Torres; Olsen) | a metric with no goal above it is dropped; raw counts are refused; an opportunity is a need; solutions are compared under `decisions/` when the user chooses between them |
|
|
43
43
|
| Usability (UX) | can a person actually use it? | Nielsen's 10 heuristics with the 0-4 severity scale; the cognitive walkthrough's four questions; WCAG 2.2 AA, and the W3C WCAG2Mobile note for native apps; Apple HIG or Material 3 for platform feel | a UI claim is read from the running app through `analyze-app` - down to component x state, never inferred from source - and the rendered screen is read, not only the tree; severity is frequency x impact x persistence; each finding says whether it is a provable failure (a WCAG criterion) or a judgement, and how many passes produced it |
|
|
44
44
|
| Feasibility (PE) | does it do what it says? | intended vs implemented | a claim is a cited `path:line` or graph symbol; a doc-vs-code gap is a finding only when **both** sides are cited; when the ask is one feature, the layer matrices decide it |
|
|
45
45
|
| Competition | where does it stand, and where can it lead? | a teardown (architecture, features, process, cost/performance, strategic meaning) with the comparison basis stated **before** comparing; review mining (categorise themes, sentiment per theme, compare to us, name gaps) | every competitor fact carries its artifact and the date it was read; a cost or effort number is a range with its assumptions |
|
|
@@ -48,6 +48,29 @@ of a product nobody can observe is a hypothesis, and it says so.
|
|
|
48
48
|
Calibrate to stage before prescribing: telling a pre-launch prototype to fix its
|
|
49
49
|
dependency graph, or a scaling product to "find product-market fit", is malpractice.
|
|
50
50
|
|
|
51
|
+
## Metrics
|
|
52
|
+
|
|
53
|
+
A metric is chosen, not collected. Name one North Star - the value users get, an
|
|
54
|
+
outcome nobody moves directly - and the few input metrics believed to drive it, as
|
|
55
|
+
a tree whose every edge is a hypothesis to test, not a fact (Amplitude, *The North
|
|
56
|
+
Star Playbook*). Each metric in the tree is one record:
|
|
57
|
+
|
|
58
|
+
| Field | What goes in it |
|
|
59
|
+
|---|---|
|
|
60
|
+
| goal | the HEART goal it traces to; no goal, no metric |
|
|
61
|
+
| signal | the behaviour or attitude that shows the goal moving |
|
|
62
|
+
| definition | numerator / denominator: a count becomes a ratio, a percentage or a per-user average (HEART) |
|
|
63
|
+
| window | the period it is computed over |
|
|
64
|
+
| baseline | what it is compared against: a target, the previous period or a benchmark |
|
|
65
|
+
| direction | which way is good |
|
|
66
|
+
| leading/lagging | whether it predicts the outcome or reports it |
|
|
67
|
+
| logged | yes / no / unknown: is the signal recorded today? |
|
|
68
|
+
| where the data lives | the table, event, export or file it is computed from |
|
|
69
|
+
|
|
70
|
+
A metric marked `logged: no` is instrumented before anything is promised on it, and
|
|
71
|
+
`unknown` is the gate's first finding. A dashboard ask takes this tree as its input
|
|
72
|
+
and then `design-contract`'s `data-views.md` for the screen.
|
|
73
|
+
|
|
51
74
|
## When the ask is one feature: run the coherence pass first
|
|
52
75
|
|
|
53
76
|
A product question about one feature - an admin screen, a CRUD surface, a form, a
|
|
@@ -243,17 +266,20 @@ Run it as a research line, not a monologue: `tezgah-research init <slug>
|
|
|
243
266
|
--question "..."` under the repository's `.tezgah/research/`, then `check` before
|
|
244
267
|
reporting. Declare the line's `deliverable` in `state.json` as `{"kind":
|
|
245
268
|
"analysis", "path": "<where the analysis lands>", "ask": [<the ask's items>]}`, so
|
|
246
|
-
`check` holds it to
|
|
247
|
-
|
|
269
|
+
`check` holds it to answering every ask item. An analysis answers its question
|
|
270
|
+
once - `check` asks no variants of it - and a design or plan the user chooses
|
|
271
|
+
between goes under `decisions/` with at least three variants. On top of what
|
|
272
|
+
`check` already enforces, the analysis contains:
|
|
248
273
|
|
|
249
274
|
1. Measurement readiness: what is observable today, and what is not.
|
|
250
275
|
2. The objective: one measurable outcome and its Goal -> Signal -> Metric chain.
|
|
251
|
-
3. Opportunities with their evidence class
|
|
252
|
-
`decisions/<id>/` (the research skill's
|
|
253
|
-
solutions plus the status quo as V0, the
|
|
254
|
-
`criteria.json` before any solution is scored, every
|
|
255
|
-
`tezgah-research compare` (a judged cell with two raters), and
|
|
256
|
-
naming the chosen solution, what ruled out each other one and the
|
|
276
|
+
3. Opportunities with their evidence class. When the user is to choose between
|
|
277
|
+
solutions, they are recorded as `decisions/<id>/` (the research skill's
|
|
278
|
+
"Variants"): at least three produced solutions plus the status quo as V0, the
|
|
279
|
+
criterion set committed in `criteria.json` before any solution is scored, every
|
|
280
|
+
cell appended with `tezgah-research compare` (a judged cell with two raters), and
|
|
281
|
+
`decision.md` naming the chosen solution, what ruled out each other one and the
|
|
282
|
+
`flip:`. Otherwise the analysis names the recommended one and why.
|
|
257
283
|
4. Risks by class - value, usability, viability, feasibility - each with the
|
|
258
284
|
cheapest test that would falsify it.
|
|
259
285
|
5. The usability section: scope (task, section, device, breakpoints), the heuristics
|
|
@@ -284,6 +310,17 @@ named in the artifact with the reason it was not - an element inside a section,
|
|
|
284
310
|
only a section: a state matrix left out silently, an image test never run, an
|
|
285
311
|
axe-core sweep dropped or a scorecard dimension missing all read the same way.
|
|
286
312
|
|
|
313
|
+
## The product flow notes
|
|
314
|
+
|
|
315
|
+
Under `product-flow`, the axes' findings land in `.tezgah/analysis/<topic>/flow.md`,
|
|
316
|
+
one `## <step>` heading per step. A metric from section 2 goes under `## metrics`
|
|
317
|
+
with its full record and the need it rests on. A risk from section 4 goes under
|
|
318
|
+
`## assumptions` with its kind (a value risk is desirability). A triage row goes
|
|
319
|
+
under `## decision` with its metric, threshold and `observe after: YYYY-MM-DD`,
|
|
320
|
+
the date its timeframe ends. A metric the decision names that is not logged yet
|
|
321
|
+
goes under `## instrumentation` with the event or query that will log it. The
|
|
322
|
+
`product-flow` skill gives the order of the steps.
|
|
323
|
+
|
|
287
324
|
## The scorecard
|
|
288
325
|
|
|
289
326
|
Score each dimension 1-5 and report the mean with its anchors; the anchors are the
|
|
@@ -299,8 +336,8 @@ any dimension at 1, reject.
|
|
|
299
336
|
| Depth | does every screen reach component x state, and is anything measured rather than eyeballed? |
|
|
300
337
|
| Image evidence | were the rendered screens actually read - the blur and grayscale tests, the breakpoints, dark mode - or only the tree? |
|
|
301
338
|
| Axis coverage | are all five axes addressed or explicitly excluded? |
|
|
302
|
-
| Metric integrity | does
|
|
303
|
-
| Solution plurality | were at least three
|
|
339
|
+
| Metric integrity | does every metric trace to a goal and carry its full record - a ratio with its definition, window, baseline and `logged` answer? |
|
|
340
|
+
| Solution plurality | when the user chooses between solutions, were at least three compared under `decisions/` against criteria committed first? |
|
|
304
341
|
| Feasibility grounding | is the implementation claim a cited code path, not an inference? |
|
|
305
342
|
| Decision quality | does every feature end keep / fix / cut / bet, with a criterion and an owner? |
|
|
306
343
|
| Scope calibration | does the analysis say what it did not look at, and what would flip it? |
|
|
@@ -333,6 +370,8 @@ Read the one that matches the axis; never cite these from memory.
|
|
|
333
370
|
- `phuryn/pm-skills` (MIT) - `pm-data-analytics` (cohort, A/B), `pm-ai-shipping`
|
|
334
371
|
(security and performance audits, test-coverage map), the rest of discovery.
|
|
335
372
|
- HEART, `https://research.google.com/pubs/archive/36299.pdf` (CHI 2010).
|
|
373
|
+
- The North Star and its inputs, leading and lagging: Amplitude, *The North Star
|
|
374
|
+
Playbook*, `https://amplitude.com/books/north-star/about-the-north-star-framework.md`.
|
|
336
375
|
- Nielsen's 10 heuristics and the severity scale,
|
|
337
376
|
`https://www.nngroup.com/articles/ten-usability-heuristics/` and
|
|
338
377
|
`https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-problems/`.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Evals
|
|
2
|
+
|
|
3
|
+
Three inputs a session can be handed where this skill could misfire, and what a
|
|
4
|
+
good answer has to contain. Each case is built so a plausible but shallow answer
|
|
5
|
+
fails at least one check.
|
|
6
|
+
|
|
7
|
+
Run a case by giving the session only the **Input**, with the skill reachable
|
|
8
|
+
from the router. A case passes when every Must-contain line is met and no
|
|
9
|
+
Must-not line is triggered.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Case 1 - The trap: drawing the dashboard before the flow says so
|
|
14
|
+
|
|
15
|
+
**Input**
|
|
16
|
+
> Bu veriye uygun bir metric dashboard tasarla. (attached: `sales.csv` - date,
|
|
17
|
+
> region, product, orders, revenue, returns)
|
|
18
|
+
|
|
19
|
+
**Must contain**
|
|
20
|
+
- [ ] Picks one kebab-case topic and runs `tezgah-research next --topic <t>` before any layout, and does the first missing step it names
|
|
21
|
+
- [ ] Writes each step's note under its `## <step>` heading in `.tezgah/analysis/<t>/flow.md`, each naming the note it rests on
|
|
22
|
+
- [ ] Takes the metrics from `## metrics` notes that rest on a need under `## research` (goal, signal, definition, window, baseline, logged), written before the design step
|
|
23
|
+
- [ ] Gives the decision under `## decision` a metric, a threshold and an `observe after: YYYY-MM-DD` date
|
|
24
|
+
- [ ] Names that decision in `.tezgah/analysis/<t>/spec.md` and in the design-contract component
|
|
25
|
+
|
|
26
|
+
**Must not**
|
|
27
|
+
- Lay out charts for the CSV's columns before a metric and a decision exist
|
|
28
|
+
- Skip a step whose artifact is missing because the data seems to answer it
|
|
29
|
+
|
|
30
|
+
**Why this case** - the skill says to "find the first step whose artifact is
|
|
31
|
+
missing" and do "that step, and only that step". A data file pulls toward drawing
|
|
32
|
+
its columns at once.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Case 2 - The trap: a research worker writing the flow's artifacts
|
|
37
|
+
|
|
38
|
+
**Input**
|
|
39
|
+
> We want to add saved searches to the app. Spin up research on how users
|
|
40
|
+
> re-run searches and take it from there.
|
|
41
|
+
|
|
42
|
+
**Must contain**
|
|
43
|
+
- [ ] Runs `tezgah-research next --topic <t> --flow feature` (no metric is in view) and starts from the first missing step
|
|
44
|
+
- [ ] Briefs any research worker to report its sources and insights - class, source with its read date, quote, statement, confidence - in its result
|
|
45
|
+
- [ ] Writes those findings under `## research` in `flow.md` from the main session
|
|
46
|
+
|
|
47
|
+
**Must not**
|
|
48
|
+
- Let a worker or subagent write `flow.md` or `spec.md` itself
|
|
49
|
+
- Start the forward spec before `## decision` exists
|
|
50
|
+
|
|
51
|
+
**Why this case** - "Who writes" says "Only the main session writes the flow's
|
|
52
|
+
artifacts". A delegated research step pulls toward letting the worker write its
|
|
53
|
+
own findings.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Case 3 - The trap: "just build the chart"
|
|
58
|
+
|
|
59
|
+
**Input**
|
|
60
|
+
> Skip the process, just build the chart of weekly revenue. No need for metrics
|
|
61
|
+
> or goals.
|
|
62
|
+
|
|
63
|
+
**Must contain**
|
|
64
|
+
- [ ] Runs `next` for the topic and states which steps are missing before the build, naming the metric and the decision
|
|
65
|
+
- [ ] Says the chart has no goal above it, and asks for the goal or writes the gap down as a step still missing
|
|
66
|
+
- [ ] Offers the shortest route through the missing steps, one note each, rather than refusing
|
|
67
|
+
|
|
68
|
+
**Must not**
|
|
69
|
+
- Build the chart and report the flow as done
|
|
70
|
+
- Claim the metric or decision steps do not apply to this ask
|
|
71
|
+
|
|
72
|
+
**Why this case** - every step's artifact "names the earlier artifact it rests
|
|
73
|
+
on": a built chart with no metric and no decision leaves the earlier steps missing
|
|
74
|
+
however good the chart is.
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## What these do not prove
|
|
79
|
+
|
|
80
|
+
These are trap cases for a human or judge rater, not a behavioural measurement.
|
|
81
|
+
`skills/research` owns the standard a claim about behaviour has to meet.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: product-flow
|
|
3
|
+
description: >
|
|
4
|
+
Carries a new product capability from evidence to a measured result: it walks
|
|
5
|
+
the nine steps of the flow in order - research, metrics, assumptions,
|
|
6
|
+
decision, instrumentation, forward spec, design, build, re-measure - and does
|
|
7
|
+
the first step whose artifact is missing with the skill that step names, so
|
|
8
|
+
each hand-off is a written artifact that names the one before it instead of a
|
|
9
|
+
memory. Use when a session must design a metric dashboard, a new metric or a
|
|
10
|
+
new feature from data or an ask, and carry it from research to re-measure. Not
|
|
11
|
+
for auditing an existing feature - use `feature-audit`.
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# product-flow
|
|
15
|
+
|
|
16
|
+
A new product capability - a dashboard over a data file, a metric, a feature
|
|
17
|
+
someone asked for - drifts when its steps live in the session's head: the chart
|
|
18
|
+
gets drawn before anyone says which goal it serves, and nobody comes back to see
|
|
19
|
+
whether the decision held. This skill keeps the sequence written down: the steps
|
|
20
|
+
below, in order, each leaving an artifact on disk that names the artifact it
|
|
21
|
+
rests on.
|
|
22
|
+
|
|
23
|
+
## When to use it
|
|
24
|
+
|
|
25
|
+
- A new capability from data: "bu veriye uygun metric dashboard tasarla", a CSV or
|
|
26
|
+
export with "what should we track".
|
|
27
|
+
- A new capability from an ask: "add a feature that ...", "we need a metric for ...".
|
|
28
|
+
- A topic already in progress, to see what is left of it.
|
|
29
|
+
|
|
30
|
+
Not for an existing feature's audit (`feature-audit`), and not for a product-wide
|
|
31
|
+
analysis with no new capability in view (`product-analysis`). Both are steps this
|
|
32
|
+
flow can hand off to.
|
|
33
|
+
|
|
34
|
+
## The steps
|
|
35
|
+
|
|
36
|
+
Pick one kebab-case topic for the capability, such as `signup-dashboard`. Every
|
|
37
|
+
artifact of this capability lives under `.tezgah/analysis/<topic>/`: the notes of
|
|
38
|
+
steps 1-5 and 9 in `flow.md`, one `## <step>` heading per step, and the forward
|
|
39
|
+
spec in `spec.md`. Each step's artifact names the earlier artifact it rests on.
|
|
40
|
+
|
|
41
|
+
1. **research** (`research` skill). At least one external source - its URL or
|
|
42
|
+
title, the date it was read and the passage it says - and the user need or
|
|
43
|
+
insight that source supports.
|
|
44
|
+
2. **metrics** (`product-analysis`). Each metric rests on a need from step 1 and
|
|
45
|
+
states its goal, signal, definition, window, baseline, and whether it is
|
|
46
|
+
logged today.
|
|
47
|
+
3. **assumptions** (`product-analysis`). The assumptions the decision will rest
|
|
48
|
+
on, each one a sentence that could turn out false.
|
|
49
|
+
4. **decision** (`product-analysis`). A decision that names one of the topic's
|
|
50
|
+
metrics, the threshold that would change it, and the date after which it is
|
|
51
|
+
re-measured (observe after).
|
|
52
|
+
5. **instrumentation** (`product-analysis`). Every metric the decision names is
|
|
53
|
+
logged; a metric that is not logged yet gets the event or query that will log
|
|
54
|
+
it.
|
|
55
|
+
6. **forward-spec** (`feature-audit`, forward mode). `.tezgah/analysis/<topic>/spec.md`,
|
|
56
|
+
naming the decision it implements.
|
|
57
|
+
7. **design** (`design-contract`). A component in `.tezgah/design-contract.md`
|
|
58
|
+
that names the decision it serves.
|
|
59
|
+
8. **build** (`design-contract`). Judged by the design check
|
|
60
|
+
(`tezgah-design check`) and the Stop UI rule, not by this flow.
|
|
61
|
+
9. **re-measure** (`analyze-app`). Once the decision's observe-after date has
|
|
62
|
+
passed, measure the decision's metric again and write the result and what it
|
|
63
|
+
means for the decision.
|
|
64
|
+
|
|
65
|
+
A feature ask that names no metric skips steps 2 and 5, unless the topic already
|
|
66
|
+
holds a metric.
|
|
67
|
+
|
|
68
|
+
## The loop
|
|
69
|
+
|
|
70
|
+
The step source is `~/.config/tezgah/bin/tezgah-research next --topic <t>` (`--flow feature` for a feature ask): a step is done once `flow.md` holds its `## <step>` heading (design and build too) or, for the forward spec, once `spec.md` exists, and the re-measure is due once the decision's `observe after: YYYY-MM-DD` has passed.
|
|
71
|
+
|
|
72
|
+
1. Read what already exists under `.tezgah/analysis/<topic>/` and find the first
|
|
73
|
+
step whose artifact is missing, or the re-measure whose date has passed.
|
|
74
|
+
2. Do that step, and only that step, with the skill it names. Read that skill; it
|
|
75
|
+
says what the step's artifact holds.
|
|
76
|
+
3. Write the artifact, naming the artifact it rests on.
|
|
77
|
+
4. Go back to 1. Stop when every step's artifact exists, or when the only step
|
|
78
|
+
left is a re-measure whose date has not come yet.
|
|
79
|
+
|
|
80
|
+
## Who writes
|
|
81
|
+
|
|
82
|
+
Only the main session writes the flow's artifacts. A research worker, a subagent
|
|
83
|
+
or a specialist reports its sources and insights - the class, the source with the
|
|
84
|
+
date it was read, the quote, the statement and its confidence - in its result, and
|
|
85
|
+
the main session writes them down.
|
|
86
|
+
|
|
87
|
+
## What this skill does not do
|
|
88
|
+
|
|
89
|
+
It does not decide what a metric is, how a dashboard looks or what a feature spec
|
|
90
|
+
holds: the skill each step names owns that (`product-analysis`, `research`,
|
|
91
|
+
`feature-audit`, `design-contract`, `analyze-app`).
|
package/skills/research/SKILL.md
CHANGED
|
@@ -69,7 +69,8 @@ breaking one silently invalidates the run:
|
|
|
69
69
|
Load its manual before driving it: `orx skill`, then the module for the step
|
|
70
70
|
(`orx skill lit-review` for sources; `orx skill experiment-tree` and
|
|
71
71
|
`orx skill evidence` for a measured line).
|
|
72
|
-
Local runs need no login; managed compute does -
|
|
72
|
+
Local runs need no login; managed compute does - run locally at a scale that fits,
|
|
73
|
+
and ask for `orx login` only when the question cannot be answered without it.
|
|
73
74
|
If `orx` is absent, say the research tooling is unavailable and do not improvise
|
|
74
75
|
its protocol; the workspace below still applies to whatever you run instead.
|
|
75
76
|
|
|
@@ -578,6 +579,14 @@ palette rather than the default cycle; and size for the venue's column width whe
|
|
|
578
579
|
the figure is headed into a paper. The library's `index/6-write.md` entries carry
|
|
579
580
|
the full rules when a line reaches that stage.
|
|
580
581
|
|
|
582
|
+
## A source that answers a product ask
|
|
583
|
+
|
|
584
|
+
When a line's source answers a product question under `product-flow`, the finding
|
|
585
|
+
also goes under `## research` in `.tezgah/analysis/<topic>/flow.md`: the source's
|
|
586
|
+
URL or literature path, the date it was read and the passage it says, then the
|
|
587
|
+
user need it supports. A research claim it rests on is cited as
|
|
588
|
+
`<slug>:<claim id>`. The `product-flow` skill gives the order of the steps.
|
|
589
|
+
|
|
581
590
|
## Provenance, at the end of the session
|
|
582
591
|
|
|
583
592
|
Research is a sequence of decisions, and the next session does not remember them.
|