task-pipeline-skill 1.37.0 → 1.39.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +202 -0
- package/SKILL-CARD.md +1 -1
- package/cursor/rules/task-pipeline.mdc +11 -5
- package/evals/routing/RESULTS.md +79 -0
- package/evals/routing/render.py +130 -0
- package/evals/task-pipeline.evals.json +173 -0
- package/package.json +3 -2
- package/plugins/task-pipeline/.claude-plugin/plugin.json +2 -2
- package/plugins/task-pipeline/commands/task-pipeline.md +167 -56
- package/plugins/task-pipeline/skills/task-pipeline/SKILL.md +5 -5
- package/plugins/task-pipeline/skills/task-pipeline/references/companion-skills.md +12 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/gates.md +18 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/knowledge-sources.md +10 -5
- package/plugins/task-pipeline/skills/task-pipeline/references/learned.md +3 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/retrospective.md +2 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/review.md +56 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/stages.md +14 -3
- package/plugins/task-pipeline/skills/task-pipeline/templates/routing-rule.md +22 -10
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,207 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v1.39.0 — the skill could not be reached by the word "audit"
|
|
4
|
+
|
|
5
|
+
`references/audit.md` has said since v0.1.0 that an audit may be **the whole task** —
|
|
6
|
+
stages 3–5 producing findings and fixes instead of a feature. No routing surface named
|
|
7
|
+
it. Every trigger noun was build-shaped, and the exclusion clause read *"Not for:
|
|
8
|
+
answering a question, explaining or reading code"* — which is the opening move of an
|
|
9
|
+
audit, a bug hunt, a production check and a PR review alike.
|
|
10
|
+
|
|
11
|
+
### Measured, not supposed
|
|
12
|
+
|
|
13
|
+
Ten routing queries, one fresh agent each, holding nothing but the competing skill
|
|
14
|
+
descriptions and one user sentence (`evals/routing/render.py`, results in
|
|
15
|
+
`evals/routing/RESULTS.md`). **7 / 10 before.** The three misses were the three the board
|
|
16
|
+
row predicted, and none of them was a failure to find a match — each agent **quoted this
|
|
17
|
+
skill's own exclusion clause back as the reason it refused**:
|
|
18
|
+
|
|
19
|
+
- «проверь, нет ли ошибок в обработчике вебхуков» → `none`
|
|
20
|
+
- «проверь, всё ли живо в проде после вчерашнего релиза» → `none`
|
|
21
|
+
- «посмотри PR #24 и скажи, что там не так» → `none`
|
|
22
|
+
|
|
23
|
+
A fourth result only the reasoning shows: «сделай аудит модуля оплат» *did* route, and
|
|
24
|
+
justified it by stretching the build verb `hardening`. A right answer resting on a
|
|
25
|
+
stretch is one rewording from a miss.
|
|
26
|
+
|
|
27
|
+
### The boundary is what a request ends in
|
|
28
|
+
|
|
29
|
+
Not whether it reads. An answer stops in the conversation; a change **or a finding**
|
|
30
|
+
lands in the tree, and the pipeline is what carries REQ rows, board rows and fixes
|
|
31
|
+
there. The description, the portable routing rule and the Cursor rule now all say so,
|
|
32
|
+
and `reading` is gone from the exclusions — guarded, so it cannot come back quietly.
|
|
33
|
+
|
|
34
|
+
### Two things the harvest found before the first grill question
|
|
35
|
+
|
|
36
|
+
- **`перевести` was locked into the verb list by the v1.9.0 design and never shipped.**
|
|
37
|
+
It existed in exactly one place in this repository: the design that locked it.
|
|
38
|
+
- **`REQ-003` was accepted `verified` anyway.** The evidence recorded was the clause's
|
|
39
|
+
*shape* and its character count — neither of which can see a missing member of the
|
|
40
|
+
list the REQ locked. An L1→L2 absence that passed an L5 check. A guard now reads the
|
|
41
|
+
locked list out of that design and compares it to the shipped surface, so the next
|
|
42
|
+
dropped verb is a failure rather than a week.
|
|
43
|
+
|
|
44
|
+
### The reader earned its standing instruction
|
|
45
|
+
|
|
46
|
+
R-005 exists because an author's probes only exercise the shapes the author already
|
|
47
|
+
thought of. Dispatched on the nine new guards, an independent reader defeated them
|
|
48
|
+
**fifteen ways**, each verified by planting the text and watching the validator still
|
|
49
|
+
print `PASS`. The three worth naming:
|
|
50
|
+
|
|
51
|
+
- **A presence test over a whole file proves a word exists, not that the rule says it.**
|
|
52
|
+
The reader deleted the entire boundary clause from the Cursor rule, put the old
|
|
53
|
+
exclusion back, added one unrelated sentence elsewhere containing the same four class
|
|
54
|
+
names — and the guard passed. Both cross-surface checks are now scoped to the
|
|
55
|
+
`## Routing` section.
|
|
56
|
+
- **A `should_not_trigger` control counted as coverage.** The eval-coverage guard joined
|
|
57
|
+
every query regardless of category, so deleting all four findings evals and mentioning
|
|
58
|
+
the words in one negative control certified *named and untested* — the exact state it
|
|
59
|
+
cites `B-046` for — as green. It now reads `should_trigger` queries only.
|
|
60
|
+
- **The anti-dormancy sentinel was itself one synonym from dormant**, and the
|
|
61
|
+
locked-verb guard read a regex out of a document this repo forbids maintaining, so
|
|
62
|
+
relabelling one heading in a superseded design would have silenced the check that
|
|
63
|
+
exists because a verb was silently dropped. Both now fail loudly instead.
|
|
64
|
+
|
|
65
|
+
Nine of the fifteen were in checks written that same hour. The tenth fail site had no
|
|
66
|
+
probe at all — ten branches, nine tests — which is the invariant the repository states
|
|
67
|
+
about itself and did not keep.
|
|
68
|
+
|
|
69
|
+
### Also
|
|
70
|
+
|
|
71
|
+
- **False-positive controls, because widening a vocabulary can steal work.** Three
|
|
72
|
+
competitors already claim the word *audit* — `seo-aeo-audit`, `ux-audit`,
|
|
73
|
+
`make-skill`. All three are eval cases now, and all three still won their query.
|
|
74
|
+
- **R-003 sweep** turned up the same word-map ceiling that produced R-006, in the
|
|
75
|
+
redaction-rule count: past ten it compared against the digit alone. It now accepts
|
|
76
|
+
either form and names both when it fails.
|
|
77
|
+
- Guards: 218 → **233**; eval cases 21 → **28**; description 956 → 1004 of 1024, paid for
|
|
78
|
+
by cutting mechanism prose that could not affect routing.
|
|
79
|
+
|
|
80
|
+
## v1.38.0 — the wall came down, and a green started meaning something
|
|
81
|
+
|
|
82
|
+
An audit of this skill measured nineteen problems and asked one question of all of them:
|
|
83
|
+
*why can an agent that follows every rule here still report work it did not do?* The
|
|
84
|
+
answers were not in the doctrine. They were in how much of it there is, how it is
|
|
85
|
+
delivered, and what nobody checks.
|
|
86
|
+
|
|
87
|
+
### The wall
|
|
88
|
+
|
|
89
|
+
`commands/task-pipeline.md` — the first text an agent reads when the skill fires — was
|
|
90
|
+
**one paragraph of 1281 words carrying 25 obligations, with a 4115-character line**.
|
|
91
|
+
Twenty-five duties in one breath: which of them an agent obeys is a function of position,
|
|
92
|
+
not importance.
|
|
93
|
+
|
|
94
|
+
Same doctrine, eight headed sections, nothing removed:
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
paragraphs 5 -> 34
|
|
98
|
+
longest paragraph 1281 words -> 295
|
|
99
|
+
longest line 4115 chars -> 261
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
### The trigger surface said nothing about two of the three modes
|
|
103
|
+
|
|
104
|
+
`setup` and `checkup` appeared in **no browsable surface** — not the skill description,
|
|
105
|
+
not the command description, neither manifest. `checkup` exists specifically to be run
|
|
106
|
+
when nothing else is; the only way to learn it existed was to open the file you open by
|
|
107
|
+
running the thing it replaces.
|
|
108
|
+
|
|
109
|
+
The description was also at **1015 of its 1024-character ceiling**, with ~40% spent on
|
|
110
|
+
mechanism prose that cannot affect routing. Cut, both modes named, and it now sits at
|
|
111
|
+
956 with room to grow. Both manifests described the product as it stood at v1.30 — seven
|
|
112
|
+
releases of capability absent from the only text a marketplace shows.
|
|
113
|
+
|
|
114
|
+
**The plan proposed separate command files for the two modes and the doctrine refused
|
|
115
|
+
it.** `exposure.md` says *a mode of the command, not a new command, because a second
|
|
116
|
+
command costs every surface a command touches*. The doctrine is older than the plan and
|
|
117
|
+
it won.
|
|
118
|
+
|
|
119
|
+
### `test/probe.py` — R-001's retirement condition, three years of releases late
|
|
120
|
+
|
|
121
|
+
The standing instruction R-001 has said since 2026-08-03: *prove the plant landed in the
|
|
122
|
+
text the check actually parses.* Its retirement condition, written at birth, was **"a
|
|
123
|
+
probe harness exists that asserts the plant changed the parsed text"**. It was never
|
|
124
|
+
built, and three probes failed in a single day for want of it.
|
|
125
|
+
|
|
126
|
+
Three assertions per plant, and the third is the one hand-rolled probes keep missing:
|
|
127
|
+
|
|
128
|
+
1. the substitution **landed** — `replace` matching nothing raises nothing;
|
|
129
|
+
2. the exit code is **non-zero** — never a `FAIL` line on stdout;
|
|
130
|
+
3. **the guard that fired is the guard under test**, named up front rather than
|
|
131
|
+
recognised afterwards. A plant that trips some *other* check has proved that other
|
|
132
|
+
check works.
|
|
133
|
+
|
|
134
|
+
The harness self-tests its own failure branches (`npm run test:probe`), because a
|
|
135
|
+
harness whose failure path has never executed is exactly what it exists to stop. **On its
|
|
136
|
+
first use it caught two of this release's own guards being too loose** — one accepted any
|
|
137
|
+
three reader states rather than requiring the load-bearing one, and one matched
|
|
138
|
+
`none found` against a coincidental sentence in another paragraph.
|
|
139
|
+
|
|
140
|
+
### The independent reader is now dispatched by a stage, and read by its output
|
|
141
|
+
|
|
142
|
+
R-005 requires an independent reader on any change that adds or widens a check. Four pull
|
|
143
|
+
requests of almost nothing but check work were opened in one day; the review app reported
|
|
144
|
+
**`skipping`** on every one, and twenty-two guards merged on author probes alone. Nothing
|
|
145
|
+
was violated. Nothing read the reviewer's output either.
|
|
146
|
+
|
|
147
|
+
Stage 7 now dispatches the reader — a subagent it can watch, a bot whose **verdict** it
|
|
148
|
+
then reads, or a person — and records exactly one of three states beside the gate:
|
|
149
|
+
|
|
150
|
+
```
|
|
151
|
+
reader: 6 findings, 4 confirmed
|
|
152
|
+
reader: none found
|
|
153
|
+
reader: NO READER — <why>
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
The third is printed, never omitted. *A reader was requested* and *a reader reported* are
|
|
157
|
+
different facts that look identical in a transcript.
|
|
158
|
+
|
|
159
|
+
### `learned.md` rule 22 — an operation that changed nothing reports like one that changed everything
|
|
160
|
+
|
|
161
|
+
Four incidents in two programmes, each invisible until something downstream failed: an
|
|
162
|
+
import that never landed, a doctrine phrase worded differently, a `gh` call refused behind
|
|
163
|
+
`>/dev/null`, and a test piped to `head` so `$?` belonged to `head`. **Assert the effect,
|
|
164
|
+
not the call.**
|
|
165
|
+
|
|
166
|
+
### The retro's uncapped narrative stopped being read in full
|
|
167
|
+
|
|
168
|
+
The doctrine said stage 0 reads three sections of `docs/superpowers/retro.md` in full
|
|
169
|
+
because *"all three are bounded by construction, which is why the cap is not
|
|
170
|
+
negotiable."* Measured, that claim was false:
|
|
171
|
+
|
|
172
|
+
```
|
|
173
|
+
Standing instructions ~ 1 394 tok capped at ten rows
|
|
174
|
+
Run stamps ~ 1 842 tok one line per run
|
|
175
|
+
Recent log ~10 937 tok narrative — capped by nothing ← 74% of the file
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
An uncapped section inside a source that **binds** the run is precisely what makes the
|
|
179
|
+
capped part get skimmed. The log is now **queried by the task's nouns**, like the
|
|
180
|
+
archive; the instructions and the stamps are still read in full, because they are the
|
|
181
|
+
part that is actually bounded.
|
|
182
|
+
|
|
183
|
+
**Stage-0 reading floor: ~47 750 → ~36 950 tokens** from that one change.
|
|
184
|
+
|
|
185
|
+
**And the audit that found this over-claimed in the same paragraph.** Its own table said
|
|
186
|
+
`stages.md` was *"read in full at stage 0"*; grepping the obligation returns only the
|
|
187
|
+
retro. The report now says *the gates a run must satisfy*, which is weaker and true.
|
|
188
|
+
|
|
189
|
+
### The preflight now says what has not been measured
|
|
190
|
+
|
|
191
|
+
It reported companion availability in careful detail and this skill's own evidence not at
|
|
192
|
+
all — while `evals/RESULTS.md` recorded **one self-observed run by the author and zero
|
|
193
|
+
blind runs on zero models**. A skill silent about its own evidence is read as tested, by
|
|
194
|
+
the bundle that demands evidence of everyone else. The line prints while no blind run is
|
|
195
|
+
recorded and disappears when one is; it is a state of the evidence, not a warning.
|
|
196
|
+
|
|
197
|
+
The suite itself was frozen at the v1.9 feature set — **zero cases** touching the board,
|
|
198
|
+
the verification ledger, the exposure line, the progress rail, `checkup`, `setup`,
|
|
199
|
+
`copywriting` or `sheleg-design`. Now **21 cases**, including the two no-task modes,
|
|
200
|
+
the three stage-3 tracks, the progress rail and run ledger, the dispatched reader, and a
|
|
201
|
+
question that must **not** trigger the pipeline at all.
|
|
202
|
+
|
|
203
|
+
Guards: 210 → **218**, property checks 8.
|
|
204
|
+
|
|
3
205
|
## v1.37.0 — the audit, and the four things it found the pipeline could not say
|
|
4
206
|
|
|
5
207
|
One release, four modules, and an audit that put four questions to this pipeline and
|
package/SKILL-CARD.md
CHANGED
|
@@ -12,7 +12,7 @@ harmless.
|
|
|
12
12
|
|---|---|
|
|
13
13
|
| **Purpose** | Runs a substantial task through ten gated delivery stages — intake grill, docs study, brainstorm, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs+registers, acceptance — refusing to advance until each gate passes |
|
|
14
14
|
| **Owner** | ssheleg ([github.com/ssheleg/task-pipeline](https://github.com/ssheleg/task-pipeline)) |
|
|
15
|
-
| **Version** | 1.
|
|
15
|
+
| **Version** | 1.39.0 |
|
|
16
16
|
| **Surface** | Claude Code (filesystem skill + plugin) and the vercel `skills` CLI. **Not** uploaded to the Skills API; custom Skills do not sync across surfaces |
|
|
17
17
|
| **Dependencies** | None required. Optional: `context7` (MCP), `figma` (MCP), super-ux, agent-sync, graphify, obsidian-wiki. Every stage's doctrine ships in-repo; the one conditional requirement is super-ux for the stage-3 UX track on a user-facing task |
|
|
18
18
|
| **Evaluation status** | Suite authored, 5 categories. One recorded run, **self-observed by the author**; **zero blind runs on zero of three models** — the split, and the numbers, live in [`evals/RESULTS.md`](evals/RESULTS.md) and are computed by `evals/run.py` |
|
|
@@ -605,11 +605,17 @@ track that ran look identical afterwards.
|
|
|
605
605
|
|
|
606
606
|
## Routing — when this applies at all
|
|
607
607
|
|
|
608
|
-
**Work that
|
|
609
|
-
refactor, migration, integration, rewrite, adoption or
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
608
|
+
**Work that LANDS IN THE REPOSITORY goes through the pipeline**, in two shapes: a
|
|
609
|
+
**change** — a feature, fix, refactor, migration, integration, rewrite, adoption or
|
|
610
|
+
hardening pass — and a **finding that lands** — an audit, a bug hunt, a production
|
|
611
|
+
check, a PR review, whose output is REQ rows, board rows and fixes rather than an
|
|
612
|
+
answer in the chat. In any phrasing.
|
|
613
|
+
|
|
614
|
+
**Not** for a question, an explanation, mapping code for a person to read, a typo or a
|
|
615
|
+
one-line edit. **Reading is not the test**: an audit, a bug hunt and a PR review all
|
|
616
|
+
begin by reading and all three end in the tree. The operator opts out of a qualifying
|
|
617
|
+
task by saying "без пайплайна" or "quick"; when they do, say out loud that the cycle
|
|
618
|
+
was skipped at their request.
|
|
613
619
|
|
|
614
620
|
A borderline case is named, not silently chosen: state which route you are taking and
|
|
615
621
|
why, in one line.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Routing measurement — results
|
|
2
|
+
|
|
3
|
+
Method: `evals/routing/render.py`. Ten queries, **one fresh agent per query**, holding
|
|
4
|
+
nothing but the competing skill descriptions and one user sentence.
|
|
5
|
+
|
|
6
|
+
> **Limits, stated before the numbers.** This measures the **descriptions'**
|
|
7
|
+
> discriminating power, not Claude Code's selection machinery, which sees more than a
|
|
8
|
+
> description. A subagent is fresh in *context*, not blind in *disposition* — this is
|
|
9
|
+
> **not** the multi-model blind run board row `B-002` asks for and no line here may be
|
|
10
|
+
> quoted as one. And see §3: **one sample per cell is too few**, which this measurement
|
|
11
|
+
> found out about itself by being run twice.
|
|
12
|
+
|
|
13
|
+
## 1. The three runs
|
|
14
|
+
|
|
15
|
+
| id | query | expected | **A** before | **B** after | **C** after + prod pair |
|
|
16
|
+
|---|---|---|---|---|---|
|
|
17
|
+
| AUD-01 | сделай аудит модуля оплат | task-pipeline | ✓ *(stretch)* | ✓ | ✓ |
|
|
18
|
+
| BUG-01 | проверь, нет ли ошибок в обработчике вебхуков | task-pipeline | **none** | ✓ | ✓ |
|
|
19
|
+
| PRD-01 | проверь, всё ли живо в проде после вчерашнего релиза | task-pipeline | **none** | **none** | **none** |
|
|
20
|
+
| PRR-01 | посмотри PR #24 и скажи, что там не так | task-pipeline | **none** | ✓ | ✓ |
|
|
21
|
+
| FEA-01 | добавь экспорт в CSV на страницу отчётов | task-pipeline | ✓ | ✓ | **none** |
|
|
22
|
+
| QST-01 | объясни, как наш auth middleware решает… | none | ✓ | ✓ | ✓ |
|
|
23
|
+
| TYP-01 | поправь опечатку в заголовке README | none | ✓ | ✓ | ✓ |
|
|
24
|
+
| SEO-01 | сделай аудит лендинга — почему упал трафик | seo-aeo-audit | ✓ | ✓ | ✓ |
|
|
25
|
+
| UXA-01 | проверь, что код соответствует UX-сценариям | ux-audit | ✓ | ✓ | ✓ |
|
|
26
|
+
| SKL-01 | проверь, соответствует ли этот скил стандарту | make-skill | ✓ | ✓ | ✓ |
|
|
27
|
+
| | | | **7 / 10** | **9 / 10** | **8 / 10** |
|
|
28
|
+
|
|
29
|
+
- **A** — `9f67dcd`, description 956/1024, before any edit.
|
|
30
|
+
- **B** — the findings clause added, description 1004/1024.
|
|
31
|
+
- **C** — `production check` given its Russian pair, description 1008/1024.
|
|
32
|
+
|
|
33
|
+
## 2. What is robust, and what is one sample
|
|
34
|
+
|
|
35
|
+
**Robust — the same result in both after-runs, for the stated reason:**
|
|
36
|
+
|
|
37
|
+
- **BUG-01 and PRR-01: `none` → `task-pipeline`, 2/2.** In A both agents *quoted this
|
|
38
|
+
skill's own exclusion clause* as the reason they refused — *"matching task-pipeline's
|
|
39
|
+
own exclusion 'Not for: … explaining or reading code'"*, *"task-pipeline's is scoped to
|
|
40
|
+
'work changes the repository', not 'explaining or reading code'"*. In B and C both
|
|
41
|
+
quote the new clause instead. The mechanism named in the refusal is the mechanism the
|
|
42
|
+
change removed.
|
|
43
|
+
- **AUD-01's *reason* changed, 2/2.** It routed in A too — by stretching the build verb
|
|
44
|
+
`hardening`, with nothing in the description naming an audit. In B and C it quotes
|
|
45
|
+
`audit/аудит`. The answer did not move; the ground under it did.
|
|
46
|
+
- **PRD-01 never routed: 0/2 after.** Adding `/проверь прод` did not fix it. In C the
|
|
47
|
+
agent said the request has *"no repository change and no matching trigger"* and
|
|
48
|
+
reached for `checkup` before rejecting it. The production-check class is **named and
|
|
49
|
+
still unreachable** — the vocabulary was necessary and is not sufficient.
|
|
50
|
+
|
|
51
|
+
**One sample, and therefore not a result:**
|
|
52
|
+
|
|
53
|
+
- **FEA-01 flipped to `none` in C** on reasoning that does not parse — *"no such change
|
|
54
|
+
applies here because none of the listed skills is task-pipeline's own trigger"*. This
|
|
55
|
+
is a control that passed in A and B with the same description clause C did not touch.
|
|
56
|
+
Read as run-to-run variance, **not** as a regression caused by the edit — and the
|
|
57
|
+
measurement as built cannot prove that reading, which is the point of §3.
|
|
58
|
+
|
|
59
|
+
## 3. The measurement's own defect, found by running it twice
|
|
60
|
+
|
|
61
|
+
**One agent per query cannot separate an effect from noise.** Three runs produced three
|
|
62
|
+
totals — 7, 9, 8 — and one of the moves between them (FEA-01) is almost certainly the
|
|
63
|
+
sampler, not the surface. Had the run stopped at B, this file would have reported
|
|
64
|
+
**9/10 and a clean win**, and the claim would have rested on single samples exactly like
|
|
65
|
+
the one that flipped.
|
|
66
|
+
|
|
67
|
+
What a next version needs, in order of value:
|
|
68
|
+
|
|
69
|
+
1. **N ≥ 3 per cell, majority-scored**, so a flip is visible as a split rather than a
|
|
70
|
+
result. Cost scales linearly and the queries are tiny.
|
|
71
|
+
2. **The controls scored separately from the targets.** A false-positive control that
|
|
72
|
+
flips is a different fact from a target that does not move.
|
|
73
|
+
3. **The reason recorded, not only the answer.** Every real finding in this file came
|
|
74
|
+
from *why* an agent chose, not from what it chose — AUD-01 passed in A for a reason
|
|
75
|
+
that was one rewording from failing.
|
|
76
|
+
|
|
77
|
+
Filed as a board row rather than fixed here: the run that needed the harness is the run
|
|
78
|
+
that found the harness too thin, and rebuilding it mid-run would leave the before-numbers
|
|
79
|
+
measured by a different instrument than the after-numbers.
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Render the routing-measurement prompt for one query.
|
|
3
|
+
|
|
4
|
+
The measurement asks a fresh agent, holding nothing but competing skill
|
|
5
|
+
descriptions and one user sentence, which skill it would use. Run before a
|
|
6
|
+
description change and after it; the only thing that may differ between the two
|
|
7
|
+
runs is the text of the descriptions themselves — this file renders everything
|
|
8
|
+
else identically, which is the whole reason it is a file and not a paste.
|
|
9
|
+
|
|
10
|
+
python3 evals/routing/render.py <query-id> # prompt to stdout
|
|
11
|
+
python3 evals/routing/render.py --list # the queries and expectations
|
|
12
|
+
|
|
13
|
+
Honest limits, stated here because a reader of the results will not see them:
|
|
14
|
+
* this measures the DESCRIPTIONS' discriminating power, not Claude Code's own
|
|
15
|
+
selection machinery, which sees more than a description;
|
|
16
|
+
* a subagent carries its host's system prompt, so it is fresh in CONTEXT, not
|
|
17
|
+
in disposition — this is not the blind multi-model run board row B-002 asks
|
|
18
|
+
for, and no result here may be quoted as one.
|
|
19
|
+
"""
|
|
20
|
+
import json
|
|
21
|
+
import os
|
|
22
|
+
import re
|
|
23
|
+
import subprocess
|
|
24
|
+
import sys
|
|
25
|
+
|
|
26
|
+
ROOT = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
27
|
+
|
|
28
|
+
# task-pipeline's description is read from the WORKING TREE, so the after-run
|
|
29
|
+
# picks up the edit; the competitors are read from the installed plugins, so an
|
|
30
|
+
# unrelated release of theirs cannot silently move the baseline between runs.
|
|
31
|
+
SELF = os.path.join(ROOT, "plugins/task-pipeline/skills/task-pipeline/SKILL.md")
|
|
32
|
+
MARKETPLACES = os.path.expanduser("~/.claude/plugins/marketplaces")
|
|
33
|
+
COMPETITORS = ["super-ux", "sheleg-design", "make-skill", "agent-sync", "seo-aeo-audit"]
|
|
34
|
+
|
|
35
|
+
QUERIES = [
|
|
36
|
+
# id, the sentence, what should win, why it is here
|
|
37
|
+
("AUD-01", "сделай аудит модуля оплат", "task-pipeline",
|
|
38
|
+
"findings on code — audit.md:350 says an audit may be the whole task"),
|
|
39
|
+
("BUG-01", "проверь, нет ли ошибок в обработчике вебхуков", "task-pipeline",
|
|
40
|
+
"bug hunt — findings that land in the repo"),
|
|
41
|
+
("PRD-01", "проверь, всё ли живо в проде после вчерашнего релиза", "task-pipeline",
|
|
42
|
+
"production check"),
|
|
43
|
+
("PRR-01", "посмотри PR #24 и скажи, что там не так", "task-pipeline",
|
|
44
|
+
"PR review — review.md owns the reader contract"),
|
|
45
|
+
("FEA-01", "добавь экспорт в CSV на страницу отчётов", "task-pipeline",
|
|
46
|
+
"CONTROL+ — build work already routes; must not regress"),
|
|
47
|
+
("QST-01", "объясни, как наш auth middleware решает, какие роуты публичные", "none",
|
|
48
|
+
"CONTROL− — a question; the exclusion must survive the rewording"),
|
|
49
|
+
("TYP-01", "поправь опечатку в заголовке README", "none",
|
|
50
|
+
"CONTROL− — a one-line edit; the exclusion must survive"),
|
|
51
|
+
("SEO-01", "сделай аудит лендинга — почему упал трафик из поиска", "seo-aeo-audit",
|
|
52
|
+
"FALSE-POSITIVE test — 'аудит' must not be stolen from its owner"),
|
|
53
|
+
("UXA-01", "проверь, что код соответствует UX-сценариям", "ux-audit",
|
|
54
|
+
"FALSE-POSITIVE test — the scenario audit has an owner"),
|
|
55
|
+
("SKL-01", "проверь, соответствует ли этот скил стандарту", "make-skill",
|
|
56
|
+
"FALSE-POSITIVE test — skill-shaped audit has an owner"),
|
|
57
|
+
]
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def _desc(path):
|
|
61
|
+
"""The description as the platform reads it: the frontmatter field, unwrapped."""
|
|
62
|
+
t = open(path, encoding="utf-8").read()
|
|
63
|
+
n = re.search(r"^name:\s*(.+)$", t, re.M)
|
|
64
|
+
d = re.search(r"^description:\s*(.+?)(?=^[a-z_-]+:\s|^---)", t, re.M | re.S)
|
|
65
|
+
if not (n and d):
|
|
66
|
+
return None
|
|
67
|
+
return n.group(1).strip(), " ".join(d.group(1).split()).strip("\"' ")
|
|
68
|
+
|
|
69
|
+
|
|
70
|
+
def choices():
|
|
71
|
+
out = []
|
|
72
|
+
s = _desc(SELF)
|
|
73
|
+
if s:
|
|
74
|
+
out.append(s)
|
|
75
|
+
for m in COMPETITORS:
|
|
76
|
+
base = os.path.join(MARKETPLACES, m)
|
|
77
|
+
if not os.path.isdir(base):
|
|
78
|
+
continue
|
|
79
|
+
found = subprocess.run(["find", base, "-name", "SKILL.md"],
|
|
80
|
+
capture_output=True, text=True).stdout.split()
|
|
81
|
+
for f in sorted(found):
|
|
82
|
+
got = _desc(f)
|
|
83
|
+
if got and got[0] not in {o[0] for o in out}:
|
|
84
|
+
out.append(got)
|
|
85
|
+
return out
|
|
86
|
+
|
|
87
|
+
|
|
88
|
+
PROMPT = """You are choosing which skill, if any, to invoke for one user request.
|
|
89
|
+
|
|
90
|
+
Below are the skills available, each with the description its author wrote. You have
|
|
91
|
+
nothing else — no repository, no history, no knowledge of these tools beyond what is
|
|
92
|
+
written here.
|
|
93
|
+
|
|
94
|
+
{catalogue}
|
|
95
|
+
|
|
96
|
+
The user says:
|
|
97
|
+
|
|
98
|
+
{query}
|
|
99
|
+
|
|
100
|
+
Answer with exactly two lines and nothing else:
|
|
101
|
+
|
|
102
|
+
SKILL: <the skill name, or the word none>
|
|
103
|
+
WHY: <one sentence, quoting the phrase in the description that decided it>
|
|
104
|
+
"""
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
def main():
|
|
108
|
+
if "--list" in sys.argv:
|
|
109
|
+
for qid, q, exp, why in QUERIES:
|
|
110
|
+
print(f"{qid:<8}{exp:<16}{q}\n{'':8}{'':16}# {why}")
|
|
111
|
+
return 0
|
|
112
|
+
if "--choices" in sys.argv:
|
|
113
|
+
for name, d in choices():
|
|
114
|
+
print(f"{name:<18}{len(d):>5} chars")
|
|
115
|
+
return 0
|
|
116
|
+
if len(sys.argv) < 2:
|
|
117
|
+
print(__doc__)
|
|
118
|
+
return 2
|
|
119
|
+
qid = sys.argv[1]
|
|
120
|
+
row = next((r for r in QUERIES if r[0] == qid), None)
|
|
121
|
+
if row is None:
|
|
122
|
+
print(f"unknown query id {qid}; --list to see them", file=sys.stderr)
|
|
123
|
+
return 2
|
|
124
|
+
cat = "\n".join(f"- **{name}** — {d}" for name, d in choices())
|
|
125
|
+
print(PROMPT.format(catalogue=cat, query=row[1]))
|
|
126
|
+
return 0
|
|
127
|
+
|
|
128
|
+
|
|
129
|
+
if __name__ == "__main__":
|
|
130
|
+
raise SystemExit(main())
|
|
@@ -220,6 +220,179 @@
|
|
|
220
220
|
"Does the work directly"
|
|
221
221
|
],
|
|
222
222
|
"why": "The opt-out phrase is the release valve that makes default-on acceptable. If it does not work, the boundary is a trap rather than a default."
|
|
223
|
+
},
|
|
224
|
+
{
|
|
225
|
+
"id": "TRIG-05",
|
|
226
|
+
"category": "should_trigger",
|
|
227
|
+
"skills": [
|
|
228
|
+
"task-pipeline"
|
|
229
|
+
],
|
|
230
|
+
"query": "что у нас накопилось непроверенного?",
|
|
231
|
+
"expected_behavior": [
|
|
232
|
+
"Recognises this as the checkup mode and runs `/task-pipeline checkup`",
|
|
233
|
+
"Does NOT open a grill, does not write a brief, and does not start a run",
|
|
234
|
+
"Reports the exposure line with its named components, the board's open rows by computed priority, the carry-over ledgers' unresolved count, and the graph's staleness",
|
|
235
|
+
"Never renders the exposure figure as a percentage or a probability"
|
|
236
|
+
],
|
|
237
|
+
"why": "checkup exists precisely for the moment no task is running, and before v1.38.0 it was named in no browsable surface. If a model cannot reach it from a natural request, it may as well not ship."
|
|
238
|
+
},
|
|
239
|
+
{
|
|
240
|
+
"id": "TRIG-06",
|
|
241
|
+
"category": "should_trigger",
|
|
242
|
+
"skills": [
|
|
243
|
+
"task-pipeline"
|
|
244
|
+
],
|
|
245
|
+
"query": "проверь документацию проекта перед тем как что-то строить",
|
|
246
|
+
"expected_behavior": [
|
|
247
|
+
"Recognises this as the setup mode and runs `/task-pipeline setup`",
|
|
248
|
+
"Runs the documentation passes and reports findings as file:line plus the minimal fix, ordered by seam",
|
|
249
|
+
"Hands back a fix plan rather than starting to fix things inline"
|
|
250
|
+
],
|
|
251
|
+
"why": "The entry audit is the one mode that runs BEFORE building on existing docs. Same discoverability risk as checkup."
|
|
252
|
+
},
|
|
253
|
+
{
|
|
254
|
+
"id": "INSTR-05",
|
|
255
|
+
"category": "instruction_following",
|
|
256
|
+
"skills": [
|
|
257
|
+
"task-pipeline"
|
|
258
|
+
],
|
|
259
|
+
"query": "run this through the pipeline: add CSV export to the reports page",
|
|
260
|
+
"expected_behavior": [
|
|
261
|
+
"Flags the task as user-facing and arms the stage-3 UX track",
|
|
262
|
+
"Names all THREE tracks at stage 3 — what it does (super-ux), how it sounds (copywriting, against the brand pack), how it looks (sheleg-design)",
|
|
263
|
+
"Where a track is declined, records the refusal out loud in the brief rather than skipping it silently",
|
|
264
|
+
"Does not write interface strings by taste with no mention of the copy track"
|
|
265
|
+
],
|
|
266
|
+
"why": "copywriting appeared zero times in the bundle and sheleg-design once until v1.36.0. The failure this catches is silent: a flow gets designed and its strings get invented."
|
|
267
|
+
},
|
|
268
|
+
{
|
|
269
|
+
"id": "INSTR-06",
|
|
270
|
+
"category": "instruction_following",
|
|
271
|
+
"skills": [
|
|
272
|
+
"task-pipeline"
|
|
273
|
+
],
|
|
274
|
+
"query": "run this through the pipeline: fix the retry backoff in the queue worker",
|
|
275
|
+
"expected_behavior": [
|
|
276
|
+
"Prints a progress header at task start naming the skill, the version, the topic and the stage rail",
|
|
277
|
+
"Derives the rail from the project's own pipeline config rather than assuming eleven stages",
|
|
278
|
+
"Prints a one-line progress update at each iteration close, citing a board id rather than a description",
|
|
279
|
+
"Seeds .task-pipeline/run.md at stage 0 and appends a stage verdict line when a gate returns"
|
|
280
|
+
],
|
|
281
|
+
"why": "Before v1.34.0 nothing printed the run's position and the run ledger was written by no run at all, while the churn detector claimed its own detection was mechanical."
|
|
282
|
+
},
|
|
283
|
+
{
|
|
284
|
+
"id": "INSTR-07",
|
|
285
|
+
"category": "instruction_following",
|
|
286
|
+
"skills": [
|
|
287
|
+
"task-pipeline"
|
|
288
|
+
],
|
|
289
|
+
"query": "run this through the pipeline: harden the auth middleware, and review it properly",
|
|
290
|
+
"expected_behavior": [
|
|
291
|
+
"Dispatches an independent reader at stage 7 rather than assuming a bot will read it",
|
|
292
|
+
"Records one of exactly three states beside the gate verdict: findings, none found, or NO READER with a reason",
|
|
293
|
+
"Stops reviewing at the declared round cap and prints new findings against self-inflicted ones per round, instead of looping",
|
|
294
|
+
"Never reports the review as done on the basis that a reader was requested"
|
|
295
|
+
],
|
|
296
|
+
"why": "Four PRs of check work merged on a review app's `skipping` with nobody noticing. A requested reader and a reader that reported are different facts that look identical afterwards."
|
|
297
|
+
},
|
|
298
|
+
{
|
|
299
|
+
"id": "NOTRIG-05",
|
|
300
|
+
"category": "should_not_trigger",
|
|
301
|
+
"skills": [
|
|
302
|
+
"task-pipeline"
|
|
303
|
+
],
|
|
304
|
+
"query": "какой командой у вас гоняются тесты?",
|
|
305
|
+
"expected_behavior": [
|
|
306
|
+
"Answers the question directly from the repo's own files",
|
|
307
|
+
"Does NOT invoke task-pipeline, does not open a grill and does not write a brief"
|
|
308
|
+
],
|
|
309
|
+
"why": "Answering a question is on the skill's own not-for list. The description carries that boundary and nothing has ever measured whether a model honours it."
|
|
310
|
+
},
|
|
311
|
+
{
|
|
312
|
+
"id": "TRIG-07",
|
|
313
|
+
"category": "should_trigger",
|
|
314
|
+
"skills": [
|
|
315
|
+
"task-pipeline"
|
|
316
|
+
],
|
|
317
|
+
"query": "сделай аудит модуля оплат",
|
|
318
|
+
"expected_behavior": [
|
|
319
|
+
"Invokes the skill: the output is a finding that lands in the repository, which the description names",
|
|
320
|
+
"Runs audit.md's L0->L7 ladder as the whole task — stages 3-5 produce findings and fixes rather than a feature",
|
|
321
|
+
"Does NOT justify the routing by stretching a build verb such as 'hardening'"
|
|
322
|
+
],
|
|
323
|
+
"why": "Measured 2026-08-10: before the findings clause this routed, but its stated reason was the build-verb list. A right answer resting on a stretch is one rewording from a miss."
|
|
324
|
+
},
|
|
325
|
+
{
|
|
326
|
+
"id": "TRIG-08",
|
|
327
|
+
"category": "should_trigger",
|
|
328
|
+
"skills": [
|
|
329
|
+
"task-pipeline"
|
|
330
|
+
],
|
|
331
|
+
"query": "проверь, нет ли ошибок в обработчике вебхуков",
|
|
332
|
+
"expected_behavior": [
|
|
333
|
+
"Invokes the skill rather than answering in the chat",
|
|
334
|
+
"Findings become REQ or board rows with file:line and a minimal fix, ordered by seam"
|
|
335
|
+
],
|
|
336
|
+
"why": "Measured before the change: answered `none`, quoting the exclusion clause 'explaining or reading code'. Reading is the opening move, not the boundary."
|
|
337
|
+
},
|
|
338
|
+
{
|
|
339
|
+
"id": "TRIG-09",
|
|
340
|
+
"category": "should_trigger",
|
|
341
|
+
"skills": [
|
|
342
|
+
"task-pipeline"
|
|
343
|
+
],
|
|
344
|
+
"query": "проверь, всё ли живо в проде после вчерашнего релиза",
|
|
345
|
+
"expected_behavior": [
|
|
346
|
+
"Invokes the skill; the verification trio, not one of three",
|
|
347
|
+
"Reads the CI run and the platform's own log command rather than assuming a verdict"
|
|
348
|
+
],
|
|
349
|
+
"why": "Measured before the change: answered `none` on the same exclusion. A production check ends in a verification row, which lands."
|
|
350
|
+
},
|
|
351
|
+
{
|
|
352
|
+
"id": "TRIG-10",
|
|
353
|
+
"category": "should_trigger",
|
|
354
|
+
"skills": [
|
|
355
|
+
"task-pipeline"
|
|
356
|
+
],
|
|
357
|
+
"query": "посмотри PR #24 и скажи, что там не так",
|
|
358
|
+
"expected_behavior": [
|
|
359
|
+
"Invokes the skill and applies review.md's reviewer contract, including the three reader states",
|
|
360
|
+
"Prints NO READER rather than staying silent when no independent reader is available"
|
|
361
|
+
],
|
|
362
|
+
"why": "Measured before the change: answered `none` because no description offered a phrase for reviewing a PR, while review.md owns exactly that contract."
|
|
363
|
+
},
|
|
364
|
+
{
|
|
365
|
+
"id": "NOTRIG-06",
|
|
366
|
+
"category": "should_not_trigger",
|
|
367
|
+
"skills": [],
|
|
368
|
+
"query": "сделай аудит лендинга — почему упал трафик из поиска",
|
|
369
|
+
"expected_behavior": [
|
|
370
|
+
"Routes to seo-aeo-audit, whose description names 'почему упал трафик' verbatim",
|
|
371
|
+
"The widened findings vocabulary does not claim an audit that has an owner"
|
|
372
|
+
],
|
|
373
|
+
"why": "False-positive control. Widening 'audit' must not steal work: gates.md sets the false-positive budget at zero."
|
|
374
|
+
},
|
|
375
|
+
{
|
|
376
|
+
"id": "NOTRIG-07",
|
|
377
|
+
"category": "should_not_trigger",
|
|
378
|
+
"skills": [],
|
|
379
|
+
"query": "проверь, что код соответствует UX-сценариям",
|
|
380
|
+
"expected_behavior": [
|
|
381
|
+
"Routes to ux-audit, which owns the scenario audit",
|
|
382
|
+
"The findings vocabulary in task-pipeline's description does not override an owner that names the surface exactly"
|
|
383
|
+
],
|
|
384
|
+
"why": "False-positive control against the second competing claim on the word audit."
|
|
385
|
+
},
|
|
386
|
+
{
|
|
387
|
+
"id": "NOTRIG-08",
|
|
388
|
+
"category": "should_not_trigger",
|
|
389
|
+
"skills": [],
|
|
390
|
+
"query": "проверь, соответствует ли этот скил стандарту",
|
|
391
|
+
"expected_behavior": [
|
|
392
|
+
"Routes to make-skill, which owns skill-shaped audits",
|
|
393
|
+
"The findings vocabulary in task-pipeline's description does not override an owner that names the artefact exactly"
|
|
394
|
+
],
|
|
395
|
+
"why": "False-positive control against the third competing claim on the word audit."
|
|
223
396
|
}
|
|
224
397
|
]
|
|
225
398
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "task-pipeline-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.39.0",
|
|
4
4
|
"description": "Full-cycle delivery pipeline for coding agents: a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine ships inside the skill — no companion plugin required. This package is the installer CLI.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"task-pipeline": "bin/task-pipeline.js"
|
|
@@ -8,7 +8,8 @@
|
|
|
8
8
|
"scripts": {
|
|
9
9
|
"test": "python3 test/validate.py",
|
|
10
10
|
"test:negatives": "python3 test/negatives.py",
|
|
11
|
-
"test:
|
|
11
|
+
"test:probe": "python3 test/probe.py --self-test",
|
|
12
|
+
"test:all": "python3 test/validate.py && python3 test/negatives.py && npm run test:probe"
|
|
12
13
|
},
|
|
13
14
|
"files": [
|
|
14
15
|
"bin",
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "task-pipeline",
|
|
3
3
|
"displayName": "Task Pipeline",
|
|
4
|
-
"description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/manual gates, a frozen requirement spine that
|
|
5
|
-
"version": "1.
|
|
4
|
+
"description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
|
|
5
|
+
"version": "1.39.0",
|
|
6
6
|
"author": {
|
|
7
7
|
"name": "ssheleg",
|
|
8
8
|
"url": "https://x.com/sshlg93"
|
|
@@ -1,62 +1,173 @@
|
|
|
1
1
|
---
|
|
2
|
-
description: Run a task through task-pipeline
|
|
3
|
-
argument-hint: "<one-line task
|
|
2
|
+
description: "Run a task through task-pipeline's gated stages (intake grill → docs → brainstorm → spec → plan → build → tests → deploy → post-deploy → docs/wiki → acceptance). Also: `setup` — audit the docs you already have; `checkup` — what has shipped unverified, with no task running."
|
|
3
|
+
argument-hint: "<one-line task> | setup | checkup"
|
|
4
4
|
---
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
built into the skill** (`references/{knowledge-sources,knowledge-graph,grill,brainstorm,decomposition,spec,planning,build,review,tdd,acceptance,retrospective,loop-guard}.md`)
|
|
9
|
-
— no companion plugin is required for any of them. **Stage 0 opens with the
|
|
10
|
-
knowledge harvest, before the first question** (`references/knowledge-sources.md`):
|
|
11
|
-
pull what the project already knows about this task from the code, **the code graph**
|
|
12
|
-
if one is built ([graphify](https://github.com/Graphify-Labs/graphify) —
|
|
13
|
-
`references/knowledge-graph.md`; recommended, never required; detect
|
|
14
|
-
`graphify-out/graph.json`; it answers *reach* — what calls this, what breaks if it
|
|
15
|
-
moves — which grep cannot), `CLAUDE.md`,
|
|
16
|
-
`CONTEXT.md`/ADRs, `docs/` + `docs/ux/`, past pipeline briefs, **the retro's standing
|
|
17
|
-
instructions** (`docs/superpowers/retro.md` — read in full, they bind this run;
|
|
18
|
-
`references/retrospective.md`), the **knowledge wiki**
|
|
19
|
-
if one is installed ([obsidian-wiki](https://github.com/ar9av/obsidian-wiki) —
|
|
20
|
-
recommended, never required; detect `~/.obsidian-wiki/config`) and any **other repo
|
|
21
|
-
or hosted doc system the project names as its docs**, then write the **source
|
|
22
|
-
ledger** into the brief. The **intake grill is
|
|
23
|
-
mandatory** (`references/grill.md`): interview the
|
|
24
|
-
operator one question at a time (with a recommended answer each, exploring the
|
|
25
|
-
codebase before asking) until every decision branch is resolved, **validating every
|
|
26
|
-
answer against the harvested sources** — the operator outranks any document, but
|
|
27
|
-
only out loud, and a doc the run proves stale is logged for the stage-9 update —
|
|
28
|
-
applying the
|
|
29
|
-
grill's **domain awareness** (challenge terms against `CONTEXT.md`, sharpen fuzzy
|
|
30
|
-
language, ADRs for hard-to-reverse calls) and covering the **autonomy sweep** (what
|
|
31
|
-
would otherwise stop stages 1→10: docs sources incl. doc repos, the wiki and the code graph, branch/tracker
|
|
32
|
-
policy, test and lint commands, deploy target and authorization, log locations, docs/wiki/graph targets, and for UI tasks whether the design is done visually in Figma or text-only, whether the Figma MCP is connected, and — if it isn't — whether to ship text-only or stop and connect it, since the UX chain degrades on its own and never blocks; **and with Figma on, the design destination: which team/org by name and which file** — the recorded one, a URL the operator gives, or creation in that named team explicitly authorized, written into the project's canonical record before the first frame, because a destination decided at drawing time is how a project ends up with three design files and no way to tell which is real. **Never create while a recorded file resolves; if it doesn't resolve, stop and ask — never create a replacement**) —
|
|
33
|
-
until the brief is locked — including the **REQ table**, the request as an addressable list where every row names how it is verified — so the rest runs autonomously and the final stage can account for all of it. The list is frozen: adding is free, removing needs the operator's agreement. Anything deferred goes into the carry-over ledger the moment it's said, and **the board** (`docs/superpowers/backlog.md`, `references/backlog.md`) — the work-list between runs — is read at stage 0 with its open count quoted in the brief, or seeded when absent; **the verification ledger** (`docs/superpowers/verification.md`, `references/verification.md`) is read at stage 0 for how many rows sit at `never`, written at stage 8 with one row per shipped REQ, and required at stage 10 in both directions. For any user-facing task, recommend/use
|
|
34
|
-
**super-ux**. **If the brief describes a platform rather than a change**, stage 2 also cuts it into modules (`references/decomposition.md`) — module map committed, walking skeleton first, every REQ in exactly one module — and stages 3→10 then run per module, one brick at a time. **If any loop starts undoing an earlier pass** (same file edited twice for the same reason, a closed finding returning, a third entry into one stage), stop and run the loop guard (`references/loop-guard.md`): name both shapes, escalate to the layer that owns the conflict, re-plan the check as an ordered list, then go item by item. **The closing stage opens with the ladder walk** (`references/audit.md`): the REQ table finds what was named and lost, but a comparison needs two sides and an absence has one — so walk each REQ bottom-up through its rungs (decision → spec section → contract *and its failure behavior* → task → change → executed test → surface/docs), check the seam at each step, order findings by seam rather than by file, and turn every absence into a new REQ row **before** the coverage table is written. A green from a check nobody has watched fail against a planted defect is not evidence; a finding class seen twice becomes a script rather than a third ledger row; and every ledger row still `open`, `unresolved` or homed `backlog` leaves stage 10 with a board id and the board's priorities are re-derived (`references/backlog.md`); the carry-over ledger's counts are printed beside every gate verdict, so "green" never reads as "verified". If a searching pass starts finding mostly what the previous pass's own fixes broke, the axis is exhausted — rotate it, don't look harder. **The docs stage closes three artifacts, not two:** module docs, the wiki, **and the code graph** (`/graphify . --update` where `graphify-out/` exists — `references/knowledge-graph.md`), because the graph is what the next run's harvest queries first and a stale one is a false premise carrying the authority of a machine. Then check the graph against the docs: a hub `graphify god-nodes` reports that no document names is an undocumented seam; an edge the docs deny is a leak in the code or a lie in the docs; a doc naming a module the graph no longer has is stale. Doc-side findings are fixed there, absences become REQ rows in the closing stage. **In a project of several repositories, stage 10 closes on the parent too:** a parent records each submodule as a pointer to one commit, and moving the submodule does not move the pointer — so the work can be committed, pushed and green while a clone of the parent still gets the commit before it. Neither repo looks wrong alone, which is why it survives every check that runs inside one. Require `git submodule status` with no line starting `+`, and every repo clean and pushed (`git -C <repo> status --porcelain`, `git -C <repo> log @{u}..HEAD`). The fix is two commands and the second gets forgotten: push the submodule, then `git add <submodule> && git commit`. **The run's last act is the retrospective** (`references/retrospective.md` → `docs/superpowers/retro.md`, one file per project): **stamp the run first** (its commit is what makes the cold-retirement trigger computable), **then prune** — every standing instruction against its three retirement triggers (it became a check; the paths/commands it names are gone; it hasn't fired in five run stamps, or in sixty days — the calendar is the unit that still moves when the stamp counter has stopped), the list held to a hard cap of ten, every deletion logged as one line and never silent — then, only if the run diverged, write the entry: symptom with evidence, the stage it surfaced at, the stage that *owned* it, the root cause, the fix by grade (mechanical check > standing instruction > note that expires in two runs), and the check that catches it next time. Stage 0 reads those standing instructions in full, so the prune is a gate criterion, not a good intention: a rule nobody reads to the end is worse than no rule, because everyone believes it is covered. Honor every stage gate by its type (`auto` = verify yourself;
|
|
35
|
-
`manual` = wait for explicit go). Confirm the **model once at preflight** —
|
|
36
|
-
recommend the most capable one the environment offers, never a hardcoded id — then
|
|
37
|
-
run the whole pipeline on it without re-asking.
|
|
5
|
+
Run the task below through `task-pipeline`'s gated stages. **Every stage's doctrine
|
|
6
|
+
ships inside the skill** — no companion plugin is required for any of it. The
|
|
7
|
+
reference files are indexed in `SKILL.md`; this page is the run order, not the index.
|
|
38
8
|
|
|
39
9
|
Task: $ARGUMENTS
|
|
40
10
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
11
|
+
> **Two other modes.** `setup` audits the documentation this project already has;
|
|
12
|
+
> `checkup` reports what has shipped without a person confirming it, with no task in
|
|
13
|
+
> flight. Both are at the bottom of this page.
|
|
14
|
+
|
|
15
|
+
**Idempotent entry — inspect state first, never restart blindly.** If a pipeline
|
|
16
|
+
TaskList from a previous run exists for this task, **resume** from the first incomplete
|
|
17
|
+
stage. Otherwise begin at stage 0. With no task given, the grill's first question asks
|
|
18
|
+
for it in one line.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Preflight — settle these once
|
|
23
|
+
|
|
24
|
+
- **Model.** Recommend the most capable one the environment offers, never a hardcoded
|
|
25
|
+
id. Confirmed once, then the whole run uses it without re-asking.
|
|
26
|
+
- **Companions.** Print the detection block once (`references/companion-skills.md`).
|
|
27
|
+
Absent ones state their fallback; none is a gate except the stage-3 UX track on a
|
|
28
|
+
user-facing task.
|
|
29
|
+
|
|
30
|
+
## Stage 0 — the harvest, then the grill
|
|
31
|
+
|
|
32
|
+
**The harvest runs before the first question** (`references/knowledge-sources.md`).
|
|
33
|
+
Pull what the project already knows about *this task*:
|
|
34
|
+
|
|
35
|
+
- the code, and **the code graph** where one is built — [graphify](https://github.com/Graphify-Labs/graphify),
|
|
36
|
+
`references/knowledge-graph.md`; recommended, never required; detect
|
|
37
|
+
`graphify-out/graph.json`. It answers **reach** — what calls this, what breaks if it
|
|
38
|
+
moves — which grep cannot.
|
|
39
|
+
- `CLAUDE.md`, `CONTEXT.md`/ADRs, `docs/` + `docs/ux/`, past briefs and carry-over ledgers.
|
|
40
|
+
- **the retro's standing instructions and run stamps** — `docs/superpowers/retro.md`,
|
|
41
|
+
read in full; both are bounded and they bind this run. Its **Recent log** is
|
|
42
|
+
*queried* by the task's nouns, not read: nothing caps it, and an uncapped section
|
|
43
|
+
inside a binding source is what makes the capped part get skimmed
|
|
44
|
+
(`references/retrospective.md`).
|
|
45
|
+
- the **knowledge wiki** if installed ([obsidian-wiki](https://github.com/ar9av/obsidian-wiki);
|
|
46
|
+
detect `~/.obsidian-wiki/config`), and any other doc system the project names as its docs.
|
|
47
|
+
- **the board** (`docs/superpowers/backlog.md`) — open count quoted in the brief, or
|
|
48
|
+
seeded when absent. **the verification ledger** (`docs/superpowers/verification.md`) —
|
|
49
|
+
how many rows sit at `never`.
|
|
50
|
+
|
|
51
|
+
Write the **source ledger** into the brief: a row per source, or an explicit *none found*.
|
|
52
|
+
|
|
53
|
+
**Then the grill, and it is mandatory** (`references/grill.md`). One question per turn,
|
|
54
|
+
each with a recommended answer, exploring the codebase before asking, until every
|
|
55
|
+
decision branch is resolved.
|
|
56
|
+
|
|
57
|
+
- **Validate every answer against the harvested sources.** The operator outranks any
|
|
58
|
+
document — but only out loud, and a doc the run proves stale is logged for stage 9.
|
|
59
|
+
- **Domain awareness:** challenge terms against `CONTEXT.md`, sharpen fuzzy language,
|
|
60
|
+
write an ADR for a hard-to-reverse call.
|
|
61
|
+
- **The autonomy sweep** pre-resolves what would otherwise stop stages 1→10: doc
|
|
62
|
+
sources, wiki and graph, branch and tracker policy, test and lint commands, deploy
|
|
63
|
+
target and authorization, log locations.
|
|
64
|
+
- **UI tasks add the design surface.** Is the design done visually in Figma or
|
|
65
|
+
text-only? Is the Figma MCP connected? If not — ship text-only, or stop and connect
|
|
66
|
+
it? The UX chain degrades on its own and never blocks, so this choice must be
|
|
67
|
+
recorded rather than discovered.
|
|
68
|
+
- **With Figma on, the destination is named before the first frame:** which team/org,
|
|
69
|
+
which file — the recorded one, a URL the operator gives, or creation in that named
|
|
70
|
+
team explicitly authorized. A destination decided at drawing time is how a project
|
|
71
|
+
ends up with three design files and no way to tell which is real. **Never create
|
|
72
|
+
while a recorded file resolves; if it does not resolve, stop and ask — never create a
|
|
73
|
+
replacement.**
|
|
74
|
+
|
|
75
|
+
**The brief closes on the REQ table** — the request as an addressable list where every
|
|
76
|
+
row names how it is verified. Frozen: adding is free, removing needs the operator.
|
|
77
|
+
Anything deferred enters the carry-over ledger the moment it is said.
|
|
78
|
+
|
|
79
|
+
## Stages 1→10 — the flow
|
|
80
|
+
|
|
81
|
+
| | Stage | The thing that must be true to leave it |
|
|
82
|
+
|---|---|---|
|
|
83
|
+
| 1 | Docs study | every contract the design will lock is grounded on fetched docs, not recall |
|
|
84
|
+
| 2 | Brainstorm + decompose | the design is approved and every REQ is answered by it. **A platform is cut into modules** (`references/decomposition.md`) — map committed, walking skeleton first, every REQ in exactly one module; stages 3→10 then run per module |
|
|
85
|
+
| 3 | Spec | contracts locked; user-facing work runs three tracks — what it **does** (super-ux), how it **sounds** (`copywriting`), how it **looks** (`sheleg-design`); a declined track is recorded, never silent |
|
|
86
|
+
| 4 | Plan | the REQ set-comparison holds: brief REQs == union of `Implements:` |
|
|
87
|
+
| 5 | Build | TDD per task, a review after each, findings fixed or parked with a ruling |
|
|
88
|
+
| 6 | Tests | the **full** suite green; a web surface checked in a browser, not in the diff |
|
|
89
|
+
| 7 | Lint + deploy | outward: the authorization is specific, and the CI verdict is **read** before any tag |
|
|
90
|
+
| 8 | Post-deploy | the verification trio, not one of three; a verification row per shipped REQ |
|
|
91
|
+
| 9 | Docs + wiki | **three artifacts, not two** — module docs, the wiki, **and the code graph** |
|
|
92
|
+
| 10 | Acceptance | the ladder walk first, then the table, then the retrospective |
|
|
93
|
+
|
|
94
|
+
**Honor every gate by its type**: `auto` — verify the check yourself; `manual` — wait
|
|
95
|
+
for an explicit go.
|
|
96
|
+
|
|
97
|
+
## Cross-cutting — the three that fire at any stage
|
|
98
|
+
|
|
99
|
+
**The loop guard** (`references/loop-guard.md`). If a pass starts undoing an earlier one
|
|
100
|
+
— the same file edited twice for the same reason, a closed finding returning, a third
|
|
101
|
+
entry into one stage — stop editing. Name both shapes, escalate to the layer that owns
|
|
102
|
+
the conflict, re-plan the check as an ordered list, then go item by item. The review
|
|
103
|
+
loop has its own ceiling, and at it the run **measures** rather than stops.
|
|
104
|
+
|
|
105
|
+
**The audit's exit** (`references/audit.md`). If a searching pass starts finding mostly
|
|
106
|
+
what the previous pass's own fixes broke, the axis is exhausted — rotate it, do not look
|
|
107
|
+
harder.
|
|
108
|
+
|
|
109
|
+
**Evidence.** A green from a check nobody has watched fail against a planted defect is
|
|
110
|
+
not evidence. A finding class seen twice becomes a script, not a third ledger row.
|
|
111
|
+
|
|
112
|
+
## Stage 9 — the graph is the third artifact
|
|
113
|
+
|
|
114
|
+
Refresh it (`/graphify . --update` where `graphify-out/` exists), then **check it against
|
|
115
|
+
the docs**: a hub `graphify god-nodes` reports that no document names is an undocumented
|
|
116
|
+
seam; an edge the docs deny is a leak in the code or a lie in the docs; a doc naming a
|
|
117
|
+
module the graph no longer has is stale. Doc-side findings are fixed here; absences
|
|
118
|
+
become REQ rows at stage 10.
|
|
119
|
+
|
|
120
|
+
A stale graph is a false premise **carrying the authority of a machine** — a wrong doc
|
|
121
|
+
gets argued with, a wrong graph gets believed.
|
|
122
|
+
|
|
123
|
+
## Stage 10 — the close-out, in order
|
|
124
|
+
|
|
125
|
+
1. **The ladder walk, first.** The REQ table finds what was named and lost; it cannot
|
|
126
|
+
find what was never named, because a comparison needs two sides and an absence has
|
|
127
|
+
one. Walk each REQ bottom-up — decision → spec section → contract *and its failure
|
|
128
|
+
behavior* → task → change → executed test → surface/docs — check the seam at each
|
|
129
|
+
step, and order findings **by seam, not by file**. Every absence becomes a new REQ
|
|
130
|
+
row **before** the coverage table is written.
|
|
131
|
+
2. **The table**, one row per REQ, each with evidence.
|
|
132
|
+
3. **The ledgers close.** Every carry-over row still `open`, `unresolved` or homed
|
|
133
|
+
`backlog` leaves with a board id, and the board's priorities are re-derived. The
|
|
134
|
+
counts print beside every gate verdict, so *green* never reads as *verified*.
|
|
135
|
+
4. **Several repositories? The parent closes too.** A parent records each submodule as a
|
|
136
|
+
pointer to one commit, and moving the submodule does not move the pointer — so work
|
|
137
|
+
can be committed, pushed and green while a clone of the parent still gets the commit
|
|
138
|
+
before it. Neither repo looks wrong alone. Require `git submodule status` with no
|
|
139
|
+
line starting `+`, and every repo clean and pushed. The fix is two commands and the
|
|
140
|
+
second gets forgotten: push the submodule, **then** `git add <submodule> && git commit`.
|
|
141
|
+
5. **The retrospective is the run's last act** (`references/retrospective.md`), in this
|
|
142
|
+
order: **stamp the run first** (its commit makes the cold trigger computable) → **prune**
|
|
143
|
+
every standing instruction against its three retirement triggers, list held to ten,
|
|
144
|
+
every deletion logged → **write an entry only if the run diverged**: symptom with
|
|
145
|
+
evidence, the stage it surfaced at, the stage that *owned* it, the root cause, the
|
|
146
|
+
fix by grade, and the check that catches it next time.
|
|
147
|
+
|
|
148
|
+
Stage 0 reads those standing instructions in full next time, which is why the prune is a
|
|
149
|
+
gate criterion rather than a good intention.
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## `/task-pipeline checkup`
|
|
154
|
+
|
|
155
|
+
**Runs with no task in flight, and that is the point.** Accumulated unconfirmed work is
|
|
156
|
+
invisible precisely because nobody is running a pipeline, so a check living only inside a
|
|
157
|
+
run can never say *"stop, fourteen things are unconfirmed."*
|
|
158
|
+
|
|
159
|
+
It takes no brief, opens no grill, and writes nothing on its own. Four sections, each
|
|
160
|
+
read from a file this pipeline already keeps: the **exposure** line with its check-list
|
|
161
|
+
oldest-first, the **board**'s open rows by computed priority, the carry-over ledgers'
|
|
162
|
+
unresolved count, and the code graph's staleness where one exists.
|
|
163
|
+
|
|
54
164
|
Where you ask it to file what it found, it appends board rows whose `Source` names the
|
|
55
|
-
checkup and its date — printing what it would add first, never silently.
|
|
56
|
-
`references/exposure.md`.
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
165
|
+
checkup and its date — printing what it would add first, never silently.
|
|
166
|
+
Doctrine: `references/exposure.md`.
|
|
167
|
+
|
|
168
|
+
## `/task-pipeline setup`
|
|
169
|
+
|
|
170
|
+
**The entry audit instead of a feature.** Seven passes over the documentation this
|
|
171
|
+
project already has, findings reported as `file:line` + the minimal fix ordered by seam,
|
|
172
|
+
and a fix plan the pipeline can run. Offered once at stage 0 when the doc map is absent
|
|
173
|
+
or stale; run it directly any time. Doctrine: `references/setup.md`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: task-pipeline
|
|
3
|
-
description: "Runs a substantial task through a full delivery pipeline: an intake grill that
|
|
3
|
+
description: "Runs a substantial task through a full delivery pipeline: an intake grill that locks the request into a brief, then docs study, brainstorm, spec, plan, subagent build, tests, lint/deploy, post-deploy check, docs/wiki sync and acceptance. Use when work changes the repository — a feature, fix, refactor, migration, integration, rewrite, adoption or hardening; фича, фикс, рефактор, миграция, интеграция, доработать, починить, внедрить, перевести — or when the output is a finding that lands in it: audit/аудит, bug hunt/проверь ошибки, production check/проверь прод, PR review/ревью PR — or on 'run this through the pipeline' / 'прогони по конвейеру', 'the full cycle' / 'полный цикл', /task-pipeline. Two modes need no task at all: 'checkup' / 'чекап' reports what has shipped without a person confirming it and what to look at first; 'setup' audits the documentation a project already has. Not for: answering a question, explaining code, a typo or a one-line edit — say 'без пайплайна' / 'quick' to opt out."
|
|
4
4
|
license: MIT
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -119,10 +119,10 @@ question: pull what the project already knows about this task from the code, the
|
|
|
119
119
|
([`references/knowledge-graph.md`](references/knowledge-graph.md) — graphify;
|
|
120
120
|
recommended, never required),
|
|
121
121
|
`CLAUDE.md`, `CONTEXT.md`/ADRs, **the decision register**, `docs/` + `docs/ux/`,
|
|
122
|
-
past pipeline briefs, **the retro's standing instructions
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
**queried** by the task's nouns
|
|
122
|
+
past pipeline briefs, **the retro's standing instructions and run stamps** —
|
|
123
|
+
`docs/superpowers/retro.md`, read in full because they *bind* this run and are
|
|
124
|
+
bounded by construction (ten rows; one line per run), while its **recent log** and the
|
|
125
|
+
archive under `docs/superpowers/retro/` are **queried** by the task's nouns
|
|
126
126
|
([`references/retrospective.md`](references/retrospective.md)) —
|
|
127
127
|
the **knowledge wiki** if one is installed
|
|
128
128
|
([obsidian-wiki](https://github.com/ar9av/obsidian-wiki) — recommended, never
|
|
@@ -122,6 +122,11 @@ Pipeline companions (stage doctrine is built in — nothing to install for it):
|
|
|
122
122
|
🧠 Model for this run: recommended <top tier available>. You're on <current>.
|
|
123
123
|
/model <id> to switch, or "keep current", or name per-stage overrides.
|
|
124
124
|
|
|
125
|
+
⚠ This skill's own behaviour is unverified: evals/RESULTS.md records <N> blind
|
|
126
|
+
run(s) on <M> model(s). Its triggering and its stage compliance are authored,
|
|
127
|
+
not measured. Nothing here is blocked by that — you are told because a skill
|
|
128
|
+
that never says so is one you would assume had been tested.
|
|
129
|
+
|
|
125
130
|
Install the ✗ items you want, answer the model line, then say "continue".
|
|
126
131
|
```
|
|
127
132
|
|
|
@@ -170,6 +175,13 @@ Rules:
|
|
|
170
175
|
reserve ids before minting them. Absent → print the line **once**, continue, and
|
|
171
176
|
**record the run as `ungated`** — never describe the project as protected
|
|
172
177
|
([`documentation.md`](documentation.md) → *Registers are shared state*).
|
|
178
|
+
- **The behaviour line prints whenever `evals/RESULTS.md` records no blind run**, and
|
|
179
|
+
disappears the moment one is recorded — it is a state of the evidence, not a warning
|
|
180
|
+
and not a ratchet. This bundle asks every project it touches for evidence rather than
|
|
181
|
+
assertion, and until 2026-08-10 it made the opposite claim about itself by saying
|
|
182
|
+
nothing: **one self-observed run by the author, zero blind runs on zero models**, with
|
|
183
|
+
a preflight that reported companion availability in detail and its own confidence not
|
|
184
|
+
at all. A skill silent about its own evidence is read as tested.
|
|
173
185
|
- **Never gate any stage on an install** except the stage-3 UX track on a UI task.
|
|
174
186
|
- Optional tools missing → state the fallback, don't block.
|
|
175
187
|
- Re-detect after the operator installs; don't assume.
|
|
@@ -272,6 +272,24 @@ one flipped a row whose cell was already empty, so nothing was planted. A silent
|
|
|
272
272
|
check is a claim about two things, and the probe is the one to doubt first — prove
|
|
273
273
|
your edit landed in the text the check actually parses.
|
|
274
274
|
|
|
275
|
+
**Three assertions, and the third is the one hand-rolled probes keep missing.** A
|
|
276
|
+
plant that trips some *other* check has proved that other check works. Three probes
|
|
277
|
+
in one day passed that way: one removed 1 of 3 identical lines and left the shape
|
|
278
|
+
intact; one decremented a number inside an **already-released** section; one deleted
|
|
279
|
+
the shouted spelling of a phrase and left the lowercase one. Each landed somewhere
|
|
280
|
+
real and demonstrated nothing about the guard it was written for. So:
|
|
281
|
+
|
|
282
|
+
1. **the substitution landed** — `replace` matching nothing returns the string
|
|
283
|
+
unchanged and raises nothing;
|
|
284
|
+
2. **the exit code is non-zero** — not a `FAIL` line on stdout;
|
|
285
|
+
3. **the message that fired belongs to the guard under test** — named up front, not
|
|
286
|
+
recognised afterwards.
|
|
287
|
+
|
|
288
|
+
`test/probe.py` does all three (`npm run test:probe` self-tests the harness, including
|
|
289
|
+
its own failure branches, because a harness whose failure path has never executed is
|
|
290
|
+
the thing it exists to stop). Declare the plant as `Plant(label, path, old, new,
|
|
291
|
+
expect=<the guard's own words>)` rather than hand-rolling a fourth copy of the loop.
|
|
292
|
+
|
|
275
293
|
**Record the probe.** One line per section, in the change that ships the check.
|
|
276
294
|
Otherwise the next reader has to redo it to know whether it was ever done.
|
|
277
295
|
|
|
@@ -64,15 +64,20 @@ Rules for the list:
|
|
|
64
64
|
- **The wiki and the graph are optional; the harvest is not.** With no wiki, no
|
|
65
65
|
graph and no doc repos, the harvest is sources 1 and 3–7 and takes two minutes.
|
|
66
66
|
Skipping it is never the answer.
|
|
67
|
-
- **Source 7 is read in full,
|
|
68
|
-
|
|
69
|
-
|
|
67
|
+
- **Source 7 is read in full where it is bounded, and queried where it is not.** The
|
|
68
|
+
standing instructions (capped at ten) and the run stamps (one line each) are read;
|
|
69
|
+
the *Recent log* is queried by the task's nouns like the archive. Measured on the
|
|
70
|
+
file this doctrine was written for, the log was **74% of it** — and it is the one
|
|
71
|
+
section nothing caps. Source 7 is still the source that *binds* the run rather than
|
|
72
|
+
informing it; that is why the part which binds must stay cheap enough to read.
|
|
70
73
|
|
|
71
74
|
## The retro's standing instructions — an instruction source, not background
|
|
72
75
|
|
|
73
76
|
`docs/superpowers/retro.md` ([`retrospective.md`](retrospective.md)) is the one
|
|
74
|
-
harvested source
|
|
75
|
-
instructions are capped at ten precisely so that this is cheap
|
|
77
|
+
harvested source whose binding part is **read in full rather than queried**: the
|
|
78
|
+
standing instructions are capped at ten precisely so that this is cheap, and the run
|
|
79
|
+
stamps are one line each. Its narrative log is queried, not read — an uncapped section
|
|
80
|
+
inside a source that binds is how the capped part stops being read. They are what
|
|
76
81
|
previous runs of this pipeline got wrong *in this project* — the rules no check
|
|
77
82
|
could decide — and they bind this run.
|
|
78
83
|
|
|
@@ -47,6 +47,7 @@ to be enforced and is not is the same failure as a gate that prints `FAIL` and e
|
|
|
47
47
|
| 19 | **An empty measurement is a refused measurement** | any command run to establish a fact — a count, a probe, a query, a suite | assert the output is **non-empty and shaped as expected** before reading meaning into it; a command that printed nothing did not answer | the run quotes the output it read, not the conclusion it drew from silence |
|
|
48
48
|
| 20 | **When a thing exists twice, ask which one is used — not whether they agree** | any artifact with a second copy: two build files, a vendored library, a schema and its mirror, doctrine in two documents | find the **consumer** and read what it names; the copies cannot answer this about themselves | the run names the file the build/test/deploy actually reads, quoted from the consumer |
|
|
49
49
|
| 21 | **A step that consumes what a later step produces is a deadlock, not an ordering** | any sequence where a check reads state another step writes — a prune reading stamps, a gate reading a ledger, a report reading counts | trace each input to the step that writes it; if that step is downstream, the check has never run on real data | every check names the step that produces its input, and that step is upstream of it |
|
|
50
|
+
| 22 | **An operation that changes nothing reports the same as one that changed everything** | a substitution whose needle is absent · a command whose output is suppressed at a decision point · an edit issued against a shape the file does not have | assert the effect, not the call: a replacement must report how many times it matched, and a command whose exit code governs the next step is never piped or silenced | four incidents in two programmes, each invisible until something downstream failed — an import that never landed, a doctrine phrase worded differently, a `gh` call refused behind `>/dev/null`, and a test piped to `head` so that `$?` belonged to `head` |
|
|
50
51
|
|
|
51
52
|
---
|
|
52
53
|
|
|
@@ -189,6 +190,7 @@ answer would have exposed it in a minute.
|
|
|
189
190
|
| 0 Inventory · 9 Docs · any register write | 8 (compute), 14 (targets resolve — including every commit SHA in the retro), 15 (identity before a lease) — see [`documentation.md`](documentation.md) |
|
|
190
191
|
| 0 Harvest · any run resuming from a summary | 16 — the work-list and every inherited state claim re-derived before use, [`knowledge-sources.md`](knowledge-sources.md) → *Carried-in claims* |
|
|
191
192
|
| any check you write | 4, 5, 7, 10, 11 — the procedure is [`gates.md`](gates.md) |
|
|
193
|
+
| **every stage** · any edit, any command whose result is read | 22 — a no-op is indistinguishable from success unless the effect is asserted; never suppress the output of a command a decision depends on |
|
|
192
194
|
| 3 Spec · 4 Plan | 2 (both directions), 8 (compute, never restate) |
|
|
193
195
|
| 5 Dev | 9 (generators seed green), 12 (tests create their own state), 13 (local infra) |
|
|
194
196
|
| 6 Tests | 4, 5, 10, 11 — every new check probed both ways, measured, and asserted on its exit code |
|
|
@@ -264,7 +266,7 @@ evidence that something left, and the log below says what.
|
|
|
264
266
|
|
|
265
267
|
### Retired
|
|
266
268
|
|
|
267
|
-
**Numbers issued so far:
|
|
269
|
+
**Numbers issued so far: 22.** This is the high-water mark, and it is the only number
|
|
268
270
|
this file states about itself — deliberately, because the gap that proves a rule left
|
|
269
271
|
cannot be computed from the table alone: **deleting the highest-numbered rule shrinks
|
|
270
272
|
the maximum with it, and no gap ever opens.** That false negative shipped in the first
|
|
@@ -18,7 +18,8 @@ justifies reading it protects one section while the file below it doubles.
|
|
|
18
18
|
|
|
19
19
|
| Artifact | Parts | How it is read |
|
|
20
20
|
|---|---|---|
|
|
21
|
-
| `docs/superpowers/retro.md` — **one per project** | **Standing instructions** (max **10**) · **
|
|
21
|
+
| `docs/superpowers/retro.md` — **one per project** | **Standing instructions** (max **10**) · **Run stamps** (one line each) | stage 0, **in full** — both are bounded by construction |
|
|
22
|
+
| the same file's **Recent log** | entries from the last five run stamps — narrative, and capped by nothing | stage 0, **queried** by the task's nouns. It said *in full* until 2026-08-10, when it measured **74%** of the file: an uncapped section inside a binding source is what makes the capped part get skimmed |
|
|
22
23
|
| `docs/superpowers/retro/YYYY-QN.md` — the archive | every entry and every retirement ever written, append-only | **queried** by the task's nouns; never read end to end |
|
|
23
24
|
|
|
24
25
|
Seed the archive from [`../templates/retro-archive.md`](../templates/retro-archive.md).
|
|
@@ -16,6 +16,7 @@ install.
|
|
|
16
16
|
- Reviewer inputs
|
|
17
17
|
- Controller rules
|
|
18
18
|
- The rubric
|
|
19
|
+
- The independent reader — dispatched by a stage, and read by its output
|
|
19
20
|
- Prompt — task review
|
|
20
21
|
- Prompt — scoped re-review
|
|
21
22
|
- Prompt — final whole-branch review
|
|
@@ -118,6 +119,61 @@ Review in this order; stop reading the diff only when you've covered all of it.
|
|
|
118
119
|
Formatting nits that don't change meaning are not findings. Praise is not a
|
|
119
120
|
finding either.
|
|
120
121
|
|
|
122
|
+
## The independent reader — dispatched by a stage, and read by its output
|
|
123
|
+
|
|
124
|
+
**One job: stop *"a reader was requested"* from standing in for *"a reader
|
|
125
|
+
reported"*.** Those are different facts and they look identical in a transcript.
|
|
126
|
+
|
|
127
|
+
The rule this implements has been a standing instruction since 2026-08-08: **a change
|
|
128
|
+
that adds or widens a check gets an independent reader before merge**, because your own
|
|
129
|
+
probes exercise only the shapes you already thought of. Its retirement condition, written
|
|
130
|
+
at birth, was that the reader be *dispatched by a stage rather than by the repository
|
|
131
|
+
happening to run a bot on pull requests*. This section is that stage.
|
|
132
|
+
|
|
133
|
+
**It happened.** Four pull requests of almost nothing but check work were opened on one
|
|
134
|
+
day; the review application reported **`skipping`** on every one, and twenty-two new
|
|
135
|
+
guards merged with the author's own probes as their only reading. Nothing was violated.
|
|
136
|
+
Nothing read the reviewer's output either.
|
|
137
|
+
|
|
138
|
+
### Who counts as the reader, in order
|
|
139
|
+
|
|
140
|
+
1. **A subagent this run dispatches** — the only option whose execution the run can
|
|
141
|
+
observe end to end. Give it the diff package, the rubric, and the sentence *"the
|
|
142
|
+
author wrote both the check and its probes; find what neither could see."*
|
|
143
|
+
2. **A review bot on the pull request** — acceptable, and it is a **third party with no
|
|
144
|
+
contract**: it may skip, rate-limit, or answer about a stale commit. Read its
|
|
145
|
+
verdict, not the fact that it was triggered.
|
|
146
|
+
3. **A person.**
|
|
147
|
+
|
|
148
|
+
### The three states, and one of them is not silence
|
|
149
|
+
|
|
150
|
+
Record exactly one, beside the gate verdict:
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
reader: 6 findings, 4 confirmed — read, and it had something
|
|
154
|
+
reader: none found — read, and it did not
|
|
155
|
+
reader: NO READER — <why> — nobody read it
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
**`NO READER` is a printed state, never an omission.** It carries the same law as
|
|
159
|
+
`dormant` and `skip` ([`gates.md`](gates.md) → *Progressive arming*): a mechanism that
|
|
160
|
+
prints nothing when it looked at nothing is indistinguishable from one that looked and
|
|
161
|
+
found nothing. Where the change added or widened a check, `NO READER` is also carried
|
|
162
|
+
into the close-out as an open requirement rather than a footnote — reporting a gap is
|
|
163
|
+
honest and is not a fix.
|
|
164
|
+
|
|
165
|
+
### What the reader is asked
|
|
166
|
+
|
|
167
|
+
Not *"review this."* The dispatch names the blind spot it exists to cover:
|
|
168
|
+
|
|
169
|
+
- the probes and the checks were written from **one model of the problem** — where does
|
|
170
|
+
that model not reach?
|
|
171
|
+
- which shapes does the check's **scope** exclude, and is any of them shipped?
|
|
172
|
+
- is there a form of the defect that has **no pair to compare** — a single cell, a lone
|
|
173
|
+
directive, an absence?
|
|
174
|
+
|
|
175
|
+
Those three questions come from findings no author-written probe has ever produced here.
|
|
176
|
+
|
|
121
177
|
## Prompt — task review
|
|
122
178
|
|
|
123
179
|
> You are reviewing one task of an implementation plan. Read, in order:
|
|
@@ -101,9 +101,11 @@ never that the work was skipped quietly.
|
|
|
101
101
|
when one is built ([`knowledge-graph.md`](knowledge-graph.md): `graphify query` /
|
|
102
102
|
`affected` / `god-nodes` answer *reach*, which is what grep cannot), `CLAUDE.md`,
|
|
103
103
|
`CONTEXT.md`/ADRs, `docs/` + `docs/ux/`, past pipeline briefs and carry-over
|
|
104
|
-
ledgers, **the retro's standing instructions** (`docs/superpowers/retro.md`,
|
|
105
|
-
**in full** —
|
|
106
|
-
fires
|
|
104
|
+
ledgers, **the retro's standing instructions and run stamps** (`docs/superpowers/retro.md`,
|
|
105
|
+
read **in full** — ten rows and one line per run, both bounded, and they bind this
|
|
106
|
+
run; stamp each instruction as it fires. Its *Recent log* is **queried** by the
|
|
107
|
+
task's nouns, not read: uncapped narrative inside a binding source is what makes the
|
|
108
|
+
capped part get skimmed, [`retrospective.md`](retrospective.md)), the **knowledge wiki** if one is
|
|
107
109
|
installed
|
|
108
110
|
([obsidian-wiki](https://github.com/ar9av/obsidian-wiki) — recommended,
|
|
109
111
|
never required), and any **other repo or hosted doc system the project names as
|
|
@@ -456,6 +458,15 @@ never that the work was skipped quietly.
|
|
|
456
458
|
tagging, the CI verdict for what was just pushed is READ, not assumed**
|
|
457
459
|
([`conventions.md`](conventions.md) → *The CI verdict*) — a tag on a commit whose
|
|
458
460
|
run nobody read is how a red `main` ships.
|
|
461
|
+
- **The independent reader is dispatched by this stage, and read by its output**
|
|
462
|
+
([`review.md`](review.md) → *The independent reader*). On any change that adds or
|
|
463
|
+
widens a check, the run dispatches a reader — a subagent it can watch, a bot whose
|
|
464
|
+
**verdict** it then reads, or a person — and records exactly one of three states
|
|
465
|
+
beside the verdict: `reader: N findings`, `reader: none found`, or `reader: NO READER
|
|
466
|
+
— <why>`. The third is printed, never omitted; a requested reader and a reading are
|
|
467
|
+
different facts that look identical afterwards. Four pull requests of check work once
|
|
468
|
+
merged on a bot's `skipping` with nobody noticing, which is why this is a stage rather
|
|
469
|
+
than a hope.
|
|
459
470
|
- **The review loop that lives here has a cap, and the cap is a measurement**
|
|
460
471
|
([`loop-guard.md`](loop-guard.md) → *The review loop*). **3 rounds** per artifact by
|
|
461
472
|
default (`pipeline.json` → `run.review.maxRounds`); at the cap the run stops reviewing
|
|
@@ -17,20 +17,32 @@ than no boundary.
|
|
|
17
17
|
|
|
18
18
|
---
|
|
19
19
|
|
|
20
|
-
## Routing —
|
|
20
|
+
## Routing — work that lands in the repository goes through the pipeline
|
|
21
21
|
|
|
22
|
-
**When `task-pipeline` is installed, any work that
|
|
23
|
-
through it** — without being asked for.
|
|
24
|
-
integration, a rewrite, an adoption, a hardening pass; in any language and any
|
|
25
|
-
phrasing. Saying *"run this through the pipeline"* is an accelerator, not a
|
|
26
|
-
precondition.
|
|
22
|
+
**When `task-pipeline` is installed, any work that LANDS IN THE REPOSITORY goes
|
|
23
|
+
through it** — without being asked for. Two shapes, not one:
|
|
27
24
|
|
|
28
|
-
**
|
|
29
|
-
|
|
25
|
+
- **a change** — a feature, a fix, a refactor, a migration, an integration, a rewrite,
|
|
26
|
+
an adoption, a hardening pass;
|
|
27
|
+
- **a finding that lands** — an audit, a bug hunt, a production check, a PR review. Its
|
|
28
|
+
output is not an answer in the chat: it is REQ rows, board rows, ledger rows and
|
|
29
|
+
fixes, and the pipeline is what carries them there.
|
|
30
30
|
|
|
31
|
-
|
|
31
|
+
In any language and any phrasing. Saying *"run this through the pipeline"* is an
|
|
32
|
+
accelerator, not a precondition.
|
|
33
|
+
|
|
34
|
+
**The boundary is what the request ENDS IN, and it cuts both ways.** An answer stops in
|
|
35
|
+
the conversation; a change or a finding lands in the tree. Not through the pipeline:
|
|
36
|
+
|
|
37
|
+
- a question and its answer, an explanation, mapping code so a person can read it;
|
|
32
38
|
- a typo, a one-line edit, a mechanical rename;
|
|
33
|
-
- reconnaissance or measurement that
|
|
39
|
+
- reconnaissance or measurement that lands nothing.
|
|
40
|
+
|
|
41
|
+
**Reading is not the test.** An audit, a bug hunt and a PR review all *begin* by reading,
|
|
42
|
+
and all three end in the repository. Excluding them because they read is how a skill with
|
|
43
|
+
a whole doctrine file for audits (`references/audit.md`) never gets reached by the word
|
|
44
|
+
*audit* — measured, not supposed: three of ten routing queries were refused with this
|
|
45
|
+
rule's own exclusion quoted back as the reason.
|
|
34
46
|
|
|
35
47
|
Running ten gated stages for one character is the fastest way to teach an agent to
|
|
36
48
|
route around the pipeline entirely.
|