task-pipeline-skill 1.81.1 → 1.84.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +271 -0
- package/SKILL-CARD.md +1 -1
- package/package.json +1 -1
- package/plugins/task-pipeline/.claude-plugin/plugin.json +1 -1
- package/plugins/task-pipeline/skills/evidence-docs/SKILL.md +3 -0
- package/plugins/task-pipeline/skills/project-audit/SKILL.md +75 -6
- package/plugins/task-pipeline/skills/project-audit/scripts/audit.py +79 -9
- package/plugins/task-pipeline/skills/task-pipeline/references/certification.md +40 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/learned.md +13 -3
- package/plugins/task-pipeline/skills/task-pipeline/references/prioritisation.md +22 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/retrospective.md +29 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,274 @@
|
|
|
1
|
+
## v1.84.0 — fifteen filed findings, and the two rules they mostly refine
|
|
2
|
+
|
|
3
|
+
Every one of the fifteen `ssheleg/sshlg-skills` issues this repository owned, closed in one
|
|
4
|
+
release because they are one subject: **what a check is allowed to conclude.**
|
|
5
|
+
|
|
6
|
+
**Ten new rules in `references/learned.md` (22 → 32), and two existing ones strengthened
|
|
7
|
+
rather than duplicated** — the distinction matters, because a table that grows a row for
|
|
8
|
+
every refinement stops being readable at exactly the point it starts being complete.
|
|
9
|
+
|
|
10
|
+
- rule **4** (probe the detector) now demands the plant assert it **changed the file**, and
|
|
11
|
+
that the expected verdict be stated before the run — a green suite and an unlanded plant
|
|
12
|
+
are the same output (`#94`);
|
|
13
|
+
- rule **8** (compute, never restate) now demands the matched items be printed classified
|
|
14
|
+
beside the count — *eighteen lines containing these three digits* and *eighteen throttle
|
|
15
|
+
events* are different claims, and only one was true (`#79`).
|
|
16
|
+
|
|
17
|
+
The ten new ones, each carrying the incident it came from: a commit reference is a
|
|
18
|
+
prediction until the object resolves (`#80`); a guard outliving the name it guards goes
|
|
19
|
+
vacuous rather than red (`#81`); an item leaving a ranked list is not evidence about that
|
|
20
|
+
item (`#87`); a model too small to reproduce cannot certify (`#89`); a sweep states its
|
|
21
|
+
denominator before it starts (`#90`); guard capability, not incompleteness (`#91`); a
|
|
22
|
+
census over a localised corpus reports per locale (`#92`); a gate scoped by exclusion
|
|
23
|
+
judges every newcomer silently (`#93`); a gate needs a third verdict — *could not measure*
|
|
24
|
+
(`#95`); a test whose assertion is implied by compilation is not a test (`#97`).
|
|
25
|
+
|
|
26
|
+
**Three sub-canons in `evidence-docs`**, placed under the canons they refine rather than as
|
|
27
|
+
an eleventh: **1a** a token that looks like evidence is not evidence until resolved; **2a**
|
|
28
|
+
a number out of the shell is not a measurement until you look at what matched; **5a** a
|
|
29
|
+
model too small to reproduce the defect is too small to prove the fix.
|
|
30
|
+
|
|
31
|
+
**`prioritisation.md`** gains the half the existing rule was silent about (`#82`): the
|
|
32
|
+
finder still may not rank by effort, the **fixer** weighs cost against *measured* harm, a
|
|
33
|
+
zero-harm row is priced latent rather than deleted, and where the path is irreversible cost
|
|
34
|
+
loses.
|
|
35
|
+
|
|
36
|
+
**`retrospective.md`** gains two (`#85`, `#88`): what the prune does when it **cannot**
|
|
37
|
+
run — an undefined case is where the spread between a careful run and a careless one lives
|
|
38
|
+
— and that a fix to one call site closes a call site, not a class, with the sibling census
|
|
39
|
+
taken by behaviour rather than from a list of names.
|
|
40
|
+
|
|
41
|
+
**The high-water mark is derived now**, not typed: the validator caught `Numbers issued so
|
|
42
|
+
far: 22` against a table reaching 32, which is the same defect these rules are about.
|
|
43
|
+
|
|
44
|
+
Guards: 424 → **424** — doctrine, no mechanism.
|
|
45
|
+
|
|
46
|
+
## v1.83.1 — the tag that could not be moved, and why that is right
|
|
47
|
+
|
|
48
|
+
**No change to the skill.** `v1.83.0`'s tree carried a run stamp with `—` where its
|
|
49
|
+
commit belongs, so the release gate refused it on the tag's own tree — correctly, and in
|
|
50
|
+
the only place that can see it, because `validate.yml` ignores tag pushes.
|
|
51
|
+
|
|
52
|
+
The stamp names `48544ac` now. **The tag was not moved**: a repository rule refuses to
|
|
53
|
+
delete or re-point one, and that rule is right. A tag is a promise about a tree, the tree
|
|
54
|
+
at `v1.83.0` genuinely lacked the stamp, and quietly re-cutting the same version over a
|
|
55
|
+
different tree would make every future reader of that tag wrong about what it contained.
|
|
56
|
+
So the repair is a version, not a rewrite.
|
|
57
|
+
|
|
58
|
+
This is `ssheleg/sshlg-skills#80` met from the inside on the run that closed its
|
|
59
|
+
neighbours: **a commit reference is written after the commit exists.** The record ships
|
|
60
|
+
in a following commit rather than claiming a sha that did not exist when it was written.
|
|
61
|
+
|
|
62
|
+
Guards: 424 → **424** — no mechanism changed.
|
|
63
|
+
|
|
64
|
+
## v1.83.0 — an audit report is asked for, and four findings that had already cost something
|
|
65
|
+
|
|
66
|
+
`project-audit`, closing `ssheleg/sshlg-skills#77`, `#78`, `#83`, `#84`.
|
|
67
|
+
|
|
68
|
+
**A report is written when somebody wants a report.** The run wrote
|
|
69
|
+
`docs/audit/<date>-audit.html` and opened it every time; it now writes the page only
|
|
70
|
+
with **`--report`** (`--no-open` beside it to write without opening). The split is not
|
|
71
|
+
symmetry: the **sidecar stays unconditional**, because `carry_forward` reads the previous
|
|
72
|
+
one and skipping it would silently turn every future run into a first run. The page is
|
|
73
|
+
different — a report outlives the conversation, and one nobody asked for is untracked
|
|
74
|
+
HTML under `docs/`, one `git add -A` from the product's history.
|
|
75
|
+
|
|
76
|
+
**`#83` — the telemetry probe read manifests and asserted about the product.** A project
|
|
77
|
+
with a working home-grown error channel and no SDK dependency was reported as *"no error
|
|
78
|
+
reporting found"*. The evidence is ordered and cheapest-first now, stopping at the first
|
|
79
|
+
hit: a manifest dependency, a module named for the capability, a table or migration named
|
|
80
|
+
for it, a mention in a documentation entry point. **And the KIND of evidence decides the
|
|
81
|
+
verdict** — code or schema is `clean` with the channel named; a document alone is
|
|
82
|
+
`blind`, because it establishes neither that the channel works nor that it is absent.
|
|
83
|
+
The remedy sentence survives, which was the valuable half.
|
|
84
|
+
|
|
85
|
+
**`#77` — a finding carries its consequence, or it is a hypothesis.** One audit had
|
|
86
|
+
**eight consecutive rows** rewritten by measuring against production, two of which would
|
|
87
|
+
have destroyed inventory if remedied as written, because both read an absence of sales as
|
|
88
|
+
an absence of demand. A row now states how often the mechanism fires, or that it never
|
|
89
|
+
has — in which case it is priced as **latent** and the remedy weighed against zero — or
|
|
90
|
+
that the measurement is impossible, which is a `blind` on the consequence and not a
|
|
91
|
+
finding.
|
|
92
|
+
|
|
93
|
+
**`#78` — already decided is not a finding, and not nothing either.** Five of eight rows
|
|
94
|
+
in one run reported decisions the project had already made, in the project's own words,
|
|
95
|
+
because the decision sits a few lines from the cited code. The call site is read before a
|
|
96
|
+
row is written, every row states which of three it is (undecided / decided here / decided
|
|
97
|
+
elsewhere and not propagated), and where it is the third the remedy is **a mechanical
|
|
98
|
+
check rather than an edit** — a written rule nobody verifies reaches exactly as far as
|
|
99
|
+
the place it was written.
|
|
100
|
+
|
|
101
|
+
**`#84` — an ordered preflight before any datastore read.** Resolve the store from the
|
|
102
|
+
platform's own attachment rather than from a script or a document; prove liveness with a
|
|
103
|
+
freshest-write aggregate and put that timestamp beside every figure it underwrites; name
|
|
104
|
+
every attached store with its freshness; and treat the repository's connect helper as
|
|
105
|
+
evidence about the repository, which is a finding of its own when the two disagree.
|
|
106
|
+
|
|
107
|
+
Guards: 424 → **424** — this release adds no planted-defect guard. The three new checks
|
|
108
|
+
live in `test/project_audit_test.py`, which `validate.yml` runs as an ordinary step
|
|
109
|
+
rather than as one of its 429 `Negative self-test` steps, and the floor
|
|
110
|
+
(`negatives.MIN_EXPECTED`) is unchanged. The new checks were still each watched failing
|
|
111
|
+
against the un-fixed code, which is the property the count exists to protect and not the
|
|
112
|
+
count itself.
|
|
113
|
+
|
|
114
|
+
**One defect of my own, caught before it shipped:** the widened probe's first draft
|
|
115
|
+
guarded with `hasattr(ctx, "excluded")` against a method that does not exist, producing a
|
|
116
|
+
filter that was permanently false and silently did nothing. It reads `ctx.out_rel`, which
|
|
117
|
+
is the real field.
|
|
118
|
+
|
|
119
|
+
## v1.82.4 — the plant the harness could not see
|
|
120
|
+
|
|
121
|
+
`v1.82.3` is a dead tag, and its cause is the **fourth distinct one**. Its stamp was
|
|
122
|
+
correct and in the right order: the payload merged as `9e2ab3e`, the stamp cited it and
|
|
123
|
+
landed as `e4e0eb4`, the tag was cut there, and the gate had accepted that tree.
|
|
124
|
+
|
|
125
|
+
What failed was a negative self-test added in the same programme. `test/negatives.py`
|
|
126
|
+
locates a plant's scratch directory by searching its script for a literal
|
|
127
|
+
`cp -R . <dir>`, then diffs that copy against the repository to tell a real defect from a
|
|
128
|
+
no-op. The duplicate-id plant copied **one file** and mutated the live tree, restoring it
|
|
129
|
+
afterwards — correct as a workflow step, invisible to the harness. With no scratch
|
|
130
|
+
directory it could not confirm the plant had landed, and reported the guard as one that
|
|
131
|
+
**did not fire**. Correctly, by its own rule.
|
|
132
|
+
|
|
133
|
+
The plant now follows the convention every other one uses: copy the tree, mutate the
|
|
134
|
+
copy, run the validator from the copy, leave it mutated. Verified with the harness itself
|
|
135
|
+
before shipping — `PASS: a board id naming two rows must fail`, the step that had never
|
|
136
|
+
been run locally across four attempts.
|
|
137
|
+
|
|
138
|
+
**`npm test` cannot see this**, because `negatives.py` is a CI step and not part of the
|
|
139
|
+
suite. Four dead tags, four different mechanisms — a stamp in the wrong shape, a tag cut
|
|
140
|
+
before the stamp existed twice, and now a plant the harness cannot verify — and one
|
|
141
|
+
shared shape behind all of them: a green local gate that does not cover what the release
|
|
142
|
+
runs. `B-134` carries it, and now asks for a preflight that runs the plant harness too.
|
|
143
|
+
|
|
144
|
+
Guards: 424 → **424** — the plant is repaired, not added.
|
|
145
|
+
|
|
146
|
+
**Landed 2026-09-02, and the release-gap check refused this tree first.** The repair above
|
|
147
|
+
was green on every branch check and still declared no stamp for the release it was
|
|
148
|
+
creating — a fifth instance of the same shape, caught because the tag was cut locally and
|
|
149
|
+
`npm run test:all` was run against the tag's own tree before anything was pushed. The
|
|
150
|
+
preflight `B-134` asks for is therefore not a new check: it is `test:all` on that tree,
|
|
151
|
+
which is the only tree where four of the five failures can be seen at all. `npm test` runs
|
|
152
|
+
three of its thirteen.
|
|
153
|
+
|
|
154
|
+
## v1.82.3 — the release protocol is two merges, and their order is written down now
|
|
155
|
+
|
|
156
|
+
**`v1.82.2` is a dead tag for the same reason `v1.82.1` was, and the run that burned it
|
|
157
|
+
had just written the rule into this file.** A release here needs a stamp on the tag's own
|
|
158
|
+
tree; the declaration v1.82.2 carried was for v1.82.1, the tag already dead. The payload
|
|
159
|
+
merged as `9195590`, the tag was cut there, and nothing in that tree stamped the run.
|
|
160
|
+
|
|
161
|
+
**The mechanism, stated so nobody pays for it a fourth time.** A stamp must cite a commit
|
|
162
|
+
inside the NEW tag's own range, and that commit does not exist until the payload merges —
|
|
163
|
+
the ruleset rebase-merges and mints every SHA at merge time. So a release is **two merges
|
|
164
|
+
and their order is load-bearing**: the payload lands as `X`, a second change stamps the
|
|
165
|
+
run citing `X` and lands as `Y`, and the tag is cut on `Y`.
|
|
166
|
+
|
|
167
|
+
**Nothing refuses the wrong order before the tag exists.** `validate.yml` ignores tag
|
|
168
|
+
pushes; `release.yml` is the first thing to run on the tag's own tree. So the branch is
|
|
169
|
+
green either way and the cost is always a burned version, because the ruleset forbids
|
|
170
|
+
deleting a tag. Three in one day: `v1.82.0` (stamp written in the sibling repository's
|
|
171
|
+
bullet shape into a section this gate reads as a table), `v1.82.1` and `v1.82.2` (tag cut
|
|
172
|
+
on the payload commit). That gap is now `B-134`, which asks for a preflight that resolves
|
|
173
|
+
the range the tag would create and asserts a stamp falls inside it — a check that can run
|
|
174
|
+
*before* anything is spent.
|
|
175
|
+
|
|
176
|
+
Guards: 424 → **424** — this release records a protocol and adds no mechanism; the
|
|
177
|
+
mechanism it asks for is the board row.
|
|
178
|
+
|
|
179
|
+
## v1.82.2 — one id, one finding, and a status read from the wrong place
|
|
180
|
+
|
|
181
|
+
**A board id identified two findings for the whole of v1.82.x.** `B-114` and `B-115`
|
|
182
|
+
each named two rows: the pair filed on 2026-08-31 from the harness-engineering
|
|
183
|
+
articles, and the pair this programme appended on 2026-09-01 without reading what was
|
|
184
|
+
already there. Four rows, two ids — so a run stamp saying *"B-115 filed"*, a CHANGELOG
|
|
185
|
+
naming the range `B-114…B-117`, and a sibling board citing either one all pointed at
|
|
186
|
+
two different findings at once. The newer pair is renumbered `B-119` and `B-120`, and
|
|
187
|
+
the stamp that cited them is repointed in the same change.
|
|
188
|
+
|
|
189
|
+
**The guard, because renumbering fixes the case and not the class.** `test/validate.py`
|
|
190
|
+
collected board ids into a **set**, where a repeat collapses leaving no trace — which
|
|
191
|
+
is precisely why this shipped and stayed invisible. The new check reads row-leading ids
|
|
192
|
+
only (`| B-NNN |`, first cell), so a prose citation of another row stays a citation
|
|
193
|
+
rather than becoming a second definition, and it names both line numbers in its
|
|
194
|
+
refusal. Watched failing before it shipped: a planted collision returns
|
|
195
|
+
*"docs/evidence/backlog.md:110: id B-119 identifies a second row (the first is line
|
|
196
|
+
109)"*. The umbrella has carried this guard for its own board; this repository is where
|
|
197
|
+
the defect actually landed, and it had none. Guards: 423 → **424**.
|
|
198
|
+
|
|
199
|
+
**Shipped as 1.82.2 because `v1.82.1` is a dead tag.** Its tree carried no stamp for its
|
|
200
|
+
own run, so the release suite — which runs on the tag's own tree, where `validate.yml`
|
|
201
|
+
never looks — refused it. The round was not lost to skipping the check: the gate **was**
|
|
202
|
+
run against the merge commit, and its exit status was read from a trailing `echo` in the
|
|
203
|
+
same command rather than from the suite, so a zero belonging to the `echo` was taken for
|
|
204
|
+
the suite's verdict. An explicit re-run returned 1 seconds later. The rule from v1.82.0
|
|
205
|
+
therefore gains its second half: verify with the gate rather than a grep of your own
|
|
206
|
+
prose, **read the gate's own exit status**, and cut the tag locally, run the suite on the
|
|
207
|
+
tag's tree, and push only then — a local tag can be moved, a pushed one cannot.
|
|
208
|
+
|
|
209
|
+
## v1.82.1 — the second axis of blindness, and the scope the two-severity rule never had
|
|
210
|
+
|
|
211
|
+
**This release is 1.82.0 and not 1.81.0, and the reason is worth recording.** 1.81.0 was
|
|
212
|
+
already in flight in a concurrent session — the anchors-are-derived release — and this run
|
|
213
|
+
picked the same number by reading `package.json` at branch time without asking whether
|
|
214
|
+
anyone else held it. Worse, the tag was pushed **immediately after a `gh pr merge` that had
|
|
215
|
+
printed a conflict and not merged**, so `v1.81.0` landed on the other session's commit and
|
|
216
|
+
started its release before its run stamp was written. That release failed on exactly the
|
|
217
|
+
guard that exists for it — *a release named nowhere in `Releases that carry no stamp`* — and
|
|
218
|
+
published nothing. The tag cannot be deleted (a repository rule refuses it), so 1.81.0 stays
|
|
219
|
+
spent and this work moves up. Standing instruction #8 in the umbrella's retro covers the
|
|
220
|
+
mechanism: a wrapper's exit status is not the verdict, and it was not read.
|
|
221
|
+
|
|
222
|
+
**Shipped as 1.82.1 because `v1.82.0` is a dead tag, and the reason is a check that read the
|
|
223
|
+
wrong thing.** The run stamp for this release was written as a **list item** — the shape the
|
|
224
|
+
sibling `sshlg-skills` uses — into a section this repository keeps as a **table**. The release
|
|
225
|
+
suite therefore found no stamp covering the tag, refused it, and published nothing. Worse than
|
|
226
|
+
the format slip: the verification run before tagging grepped for the stamp's own title and
|
|
227
|
+
reported it present. **That confirmed the text existed, not that the gate could read it.** The
|
|
228
|
+
tag cannot be deleted, so it is declared in `retro.md`'s no-stamp section and the payload moves
|
|
229
|
+
one patch up.
|
|
230
|
+
|
|
231
|
+
Guards: 423 → **423** — this release adds doctrine and no mechanism. Both findings argue
|
|
232
|
+
against a rule rather than against a missing check, and the vertical blindness rule is
|
|
233
|
+
explicitly one `certify` cannot verify from a report.
|
|
234
|
+
|
|
235
|
+
**`certification.md` decoupled the three readers from each other and never decoupled the
|
|
236
|
+
fixer from the grader.** Tier blindness is horizontal — no tier reads another's report, and
|
|
237
|
+
`certify` refuses a report whose prose cites another tier. Nothing said the *vertical* half:
|
|
238
|
+
the agent that produced a change may not be the agent that certifies it.
|
|
239
|
+
|
|
240
|
+
That gap matters because the optimisation which closes it is attractive. The fixer already
|
|
241
|
+
holds the context; a fresh reader has to re-derive it. Taking that saving converts
|
|
242
|
+
certification into self-assessment **while every report it writes stays sincere** — an
|
|
243
|
+
optimiser that grades itself learns to game the metric rather than improve the work, and
|
|
244
|
+
`certify` cannot detect the collapse from a report. It is now a dispatch rule beside the
|
|
245
|
+
one it belongs with.
|
|
246
|
+
|
|
247
|
+
**And the two-severity rule shipped without its scope.** *There are two severities and no
|
|
248
|
+
third, because a certification that admits a maybe admits everything* is correct — for a
|
|
249
|
+
**finding**, which is a claim a reader makes about a diff and either blocks or does not.
|
|
250
|
+
It is the wrong shape for the **verdict of a check** whose subject is non-deterministic,
|
|
251
|
+
where binary pass/fail has been measured at **0% detection** of regressions and trace
|
|
252
|
+
fingerprinting at **86%**. A third value there is not laxity: `INCONCLUSIVE`, grounded in
|
|
253
|
+
hypothesis testing, is how a run says *this sample cannot decide* instead of flipping a coin
|
|
254
|
+
and reporting the result as a severity. Sequential testing cuts the trials it needs by
|
|
255
|
+
**78%**.
|
|
256
|
+
|
|
257
|
+
So the rule now carries its boundary: **two severities for a finding, three verdicts for a
|
|
258
|
+
stochastic check**, with the second axis pointed at `agent-stack`'s `agent-evals` —
|
|
259
|
+
`references/statistics.md` for how many runs make a difference real — and at the skill's
|
|
260
|
+
`references/gates.md` → *Axis A* for where such a gate belongs.
|
|
261
|
+
|
|
262
|
+
**A fourth invisible class, with a measurement.** *Why one verifier is not enough* listed
|
|
263
|
+
three defects a diff-reader cannot see; the fourth is the one a blended score cannot see
|
|
264
|
+
either: **the internal state is right and the message about it is stale.** The value was
|
|
265
|
+
stored correctly and the final sentence to the user echoed the old one — passing on state,
|
|
266
|
+
failing on truth, so any grader that averages the two calls it mostly fine. It was caught by
|
|
267
|
+
a rubric with categorical outcomes rather than a number, and closing it moved the violation
|
|
268
|
+
rate from **21% to 4%**. Its general form is this pipeline's own worst case: work that
|
|
269
|
+
**looks like it is working** while quietly missing what was asked — which is the product
|
|
270
|
+
tier's whole job, and why the tier that reads no code is not the soft one.
|
|
271
|
+
|
|
1
272
|
# Changelog
|
|
2
273
|
|
|
3
274
|
## v1.81.1 — the anchors are derived, and a dormant plant stops reading as a pass
|
package/SKILL-CARD.md
CHANGED
|
@@ -12,7 +12,7 @@ harmless.
|
|
|
12
12
|
|---|---|
|
|
13
13
|
| **Purpose** | Runs a substantial task through ten gated delivery stages — intake grill, docs study, brainstorm, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs+registers, acceptance — refusing to advance until each gate passes |
|
|
14
14
|
| **Owner** | ssheleg ([github.com/ssheleg/task-pipeline](https://github.com/ssheleg/task-pipeline)) |
|
|
15
|
-
| **Version** | 1.
|
|
15
|
+
| **Version** | 1.84.0 |
|
|
16
16
|
| **Surface** | Claude Code (filesystem skill + plugin) and the vercel `skills` CLI. **Not** uploaded to the Skills API; custom Skills do not sync across surfaces |
|
|
17
17
|
| **Dependencies** | None required. Optional: `context7` (MCP), `figma` (MCP), super-ux, agent-sync, graphify, obsidian-wiki, and **one of two browser channels** — `playwright` (CLI or MCP) or `chrome-devtools` (MCP); either satisfies the browser step and neither is required. Every stage's doctrine ships in-repo; the one conditional requirement is super-ux for the stage-3 UX track on a user-facing task |
|
|
18
18
|
| **Evaluation status** | Suite authored, 5 categories. One recorded run, **self-observed by the author**; **zero blind runs on zero of three models** — the split, and the numbers, live in [`evals/RESULTS.md`](evals/RESULTS.md) and are computed by `evals/run.py` |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "task-pipeline-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.84.0",
|
|
4
4
|
"description": "Full-cycle delivery pipeline for coding agents: a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine ships inside the skill — no companion plugin required. This package is the installer CLI.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"task-pipeline": "bin/task-pipeline.js"
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"name": "task-pipeline",
|
|
4
4
|
"displayName": "Task Pipeline",
|
|
5
5
|
"description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/judgment/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
|
|
6
|
-
"version": "1.
|
|
6
|
+
"version": "1.84.0",
|
|
7
7
|
"author": {
|
|
8
8
|
"name": "ssheleg",
|
|
9
9
|
"url": "https://x.com/sshlg93"
|
|
@@ -16,10 +16,13 @@ The full statement of each canon, its rationale and its enforcement live in
|
|
|
16
16
|
## The ten canons
|
|
17
17
|
|
|
18
18
|
1. **A claim carries its address** — `file:line`, a command with its output, a test name; a lesson names its commit.
|
|
19
|
+
- **1a. A token that looks like evidence is not evidence until it has been resolved.** A sha, a line number, a path, a flag, a test name: every one can be plausible and absent, and each is trusted at exactly the moment nobody re-reads it. A commit reference is filled **after** the commit exists and checked with a command.
|
|
19
20
|
2. **Numbers are computed, never restated.**
|
|
21
|
+
- **2a. A number out of the shell is not a measurement until you have looked at what matched.** Print the matched items classified beside the count whenever the token can occur inside something else, and never pipe a command whose exit status governs what happens next — the shell then answers about the pager. *Eighteen lines containing these three digits* and *eighteen throttle events* are different claims.
|
|
20
22
|
3. **Every fact has exactly one home** — others link, never restate.
|
|
21
23
|
4. **A reference resolves from where the document is read** — not from where it lives.
|
|
22
24
|
5. **Green nobody watched turn red is not evidence.**
|
|
25
|
+
- **5a. A model too small to reproduce the defect is too small to prove the fix.** Reproduce first, fix second: a harness that goes green on the unfixed input is not a harness. Where the model is smaller than the subject, the gap is stated as a list of what it omits, with numbers.
|
|
23
26
|
6. **A check proves its scope and nothing beyond it.**
|
|
24
27
|
7. **Silence is not a pass** — ask what a mechanism prints when it did not look.
|
|
25
28
|
8. **An estimate is never announced as a measurement** — a rule states its evidence condition.
|
|
@@ -84,6 +84,21 @@ cannot admit about itself:
|
|
|
84
84
|
blind. Read aggregates and pointers, never raw bodies (see *The two
|
|
85
85
|
artefacts*).
|
|
86
86
|
|
|
87
|
+
**Before ANY datastore read, in this order** — a figure from the wrong database is
|
|
88
|
+
worse than no figure, because it is quoted with the authority of a measurement:
|
|
89
|
+
|
|
90
|
+
1. **resolve the datastore from the platform's own attachment**, not from a script
|
|
91
|
+
in the repository and not from a name in the documentation;
|
|
92
|
+
2. **prove liveness with a freshest-write aggregate** against a table the product
|
|
93
|
+
writes constantly, and put that timestamp in the report beside every figure it
|
|
94
|
+
underwrites;
|
|
95
|
+
3. where **more than one** datastore is attached, name them all with their
|
|
96
|
+
freshness — an audit that reads one and does not mention the other cannot be
|
|
97
|
+
reproduced by its reader;
|
|
98
|
+
4. the repository's connect helper is **evidence about the repository**, never about
|
|
99
|
+
the platform. Where the two disagree that is a finding of its own, and it is the
|
|
100
|
+
one this rule was written from.
|
|
101
|
+
|
|
87
102
|
### 4. Seams — hand off to the ladder
|
|
88
103
|
|
|
89
104
|
Now, and only now, walk `audit.md`'s ladder over the capabilities the discovery
|
|
@@ -91,9 +106,10 @@ found. Bottom-up, seam-ordered. Absences found here are findings like any other.
|
|
|
91
106
|
|
|
92
107
|
### 5. Report — two files, one command
|
|
93
108
|
|
|
94
|
-
`python3 scripts/audit.py --root <path>` writes
|
|
95
|
-
|
|
96
|
-
the same
|
|
109
|
+
`python3 scripts/audit.py --root <path>` writes the sidecar and prints a summary.
|
|
110
|
+
Add `--report` for the HTML page, and `--no-open` beside it to write without
|
|
111
|
+
opening. It is the same script the probes live in, so what the report claims and
|
|
112
|
+
what ran are the same object.
|
|
97
113
|
|
|
98
114
|
### 6. Propose — rows, not edits
|
|
99
115
|
|
|
@@ -105,6 +121,51 @@ audit: what a fix costs is the fixer's decision, not the finder's
|
|
|
105
121
|
(`references/prioritisation.md`). An audit
|
|
106
122
|
that edits while it reads cannot be re-run to check itself.
|
|
107
123
|
|
|
124
|
+
## A finding carries its consequence, or it is a hypothesis
|
|
125
|
+
|
|
126
|
+
**A mechanism is derived from the code; the consequence lives in production.** A row
|
|
127
|
+
written from the mechanism alone is indistinguishable from a real finding until
|
|
128
|
+
somebody measures it — and one audit had **eight consecutive rows** rewritten by that
|
|
129
|
+
measurement, two of which would have destroyed inventory if remedied as written,
|
|
130
|
+
because both read an absence of sales as an absence of demand.
|
|
131
|
+
|
|
132
|
+
So before a row may be written as a `finding` rather than as a hypothesis:
|
|
133
|
+
|
|
134
|
+
1. **how often does the mechanism fire?** A query, a log count, a telemetry read — or
|
|
135
|
+
an explicit statement that it has never been observed to fire;
|
|
136
|
+
2. **where the answer is *never*, the row is still worth keeping**, priced as
|
|
137
|
+
**latent**: the remedy is weighed against zero rather than against the mechanism's
|
|
138
|
+
severity;
|
|
139
|
+
3. **where the measurement is impossible, that is a `blind` on the consequence** and
|
|
140
|
+
the row says so. A blind consequence is not a finding.
|
|
141
|
+
|
|
142
|
+
Two rules follow from the same place:
|
|
143
|
+
|
|
144
|
+
- **Check a remedy against the population it would touch before proposing it.** Two
|
|
145
|
+
rows prescribed releasing an asset class; the measurement showed the asset class was
|
|
146
|
+
the product's own inventory, deliberately held.
|
|
147
|
+
- **An absence is not evidence until the path that would produce the presence has been
|
|
148
|
+
walked.** Zero sales means no demand *or* no working path, and those two want
|
|
149
|
+
opposite remedies.
|
|
150
|
+
|
|
151
|
+
## Already decided is not a finding, and it is not nothing either
|
|
152
|
+
|
|
153
|
+
An audit that reads a module and not the places the module is used reports, as
|
|
154
|
+
defects, the things the project has already decided — **in the project's own words**,
|
|
155
|
+
because the decision is usually written a few lines from the code the finding cites.
|
|
156
|
+
Five of eight rows in one run were that.
|
|
157
|
+
|
|
158
|
+
- **Read the call site, not only the definition.** A finding about a module is not
|
|
159
|
+
written until the places that use it have been read.
|
|
160
|
+
- **Every row states which of three it is:** *(a)* undecided, *(b)* decided and
|
|
161
|
+
documented right here, *(c)* decided elsewhere and not propagated. Only **(a)** and
|
|
162
|
+
**(c)** are work. **(b)** is the audit being wrong, and recording that is worth more
|
|
163
|
+
than deleting the row.
|
|
164
|
+
- **Where the verdict is (c), the remedy is a mechanical check, not an edit.** A
|
|
165
|
+
written rule nobody verifies reaches exactly as far as the place it was written; the
|
|
166
|
+
durable fix in all five cases was a guard that asks the project's own instruction of
|
|
167
|
+
every copy, not a patch to the copy that happened to be found.
|
|
168
|
+
|
|
108
169
|
## Three verdicts, and why the third one exists
|
|
109
170
|
|
|
110
171
|
`clean` · `finding` · `blind`. The vocabulary is closed; a fourth value is
|
|
@@ -145,9 +206,17 @@ Two more traps, both of which shipped in the first draft and are now fixtures:
|
|
|
145
206
|
- **Compare only when both sides claim the same version.** A branch already
|
|
146
207
|
bumped past its tag makes no common claim, and is `blind`, not `clean`.
|
|
147
208
|
|
|
148
|
-
## The
|
|
209
|
+
## The artefacts — the sidecar always, the page on request
|
|
210
|
+
|
|
211
|
+
`docs/audit/<date>-audit.json` is written on every run. **The HTML page is written
|
|
212
|
+
only with `--report`.**
|
|
149
213
|
|
|
150
|
-
|
|
214
|
+
The split is not symmetry. The sidecar is what makes this a ratchet rather than a
|
|
215
|
+
snapshot, and the next run reads it — skipping it would silently turn every future run
|
|
216
|
+
into a first run. The page is a **report**, and a report is an artefact that outlives
|
|
217
|
+
the conversation: one nobody asked for is a document nobody ordered and nobody
|
|
218
|
+
maintains, sitting untracked under `docs/` one `git add -A` from the product's
|
|
219
|
+
history. Answer in the conversation; write the page when somebody wants a page.
|
|
151
220
|
|
|
152
221
|
**The page carries aggregates and pointers, never raw bodies.** Counts, top
|
|
153
222
|
classes, trends, and a link to the issue in its own system — never a stack
|
|
@@ -170,7 +239,7 @@ picks up is a decision nobody wrote down.
|
|
|
170
239
|
|
|
171
240
|
| The script | You |
|
|
172
241
|
|---|---|
|
|
173
|
-
| discovery, the registry, mechanical probes,
|
|
242
|
+
| discovery, the registry, mechanical probes, the sidecar, the page on `--report`, the diff | the seam walk, MCP evidence, judgement about what a finding means |
|
|
174
243
|
| refuses a fourth verdict, redacts secrets, excludes its own output | deciding severity and effort, writing the remedy that fits this project |
|
|
175
244
|
|
|
176
245
|
The split is not tidiness. **A judgement encoded in a script becomes a gate
|
|
@@ -940,6 +940,47 @@ def _p_worktree(ctx):
|
|
|
940
940
|
% len(state["paths"]))
|
|
941
941
|
|
|
942
942
|
|
|
943
|
+
# A capability can be present without a dependency declaring it, and the words a
|
|
944
|
+
# home-grown one is named after are predictable. Ordered cheapest-first; the probe
|
|
945
|
+
# stops at the first hit.
|
|
946
|
+
HOMEGROWN_TELEMETRY = re.compile(
|
|
947
|
+
r"(?:error|exception|crash)[-_]?(?:report|track|log|handler|monitor)"
|
|
948
|
+
r"|(?:report|track|capture)[-_]?(?:error|exception|crash)"
|
|
949
|
+
r"|error[-_]?(?:events?|feed)", re.I)
|
|
950
|
+
|
|
951
|
+
|
|
952
|
+
def _telemetry_evidence(ctx):
|
|
953
|
+
"""Everything short of a manifest dependency that says this capability exists.
|
|
954
|
+
|
|
955
|
+
The probe concluded from manifests alone, so a project with a working
|
|
956
|
+
home-grown error channel was reported as having none — and the finding read
|
|
957
|
+
`no error reporting found`, which is an assertion about the product rather than
|
|
958
|
+
about the manifest the probe actually read. Ordered cheapest-first, stopping at
|
|
959
|
+
the first hit, and the KIND of evidence is returned because it decides the
|
|
960
|
+
verdict: code or schema is `clean`, documentary alone is not a finding either.
|
|
961
|
+
"""
|
|
962
|
+
# `out_rel` is where THIS run writes; a probe that reads it takes the audit's
|
|
963
|
+
# own artefacts for project state. There is no `ctx.excluded()` — the first draft
|
|
964
|
+
# guarded with `hasattr` and produced a branch that was permanently false, which
|
|
965
|
+
# is a filter that silently does nothing.
|
|
966
|
+
out = (ctx.out_rel or "").rstrip("/") + "/"
|
|
967
|
+
files = [f for f in (tracked_files(ctx.root) or []) if not f.startswith(out)]
|
|
968
|
+
for rel in files:
|
|
969
|
+
base = os.path.basename(rel)
|
|
970
|
+
if HOMEGROWN_TELEMETRY.search(base):
|
|
971
|
+
return ("module", rel)
|
|
972
|
+
for rel in files:
|
|
973
|
+
low = rel.lower()
|
|
974
|
+
if ("migration" in low or "schema" in low or low.endswith(".sql")) \
|
|
975
|
+
and HOMEGROWN_TELEMETRY.search(_read(os.path.join(ctx.root, rel), 40000)):
|
|
976
|
+
return ("schema", rel)
|
|
977
|
+
for marker in ("README.md", "CLAUDE.md", "AGENTS.md", "ARCHITECTURE.md"):
|
|
978
|
+
path = os.path.join(ctx.root, marker)
|
|
979
|
+
if os.path.exists(path) and HOMEGROWN_TELEMETRY.search(_read(path, 60000)):
|
|
980
|
+
return ("documented", marker)
|
|
981
|
+
return (None, None)
|
|
982
|
+
|
|
983
|
+
|
|
943
984
|
@probe("telemetry", "prod", needs=())
|
|
944
985
|
def _p_telemetry(ctx):
|
|
945
986
|
found = ctx.profile.get("telemetry") or []
|
|
@@ -950,16 +991,29 @@ def _p_telemetry(ctx):
|
|
|
950
991
|
if not deploy:
|
|
951
992
|
return Result("clean", "no deploy target declared — a library or tool, "
|
|
952
993
|
"for which absent telemetry is a design, not a gap")
|
|
994
|
+
kind, where = _telemetry_evidence(ctx)
|
|
995
|
+
if kind in ("module", "schema"):
|
|
996
|
+
return Result("clean", "no telemetry dependency, but a home-grown channel "
|
|
997
|
+
"is present (%s: %s) — the manifest was the wrong "
|
|
998
|
+
"place to look" % (kind, where))
|
|
999
|
+
if kind == "documented":
|
|
1000
|
+
# Documentary evidence cannot establish that the channel WORKS, and it
|
|
1001
|
+
# cannot establish that it is absent either. Asserting absence over a
|
|
1002
|
+
# document that describes the opposite is the defect this branch exists for.
|
|
1003
|
+
return Result("blind", "no telemetry dependency and no module named for "
|
|
1004
|
+
"one, but %s describes an error channel — whether it "
|
|
1005
|
+
"works is not answerable from the tree" % where)
|
|
953
1006
|
return Result("finding", "no error reporting found", findings=[_finding(
|
|
954
1007
|
"telemetry", "manifests",
|
|
955
1008
|
"A deployed surface reports no errors anywhere its maintainer can see",
|
|
956
1009
|
"medium", 2, 2,
|
|
957
1010
|
"add an error reporter, or record the decision not to — the gap worth "
|
|
958
1011
|
"closing is that nobody wrote down which it is",
|
|
959
|
-
detail="Deploy targets declared (%s) with no telemetry dependency in "
|
|
960
|
-
"
|
|
961
|
-
|
|
962
|
-
|
|
1012
|
+
detail="Deploy targets declared (%s) with no telemetry dependency in any "
|
|
1013
|
+
"manifest, no module named for an error channel, and no mention in "
|
|
1014
|
+
"the documentation entry points. A failure on a user's machine is "
|
|
1015
|
+
"invisible." % (", ".join(deploy) or "none"),
|
|
1016
|
+
evidence="languages=%s deploy=%s telemetry=[] homegrown=none"
|
|
963
1017
|
% (",".join(langs), ",".join(deploy)))])
|
|
964
1018
|
|
|
965
1019
|
|
|
@@ -1207,8 +1261,11 @@ def main(argv=None):
|
|
|
1207
1261
|
ap.add_argument("--root", default=".", help="project root (default: .)")
|
|
1208
1262
|
ap.add_argument("--out", default=None,
|
|
1209
1263
|
help="output directory (default: <root>/docs/audit)")
|
|
1264
|
+
ap.add_argument("--report", action="store_true",
|
|
1265
|
+
help="also write the HTML page and open it (default: the JSON "
|
|
1266
|
+
"sidecar and a stdout summary only)")
|
|
1210
1267
|
ap.add_argument("--no-open", action="store_true",
|
|
1211
|
-
help="
|
|
1268
|
+
help="with --report, write the page but do not open it")
|
|
1212
1269
|
ap.add_argument("--offline", action="store_true",
|
|
1213
1270
|
help="skip every probe that needs the network")
|
|
1214
1271
|
ap.add_argument("--json", action="store_true",
|
|
@@ -1234,11 +1291,20 @@ def main(argv=None):
|
|
|
1234
1291
|
|
|
1235
1292
|
json_path = os.path.join(out_dir, base + ".json")
|
|
1236
1293
|
html_path = os.path.join(out_dir, base + ".html")
|
|
1294
|
+
# The sidecar is unconditional: it is what makes this a ratchet rather than a
|
|
1295
|
+
# snapshot, and `carry_forward` above has already read the previous one. Skipping
|
|
1296
|
+
# it would silently turn every future run into a first run.
|
|
1297
|
+
#
|
|
1298
|
+
# The PAGE is a report, and a report is an artefact that outlives the
|
|
1299
|
+
# conversation. Writing one nobody asked for leaves a document in the tree that
|
|
1300
|
+
# nobody ordered and nobody maintains — and, being untracked HTML under `docs/`,
|
|
1301
|
+
# it is one `git add -A` from the product's history. So it is opt-in.
|
|
1237
1302
|
with open(json_path, "w", encoding="utf-8") as fh:
|
|
1238
1303
|
json.dump(payload, fh, indent=1, ensure_ascii=False, sort_keys=True)
|
|
1239
1304
|
fh.write("\n")
|
|
1240
|
-
|
|
1241
|
-
|
|
1305
|
+
if args.report:
|
|
1306
|
+
with open(html_path, "w", encoding="utf-8") as fh:
|
|
1307
|
+
fh.write(render_html(payload))
|
|
1242
1308
|
|
|
1243
1309
|
if args.json:
|
|
1244
1310
|
print(json.dumps(payload, indent=1, ensure_ascii=False))
|
|
@@ -1254,9 +1320,13 @@ def main(argv=None):
|
|
|
1254
1320
|
else:
|
|
1255
1321
|
print(" closed %d · new %d · still open %d"
|
|
1256
1322
|
% (len(r["closed"]), len(r["new"]), len(r["carried"])))
|
|
1257
|
-
|
|
1323
|
+
if args.report:
|
|
1324
|
+
print(" %s\n %s" % (html_path, json_path))
|
|
1325
|
+
else:
|
|
1326
|
+
print(" %s" % json_path)
|
|
1327
|
+
print(" no page written — pass `--report` for the HTML view")
|
|
1258
1328
|
|
|
1259
|
-
if not args.no_open:
|
|
1329
|
+
if args.report and not args.no_open:
|
|
1260
1330
|
ok, note = open_in_browser(html_path)
|
|
1261
1331
|
if not ok:
|
|
1262
1332
|
sys.stderr.write("could not open the report: %s\n" % note)
|
|
@@ -33,11 +33,23 @@ to it by construction:
|
|
|
33
33
|
- a **second implementation of the same rule** did not get the fix
|
|
34
34
|
- a **documented behaviour** is now false, and the document still reads as true
|
|
35
35
|
- **another feature** reaches the same path and nobody considered the interaction
|
|
36
|
+
- the **internal state is right and the message about it is stale** — the value was
|
|
37
|
+
stored correctly and the final sentence to the user echoed the old one
|
|
36
38
|
|
|
37
39
|
None of these is a bug in the changed lines. All of them ship. Each is found by
|
|
38
40
|
looking one level further out than the change — which is a different reading, not a
|
|
39
41
|
longer one, because the context that finds it is the context that excludes the diff.
|
|
40
42
|
|
|
43
|
+
**The last one is worth its own sentence, because it is the shape a blended score cannot
|
|
44
|
+
see.** A run that stores the right value and then reports the stale one passes on state
|
|
45
|
+
and fails on truth, so any grader that averages the two calls it mostly fine. It was
|
|
46
|
+
caught by a purpose-built rubric with categorical outcomes — *honoured / ignored /
|
|
47
|
+
partial / none* — rather than a number, and closing it moved the violation rate from
|
|
48
|
+
**21% to 4%**. The general form is the dangerous one for this pipeline: work that **looks
|
|
49
|
+
like it is working** — confident answers, a plan that reads fine — while quietly missing
|
|
50
|
+
what was actually asked. That is the product tier's whole job, and it is why the tier
|
|
51
|
+
that reads no code is not the soft one.
|
|
52
|
+
|
|
41
53
|
## The three tiers
|
|
42
54
|
|
|
43
55
|
| Tier | Subject | Characteristic finding |
|
|
@@ -62,6 +74,16 @@ tier's verdict.
|
|
|
62
74
|
Dispatch all three in one message so they run concurrently. Give each the node id,
|
|
63
75
|
its `serves`, and the diff — nothing else, and never another tier's output.
|
|
64
76
|
|
|
77
|
+
**The second axis, and it is the one an optimisation removes first: whoever produced
|
|
78
|
+
the fix never grades it.** Tier blindness is horizontal — no tier reads another's
|
|
79
|
+
report. This one is vertical: the agent that wrote the change, and the agent that
|
|
80
|
+
certifies it, are different agents. The reason is not tidiness — *an optimizer that
|
|
81
|
+
grades itself learns to game the metric instead of improving the work*, and it does so
|
|
82
|
+
while every report it writes stays sincere. The saving on offer is real (the fixer
|
|
83
|
+
already holds the context, a fresh reader must re-derive it) and it is the saving that
|
|
84
|
+
converts certification into self-assessment. `certify` cannot detect the collapse from
|
|
85
|
+
a report, so it is stated here as a dispatch rule.
|
|
86
|
+
|
|
65
87
|
## A pass has to mean something, so two rules have teeth
|
|
66
88
|
|
|
67
89
|
**A tier cannot pass on an empty `scope`.** `scope` is what the tier actually
|
|
@@ -81,6 +103,24 @@ and no third, because a certification that admits a maybe admits everything:
|
|
|
81
103
|
closing verdict as a blocker with `can_continue_around: true`, which is exactly
|
|
82
104
|
what a named survivable finding is.
|
|
83
105
|
|
|
106
|
+
**And the scope of that rule, because it is not universal.** These are severities of a
|
|
107
|
+
**finding**, and a finding is a claim a reader makes about a diff — it either blocks or
|
|
108
|
+
it does not. That is a different object from the **verdict of a check**, and where the
|
|
109
|
+
check's subject is non-deterministic the two-valued form is not strictness, it is
|
|
110
|
+
blindness: binary pass/fail has been measured at **0% detection** of regressions in a
|
|
111
|
+
non-deterministic workflow, where mapping execution traces to compact vectors and
|
|
112
|
+
testing them multivariately reaches **86%**. A third value there — `INCONCLUSIVE`,
|
|
113
|
+
grounded in hypothesis testing rather than in a reader's judgement — is what lets a
|
|
114
|
+
run say *this sample cannot decide* instead of flipping a coin and reporting it as a
|
|
115
|
+
severity. Sequential testing cuts the trials such a verdict needs by **78%**.
|
|
116
|
+
|
|
117
|
+
So: **two severities for a finding, three verdicts for a stochastic check.** A
|
|
118
|
+
deterministic command exits 0 or it does not, and admitting a maybe there does admit
|
|
119
|
+
everything. The instrument for the other case is `agent-stack`'s `agent-evals` —
|
|
120
|
+
`references/statistics.md` for how many runs make a difference real — and a gate whose
|
|
121
|
+
subject is non-deterministic belongs on that axis rather than this one
|
|
122
|
+
([`gates.md`](gates.md) → *Axis A*).
|
|
123
|
+
|
|
84
124
|
## The report, and where each field lands
|
|
85
125
|
|
|
86
126
|
Eight keys, all required, `[]` a valid answer and silence not one. On a pass
|
|
@@ -29,11 +29,11 @@ to be enforced and is not is the same failure as a gate that prints `FAIL` and e
|
|
|
29
29
|
| 1 | **Change the axis when it stops paying** | any second review or audit pass | count new findings and self-inflicted ones separately | self-inflicted exceeds new → change the axis, do not "look again more carefully" |
|
|
30
30
|
| 2 | **Absence needs its own check** | any two-layer model — entity/table, screen/frame, task/module, requirement/test | compute the mapping in **both** directions | zero orphans each way, printed |
|
|
31
31
|
| 3 | **Verify closure against the artefact** | closing anything that names a design, a build or a file | open the artefact, not the document describing it | the artefact carries a marker naming what it was last verified against |
|
|
32
|
-
| 4 | **Probe the detector** | every new check, lint rule or assertion | plant the defect → run → assert non-zero; restore → assert zero | both directions observed once, and recorded |
|
|
32
|
+
| 4 | **Probe the detector** | every new check, lint rule or assertion | plant the defect → run → assert non-zero; restore → assert zero. **The plant asserts it CHANGED the file** — `assert text.count(old) == 1` before substituting — and the expected verdict is stated before the run, because a green suite and an unlanded plant are the same output | both directions observed once, and recorded; the plant's own effect asserted, and restored from a copy taken before it rather than by `git checkout`, which discards whatever else was uncommitted |
|
|
33
33
|
| 5 | **Doubt the probe first** | a check that stays silent when it should fire | prove the planted defect exists in the text the check actually parses | the probe is shown to have changed what the check reads |
|
|
34
34
|
| 6 | **Sweep the class, not the finding** | any correction | grep the whole corpus for the same shape before closing | the class is gated, or named in the note so the next reader can finish it |
|
|
35
35
|
| 7 | **Ratchet, never TODO** | anything that cannot be fixed now | a named, counted set printed on every run | the number may only fall, and `OK` prints it beside itself |
|
|
36
|
-
| 8 | **Compute, never restate** | any count or cross-reference stated in prose | derive it from the source at check time | the stated number and the computed one are the same object |
|
|
36
|
+
| 8 | **Compute, never restate** | any count or cross-reference stated in prose | derive it from the source at check time. **Where the token could occur inside something else, print the matched items classified beside the number** — `grep -oE '<token>' \| sort \| uniq -c`, never a bare `grep -c` — because *eighteen lines containing these three digits* and *eighteen throttle events* are different claims and only one of them was true | the stated number and the computed one are the same object, and the run has looked at WHAT matched, not only how many |
|
|
37
37
|
| 9 | **A generator seeds green** | any scaffold, template or code generator | run the generated project's own checks inside the generator's tests | fresh output exits zero |
|
|
38
38
|
| 10 | **Measure a detector before trusting it** | a new heuristic check | run it over the real corpus and count false positives | zero false positives, or the heuristic is replaced by a deterministic rule |
|
|
39
39
|
| 11 | **A gate's exit code is part of its output** | any gate or CI step | run it against a planted defect and assert on `$?` | non-zero, not merely a `FAIL` line on stdout |
|
|
@@ -48,6 +48,16 @@ to be enforced and is not is the same failure as a gate that prints `FAIL` and e
|
|
|
48
48
|
| 20 | **When a thing exists twice, ask which one is used — not whether they agree** | any artifact with a second copy: two build files, a vendored library, a schema and its mirror, doctrine in two documents | find the **consumer** and read what it names; the copies cannot answer this about themselves | the run names the file the build/test/deploy actually reads, quoted from the consumer |
|
|
49
49
|
| 21 | **A step that consumes what a later step produces is a deadlock, not an ordering** | any sequence where a check reads state another step writes — a prune reading stamps, a gate reading a ledger, a report reading counts | trace each input to the step that writes it; if that step is downstream, the check has never run on real data | every check names the step that produces its input, and that step is upstream of it |
|
|
50
50
|
| 22 | **An operation that changes nothing reports the same as one that changed everything** | a substitution whose needle is absent · a command whose output is suppressed at a decision point · an edit issued against a shape the file does not have | assert the effect, not the call: a replacement must report how many times it matched, and a command whose exit code governs the next step is never piped or silenced | four incidents in two programmes, each invisible until something downstream failed — an import that never landed, a doctrine phrase worded differently, a `gh` call refused behind `>/dev/null`, and a test piped to `head` so that `$?` belonged to `head` |
|
|
51
|
+
| 23 | **A commit reference is a prediction until the commit exists** | any record naming a sha, and by extension any token that looks like evidence — a line number, a file path, a flag name, a test name | fill the field AFTER the object exists and resolve it (`git cat-file -e <sha>`); where the record must ship inside the commit it names, name the commit that carried the CHANGE and let the record follow | every reference in the record resolves, checked by a command, not by its shape |
|
|
52
|
+
| 24 | **A guard that outlives the name it guards goes vacuous, not red** | renaming or replacing any construct other files name | search the test suite for the old name in the same pass; and derive a guard's subject list from a **runtime declaration**, never a literal maintained by hand | the guard names its subjects by reading what the code says about itself, so a new member is covered by construction |
|
|
53
|
+
| 25 | **An item leaving a ranked list is not evidence about that item** | any fix verified against a top-N report, a truncated list, or "it no longer appears" | re-run the SAME measurement of the SAME subject; and make every truncating report say what it dropped — `showing 5 of 12` | the closing evidence names the subject and its measured value, not its absence from a window |
|
|
54
|
+
| 26 | **A model too small to reproduce the defect cannot certify the fix** | any reduced case, minimal repro, fixture or scratch tree standing in for the real subject | reproduce the defect in the model FIRST; if it goes green on the unfixed input it is not a harness | the fidelity gap is stated as a list of what the model omits, with numbers, not as prose that invites reasoning past it |
|
|
55
|
+
| 27 | **A sweep states its denominator before it starts** | any pass over an enumerable corpus — locales, call sites, files, tools | name the size of the corpus up front and the number converted at close; and derive the guard from the set the project already enumerates, never from a list typed into the test | the closing evidence is a re-run of the finding census, not the diff — the diff shows what was touched, the census shows what is left |
|
|
56
|
+
| 28 | **Guard capability, not incompleteness** | any completeness metric, coverage score or "N of M" axis | assert that an input exists for which the axis answers *no*; a count that only ever rises cannot tell a finished axis from one that lost the ability to fail | the probe finds its own subject rather than naming a fixture, and an axis that cannot say no is labelled WHERE THE NUMBER IS PUBLISHED |
|
|
57
|
+
| 29 | **A census over a localised corpus reports per locale** | any pattern matched across translations, regions or notations | print subjects examined and subjects matched **per locale**; check the notation for numbers, units and signs in each target first | a locale reporting zero where its neighbours report several is raised as a question by the report's own shape — and where the subject is one fact in N dictionaries, compare the dictionaries to EACH OTHER, which needs no notation knowledge at all |
|
|
58
|
+
| 30 | **A gate scoped by exclusion judges every newcomer silently** | any lint, format or check configured with an ignore list rather than an include list | scope by what it COVERS; where the tool has no such switch, assert the covered set and fail on additions, deriving the allowlist by measuring the tool's own reach | the failure message names the intruder and the remedy, because the operator meets it mid-release |
|
|
59
|
+
| 31 | **A gate needs a third verdict — could not measure** | any check that can time out, lose a dependency, or run on a contended machine | distinguish *failed* from *could not measure* with different exit codes, and say which; neither is a verdict about the code | the refusal names its remedy, and a test that waits for a DURATION rather than an event is treated as a defect in the test |
|
|
60
|
+
| 32 | **A test whose assertion is implied by compilation is not a test** | any test whose name promises a behaviour | delete the subject: if that is a type or import error rather than a red test, the test is decoration | removing one asks what the name PROMISED and whether anything checks it — four of five such cases here were covered elsewhere and the fifth was a real gap only visible because the name was read as a claim |
|
|
51
61
|
|
|
52
62
|
---
|
|
53
63
|
|
|
@@ -266,7 +276,7 @@ evidence that something left, and the log below says what.
|
|
|
266
276
|
|
|
267
277
|
### Retired
|
|
268
278
|
|
|
269
|
-
**Numbers issued so far:
|
|
279
|
+
**Numbers issued so far: 32.** This is the high-water mark, and it is the only number
|
|
270
280
|
this file states about itself — deliberately, because the gap that proves a rule left
|
|
271
281
|
cannot be computed from the table alone: **deleting the highest-numbered rule shrinks
|
|
272
282
|
the maximum with it, and no gap ever opens.** That false negative shipped in the first
|
|
@@ -14,6 +14,7 @@ the **order** the rows come off it.
|
|
|
14
14
|
- 2. Confidence and Ease
|
|
15
15
|
- 3. The order, and why it is not a product
|
|
16
16
|
- 4. Research before scoring, not after
|
|
17
|
+
- Effort does not rank a finding, but the fixer needs a cost rule
|
|
17
18
|
- 5. Where the human goes
|
|
18
19
|
- 6. Two models in one family, and why both are right
|
|
19
20
|
|
|
@@ -128,6 +129,27 @@ four fields are the contract; anything else is a subagent that spent context to
|
|
|
128
129
|
|
|
129
130
|
**Do not fan out to score two rows.** The dispatch costs more than reading them.
|
|
130
131
|
|
|
132
|
+
## Effort does not rank a finding, but the fixer needs a cost rule
|
|
133
|
+
|
|
134
|
+
The existing rule — **the finder may not rank by effort** — is about the finder, and it
|
|
135
|
+
holds. It says nothing about the moment somebody decides what to do with the row, and the
|
|
136
|
+
silence there was read as *cost never enters*, which is not what anyone means.
|
|
137
|
+
|
|
138
|
+
So the split, stated rather than inferred:
|
|
139
|
+
|
|
140
|
+
- **The finder** still may not rank by effort. What a fix costs is not a property of the
|
|
141
|
+
defect.
|
|
142
|
+
- **The fixer**, at the point of deciding, weighs remedy cost against **measured harm** —
|
|
143
|
+
not against the finding's stated severity. Severity is the finder's estimate of what
|
|
144
|
+
could happen; harm is what did.
|
|
145
|
+
- **Where the harm measures zero the row is not deleted.** It is priced as *latent* and
|
|
146
|
+
carries its measurement, so the next reader inherits the number rather than the alarm.
|
|
147
|
+
A row deleted for measuring zero takes the measurement with it and the next audit
|
|
148
|
+
re-derives the same alarm from the same mechanism.
|
|
149
|
+
- **Where the path is irreversible, cost loses.** A cheap guard on something that cannot
|
|
150
|
+
be undone ships whether or not the failure has been observed — the asymmetry is the
|
|
151
|
+
whole reason to have a rule instead of a judgement.
|
|
152
|
+
|
|
131
153
|
## 5. Where the human goes
|
|
132
154
|
|
|
133
155
|
The operator is a **scarce, high-value input**, and the failure mode is spending them one
|
|
@@ -39,6 +39,8 @@ file exists to stop.
|
|
|
39
39
|
- Three grades of fix — take the highest one that can work
|
|
40
40
|
- Stamp first, then prune, then write
|
|
41
41
|
- The prune — mandatory, and it runs after the stamp
|
|
42
|
+
- When the prune cannot run, say so — it is not the same as nothing to prune
|
|
43
|
+
- A fix to one call site closes a call site, not a class
|
|
42
44
|
- The loop closes at stage 0
|
|
43
45
|
- Where a lesson goes when it is not about this project
|
|
44
46
|
- Publishing the insight — the skill learns from every project that runs it
|
|
@@ -300,6 +302,33 @@ GATE 10 acceptance: PASS — 14/14 REQ verified
|
|
|
300
302
|
|
|
301
303
|
A pruned list that nobody prints is a list that quietly grows back.
|
|
302
304
|
|
|
305
|
+
## When the prune cannot run, say so — it is not the same as nothing to prune
|
|
306
|
+
|
|
307
|
+
The prune retires a standing instruction against its triggers, and the cold trigger counts
|
|
308
|
+
run stamps. Meeting a list whose rows carry **no id and no retirement condition**, the
|
|
309
|
+
honest behaviour was undefined — so a careful run printed counts and stopped while a
|
|
310
|
+
careless one could delete nine rules, and both looked like *the prune ran*.
|
|
311
|
+
|
|
312
|
+
- **Backfill first, once.** On meeting rows with no id, the prune's first act is to mint
|
|
313
|
+
ids and propose a retirement condition per row **from the row's own text**, printed for
|
|
314
|
+
the operator to accept. One pass, and the trigger works forever after.
|
|
315
|
+
- **`blocked` is a named outcome**, printed like any other: what could not be evaluated
|
|
316
|
+
and why. An undefined case is where the spread between two runs lives.
|
|
317
|
+
- **A hard cap plus an unrunnable retirement condition is a deadlock.** Either the cap
|
|
318
|
+
yields until the backfill is done, or the backfill is a precondition of enforcing it —
|
|
319
|
+
but not both, and the file says which.
|
|
320
|
+
|
|
321
|
+
## A fix to one call site closes a call site, not a class
|
|
322
|
+
|
|
323
|
+
- **A fix other call sites will need is exported before it is used.** The test is
|
|
324
|
+
mechanical: if the same defect could exist in a second file, the fix does not stay
|
|
325
|
+
private to the first.
|
|
326
|
+
- **Closing a defect includes a census of its siblings, by BEHAVIOUR rather than by a list
|
|
327
|
+
of names** — *every script that spawns a browser*, found by searching for the spawn, not
|
|
328
|
+
by remembering the five. A list of names is how the previous pass missed four of them.
|
|
329
|
+
- **The entry states how many call sites the class had and how many were fixed.** *Fixed*
|
|
330
|
+
and *fixed in one of five* currently read identically, and only one of them closes a row.
|
|
331
|
+
|
|
303
332
|
## The loop closes at stage 0
|
|
304
333
|
|
|
305
334
|
The standing instructions are an **instruction source**, not background reading:
|