task-pipeline-skill 1.82.4 → 1.84.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +156 -0
- package/README.md +3 -3
- package/SKILL-CARD.md +1 -1
- package/package.json +1 -1
- package/plugins/task-pipeline/.claude-plugin/plugin.json +1 -1
- package/plugins/task-pipeline/skills/evidence-docs/SKILL.md +3 -0
- package/plugins/task-pipeline/skills/project-audit/SKILL.md +75 -6
- package/plugins/task-pipeline/skills/project-audit/scripts/audit.py +79 -9
- package/plugins/task-pipeline/skills/task-pipeline/references/learned.md +13 -3
- package/plugins/task-pipeline/skills/task-pipeline/references/prioritisation.md +22 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/retrospective.md +29 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,3 +1,159 @@
|
|
|
1
|
+
## v1.84.1 — the conformance sentence is computed, and two of its five figures were wrong
|
|
2
|
+
|
|
3
|
+
**A sentence of five measured figures, and nothing derived any of them.** The README's
|
|
4
|
+
*Held to Anthropic's own Skill authoring guidance* paragraph states `name` length,
|
|
5
|
+
`description` length, `SKILL.md` lines, the reference count and the bundle size. Measured
|
|
6
|
+
2026-09-05:
|
|
7
|
+
|
|
8
|
+
| stated | measured |
|
|
9
|
+
|---|---|
|
|
10
|
+
| `name` 13/64 | 13 ✓ |
|
|
11
|
+
| `description` inside 1024 | 899 ✓ |
|
|
12
|
+
| `SKILL.md` **334**/500 lines | **262** |
|
|
13
|
+
| all **38** references | 38 ✓ |
|
|
14
|
+
| **436 KB** against 30 MB | **1128 KB** |
|
|
15
|
+
|
|
16
|
+
**The line figure was wrong when it was written.** At commit `297a0cb`, which added the
|
|
17
|
+
sentence, the file was **339** lines — not 334. It was never a measurement, and a reader
|
|
18
|
+
had no way to tell which of the five to trust.
|
|
19
|
+
|
|
20
|
+
**The size figure changed SHAPE rather than value, and that is the decision worth
|
|
21
|
+
recording.** It moved on four of the eight commits before this one — 950 → 970 → 980 →
|
|
22
|
+
1128 KB — so an exact figure in prose goes stale by construction, and a check over it
|
|
23
|
+
would fire on nearly every edit. *A check that fires constantly is one somebody turns
|
|
24
|
+
off.* It is a bound now — **far under the 30 MB ceiling** — which stays true across three
|
|
25
|
+
orders of magnitude and still says what the sentence exists to say.
|
|
26
|
+
|
|
27
|
+
The other four are derived and compared on every run. The parser **asserts it found all
|
|
28
|
+
four**: a sentence reworded past it fails here saying so rather than passing quietly,
|
|
29
|
+
which is the shape that has twice made a healthy guard in this family report itself
|
|
30
|
+
broken. Line count is `splitlines()`, stated in the failure message because `wc -l` over
|
|
31
|
+
a file without a trailing newline gives a different number.
|
|
32
|
+
|
|
33
|
+
Both directions watched failing, each plant asserting it landed first: a wrong figure
|
|
34
|
+
gives *states lines = 999 and the tree gives 262*; deleting the reference clause gives
|
|
35
|
+
*no longer states references in a shape this check can read*.
|
|
36
|
+
|
|
37
|
+
Guards: 424 → **424** — the new check is an ordinary step, not a planted-defect guard.
|
|
38
|
+
|
|
39
|
+
## v1.84.0 — fifteen filed findings, and the two rules they mostly refine
|
|
40
|
+
|
|
41
|
+
Every one of the fifteen `ssheleg/sshlg-skills` issues this repository owned, closed in one
|
|
42
|
+
release because they are one subject: **what a check is allowed to conclude.**
|
|
43
|
+
|
|
44
|
+
**Ten new rules in `references/learned.md` (22 → 32), and two existing ones strengthened
|
|
45
|
+
rather than duplicated** — the distinction matters, because a table that grows a row for
|
|
46
|
+
every refinement stops being readable at exactly the point it starts being complete.
|
|
47
|
+
|
|
48
|
+
- rule **4** (probe the detector) now demands the plant assert it **changed the file**, and
|
|
49
|
+
that the expected verdict be stated before the run — a green suite and an unlanded plant
|
|
50
|
+
are the same output (`#94`);
|
|
51
|
+
- rule **8** (compute, never restate) now demands the matched items be printed classified
|
|
52
|
+
beside the count — *eighteen lines containing these three digits* and *eighteen throttle
|
|
53
|
+
events* are different claims, and only one was true (`#79`).
|
|
54
|
+
|
|
55
|
+
The ten new ones, each carrying the incident it came from: a commit reference is a
|
|
56
|
+
prediction until the object resolves (`#80`); a guard outliving the name it guards goes
|
|
57
|
+
vacuous rather than red (`#81`); an item leaving a ranked list is not evidence about that
|
|
58
|
+
item (`#87`); a model too small to reproduce cannot certify (`#89`); a sweep states its
|
|
59
|
+
denominator before it starts (`#90`); guard capability, not incompleteness (`#91`); a
|
|
60
|
+
census over a localised corpus reports per locale (`#92`); a gate scoped by exclusion
|
|
61
|
+
judges every newcomer silently (`#93`); a gate needs a third verdict — *could not measure*
|
|
62
|
+
(`#95`); a test whose assertion is implied by compilation is not a test (`#97`).
|
|
63
|
+
|
|
64
|
+
**Three sub-canons in `evidence-docs`**, placed under the canons they refine rather than as
|
|
65
|
+
an eleventh: **1a** a token that looks like evidence is not evidence until resolved; **2a**
|
|
66
|
+
a number out of the shell is not a measurement until you look at what matched; **5a** a
|
|
67
|
+
model too small to reproduce the defect is too small to prove the fix.
|
|
68
|
+
|
|
69
|
+
**`prioritisation.md`** gains the half the existing rule was silent about (`#82`): the
|
|
70
|
+
finder still may not rank by effort, the **fixer** weighs cost against *measured* harm, a
|
|
71
|
+
zero-harm row is priced latent rather than deleted, and where the path is irreversible cost
|
|
72
|
+
loses.
|
|
73
|
+
|
|
74
|
+
**`retrospective.md`** gains two (`#85`, `#88`): what the prune does when it **cannot**
|
|
75
|
+
run — an undefined case is where the spread between a careful run and a careless one lives
|
|
76
|
+
— and that a fix to one call site closes a call site, not a class, with the sibling census
|
|
77
|
+
taken by behaviour rather than from a list of names.
|
|
78
|
+
|
|
79
|
+
**The high-water mark is derived now**, not typed: the validator caught `Numbers issued so
|
|
80
|
+
far: 22` against a table reaching 32, which is the same defect these rules are about.
|
|
81
|
+
|
|
82
|
+
Guards: 424 → **424** — doctrine, no mechanism.
|
|
83
|
+
|
|
84
|
+
## v1.83.1 — the tag that could not be moved, and why that is right
|
|
85
|
+
|
|
86
|
+
**No change to the skill.** `v1.83.0`'s tree carried a run stamp with `—` where its
|
|
87
|
+
commit belongs, so the release gate refused it on the tag's own tree — correctly, and in
|
|
88
|
+
the only place that can see it, because `validate.yml` ignores tag pushes.
|
|
89
|
+
|
|
90
|
+
The stamp names `48544ac` now. **The tag was not moved**: a repository rule refuses to
|
|
91
|
+
delete or re-point one, and that rule is right. A tag is a promise about a tree, the tree
|
|
92
|
+
at `v1.83.0` genuinely lacked the stamp, and quietly re-cutting the same version over a
|
|
93
|
+
different tree would make every future reader of that tag wrong about what it contained.
|
|
94
|
+
So the repair is a version, not a rewrite.
|
|
95
|
+
|
|
96
|
+
This is `ssheleg/sshlg-skills#80` met from the inside on the run that closed its
|
|
97
|
+
neighbours: **a commit reference is written after the commit exists.** The record ships
|
|
98
|
+
in a following commit rather than claiming a sha that did not exist when it was written.
|
|
99
|
+
|
|
100
|
+
Guards: 424 → **424** — no mechanism changed.
|
|
101
|
+
|
|
102
|
+
## v1.83.0 — an audit report is asked for, and four findings that had already cost something
|
|
103
|
+
|
|
104
|
+
`project-audit`, closing `ssheleg/sshlg-skills#77`, `#78`, `#83`, `#84`.
|
|
105
|
+
|
|
106
|
+
**A report is written when somebody wants a report.** The run wrote
|
|
107
|
+
`docs/audit/<date>-audit.html` and opened it every time; it now writes the page only
|
|
108
|
+
with **`--report`** (`--no-open` beside it to write without opening). The split is not
|
|
109
|
+
symmetry: the **sidecar stays unconditional**, because `carry_forward` reads the previous
|
|
110
|
+
one and skipping it would silently turn every future run into a first run. The page is
|
|
111
|
+
different — a report outlives the conversation, and one nobody asked for is untracked
|
|
112
|
+
HTML under `docs/`, one `git add -A` from the product's history.
|
|
113
|
+
|
|
114
|
+
**`#83` — the telemetry probe read manifests and asserted about the product.** A project
|
|
115
|
+
with a working home-grown error channel and no SDK dependency was reported as *"no error
|
|
116
|
+
reporting found"*. The evidence is ordered and cheapest-first now, stopping at the first
|
|
117
|
+
hit: a manifest dependency, a module named for the capability, a table or migration named
|
|
118
|
+
for it, a mention in a documentation entry point. **And the KIND of evidence decides the
|
|
119
|
+
verdict** — code or schema is `clean` with the channel named; a document alone is
|
|
120
|
+
`blind`, because it establishes neither that the channel works nor that it is absent.
|
|
121
|
+
The remedy sentence survives, which was the valuable half.
|
|
122
|
+
|
|
123
|
+
**`#77` — a finding carries its consequence, or it is a hypothesis.** One audit had
|
|
124
|
+
**eight consecutive rows** rewritten by measuring against production, two of which would
|
|
125
|
+
have destroyed inventory if remedied as written, because both read an absence of sales as
|
|
126
|
+
an absence of demand. A row now states how often the mechanism fires, or that it never
|
|
127
|
+
has — in which case it is priced as **latent** and the remedy weighed against zero — or
|
|
128
|
+
that the measurement is impossible, which is a `blind` on the consequence and not a
|
|
129
|
+
finding.
|
|
130
|
+
|
|
131
|
+
**`#78` — already decided is not a finding, and not nothing either.** Five of eight rows
|
|
132
|
+
in one run reported decisions the project had already made, in the project's own words,
|
|
133
|
+
because the decision sits a few lines from the cited code. The call site is read before a
|
|
134
|
+
row is written, every row states which of three it is (undecided / decided here / decided
|
|
135
|
+
elsewhere and not propagated), and where it is the third the remedy is **a mechanical
|
|
136
|
+
check rather than an edit** — a written rule nobody verifies reaches exactly as far as
|
|
137
|
+
the place it was written.
|
|
138
|
+
|
|
139
|
+
**`#84` — an ordered preflight before any datastore read.** Resolve the store from the
|
|
140
|
+
platform's own attachment rather than from a script or a document; prove liveness with a
|
|
141
|
+
freshest-write aggregate and put that timestamp beside every figure it underwrites; name
|
|
142
|
+
every attached store with its freshness; and treat the repository's connect helper as
|
|
143
|
+
evidence about the repository, which is a finding of its own when the two disagree.
|
|
144
|
+
|
|
145
|
+
Guards: 424 → **424** — this release adds no planted-defect guard. The three new checks
|
|
146
|
+
live in `test/project_audit_test.py`, which `validate.yml` runs as an ordinary step
|
|
147
|
+
rather than as one of its 429 `Negative self-test` steps, and the floor
|
|
148
|
+
(`negatives.MIN_EXPECTED`) is unchanged. The new checks were still each watched failing
|
|
149
|
+
against the un-fixed code, which is the property the count exists to protect and not the
|
|
150
|
+
count itself.
|
|
151
|
+
|
|
152
|
+
**One defect of my own, caught before it shipped:** the widened probe's first draft
|
|
153
|
+
guarded with `hasattr(ctx, "excluded")` against a method that does not exist, producing a
|
|
154
|
+
filter that was permanently false and silently did nothing. It reads `ctx.out_rel`, which
|
|
155
|
+
is the real field.
|
|
156
|
+
|
|
1
157
|
## v1.82.4 — the plant the harness could not see
|
|
2
158
|
|
|
3
159
|
`v1.82.3` is a dead tag, and its cause is the **fourth distinct one**. Its stamp was
|
package/README.md
CHANGED
|
@@ -587,9 +587,9 @@ must say so.
|
|
|
587
587
|
### Held to Anthropic's own Skill authoring guidance
|
|
588
588
|
|
|
589
589
|
Audited against the four Agent Skills pages. Most of it already held — `name`
|
|
590
|
-
13/64 chars, `description`
|
|
591
|
-
linked **directly** from `SKILL.md`,
|
|
592
|
-
now does:
|
|
590
|
+
13/64 chars, `description` 899/1024 chars, `SKILL.md` 262/500 lines, all 38 references
|
|
591
|
+
linked **directly** from `SKILL.md`, and the bundle far under the 30 MB ceiling. What
|
|
592
|
+
did not, now does:
|
|
593
593
|
|
|
594
594
|
- **Every reference over 100 lines carries a `## Contents` list**, and the list is
|
|
595
595
|
*compared against the file's own headings* rather than trusted. The guidance is
|
package/SKILL-CARD.md
CHANGED
|
@@ -12,7 +12,7 @@ harmless.
|
|
|
12
12
|
|---|---|
|
|
13
13
|
| **Purpose** | Runs a substantial task through ten gated delivery stages — intake grill, docs study, brainstorm, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs+registers, acceptance — refusing to advance until each gate passes |
|
|
14
14
|
| **Owner** | ssheleg ([github.com/ssheleg/task-pipeline](https://github.com/ssheleg/task-pipeline)) |
|
|
15
|
-
| **Version** | 1.
|
|
15
|
+
| **Version** | 1.84.1 |
|
|
16
16
|
| **Surface** | Claude Code (filesystem skill + plugin) and the vercel `skills` CLI. **Not** uploaded to the Skills API; custom Skills do not sync across surfaces |
|
|
17
17
|
| **Dependencies** | None required. Optional: `context7` (MCP), `figma` (MCP), super-ux, agent-sync, graphify, obsidian-wiki, and **one of two browser channels** — `playwright` (CLI or MCP) or `chrome-devtools` (MCP); either satisfies the browser step and neither is required. Every stage's doctrine ships in-repo; the one conditional requirement is super-ux for the stage-3 UX track on a user-facing task |
|
|
18
18
|
| **Evaluation status** | Suite authored, 5 categories. One recorded run, **self-observed by the author**; **zero blind runs on zero of three models** — the split, and the numbers, live in [`evals/RESULTS.md`](evals/RESULTS.md) and are computed by `evals/run.py` |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "task-pipeline-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.84.1",
|
|
4
4
|
"description": "Full-cycle delivery pipeline for coding agents: a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine ships inside the skill — no companion plugin required. This package is the installer CLI.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"task-pipeline": "bin/task-pipeline.js"
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
"name": "task-pipeline",
|
|
4
4
|
"displayName": "Task Pipeline",
|
|
5
5
|
"description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/judgment/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
|
|
6
|
-
"version": "1.
|
|
6
|
+
"version": "1.84.1",
|
|
7
7
|
"author": {
|
|
8
8
|
"name": "ssheleg",
|
|
9
9
|
"url": "https://x.com/sshlg93"
|
|
@@ -16,10 +16,13 @@ The full statement of each canon, its rationale and its enforcement live in
|
|
|
16
16
|
## The ten canons
|
|
17
17
|
|
|
18
18
|
1. **A claim carries its address** — `file:line`, a command with its output, a test name; a lesson names its commit.
|
|
19
|
+
- **1a. A token that looks like evidence is not evidence until it has been resolved.** A sha, a line number, a path, a flag, a test name: every one can be plausible and absent, and each is trusted at exactly the moment nobody re-reads it. A commit reference is filled **after** the commit exists and checked with a command.
|
|
19
20
|
2. **Numbers are computed, never restated.**
|
|
21
|
+
- **2a. A number out of the shell is not a measurement until you have looked at what matched.** Print the matched items classified beside the count whenever the token can occur inside something else, and never pipe a command whose exit status governs what happens next — the shell then answers about the pager. *Eighteen lines containing these three digits* and *eighteen throttle events* are different claims.
|
|
20
22
|
3. **Every fact has exactly one home** — others link, never restate.
|
|
21
23
|
4. **A reference resolves from where the document is read** — not from where it lives.
|
|
22
24
|
5. **Green nobody watched turn red is not evidence.**
|
|
25
|
+
- **5a. A model too small to reproduce the defect is too small to prove the fix.** Reproduce first, fix second: a harness that goes green on the unfixed input is not a harness. Where the model is smaller than the subject, the gap is stated as a list of what it omits, with numbers.
|
|
23
26
|
6. **A check proves its scope and nothing beyond it.**
|
|
24
27
|
7. **Silence is not a pass** — ask what a mechanism prints when it did not look.
|
|
25
28
|
8. **An estimate is never announced as a measurement** — a rule states its evidence condition.
|
|
@@ -84,6 +84,21 @@ cannot admit about itself:
|
|
|
84
84
|
blind. Read aggregates and pointers, never raw bodies (see *The two
|
|
85
85
|
artefacts*).
|
|
86
86
|
|
|
87
|
+
**Before ANY datastore read, in this order** — a figure from the wrong database is
|
|
88
|
+
worse than no figure, because it is quoted with the authority of a measurement:
|
|
89
|
+
|
|
90
|
+
1. **resolve the datastore from the platform's own attachment**, not from a script
|
|
91
|
+
in the repository and not from a name in the documentation;
|
|
92
|
+
2. **prove liveness with a freshest-write aggregate** against a table the product
|
|
93
|
+
writes constantly, and put that timestamp in the report beside every figure it
|
|
94
|
+
underwrites;
|
|
95
|
+
3. where **more than one** datastore is attached, name them all with their
|
|
96
|
+
freshness — an audit that reads one and does not mention the other cannot be
|
|
97
|
+
reproduced by its reader;
|
|
98
|
+
4. the repository's connect helper is **evidence about the repository**, never about
|
|
99
|
+
the platform. Where the two disagree that is a finding of its own, and it is the
|
|
100
|
+
one this rule was written from.
|
|
101
|
+
|
|
87
102
|
### 4. Seams — hand off to the ladder
|
|
88
103
|
|
|
89
104
|
Now, and only now, walk `audit.md`'s ladder over the capabilities the discovery
|
|
@@ -91,9 +106,10 @@ found. Bottom-up, seam-ordered. Absences found here are findings like any other.
|
|
|
91
106
|
|
|
92
107
|
### 5. Report — two files, one command
|
|
93
108
|
|
|
94
|
-
`python3 scripts/audit.py --root <path>` writes
|
|
95
|
-
|
|
96
|
-
the same
|
|
109
|
+
`python3 scripts/audit.py --root <path>` writes the sidecar and prints a summary.
|
|
110
|
+
Add `--report` for the HTML page, and `--no-open` beside it to write without
|
|
111
|
+
opening. It is the same script the probes live in, so what the report claims and
|
|
112
|
+
what ran are the same object.
|
|
97
113
|
|
|
98
114
|
### 6. Propose — rows, not edits
|
|
99
115
|
|
|
@@ -105,6 +121,51 @@ audit: what a fix costs is the fixer's decision, not the finder's
|
|
|
105
121
|
(`references/prioritisation.md`). An audit
|
|
106
122
|
that edits while it reads cannot be re-run to check itself.
|
|
107
123
|
|
|
124
|
+
## A finding carries its consequence, or it is a hypothesis
|
|
125
|
+
|
|
126
|
+
**A mechanism is derived from the code; the consequence lives in production.** A row
|
|
127
|
+
written from the mechanism alone is indistinguishable from a real finding until
|
|
128
|
+
somebody measures it — and one audit had **eight consecutive rows** rewritten by that
|
|
129
|
+
measurement, two of which would have destroyed inventory if remedied as written,
|
|
130
|
+
because both read an absence of sales as an absence of demand.
|
|
131
|
+
|
|
132
|
+
So before a row may be written as a `finding` rather than as a hypothesis:
|
|
133
|
+
|
|
134
|
+
1. **how often does the mechanism fire?** A query, a log count, a telemetry read — or
|
|
135
|
+
an explicit statement that it has never been observed to fire;
|
|
136
|
+
2. **where the answer is *never*, the row is still worth keeping**, priced as
|
|
137
|
+
**latent**: the remedy is weighed against zero rather than against the mechanism's
|
|
138
|
+
severity;
|
|
139
|
+
3. **where the measurement is impossible, that is a `blind` on the consequence** and
|
|
140
|
+
the row says so. A blind consequence is not a finding.
|
|
141
|
+
|
|
142
|
+
Two rules follow from the same place:
|
|
143
|
+
|
|
144
|
+
- **Check a remedy against the population it would touch before proposing it.** Two
|
|
145
|
+
rows prescribed releasing an asset class; the measurement showed the asset class was
|
|
146
|
+
the product's own inventory, deliberately held.
|
|
147
|
+
- **An absence is not evidence until the path that would produce the presence has been
|
|
148
|
+
walked.** Zero sales means no demand *or* no working path, and those two want
|
|
149
|
+
opposite remedies.
|
|
150
|
+
|
|
151
|
+
## Already decided is not a finding, and it is not nothing either
|
|
152
|
+
|
|
153
|
+
An audit that reads a module and not the places the module is used reports, as
|
|
154
|
+
defects, the things the project has already decided — **in the project's own words**,
|
|
155
|
+
because the decision is usually written a few lines from the code the finding cites.
|
|
156
|
+
Five of eight rows in one run were that.
|
|
157
|
+
|
|
158
|
+
- **Read the call site, not only the definition.** A finding about a module is not
|
|
159
|
+
written until the places that use it have been read.
|
|
160
|
+
- **Every row states which of three it is:** *(a)* undecided, *(b)* decided and
|
|
161
|
+
documented right here, *(c)* decided elsewhere and not propagated. Only **(a)** and
|
|
162
|
+
**(c)** are work. **(b)** is the audit being wrong, and recording that is worth more
|
|
163
|
+
than deleting the row.
|
|
164
|
+
- **Where the verdict is (c), the remedy is a mechanical check, not an edit.** A
|
|
165
|
+
written rule nobody verifies reaches exactly as far as the place it was written; the
|
|
166
|
+
durable fix in all five cases was a guard that asks the project's own instruction of
|
|
167
|
+
every copy, not a patch to the copy that happened to be found.
|
|
168
|
+
|
|
108
169
|
## Three verdicts, and why the third one exists
|
|
109
170
|
|
|
110
171
|
`clean` · `finding` · `blind`. The vocabulary is closed; a fourth value is
|
|
@@ -145,9 +206,17 @@ Two more traps, both of which shipped in the first draft and are now fixtures:
|
|
|
145
206
|
- **Compare only when both sides claim the same version.** A branch already
|
|
146
207
|
bumped past its tag makes no common claim, and is `blind`, not `clean`.
|
|
147
208
|
|
|
148
|
-
## The
|
|
209
|
+
## The artefacts — the sidecar always, the page on request
|
|
210
|
+
|
|
211
|
+
`docs/audit/<date>-audit.json` is written on every run. **The HTML page is written
|
|
212
|
+
only with `--report`.**
|
|
149
213
|
|
|
150
|
-
|
|
214
|
+
The split is not symmetry. The sidecar is what makes this a ratchet rather than a
|
|
215
|
+
snapshot, and the next run reads it — skipping it would silently turn every future run
|
|
216
|
+
into a first run. The page is a **report**, and a report is an artefact that outlives
|
|
217
|
+
the conversation: one nobody asked for is a document nobody ordered and nobody
|
|
218
|
+
maintains, sitting untracked under `docs/` one `git add -A` from the product's
|
|
219
|
+
history. Answer in the conversation; write the page when somebody wants a page.
|
|
151
220
|
|
|
152
221
|
**The page carries aggregates and pointers, never raw bodies.** Counts, top
|
|
153
222
|
classes, trends, and a link to the issue in its own system — never a stack
|
|
@@ -170,7 +239,7 @@ picks up is a decision nobody wrote down.
|
|
|
170
239
|
|
|
171
240
|
| The script | You |
|
|
172
241
|
|---|---|
|
|
173
|
-
| discovery, the registry, mechanical probes,
|
|
242
|
+
| discovery, the registry, mechanical probes, the sidecar, the page on `--report`, the diff | the seam walk, MCP evidence, judgement about what a finding means |
|
|
174
243
|
| refuses a fourth verdict, redacts secrets, excludes its own output | deciding severity and effort, writing the remedy that fits this project |
|
|
175
244
|
|
|
176
245
|
The split is not tidiness. **A judgement encoded in a script becomes a gate
|
|
@@ -940,6 +940,47 @@ def _p_worktree(ctx):
|
|
|
940
940
|
% len(state["paths"]))
|
|
941
941
|
|
|
942
942
|
|
|
943
|
+
# A capability can be present without a dependency declaring it, and the words a
|
|
944
|
+
# home-grown one is named after are predictable. Ordered cheapest-first; the probe
|
|
945
|
+
# stops at the first hit.
|
|
946
|
+
HOMEGROWN_TELEMETRY = re.compile(
|
|
947
|
+
r"(?:error|exception|crash)[-_]?(?:report|track|log|handler|monitor)"
|
|
948
|
+
r"|(?:report|track|capture)[-_]?(?:error|exception|crash)"
|
|
949
|
+
r"|error[-_]?(?:events?|feed)", re.I)
|
|
950
|
+
|
|
951
|
+
|
|
952
|
+
def _telemetry_evidence(ctx):
|
|
953
|
+
"""Everything short of a manifest dependency that says this capability exists.
|
|
954
|
+
|
|
955
|
+
The probe concluded from manifests alone, so a project with a working
|
|
956
|
+
home-grown error channel was reported as having none — and the finding read
|
|
957
|
+
`no error reporting found`, which is an assertion about the product rather than
|
|
958
|
+
about the manifest the probe actually read. Ordered cheapest-first, stopping at
|
|
959
|
+
the first hit, and the KIND of evidence is returned because it decides the
|
|
960
|
+
verdict: code or schema is `clean`, documentary alone is not a finding either.
|
|
961
|
+
"""
|
|
962
|
+
# `out_rel` is where THIS run writes; a probe that reads it takes the audit's
|
|
963
|
+
# own artefacts for project state. There is no `ctx.excluded()` — the first draft
|
|
964
|
+
# guarded with `hasattr` and produced a branch that was permanently false, which
|
|
965
|
+
# is a filter that silently does nothing.
|
|
966
|
+
out = (ctx.out_rel or "").rstrip("/") + "/"
|
|
967
|
+
files = [f for f in (tracked_files(ctx.root) or []) if not f.startswith(out)]
|
|
968
|
+
for rel in files:
|
|
969
|
+
base = os.path.basename(rel)
|
|
970
|
+
if HOMEGROWN_TELEMETRY.search(base):
|
|
971
|
+
return ("module", rel)
|
|
972
|
+
for rel in files:
|
|
973
|
+
low = rel.lower()
|
|
974
|
+
if ("migration" in low or "schema" in low or low.endswith(".sql")) \
|
|
975
|
+
and HOMEGROWN_TELEMETRY.search(_read(os.path.join(ctx.root, rel), 40000)):
|
|
976
|
+
return ("schema", rel)
|
|
977
|
+
for marker in ("README.md", "CLAUDE.md", "AGENTS.md", "ARCHITECTURE.md"):
|
|
978
|
+
path = os.path.join(ctx.root, marker)
|
|
979
|
+
if os.path.exists(path) and HOMEGROWN_TELEMETRY.search(_read(path, 60000)):
|
|
980
|
+
return ("documented", marker)
|
|
981
|
+
return (None, None)
|
|
982
|
+
|
|
983
|
+
|
|
943
984
|
@probe("telemetry", "prod", needs=())
|
|
944
985
|
def _p_telemetry(ctx):
|
|
945
986
|
found = ctx.profile.get("telemetry") or []
|
|
@@ -950,16 +991,29 @@ def _p_telemetry(ctx):
|
|
|
950
991
|
if not deploy:
|
|
951
992
|
return Result("clean", "no deploy target declared — a library or tool, "
|
|
952
993
|
"for which absent telemetry is a design, not a gap")
|
|
994
|
+
kind, where = _telemetry_evidence(ctx)
|
|
995
|
+
if kind in ("module", "schema"):
|
|
996
|
+
return Result("clean", "no telemetry dependency, but a home-grown channel "
|
|
997
|
+
"is present (%s: %s) — the manifest was the wrong "
|
|
998
|
+
"place to look" % (kind, where))
|
|
999
|
+
if kind == "documented":
|
|
1000
|
+
# Documentary evidence cannot establish that the channel WORKS, and it
|
|
1001
|
+
# cannot establish that it is absent either. Asserting absence over a
|
|
1002
|
+
# document that describes the opposite is the defect this branch exists for.
|
|
1003
|
+
return Result("blind", "no telemetry dependency and no module named for "
|
|
1004
|
+
"one, but %s describes an error channel — whether it "
|
|
1005
|
+
"works is not answerable from the tree" % where)
|
|
953
1006
|
return Result("finding", "no error reporting found", findings=[_finding(
|
|
954
1007
|
"telemetry", "manifests",
|
|
955
1008
|
"A deployed surface reports no errors anywhere its maintainer can see",
|
|
956
1009
|
"medium", 2, 2,
|
|
957
1010
|
"add an error reporter, or record the decision not to — the gap worth "
|
|
958
1011
|
"closing is that nobody wrote down which it is",
|
|
959
|
-
detail="Deploy targets declared (%s) with no telemetry dependency in "
|
|
960
|
-
"
|
|
961
|
-
|
|
962
|
-
|
|
1012
|
+
detail="Deploy targets declared (%s) with no telemetry dependency in any "
|
|
1013
|
+
"manifest, no module named for an error channel, and no mention in "
|
|
1014
|
+
"the documentation entry points. A failure on a user's machine is "
|
|
1015
|
+
"invisible." % (", ".join(deploy) or "none"),
|
|
1016
|
+
evidence="languages=%s deploy=%s telemetry=[] homegrown=none"
|
|
963
1017
|
% (",".join(langs), ",".join(deploy)))])
|
|
964
1018
|
|
|
965
1019
|
|
|
@@ -1207,8 +1261,11 @@ def main(argv=None):
|
|
|
1207
1261
|
ap.add_argument("--root", default=".", help="project root (default: .)")
|
|
1208
1262
|
ap.add_argument("--out", default=None,
|
|
1209
1263
|
help="output directory (default: <root>/docs/audit)")
|
|
1264
|
+
ap.add_argument("--report", action="store_true",
|
|
1265
|
+
help="also write the HTML page and open it (default: the JSON "
|
|
1266
|
+
"sidecar and a stdout summary only)")
|
|
1210
1267
|
ap.add_argument("--no-open", action="store_true",
|
|
1211
|
-
help="
|
|
1268
|
+
help="with --report, write the page but do not open it")
|
|
1212
1269
|
ap.add_argument("--offline", action="store_true",
|
|
1213
1270
|
help="skip every probe that needs the network")
|
|
1214
1271
|
ap.add_argument("--json", action="store_true",
|
|
@@ -1234,11 +1291,20 @@ def main(argv=None):
|
|
|
1234
1291
|
|
|
1235
1292
|
json_path = os.path.join(out_dir, base + ".json")
|
|
1236
1293
|
html_path = os.path.join(out_dir, base + ".html")
|
|
1294
|
+
# The sidecar is unconditional: it is what makes this a ratchet rather than a
|
|
1295
|
+
# snapshot, and `carry_forward` above has already read the previous one. Skipping
|
|
1296
|
+
# it would silently turn every future run into a first run.
|
|
1297
|
+
#
|
|
1298
|
+
# The PAGE is a report, and a report is an artefact that outlives the
|
|
1299
|
+
# conversation. Writing one nobody asked for leaves a document in the tree that
|
|
1300
|
+
# nobody ordered and nobody maintains — and, being untracked HTML under `docs/`,
|
|
1301
|
+
# it is one `git add -A` from the product's history. So it is opt-in.
|
|
1237
1302
|
with open(json_path, "w", encoding="utf-8") as fh:
|
|
1238
1303
|
json.dump(payload, fh, indent=1, ensure_ascii=False, sort_keys=True)
|
|
1239
1304
|
fh.write("\n")
|
|
1240
|
-
|
|
1241
|
-
|
|
1305
|
+
if args.report:
|
|
1306
|
+
with open(html_path, "w", encoding="utf-8") as fh:
|
|
1307
|
+
fh.write(render_html(payload))
|
|
1242
1308
|
|
|
1243
1309
|
if args.json:
|
|
1244
1310
|
print(json.dumps(payload, indent=1, ensure_ascii=False))
|
|
@@ -1254,9 +1320,13 @@ def main(argv=None):
|
|
|
1254
1320
|
else:
|
|
1255
1321
|
print(" closed %d · new %d · still open %d"
|
|
1256
1322
|
% (len(r["closed"]), len(r["new"]), len(r["carried"])))
|
|
1257
|
-
|
|
1323
|
+
if args.report:
|
|
1324
|
+
print(" %s\n %s" % (html_path, json_path))
|
|
1325
|
+
else:
|
|
1326
|
+
print(" %s" % json_path)
|
|
1327
|
+
print(" no page written — pass `--report` for the HTML view")
|
|
1258
1328
|
|
|
1259
|
-
if not args.no_open:
|
|
1329
|
+
if args.report and not args.no_open:
|
|
1260
1330
|
ok, note = open_in_browser(html_path)
|
|
1261
1331
|
if not ok:
|
|
1262
1332
|
sys.stderr.write("could not open the report: %s\n" % note)
|
|
@@ -29,11 +29,11 @@ to be enforced and is not is the same failure as a gate that prints `FAIL` and e
|
|
|
29
29
|
| 1 | **Change the axis when it stops paying** | any second review or audit pass | count new findings and self-inflicted ones separately | self-inflicted exceeds new → change the axis, do not "look again more carefully" |
|
|
30
30
|
| 2 | **Absence needs its own check** | any two-layer model — entity/table, screen/frame, task/module, requirement/test | compute the mapping in **both** directions | zero orphans each way, printed |
|
|
31
31
|
| 3 | **Verify closure against the artefact** | closing anything that names a design, a build or a file | open the artefact, not the document describing it | the artefact carries a marker naming what it was last verified against |
|
|
32
|
-
| 4 | **Probe the detector** | every new check, lint rule or assertion | plant the defect → run → assert non-zero; restore → assert zero | both directions observed once, and recorded |
|
|
32
|
+
| 4 | **Probe the detector** | every new check, lint rule or assertion | plant the defect → run → assert non-zero; restore → assert zero. **The plant asserts it CHANGED the file** — `assert text.count(old) == 1` before substituting — and the expected verdict is stated before the run, because a green suite and an unlanded plant are the same output | both directions observed once, and recorded; the plant's own effect asserted, and restored from a copy taken before it rather than by `git checkout`, which discards whatever else was uncommitted |
|
|
33
33
|
| 5 | **Doubt the probe first** | a check that stays silent when it should fire | prove the planted defect exists in the text the check actually parses | the probe is shown to have changed what the check reads |
|
|
34
34
|
| 6 | **Sweep the class, not the finding** | any correction | grep the whole corpus for the same shape before closing | the class is gated, or named in the note so the next reader can finish it |
|
|
35
35
|
| 7 | **Ratchet, never TODO** | anything that cannot be fixed now | a named, counted set printed on every run | the number may only fall, and `OK` prints it beside itself |
|
|
36
|
-
| 8 | **Compute, never restate** | any count or cross-reference stated in prose | derive it from the source at check time | the stated number and the computed one are the same object |
|
|
36
|
+
| 8 | **Compute, never restate** | any count or cross-reference stated in prose | derive it from the source at check time. **Where the token could occur inside something else, print the matched items classified beside the number** — `grep -oE '<token>' \| sort \| uniq -c`, never a bare `grep -c` — because *eighteen lines containing these three digits* and *eighteen throttle events* are different claims and only one of them was true | the stated number and the computed one are the same object, and the run has looked at WHAT matched, not only how many |
|
|
37
37
|
| 9 | **A generator seeds green** | any scaffold, template or code generator | run the generated project's own checks inside the generator's tests | fresh output exits zero |
|
|
38
38
|
| 10 | **Measure a detector before trusting it** | a new heuristic check | run it over the real corpus and count false positives | zero false positives, or the heuristic is replaced by a deterministic rule |
|
|
39
39
|
| 11 | **A gate's exit code is part of its output** | any gate or CI step | run it against a planted defect and assert on `$?` | non-zero, not merely a `FAIL` line on stdout |
|
|
@@ -48,6 +48,16 @@ to be enforced and is not is the same failure as a gate that prints `FAIL` and e
|
|
|
48
48
|
| 20 | **When a thing exists twice, ask which one is used — not whether they agree** | any artifact with a second copy: two build files, a vendored library, a schema and its mirror, doctrine in two documents | find the **consumer** and read what it names; the copies cannot answer this about themselves | the run names the file the build/test/deploy actually reads, quoted from the consumer |
|
|
49
49
|
| 21 | **A step that consumes what a later step produces is a deadlock, not an ordering** | any sequence where a check reads state another step writes — a prune reading stamps, a gate reading a ledger, a report reading counts | trace each input to the step that writes it; if that step is downstream, the check has never run on real data | every check names the step that produces its input, and that step is upstream of it |
|
|
50
50
|
| 22 | **An operation that changes nothing reports the same as one that changed everything** | a substitution whose needle is absent · a command whose output is suppressed at a decision point · an edit issued against a shape the file does not have | assert the effect, not the call: a replacement must report how many times it matched, and a command whose exit code governs the next step is never piped or silenced | four incidents in two programmes, each invisible until something downstream failed — an import that never landed, a doctrine phrase worded differently, a `gh` call refused behind `>/dev/null`, and a test piped to `head` so that `$?` belonged to `head` |
|
|
51
|
+
| 23 | **A commit reference is a prediction until the commit exists** | any record naming a sha, and by extension any token that looks like evidence — a line number, a file path, a flag name, a test name | fill the field AFTER the object exists and resolve it (`git cat-file -e <sha>`); where the record must ship inside the commit it names, name the commit that carried the CHANGE and let the record follow | every reference in the record resolves, checked by a command, not by its shape |
|
|
52
|
+
| 24 | **A guard that outlives the name it guards goes vacuous, not red** | renaming or replacing any construct other files name | search the test suite for the old name in the same pass; and derive a guard's subject list from a **runtime declaration**, never a literal maintained by hand | the guard names its subjects by reading what the code says about itself, so a new member is covered by construction |
|
|
53
|
+
| 25 | **An item leaving a ranked list is not evidence about that item** | any fix verified against a top-N report, a truncated list, or "it no longer appears" | re-run the SAME measurement of the SAME subject; and make every truncating report say what it dropped — `showing 5 of 12` | the closing evidence names the subject and its measured value, not its absence from a window |
|
|
54
|
+
| 26 | **A model too small to reproduce the defect cannot certify the fix** | any reduced case, minimal repro, fixture or scratch tree standing in for the real subject | reproduce the defect in the model FIRST; if it goes green on the unfixed input it is not a harness | the fidelity gap is stated as a list of what the model omits, with numbers, not as prose that invites reasoning past it |
|
|
55
|
+
| 27 | **A sweep states its denominator before it starts** | any pass over an enumerable corpus — locales, call sites, files, tools | name the size of the corpus up front and the number converted at close; and derive the guard from the set the project already enumerates, never from a list typed into the test | the closing evidence is a re-run of the finding census, not the diff — the diff shows what was touched, the census shows what is left |
|
|
56
|
+
| 28 | **Guard capability, not incompleteness** | any completeness metric, coverage score or "N of M" axis | assert that an input exists for which the axis answers *no*; a count that only ever rises cannot tell a finished axis from one that lost the ability to fail | the probe finds its own subject rather than naming a fixture, and an axis that cannot say no is labelled WHERE THE NUMBER IS PUBLISHED |
|
|
57
|
+
| 29 | **A census over a localised corpus reports per locale** | any pattern matched across translations, regions or notations | print subjects examined and subjects matched **per locale**; check the notation for numbers, units and signs in each target first | a locale reporting zero where its neighbours report several is raised as a question by the report's own shape — and where the subject is one fact in N dictionaries, compare the dictionaries to EACH OTHER, which needs no notation knowledge at all |
|
|
58
|
+
| 30 | **A gate scoped by exclusion judges every newcomer silently** | any lint, format or check configured with an ignore list rather than an include list | scope by what it COVERS; where the tool has no such switch, assert the covered set and fail on additions, deriving the allowlist by measuring the tool's own reach | the failure message names the intruder and the remedy, because the operator meets it mid-release |
|
|
59
|
+
| 31 | **A gate needs a third verdict — could not measure** | any check that can time out, lose a dependency, or run on a contended machine | distinguish *failed* from *could not measure* with different exit codes, and say which; neither is a verdict about the code | the refusal names its remedy, and a test that waits for a DURATION rather than an event is treated as a defect in the test |
|
|
60
|
+
| 32 | **A test whose assertion is implied by compilation is not a test** | any test whose name promises a behaviour | delete the subject: if that is a type or import error rather than a red test, the test is decoration | removing one asks what the name PROMISED and whether anything checks it — four of five such cases here were covered elsewhere and the fifth was a real gap only visible because the name was read as a claim |
|
|
51
61
|
|
|
52
62
|
---
|
|
53
63
|
|
|
@@ -266,7 +276,7 @@ evidence that something left, and the log below says what.
|
|
|
266
276
|
|
|
267
277
|
### Retired
|
|
268
278
|
|
|
269
|
-
**Numbers issued so far:
|
|
279
|
+
**Numbers issued so far: 32.** This is the high-water mark, and it is the only number
|
|
270
280
|
this file states about itself — deliberately, because the gap that proves a rule left
|
|
271
281
|
cannot be computed from the table alone: **deleting the highest-numbered rule shrinks
|
|
272
282
|
the maximum with it, and no gap ever opens.** That false negative shipped in the first
|
|
@@ -14,6 +14,7 @@ the **order** the rows come off it.
|
|
|
14
14
|
- 2. Confidence and Ease
|
|
15
15
|
- 3. The order, and why it is not a product
|
|
16
16
|
- 4. Research before scoring, not after
|
|
17
|
+
- Effort does not rank a finding, but the fixer needs a cost rule
|
|
17
18
|
- 5. Where the human goes
|
|
18
19
|
- 6. Two models in one family, and why both are right
|
|
19
20
|
|
|
@@ -128,6 +129,27 @@ four fields are the contract; anything else is a subagent that spent context to
|
|
|
128
129
|
|
|
129
130
|
**Do not fan out to score two rows.** The dispatch costs more than reading them.
|
|
130
131
|
|
|
132
|
+
## Effort does not rank a finding, but the fixer needs a cost rule
|
|
133
|
+
|
|
134
|
+
The existing rule — **the finder may not rank by effort** — is about the finder, and it
|
|
135
|
+
holds. It says nothing about the moment somebody decides what to do with the row, and the
|
|
136
|
+
silence there was read as *cost never enters*, which is not what anyone means.
|
|
137
|
+
|
|
138
|
+
So the split, stated rather than inferred:
|
|
139
|
+
|
|
140
|
+
- **The finder** still may not rank by effort. What a fix costs is not a property of the
|
|
141
|
+
defect.
|
|
142
|
+
- **The fixer**, at the point of deciding, weighs remedy cost against **measured harm** —
|
|
143
|
+
not against the finding's stated severity. Severity is the finder's estimate of what
|
|
144
|
+
could happen; harm is what did.
|
|
145
|
+
- **Where the harm measures zero the row is not deleted.** It is priced as *latent* and
|
|
146
|
+
carries its measurement, so the next reader inherits the number rather than the alarm.
|
|
147
|
+
A row deleted for measuring zero takes the measurement with it and the next audit
|
|
148
|
+
re-derives the same alarm from the same mechanism.
|
|
149
|
+
- **Where the path is irreversible, cost loses.** A cheap guard on something that cannot
|
|
150
|
+
be undone ships whether or not the failure has been observed — the asymmetry is the
|
|
151
|
+
whole reason to have a rule instead of a judgement.
|
|
152
|
+
|
|
131
153
|
## 5. Where the human goes
|
|
132
154
|
|
|
133
155
|
The operator is a **scarce, high-value input**, and the failure mode is spending them one
|
|
@@ -39,6 +39,8 @@ file exists to stop.
|
|
|
39
39
|
- Three grades of fix — take the highest one that can work
|
|
40
40
|
- Stamp first, then prune, then write
|
|
41
41
|
- The prune — mandatory, and it runs after the stamp
|
|
42
|
+
- When the prune cannot run, say so — it is not the same as nothing to prune
|
|
43
|
+
- A fix to one call site closes a call site, not a class
|
|
42
44
|
- The loop closes at stage 0
|
|
43
45
|
- Where a lesson goes when it is not about this project
|
|
44
46
|
- Publishing the insight — the skill learns from every project that runs it
|
|
@@ -300,6 +302,33 @@ GATE 10 acceptance: PASS — 14/14 REQ verified
|
|
|
300
302
|
|
|
301
303
|
A pruned list that nobody prints is a list that quietly grows back.
|
|
302
304
|
|
|
305
|
+
## When the prune cannot run, say so — it is not the same as nothing to prune
|
|
306
|
+
|
|
307
|
+
The prune retires a standing instruction against its triggers, and the cold trigger counts
|
|
308
|
+
run stamps. Meeting a list whose rows carry **no id and no retirement condition**, the
|
|
309
|
+
honest behaviour was undefined — so a careful run printed counts and stopped while a
|
|
310
|
+
careless one could delete nine rules, and both looked like *the prune ran*.
|
|
311
|
+
|
|
312
|
+
- **Backfill first, once.** On meeting rows with no id, the prune's first act is to mint
|
|
313
|
+
ids and propose a retirement condition per row **from the row's own text**, printed for
|
|
314
|
+
the operator to accept. One pass, and the trigger works forever after.
|
|
315
|
+
- **`blocked` is a named outcome**, printed like any other: what could not be evaluated
|
|
316
|
+
and why. An undefined case is where the spread between two runs lives.
|
|
317
|
+
- **A hard cap plus an unrunnable retirement condition is a deadlock.** Either the cap
|
|
318
|
+
yields until the backfill is done, or the backfill is a precondition of enforcing it —
|
|
319
|
+
but not both, and the file says which.
|
|
320
|
+
|
|
321
|
+
## A fix to one call site closes a call site, not a class
|
|
322
|
+
|
|
323
|
+
- **A fix other call sites will need is exported before it is used.** The test is
|
|
324
|
+
mechanical: if the same defect could exist in a second file, the fix does not stay
|
|
325
|
+
private to the first.
|
|
326
|
+
- **Closing a defect includes a census of its siblings, by BEHAVIOUR rather than by a list
|
|
327
|
+
of names** — *every script that spawns a browser*, found by searching for the spawn, not
|
|
328
|
+
by remembering the five. A list of names is how the previous pass missed four of them.
|
|
329
|
+
- **The entry states how many call sites the class had and how many were fixed.** *Fixed*
|
|
330
|
+
and *fixed in one of five* currently read identically, and only one of them closes a row.
|
|
331
|
+
|
|
303
332
|
## The loop closes at stage 0
|
|
304
333
|
|
|
305
334
|
The standing instructions are an **instruction source**, not background reading:
|