task-pipeline-skill 1.79.1 → 1.81.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +224 -0
- package/CONTRIBUTING.md +15 -0
- package/README.md +4 -3
- package/SKILL-CARD.md +1 -1
- package/cursor/rules/task-pipeline.mdc +3 -1
- package/evals/RESULTS.md +212 -9
- package/evals/evidence-docs.evals.json +109 -0
- package/evals/project-audit.evals.json +108 -0
- package/evals/run.py +47 -19
- package/package.json +4 -2
- package/plugins/task-pipeline/.claude-plugin/plugin.json +3 -2
- package/plugins/task-pipeline/commands/task-pipeline.md +2 -1
- package/plugins/task-pipeline/hooks/gate-observer.sh +19 -2
- package/plugins/task-pipeline/skills/evidence-docs/SKILL.md +1 -0
- package/plugins/task-pipeline/skills/project-audit/SKILL.md +7 -2
- package/plugins/task-pipeline/skills/task-pipeline/SKILL.md +49 -56
- package/plugins/task-pipeline/skills/task-pipeline/pipeline.example.json +1 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/acceptance.md +1 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/adoption.md +15 -4
- package/plugins/task-pipeline/skills/task-pipeline/references/artifacts.md +4 -2
- package/plugins/task-pipeline/skills/task-pipeline/references/brainstorm.md +6 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/build.md +11 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/certification.md +7 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/companion-skills.md +33 -2
- package/plugins/task-pipeline/skills/task-pipeline/references/documentation.md +2 -2
- package/plugins/task-pipeline/skills/task-pipeline/references/exposure.md +7 -3
- package/plugins/task-pipeline/skills/task-pipeline/references/gates.md +17 -185
- package/plugins/task-pipeline/skills/task-pipeline/references/model-tiering.md +13 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/portability.md +1 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/probing.md +202 -0
- package/plugins/task-pipeline/skills/task-pipeline/references/progress.md +8 -4
- package/plugins/task-pipeline/skills/task-pipeline/references/spec.md +60 -1
- package/plugins/task-pipeline/skills/task-pipeline/references/stages.md +32 -54
- package/plugins/task-pipeline/skills/task-pipeline/references/work-graph.md +1 -1
- package/plugins/task-pipeline/skills/task-pipeline/scripts/graph.py +28 -5
- package/plugins/task-pipeline/skills/task-pipeline/templates/backlog.md +6 -2
- package/plugins/task-pipeline/skills/task-pipeline/templates/run.md +2 -2
|
@@ -30,13 +30,9 @@ elsewhere and is not restated here:
|
|
|
30
30
|
- False success — when a mechanism reports a win it never checked
|
|
31
31
|
- Anatomy of a project gate
|
|
32
32
|
- Writing the check itself
|
|
33
|
-
- Probing — plant, run, restore
|
|
34
|
-
- A probe rots, and every way it rots reports green
|
|
35
33
|
- A ratchet prices the rule, not the exception
|
|
36
34
|
- Run the whole suite locally before you push the tag
|
|
37
|
-
- The neighbour probe — plant the evidence outside the subject
|
|
38
35
|
- A ratchet's matcher is itself a check, and it needs a near-miss
|
|
39
|
-
- A green probe is evidence only if the mutation is known to have landed
|
|
40
36
|
- The false-positive budget
|
|
41
37
|
- Ratchets
|
|
42
38
|
- The result is the goal; the check is how you know
|
|
@@ -107,6 +103,15 @@ Gate assignment is the operator's call and the framework fixes none — so shipp
|
|
|
107
103
|
reclassified stage list would contradict the sentence above it. The type exists; the
|
|
108
104
|
project chooses where it applies.
|
|
109
105
|
|
|
106
|
+
**The rubric steers the builder before it judges the work.** A judgment gate's
|
|
107
|
+
criteria are read twice — by the judge at the gate, and earlier by whoever builds
|
|
108
|
+
toward it — and the second reading is absorbed as direction: Anthropic reports
|
|
109
|
+
agents converging on one aesthetic because a rubric's adjective ("museum
|
|
110
|
+
quality") was obeyed as a target rather than weighed as a bar (read 2026-08-30).
|
|
111
|
+
So write criteria as the direction you want taken, not only as the bar to clear;
|
|
112
|
+
the words will be obeyed either way, and a rubric written carelessly is an
|
|
113
|
+
instruction issued accidentally.
|
|
114
|
+
|
|
110
115
|
| Rationalization | Why it is wrong |
|
|
111
116
|
|---|---|
|
|
112
117
|
| *"The reviewer approved it, so the gate passed."* | It did — as a judgement. Type it as one, or the table claims a machine agreed |
|
|
@@ -191,7 +196,10 @@ Four preconditions. Skipping any of them turns a run into a claim.
|
|
|
191
196
|
added to an already-red base passes for the wrong reason and proves nothing.
|
|
192
197
|
2. **The check has been probed.** Green from a check nobody has watched fail is
|
|
193
198
|
worth nothing. If you did not plant the defect, you do not know what the green
|
|
194
|
-
means.
|
|
199
|
+
means. How to plant, what a probe owes, and every way one rots is
|
|
200
|
+
[`probing.md`](probing.md) — the harness is `test/probe.py`
|
|
201
|
+
(`npm run test:probe`), and hand-rolling a fourth copy of its loop is how
|
|
202
|
+
three probes in one day proved the wrong guard.
|
|
195
203
|
3. **You have read its scope header** and know what it does **not** cover. A gate
|
|
196
204
|
is evidence for exactly the surface it walks; quoting it beyond that is how
|
|
197
205
|
"the gate is green" becomes a false statement made in good faith.
|
|
@@ -338,89 +346,6 @@ exit 0
|
|
|
338
346
|
|
|
339
347
|
---
|
|
340
348
|
|
|
341
|
-
## Probing — plant, run, restore
|
|
342
|
-
|
|
343
|
-
The law is [`audit.md`](audit.md)'s exit criterion. The procedure is this, and it is
|
|
344
|
-
not optional for a check you intend to trust:
|
|
345
|
-
|
|
346
|
-
```bash
|
|
347
|
-
cp -R . /tmp/probe && cd /tmp/probe
|
|
348
|
-
python3 - <<'PY' # plant IN PYTHON, never sed -i
|
|
349
|
-
p = "docs/DECISIONS.md"
|
|
350
|
-
s = open(p).read().replace("DEC-0001", "DEC-9999", 1)
|
|
351
|
-
open(p, "w").write(s)
|
|
352
|
-
PY
|
|
353
|
-
bash scripts/check-docs.sh; echo "planted -> exit=$?" # MUST be non-zero
|
|
354
|
-
cd - && bash scripts/check-docs.sh; echo "clean -> exit=$?" # MUST be 0
|
|
355
|
-
```
|
|
356
|
-
|
|
357
|
-
**Assert on `$?`, not on a `FAIL` line in the output.** A line on stdout is a
|
|
358
|
-
decoration; the exit code is what CI reads.
|
|
359
|
-
|
|
360
|
-
**Doubt the probe before you doubt the check.** On the project this comes from,
|
|
361
|
-
**four of five** silent probes were the probe's fault: one added a *definition*
|
|
362
|
-
where the check looks for an unresolved *reference*; one edited a string whose
|
|
363
|
-
whitespace did not match; one hit the first prose mention instead of the table row;
|
|
364
|
-
one flipped a row whose cell was already empty, so nothing was planted. A silent
|
|
365
|
-
check is a claim about two things, and the probe is the one to doubt first — prove
|
|
366
|
-
your edit landed in the text the check actually parses.
|
|
367
|
-
|
|
368
|
-
**Three assertions, and the third is the one hand-rolled probes keep missing.** A
|
|
369
|
-
plant that trips some *other* check has proved that other check works. Three probes
|
|
370
|
-
in one day passed that way: one removed 1 of 3 identical lines and left the shape
|
|
371
|
-
intact; one decremented a number inside an **already-released** section; one deleted
|
|
372
|
-
the shouted spelling of a phrase and left the lowercase one. Each landed somewhere
|
|
373
|
-
real and demonstrated nothing about the guard it was written for. So:
|
|
374
|
-
|
|
375
|
-
1. **the substitution landed** — `replace` matching nothing returns the string
|
|
376
|
-
unchanged and raises nothing;
|
|
377
|
-
2. **the exit code is non-zero** — not a `FAIL` line on stdout;
|
|
378
|
-
3. **the message that fired belongs to the guard under test** — named up front, not
|
|
379
|
-
recognised afterwards.
|
|
380
|
-
|
|
381
|
-
`test/probe.py` does all three (`npm run test:probe` self-tests the harness, including
|
|
382
|
-
its own failure branches, because a harness whose failure path has never executed is
|
|
383
|
-
the thing it exists to stop). Declare the plant as `Plant(label, path, old, new,
|
|
384
|
-
expect=<the guard's own words>)` rather than hand-rolling a fourth copy of the loop.
|
|
385
|
-
|
|
386
|
-
**Record the probe.** One line per section, in the change that ships the check.
|
|
387
|
-
Otherwise the next reader has to redo it to know whether it was ever done.
|
|
388
|
-
|
|
389
|
-
---
|
|
390
|
-
|
|
391
|
-
## A probe rots, and every way it rots reports green
|
|
392
|
-
|
|
393
|
-
The three assertions above prove a probe works **today**. They say nothing about the
|
|
394
|
-
day after, and a probe is uniquely exposed: the thing it guards is the thing that moves
|
|
395
|
-
it. Four rotted in one release on 2026-08-22, and none of them failed loudly — two
|
|
396
|
-
reported `caught`, two reported nothing at all.
|
|
397
|
-
|
|
398
|
-
**1. The anchor is a literal the guarded thing moves.** A probe pinned to *"the bundle
|
|
399
|
-
is N reference files"* stops landing the day a reference file is added — on the release
|
|
400
|
-
that changes the very number it guards. Same for a version, a count word, a phrase a
|
|
401
|
-
release rewrites. **Derive the anchor**: read whatever the text currently says and make
|
|
402
|
-
*that* wrong. A probe that computes `wrong = stated - 1` never needs maintaining.
|
|
403
|
-
|
|
404
|
-
**2. The precondition is inherited from the tree rather than created.** This is the
|
|
405
|
-
subtle one, because it is triggered by the system working correctly. A probe that
|
|
406
|
-
narrows a declared gap to expose *releases after the newest run stamp* has nothing to
|
|
407
|
-
expose the moment a release writes an honest stamp — the newest release is now the
|
|
408
|
-
newest stamp. A probe requiring an `## Unreleased` section has none the moment a release
|
|
409
|
-
absorbs it. **Both landed. Both proved nothing.** A probe must construct the state it
|
|
410
|
-
needs: remove the stamp, write the section, then plant the defect.
|
|
411
|
-
|
|
412
|
-
**3. The document quotes the form the probe removes.** Covered as
|
|
413
|
-
[`documentation.md`](documentation.md) canon 2's second half, and it belongs here too
|
|
414
|
-
because the probe is where it surfaces: prose describing the wrong shape *with real
|
|
415
|
-
values in it* is a second instance of the shape. The probe deletes the real one, the
|
|
416
|
-
narrative still matches, the guard is silent.
|
|
417
|
-
|
|
418
|
-
**How to see it before CI does.** Ask one question per probe: *what does this probe
|
|
419
|
-
LOOK FOR, and who is allowed to change it?* Where the answer is "the thing it guards",
|
|
420
|
-
it is rotting already. A repository-wide sweep is one grep — a two-digit literal inside
|
|
421
|
-
a needle, an `assert` or a `replace` — and the triage is: the number a probe **writes**
|
|
422
|
-
is correct, the number it **looks for** is the defect.
|
|
423
|
-
|
|
424
349
|
## A ratchet prices the rule, not the exception
|
|
425
350
|
|
|
426
351
|
A floor that counts assertions is a floor that can be lowered by improving the code, if
|
|
@@ -477,61 +402,11 @@ the tag-ancestry check, the version-sync check and the run-stamp check have no e
|
|
|
477
402
|
opportunity to fire. Locally, run them the way the release does — against the tree you are
|
|
478
403
|
about to tag, with the suite the release claims.
|
|
479
404
|
|
|
480
|
-
## The neighbour probe — plant the evidence outside the subject
|
|
481
|
-
|
|
482
|
-
A probe proves a guard rejects **the phrasing its author had in mind**. That is less than
|
|
483
|
-
it looks, and the gap has one shape: **a check answered by text that is not its subject.**
|
|
484
|
-
|
|
485
|
-
Measured on one project in one session, six times, each found by a reader planting a
|
|
486
|
-
defect and watching `PASS`:
|
|
487
|
-
|
|
488
|
-
| The check's subject | The text that answered it instead |
|
|
489
|
-
|---|---|
|
|
490
|
-
| stage 2 names the loop's arming | *"it **arms** the UX track"*, present since an earlier release |
|
|
491
|
-
| the section states the authorization floor | the same phrase in a Rationalizations row, and in a section written twenty-nine releases before |
|
|
492
|
-
| the run stamps have a cap | the **standing instructions'** `max 10`, in the same table cell — and, once the check was narrowed past it, the same cap moved to the other side of the `·` |
|
|
493
|
-
| a section read in full stays capped | rows under **one** heading, while a second heading in the same file held forty more |
|
|
494
|
-
|
|
495
|
-
Every one of those guards had a probe. Every probe fired. The probes and the guards were
|
|
496
|
-
written in the same hour from the same reading, so they shared the same blind spot.
|
|
497
|
-
|
|
498
|
-
**So a guard that reads a scoped span owes a second probe, and it plants in two places at
|
|
499
|
-
once:**
|
|
500
|
-
|
|
501
|
-
1. **break the subject** — remove the thing the guard is about;
|
|
502
|
-
2. **plant the guard's own evidence next door** — in a sibling section, an adjacent table
|
|
503
|
-
cell, a rationalizations row, the other side of a separator;
|
|
504
|
-
3. require the guard to **still fail**.
|
|
505
|
-
|
|
506
|
-
A guard that passes step 3 is reading its subject. A guard that goes green is reading the
|
|
507
|
-
neighbourhood, and the ordinary probe cannot tell the difference — which is why this one
|
|
508
|
-
is separate rather than a stricter version of it.
|
|
509
|
-
|
|
510
|
-
**Step 2 means the literal the predicate matches *today*, read out of the guard.** The
|
|
511
|
-
first three neighbour probes written against this section got that wrong on their first
|
|
512
|
-
run: one planted the needles of two **retired** predicates, which proves the guards that
|
|
513
|
-
used to exist were neighbour-answerable and says nothing about the one that does. A
|
|
514
|
-
neighbour probe keyed to a needle the guard no longer reads is the same defect it was
|
|
515
|
-
written to catch, one level up.
|
|
516
|
-
|
|
517
|
-
**A probe that only deletes is not a neighbour probe.** It may still be a correct probe —
|
|
518
|
-
but if it relies on copies that already sit next door, it must **assert they are there**.
|
|
519
|
-
Otherwise a later edit removes them and the probe quietly becomes a delete-only test that
|
|
520
|
-
still passes, having stopped testing the thing it is named for.
|
|
521
|
-
|
|
522
|
-
**Positional narrowing is not scoping.** Three of the six were "fixed" by cutting the
|
|
523
|
-
search down to a row, then to everything after a phrase, and fell each time to text that
|
|
524
|
-
was still inside the cut. Scope by *what the span is about*: split to the cell, then to
|
|
525
|
-
the item, then match on flattened text so an emphasis marker cannot hide the boundary.
|
|
526
|
-
|
|
527
|
-
**And state the span in the guard.** One line above the predicate — *what it reads, and
|
|
528
|
-
where that ends*. It costs nothing and it is the only part of a check a later reader can
|
|
529
|
-
disagree with before the defect arrives.
|
|
530
|
-
|
|
531
405
|
## A ratchet's matcher is itself a check, and it needs a near-miss
|
|
532
406
|
|
|
533
|
-
Reported from another project through `retro.publish`, and it is the neighbour
|
|
534
|
-
own class
|
|
407
|
+
Reported from another project through `retro.publish`, and it is the neighbour
|
|
408
|
+
probe's own class ([`probing.md`](probing.md) → *The neighbour probe*) arrived at
|
|
409
|
+
independently — which is the strongest evidence either has.
|
|
535
410
|
|
|
536
411
|
A run built a ratchet to hold a coverage debt: a list of units with no test, a guard that
|
|
537
412
|
fails when the list grows, a count printed at the gate. Exactly the shape
|
|
@@ -558,49 +433,6 @@ the reason.** In the reporting project the corrected count was *identical* to th
|
|
|
558
433
|
and the composition was not: rows credited falsely came back in as rows genuinely paid off
|
|
559
434
|
went out. A single number with no delta reads as a run where nothing happened.
|
|
560
435
|
|
|
561
|
-
## A green probe is evidence only if the mutation is known to have landed
|
|
562
|
-
|
|
563
|
-
Also reported from another project, three times in one day, each caught only because the
|
|
564
|
-
result was too good:
|
|
565
|
-
|
|
566
|
-
1. a scripted substitution missed on indentation — the file was unchanged and the probe
|
|
567
|
-
measured nothing;
|
|
568
|
-
2. an assertion written against a bare identifier kept matching the **import line** after
|
|
569
|
-
the field it guarded was deleted;
|
|
570
|
-
3. a file-extension alternation matched the longer extension as though it were the
|
|
571
|
-
shorter, reporting nine live files as missing.
|
|
572
|
-
|
|
573
|
-
In all three the observable was identical to success. *"See it fail once"* has an unstated
|
|
574
|
-
precondition — **that the thing you changed is the thing the check reads** — and a planted
|
|
575
|
-
defect that did not land produces the same green as a check that cannot fail.
|
|
576
|
-
|
|
577
|
-
**So a probe that mutates an existing file asserts its plant landed, in the same breath as
|
|
578
|
-
planting it.** A probe that writes a whole file has no such question: the file exists or
|
|
579
|
-
the command failed. This repository measured itself while writing this section and got the
|
|
580
|
-
number wrong three times. A hand-rolled classifier said *206 of 206 already carry it*.
|
|
581
|
-
The guard written from the rule said **22 did not** — and was itself too narrow, matching
|
|
582
|
-
one spelling of the assertion, so six probes that already had it in lower case were
|
|
583
|
-
called defective. A sweep then "fixed" those six and **corrupted five**, splitting live
|
|
584
|
-
statements. The true figure was **16**, and it took the guard, a compile check and a
|
|
585
|
-
restore from git to find it.
|
|
586
|
-
|
|
587
|
-
Two things are worth keeping from that. **The check corrected the measurement that
|
|
588
|
-
motivated it** — which is the argument for writing checks rather than counting by hand.
|
|
589
|
-
And **a check keyed to one spelling of a rule is the class two sections above**: it
|
|
590
|
-
reported as defective the probes that obeyed the rule in different words. **Every**
|
|
591
|
-
mutating probe carries the assertion now — the figure is deliberately not
|
|
592
|
-
written here. A first draft said *201*, which was true of the branch point and false in
|
|
593
|
-
the same commit, because the twelve probes added for this release are themselves mutating
|
|
594
|
-
probes. The guard computes it; a number in prose beside a check that can count is the
|
|
595
|
-
class this bundle calls restating instead of computing. Probes that write a whole file
|
|
596
|
-
need none: the file exists or the command
|
|
597
|
-
failed.
|
|
598
|
-
|
|
599
|
-
**Prefer an assertion that names the construct over one that names a substring of it.** A
|
|
600
|
-
guard written against a bare identifier survives the deletion of everything it guarded,
|
|
601
|
-
because the identifier still appears in an import. That is case 2 above and it is the same
|
|
602
|
-
class as the section before this one, one level down.
|
|
603
|
-
|
|
604
436
|
## The false-positive budget
|
|
605
437
|
|
|
606
438
|
Run a new heuristic over the **real corpus** before shipping it and count the false
|
|
@@ -772,7 +604,7 @@ of is deleted in the next refactor by someone who assumed it was dead.
|
|
|
772
604
|
|
|
773
605
|
## Cross-cutting, at every stage
|
|
774
606
|
|
|
775
|
-
|
|
607
|
+
**When anything is settled — scope, a contract, a
|
|
776
608
|
name, a policy, a vocabulary — run the Doc Loop
|
|
777
609
|
(`references/documentation.md`) before the run moves on**: reserve the id,
|
|
778
610
|
record it, resolve the question it answers, propagate by the matrix, commit
|
|
@@ -63,3 +63,16 @@ and no silent downgrade to a cheaper tier.
|
|
|
63
63
|
The recommendation is a **reminder, not a block**. If the recommended tier isn't
|
|
64
64
|
available, keep the current model, state plainly which one is in use, and run. The
|
|
65
65
|
pipeline never stalls on a model it can't get.
|
|
66
|
+
|
|
67
|
+
## A harness clause encodes a model's weakness — stress-test it per generation
|
|
68
|
+
|
|
69
|
+
Every fixed control in this pipeline — a cap, a mandatory checkpoint, a fix-loop
|
|
70
|
+
ceiling — encodes an assumption about what the confirmed model cannot yet do
|
|
71
|
+
reliably, and those assumptions expire: Anthropic reports removing an entire
|
|
72
|
+
construct from its own long-running harness when a newer model generation stopped
|
|
73
|
+
needing it (*Harness design for long-running agents*,
|
|
74
|
+
anthropic.com/engineering, read 2026-08-30). So when the confirmed tier moves a
|
|
75
|
+
generation, re-test the clauses that exist to compensate the old one instead of
|
|
76
|
+
carrying them as doctrine — a control the model has outgrown is not free, it is a
|
|
77
|
+
stop nobody can name the reason for. The retro's cold-retirement trigger is the
|
|
78
|
+
same rule applied to standing instructions; this is it applied to the harness.
|
|
@@ -63,6 +63,7 @@ a row pointing outside the bundle is the defect this file exists to catch.
|
|
|
63
63
|
| The retro: prune, cap, commits, archive | `references/retrospective.md` |
|
|
64
64
|
| Rules earned by failure | `references/learned.md` |
|
|
65
65
|
| **The routing default and its boundary** | `templates/routing-rule.md` |
|
|
66
|
+
| **Probing** — plant, run, restore; how a probe rots; the neighbour probe; the landed-mutation rule | `references/probing.md` |
|
|
66
67
|
| The seeded doc map, registers and gate | `templates/docmap.md`, `templates/decisions.md`, `templates/open-questions.md`, `templates/docgate.sh` |
|
|
67
68
|
| Which agent-introduced defects are found, and that the agent fixes them rather than the script | `templates/hygiene.sh`, `references/build.md` |
|
|
68
69
|
| What a stage-3/4 self-review must read back, and that its trace is computed numbers | `references/spec.md`, `references/planning.md`, `references/learned.md` |
|
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
# Probing — how a check is proven before it is trusted
|
|
2
|
+
|
|
3
|
+
The authoring doctrine for **probes**: planting a defect, watching the guard
|
|
4
|
+
refuse it, and keeping that proof alive after the release that ships it. It
|
|
5
|
+
lived inside [`gates.md`](gates.md) until 2026-08-31 and moved here whole —
|
|
6
|
+
one home, routed from the gate doctrine it proves. The law it serves is
|
|
7
|
+
[`audit.md`](audit.md)'s exit criterion: **a green from a check nobody has
|
|
8
|
+
watched fail is not evidence.**
|
|
9
|
+
|
|
10
|
+
## Contents
|
|
11
|
+
|
|
12
|
+
- Probing — plant, run, restore
|
|
13
|
+
- A probe rots, and every way it rots reports green
|
|
14
|
+
- The neighbour probe — plant the evidence outside the subject
|
|
15
|
+
- A green probe is evidence only if the mutation is known to have landed
|
|
16
|
+
- Rationalizations
|
|
17
|
+
|
|
18
|
+
## Probing — plant, run, restore
|
|
19
|
+
|
|
20
|
+
The law is [`audit.md`](audit.md)'s exit criterion. The procedure is this, and it is
|
|
21
|
+
not optional for a check you intend to trust:
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
cp -R . /tmp/probe && cd /tmp/probe
|
|
25
|
+
python3 - <<'PY' # plant IN PYTHON, never sed -i
|
|
26
|
+
p = "docs/DECISIONS.md"
|
|
27
|
+
s = open(p).read().replace("DEC-0001", "DEC-9999", 1)
|
|
28
|
+
open(p, "w").write(s)
|
|
29
|
+
PY
|
|
30
|
+
bash scripts/check-docs.sh; echo "planted -> exit=$?" # MUST be non-zero
|
|
31
|
+
cd - && bash scripts/check-docs.sh; echo "clean -> exit=$?" # MUST be 0
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
**Assert on `$?`, not on a `FAIL` line in the output.** A line on stdout is a
|
|
35
|
+
decoration; the exit code is what CI reads.
|
|
36
|
+
|
|
37
|
+
**Doubt the probe before you doubt the check.** On the project this comes from,
|
|
38
|
+
**four of five** silent probes were the probe's fault: one added a *definition*
|
|
39
|
+
where the check looks for an unresolved *reference*; one edited a string whose
|
|
40
|
+
whitespace did not match; one hit the first prose mention instead of the table row;
|
|
41
|
+
one flipped a row whose cell was already empty, so nothing was planted. A silent
|
|
42
|
+
check is a claim about two things, and the probe is the one to doubt first — prove
|
|
43
|
+
your edit landed in the text the check actually parses.
|
|
44
|
+
|
|
45
|
+
**Three assertions, and the third is the one hand-rolled probes keep missing.** A
|
|
46
|
+
plant that trips some *other* check has proved that other check works. Three probes
|
|
47
|
+
in one day passed that way: one removed 1 of 3 identical lines and left the shape
|
|
48
|
+
intact; one decremented a number inside an **already-released** section; one deleted
|
|
49
|
+
the shouted spelling of a phrase and left the lowercase one. Each landed somewhere
|
|
50
|
+
real and demonstrated nothing about the guard it was written for. So:
|
|
51
|
+
|
|
52
|
+
1. **the substitution landed** — `replace` matching nothing returns the string
|
|
53
|
+
unchanged and raises nothing;
|
|
54
|
+
2. **the exit code is non-zero** — not a `FAIL` line on stdout;
|
|
55
|
+
3. **the message that fired belongs to the guard under test** — named up front, not
|
|
56
|
+
recognised afterwards.
|
|
57
|
+
|
|
58
|
+
`test/probe.py` does all three (`npm run test:probe` self-tests the harness, including
|
|
59
|
+
its own failure branches, because a harness whose failure path has never executed is
|
|
60
|
+
the thing it exists to stop). Declare the plant as `Plant(label, path, old, new,
|
|
61
|
+
expect=<the guard's own words>)` rather than hand-rolling a fourth copy of the loop.
|
|
62
|
+
|
|
63
|
+
**Record the probe.** One line per section, in the change that ships the check.
|
|
64
|
+
Otherwise the next reader has to redo it to know whether it was ever done.
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## A probe rots, and every way it rots reports green
|
|
69
|
+
|
|
70
|
+
The three assertions above prove a probe works **today**. They say nothing about the
|
|
71
|
+
day after, and a probe is uniquely exposed: the thing it guards is the thing that moves
|
|
72
|
+
it. Four rotted in one release on 2026-08-22, and none of them failed loudly — two
|
|
73
|
+
reported `caught`, two reported nothing at all.
|
|
74
|
+
|
|
75
|
+
**1. The anchor is a literal the guarded thing moves.** A probe pinned to *"the bundle
|
|
76
|
+
is N reference files"* stops landing the day a reference file is added — on the release
|
|
77
|
+
that changes the very number it guards. Same for a version, a count word, a phrase a
|
|
78
|
+
release rewrites. **Derive the anchor**: read whatever the text currently says and make
|
|
79
|
+
*that* wrong. A probe that computes `wrong = stated - 1` never needs maintaining.
|
|
80
|
+
|
|
81
|
+
**2. The precondition is inherited from the tree rather than created.** This is the
|
|
82
|
+
subtle one, because it is triggered by the system working correctly. A probe that
|
|
83
|
+
narrows a declared gap to expose *releases after the newest run stamp* has nothing to
|
|
84
|
+
expose the moment a release writes an honest stamp — the newest release is now the
|
|
85
|
+
newest stamp. A probe requiring an `## Unreleased` section has none the moment a release
|
|
86
|
+
absorbs it. **Both landed. Both proved nothing.** A probe must construct the state it
|
|
87
|
+
needs: remove the stamp, write the section, then plant the defect.
|
|
88
|
+
|
|
89
|
+
**3. The document quotes the form the probe removes.** Covered as
|
|
90
|
+
[`documentation.md`](documentation.md) canon 2's second half, and it belongs here too
|
|
91
|
+
because the probe is where it surfaces: prose describing the wrong shape *with real
|
|
92
|
+
values in it* is a second instance of the shape. The probe deletes the real one, the
|
|
93
|
+
narrative still matches, the guard is silent.
|
|
94
|
+
|
|
95
|
+
**How to see it before CI does.** Ask one question per probe: *what does this probe
|
|
96
|
+
LOOK FOR, and who is allowed to change it?* Where the answer is "the thing it guards",
|
|
97
|
+
it is rotting already. A repository-wide sweep is one grep — a two-digit literal inside
|
|
98
|
+
a needle, an `assert` or a `replace` — and the triage is: the number a probe **writes**
|
|
99
|
+
is correct, the number it **looks for** is the defect.
|
|
100
|
+
|
|
101
|
+
## The neighbour probe — plant the evidence outside the subject
|
|
102
|
+
|
|
103
|
+
A probe proves a guard rejects **the phrasing its author had in mind**. That is less than
|
|
104
|
+
it looks, and the gap has one shape: **a check answered by text that is not its subject.**
|
|
105
|
+
|
|
106
|
+
Measured on one project in one session, six times, each found by a reader planting a
|
|
107
|
+
defect and watching `PASS`:
|
|
108
|
+
|
|
109
|
+
| The check's subject | The text that answered it instead |
|
|
110
|
+
|---|---|
|
|
111
|
+
| stage 2 names the loop's arming | *"it **arms** the UX track"*, present since an earlier release |
|
|
112
|
+
| the section states the authorization floor | the same phrase in a Rationalizations row, and in a section written twenty-nine releases before |
|
|
113
|
+
| the run stamps have a cap | the **standing instructions'** `max 10`, in the same table cell — and, once the check was narrowed past it, the same cap moved to the other side of the `·` |
|
|
114
|
+
| a section read in full stays capped | rows under **one** heading, while a second heading in the same file held forty more |
|
|
115
|
+
|
|
116
|
+
Every one of those guards had a probe. Every probe fired. The probes and the guards were
|
|
117
|
+
written in the same hour from the same reading, so they shared the same blind spot.
|
|
118
|
+
|
|
119
|
+
**So a guard that reads a scoped span owes a second probe, and it plants in two places at
|
|
120
|
+
once:**
|
|
121
|
+
|
|
122
|
+
1. **break the subject** — remove the thing the guard is about;
|
|
123
|
+
2. **plant the guard's own evidence next door** — in a sibling section, an adjacent table
|
|
124
|
+
cell, a rationalizations row, the other side of a separator;
|
|
125
|
+
3. require the guard to **still fail**.
|
|
126
|
+
|
|
127
|
+
A guard that passes step 3 is reading its subject. A guard that goes green is reading the
|
|
128
|
+
neighbourhood, and the ordinary probe cannot tell the difference — which is why this one
|
|
129
|
+
is separate rather than a stricter version of it.
|
|
130
|
+
|
|
131
|
+
**Step 2 means the literal the predicate matches *today*, read out of the guard.** The
|
|
132
|
+
first three neighbour probes written against this section got that wrong on their first
|
|
133
|
+
run: one planted the needles of two **retired** predicates, which proves the guards that
|
|
134
|
+
used to exist were neighbour-answerable and says nothing about the one that does. A
|
|
135
|
+
neighbour probe keyed to a needle the guard no longer reads is the same defect it was
|
|
136
|
+
written to catch, one level up.
|
|
137
|
+
|
|
138
|
+
**A probe that only deletes is not a neighbour probe.** It may still be a correct probe —
|
|
139
|
+
but if it relies on copies that already sit next door, it must **assert they are there**.
|
|
140
|
+
Otherwise a later edit removes them and the probe quietly becomes a delete-only test that
|
|
141
|
+
still passes, having stopped testing the thing it is named for.
|
|
142
|
+
|
|
143
|
+
**Positional narrowing is not scoping.** Three of the six were "fixed" by cutting the
|
|
144
|
+
search down to a row, then to everything after a phrase, and fell each time to text that
|
|
145
|
+
was still inside the cut. Scope by *what the span is about*: split to the cell, then to
|
|
146
|
+
the item, then match on flattened text so an emphasis marker cannot hide the boundary.
|
|
147
|
+
|
|
148
|
+
**And state the span in the guard.** One line above the predicate — *what it reads, and
|
|
149
|
+
where that ends*. It costs nothing and it is the only part of a check a later reader can
|
|
150
|
+
disagree with before the defect arrives.
|
|
151
|
+
|
|
152
|
+
## A green probe is evidence only if the mutation is known to have landed
|
|
153
|
+
|
|
154
|
+
Also reported from another project, three times in one day, each caught only because the
|
|
155
|
+
result was too good:
|
|
156
|
+
|
|
157
|
+
1. a scripted substitution missed on indentation — the file was unchanged and the probe
|
|
158
|
+
measured nothing;
|
|
159
|
+
2. an assertion written against a bare identifier kept matching the **import line** after
|
|
160
|
+
the field it guarded was deleted;
|
|
161
|
+
3. a file-extension alternation matched the longer extension as though it were the
|
|
162
|
+
shorter, reporting nine live files as missing.
|
|
163
|
+
|
|
164
|
+
In all three the observable was identical to success. *"See it fail once"* has an unstated
|
|
165
|
+
precondition — **that the thing you changed is the thing the check reads** — and a planted
|
|
166
|
+
defect that did not land produces the same green as a check that cannot fail.
|
|
167
|
+
|
|
168
|
+
**So a probe that mutates an existing file asserts its plant landed, in the same breath as
|
|
169
|
+
planting it.** A probe that writes a whole file has no such question: the file exists or
|
|
170
|
+
the command failed. This repository measured itself while writing this section and got the
|
|
171
|
+
number wrong three times. A hand-rolled classifier said *206 of 206 already carry it*.
|
|
172
|
+
The guard written from the rule said **22 did not** — and was itself too narrow, matching
|
|
173
|
+
one spelling of the assertion, so six probes that already had it in lower case were
|
|
174
|
+
called defective. A sweep then "fixed" those six and **corrupted five**, splitting live
|
|
175
|
+
statements. The true figure was **16**, and it took the guard, a compile check and a
|
|
176
|
+
restore from git to find it.
|
|
177
|
+
|
|
178
|
+
Two things are worth keeping from that. **The check corrected the measurement that
|
|
179
|
+
motivated it** — which is the argument for writing checks rather than counting by hand.
|
|
180
|
+
And **a check keyed to one spelling of a rule is *The neighbour probe*'s class, above**: it
|
|
181
|
+
reported as defective the probes that obeyed the rule in different words. **Every**
|
|
182
|
+
mutating probe carries the assertion now — the figure is deliberately not
|
|
183
|
+
written here. A first draft said *201*, which was true of the branch point and false in
|
|
184
|
+
the same commit, because the twelve probes added for this release are themselves mutating
|
|
185
|
+
probes. The guard computes it; a number in prose beside a check that can count is the
|
|
186
|
+
class this bundle calls restating instead of computing. Probes that write a whole file
|
|
187
|
+
need none: the file exists or the command
|
|
188
|
+
failed.
|
|
189
|
+
|
|
190
|
+
**Prefer an assertion that names the construct over one that names a substring of it.** A
|
|
191
|
+
guard written against a bare identifier survives the deletion of everything it guarded,
|
|
192
|
+
because the identifier still appears in an import. That is case 2 above and it is the same class as
|
|
193
|
+
[`gates.md`](gates.md) → *A ratchet's matcher is itself a check, and it needs a near-miss*, one level down.
|
|
194
|
+
|
|
195
|
+
## Rationalizations
|
|
196
|
+
|
|
197
|
+
| Excuse | Reality |
|
|
198
|
+
|---|---|
|
|
199
|
+
| "The check is green, that's evidence" | Only if you have seen it red. An unproven check is a decoration that reports success. |
|
|
200
|
+
| "I probed it when I wrote it" | A probe proves the check works today. The thing it guards is the thing that moves it — re-read the anchor on every release that touches the subject. |
|
|
201
|
+
| "The plant obviously landed, the file is smaller" | `replace` matching nothing returns the string unchanged and raises nothing. Assert the mutation landed, in the same breath as planting it. |
|
|
202
|
+
| "Some guard fired, so the probe passed" | A plant that trips some *other* check has proved that other check works. The message that fired must belong to the guard under test, named up front. |
|
|
@@ -63,7 +63,7 @@ changes:
|
|
|
63
63
|
task-pipeline v1.34.0 · pipeline-audit · module P1 «the progress print» (1 of 4)
|
|
64
64
|
0 ✓ 1 ✓ 2 ✓ 3 ▶ 4 · 5 · 6 · 7 · 8 · 9 · 10 ·
|
|
65
65
|
███████░░░░░░░░░░░░░░░░░░░ gates 3/11 · now 3 Spec · manual
|
|
66
|
-
board B-028 · carry-over 0 rows · exposure
|
|
66
|
+
board B-028 · carry-over 0 rows · exposure N never · unlooked 0
|
|
67
67
|
```
|
|
68
68
|
|
|
69
69
|
Four lines, and each one answers a question an operator otherwise has to ask:
|
|
@@ -142,7 +142,8 @@ its pair is task start and iteration close. The hand-back shares one and adds th
|
|
|
142
142
|
end — a rail at task start has nothing to report, and a run that ends without a hand-back
|
|
143
143
|
is the case this section exists for. A reader resolving *"both boundaries"* against the
|
|
144
144
|
other section wrote one at task start, where TASK is the only field with content. The run
|
|
145
|
-
writes a hand-back with **
|
|
145
|
+
writes a hand-back with **the sections and the two counted lists below** — the
|
|
146
|
+
block is the count, and a number restated here said *four* over a block of six. It is a gate criterion **at stage
|
|
146
147
|
10**, not a good intention: this file already carried one instruction with no gate behind
|
|
147
148
|
it (*"copy it, tick it"*), and the v1.37.0 audit found no run had ever obeyed it.
|
|
148
149
|
|
|
@@ -380,7 +381,7 @@ And why the verb prints
|
|
|
380
381
|
`unmeasured` rather than `0` when there are no such lines is the same reason again — the
|
|
381
382
|
hook being absent and the run reading nothing are **opposite facts** the ledger cannot
|
|
382
383
|
separate, so it claims neither. A `0` there would be the reassuring answer to a question
|
|
383
|
-
nobody asked, over
|
|
384
|
+
nobody asked, over a directory of reference files nobody checked.
|
|
384
385
|
|
|
385
386
|
It is a disclosure: no floor, no direction, never a target. The moment the number becomes
|
|
386
387
|
something to raise, a run will open files to raise it.
|
|
@@ -416,7 +417,10 @@ written by any run — the detector had no input, and the guard was doctrine wea
|
|
|
416
417
|
script's clothes. One file serves both readers: the guard reads the `touch:` lines,
|
|
417
418
|
this block reads the verdict rows and the iteration counter.
|
|
418
419
|
|
|
419
|
-
|
|
420
|
+
Appended, never rewritten. The full set of line shapes is declared under
|
|
421
|
+
`## Lines` in [`../templates/run.md`](../templates/run.md) — the list is the count,
|
|
422
|
+
and a count restated here drifted once already. The three these two readers
|
|
423
|
+
consume:
|
|
420
424
|
|
|
421
425
|
```
|
|
422
426
|
stage: 3 Spec — gate manual — verdict pass — 2026-08-10T14:02Z
|
|
@@ -13,6 +13,7 @@ Writing the approved design down so a zero-context implementer — human or suba
|
|
|
13
13
|
|
|
14
14
|
- Order of operations
|
|
15
15
|
- UX track (user-facing tasks only)
|
|
16
|
+
- The COPY and VISUAL tracks, and their convergence
|
|
16
17
|
- Write the spec
|
|
17
18
|
- Module dossier — when the run is one brick of a platform
|
|
18
19
|
- Self-review — before showing it
|
|
@@ -65,6 +66,60 @@ These skills are **idempotent** — extend the existing `docs/ux/` layers, never
|
|
|
65
66
|
rebuild them. If the chain already exists and is validated (typically when the run
|
|
66
67
|
entered from super-ux), verify it and embed it; build only what's missing.
|
|
67
68
|
|
|
69
|
+
## The COPY and VISUAL tracks, and their convergence
|
|
70
|
+
|
|
71
|
+
Three tracks, three questions, and they do not substitute for each other:
|
|
72
|
+
super-ux decides what the interface must **do**; `copywriting` how it
|
|
73
|
+
**sounds**; `sheleg-design` how it **looks**. Until 2026-08-10 stage 3 named
|
|
74
|
+
only the first, so a run designed a flow, then wrote its strings by taste and
|
|
75
|
+
picked its values at the keyboard — and every gate in the pipeline reported
|
|
76
|
+
green over both.
|
|
77
|
+
|
|
78
|
+
- **COPY track — how it sounds.** Every string a product's user will read is written
|
|
79
|
+
through super-ux's `copywriting`, against the brand pack (`docs/brand/voice.md`,
|
|
80
|
+
`terminology.md`, `facts.md`). No pack ⇒ `/brand-init` **before** the first string,
|
|
81
|
+
not after: a voice reverse-engineered from copy already written is a description of
|
|
82
|
+
what happened, not a decision. In scope: interface strings, errors, empty states,
|
|
83
|
+
the landing, pricing, the user-facing changelog. **Out of scope, and saying so is
|
|
84
|
+
what keeps the track honest:** commit messages, PR descriptions, code comments, a
|
|
85
|
+
developer README, internal docs. Running a brand pack over a line in a contributors'
|
|
86
|
+
changelog is the fastest way to teach an agent to route around the track.
|
|
87
|
+
- **VISUAL track — how it looks.** Where the task has a visual surface, the visual
|
|
88
|
+
layer goes through `sheleg-design` ([`companion-skills.md`](companion-skills.md)):
|
|
89
|
+
tokens and themes, typography and rhythm, motion and how it degrades to rest, the
|
|
90
|
+
boundary with Figma (tokens as variables, never raw values carried across). Not
|
|
91
|
+
through it: a purely structural change — what sits where is the UX track's — text,
|
|
92
|
+
a backend, an internal script.
|
|
93
|
+
- **Each track's refusal is a sentence, never a silence.** *"Без дизайна" / "as is"*
|
|
94
|
+
ends the visual track; *"без бренда" / "draft"* ends the copy track. Either one is
|
|
95
|
+
the operator's to make and costs nothing — but it is **recorded in the brief and
|
|
96
|
+
said out loud in the close-out**, because a track skipped silently and a track that
|
|
97
|
+
ran are the same thing in a transcript. This is the rail's `⊘` rule one layer up: a
|
|
98
|
+
skip nobody can see is indistinguishable from work that happened.
|
|
99
|
+
- **Two of the three are a parallel layer, and the third is their only real dependency.**
|
|
100
|
+
COPY and VISUAL both consume the UX track's scenarios; **neither consumes the other**.
|
|
101
|
+
Copy is written against the brand pack and the scenarios, not against tokens; the visual
|
|
102
|
+
is built from the frame and the style pack, not from strings. Writing them in a line —
|
|
103
|
+
which this doctrine did until 2026-08-15 — teaches a run to wait for a result that never
|
|
104
|
+
arrives. The order is `UX → { COPY ∥ VISUAL }`, and the only thing crossing each of
|
|
105
|
+
those two arrows is **the scenario set**.
|
|
106
|
+
- **Their convergence needs a check, and it has a real contradiction to catch.** Both land
|
|
107
|
+
on the same screen, so the failure is not that one is wrong: it is that each is right
|
|
108
|
+
alone and they disagree together. Before the spec is committed, compare the two outputs
|
|
109
|
+
and record the answer:
|
|
110
|
+
1. **A string the layout has no room for** — a label, an error or an empty state longer
|
|
111
|
+
than the frame's element, at the frame's own width.
|
|
112
|
+
2. **A state one track has and the other does not** — copy for an empty state the design
|
|
113
|
+
never drew, or a loading state drawn with no string.
|
|
114
|
+
3. **Two names for one thing** — the design system's component name against the
|
|
115
|
+
terminology file's noun, where a user reads both.
|
|
116
|
+
4. **A tone the visual contradicts** — a calm, plain register on a screen whose motion
|
|
117
|
+
and colour say urgency.
|
|
118
|
+
`Tracks converge: clean` is the answer most runs write, and writing it is the point —
|
|
119
|
+
a check whose silence is indistinguishable from not having run is not evidence. This is
|
|
120
|
+
the same rule the harvest applies at stage 0 and the build applies to a fanned-out group
|
|
121
|
+
at stage 5 ([`build.md`](build.md) §4.2a); one shape, three places.
|
|
122
|
+
|
|
68
123
|
## Write the spec
|
|
69
124
|
|
|
70
125
|
Path: `<artifacts>/specs/YYYY-MM-DD-<topic>-design.md`, committed, **same
|
|
@@ -212,7 +267,11 @@ and every REQ in the brief appears in at least one section.** For UI tasks it
|
|
|
212
267
|
additionally requires: the chain (foundation → flows → screens → scenarios)
|
|
213
268
|
designed, validated and approved; `/ux-lint` green; every user-facing requirement
|
|
214
269
|
traced to a scenario ID — or an explicit waiver from the operator recorded in the
|
|
215
|
-
spec.
|
|
270
|
+
spec. **Every user-facing string went through the COPY track or the refusal is
|
|
271
|
+
recorded, the visual layer went through the VISUAL track or the refusal is
|
|
272
|
+
recorded** — a recorded refusal passes and an unmentioned one does not — **and
|
|
273
|
+
where both tracks ran, their convergence check is recorded**: findings with the
|
|
274
|
+
ruling, or `Tracks converge: clean` (*The COPY and VISUAL tracks*, above).
|
|
216
275
|
|
|
217
276
|
With Figma on, one more, and it is mechanical: **the canonical record names a file,
|
|
218
277
|
and every frame link in `screens.md` carries that same file key.** Deep links are
|