task-pipeline-skill 1.46.0 → 1.47.1

package/CHANGELOG.md

@@ -1,5 +1,90 @@
 # Changelog
 
+## v1.47.1 — the fixes a reader found, which three releases shipped without
+
+v1.46.0 was tagged and published from a commit that carried this branch's doctrine and
+none of the three commits answering the reviewer. The defects below were already found
+and already fixed when the release went out; this is them arriving.
+
+**`docs/DOCMAP.md` forbade the register shipping beside it.** It said, in as many words,
+that no `docs/DECISIONS.md` is created here deliberately — and the release shipped that
+file. The old rule was right about the risk and wrong about the mechanism: `OQ-####`
+closes with `Resolved→DEC-####`, and a CHANGELOG version heading cannot be that target,
+because two decisions in one release collapse to a single pointer. SSOT is kept by
+direction now — the reason lives in `DECISIONS.md`, the CHANGELOG points at the id. The
+reversal is `DEC-0003`, because a register that appears without a decision is exactly
+the fork the old rule feared.
+
+**`HOW-IT-WORKS.md` called itself the version it was written under.** Its first line
+promises a rewrite every release and names the banner as the freshness signal; the
+banner read 1.45.0 while every other surface read 1.46.0. Corrected, and the two
+releases that landed from another session are named for what they were, so the gap in
+the version history is explained rather than silent.
+
+**An absolute rule keeps every keyword while an appended clause reverses it.** This
+release found that class, guarded it on stage 10's criterion 13, and left `residue.md`'s
+two absolute rules on plain substring presence — *"never released by this run, unless it
+has clearly expired"* passed. Sweeping the fix to its siblings then produced the reason
+to sweep it structurally: the second copy of the carve-out pattern had its escaping
+doubled, so it matched a literal backslash and walked past every inversion while looking
+correct. One home now — `_EXCEPTION_MARKER` and the `_carve_out` helper that reads it —
+with two call sites.
+
+**A probe demanded a dependency the harness disclaims.** `res8` required PyYAML to be
+installed; the guard it tests degrades honestly without it. `test:all` was red on any
+machine lacking it. Three branches now: the guard fired, the guard said it could not
+look, or neither — and only the third is an error.
+
+Also: the owner-row check went through `_row_cells` instead of a hand-rolled regex, as
+that helper's own docstring asks; and a reflow that fixed a 157-character line had
+orphaned two words onto a five-character one.
+
+Guards: 291 → **294**.
+
+## v1.47.0 — a green suite that cannot speak for an agent
+
+The suite gate at stage 6 assumes the thing under test is deterministic: run it twice,
+get the same answer, and a pass means something. An agent breaks that assumption, and
+until now this pipeline had nothing to say about it — a grep across the plugin for
+`llm-as-judge`, `eval suite`, `regression fixture` and `trace id` returned nothing.
+
+**`tdd.md` gains *When the thing under test is an agent*.** The artifact under test is
+the execution record, not the source: the code says what the agent is allowed to do,
+only a run says what it did. Three tiers with unequal authority at the gate — step and
+turn **block**, thread **reports** — plus the two rules an ordinary suite never needs.
+**Assert the side effect, not the sentence**, because an agent that says it saved the
+preference and did not passes trajectory and response and is broken. And **a model
+judging a model is not a check until it has been calibrated** against human labels on
+cases known to be bad. The suite is grown from production failures, minimised, and kept
+permanently — a fixed defect that silently returns is the whole reason.
+
+The gate states what it does **not** cover, as canon 6 requires: an offline suite speaks
+for the cases already known and for nothing else, which is why production observation
+stays a stage-8 concern.
+
+**Three canons extended rather than an eleventh added.** Canon 3 says every fact has one
+home, so the agent case belongs inside the canons it is an instance of, not beside them:
+
+- **Canon 1** — where the subject is non-deterministic, the address is a **trace id and
+  the assertion that ran against it**; a rerun is not the same run.
+- **Canon 5** — a model used as a judge is the same object as a check: until it has been
+  seen disagreeing with a human on a known-bad case, its pass is an opinion with a
+  number attached.
+- **Canon 8** — a score produced by a model is an estimate in every report that quotes
+  it, however many decimal places it carries.
+
+**`evidence-docs` gains one routing row** and no doctrine — it is a navigator. The row
+names its target instead of linking it: this navigator already carries eleven
+out-of-directory links that break wherever a packager ships the skill alone, and a
+twelfth would widen a known defect. Stated rather than fixed, and the count is unchanged
+at eleven — measured before and after, not assumed.
+
+`Guards: 291 → 291`. This release adds doctrine, not enforcement: nothing here is
+machine-checkable yet, and the count says so rather than leaving a reader to infer it.
+The gate written into `tdd.md` is a specification a host project runs against its own
+agent — this repository has no agent to run it against, so arming it here would be a
+check with nothing to look at, which `gates.md` calls dormant and forbids passing.
+
 ## v1.46.0 — what a run leaves running, what "done" costs to say, and what a check is for
 
 Three rules this pipeline had been following by disposition rather than by doctrine,

package/HOW-IT-WORKS.md

@@ -5,7 +5,7 @@ explains the machine — what routes a request into it, what each stage refuses
 let past, and what makes any claim it prints believable. The README sells it and
 tells you how to install it; this file tells you how it thinks.
 
-> **Version 1.45.0.** The "What changed" section at the bottom carries the last
+> **Version 1.47.1.** The "What changed" section at the bottom carries the last
 > few releases. Everything above it describes the pipeline as it is *now*, not as
 > it was designed.
 
@@ -238,7 +238,7 @@ This is the part most pipelines skip, and it is the reason this one is trustwort
 rather than merely tidy.
 
 **Every guard is proven against a planted defect.** `npm run test:all` breaks the
-thing each of the **291** checks is about, and requires that check to reject it. A
+thing each of the **294** checks is about, and requires that check to reject it. A
 guard that has never been watched failing is not a guard; it is a comment.
 
 **The neighbour probe.** A check has a *subject* — the rule it is about — and
@@ -291,7 +291,28 @@ a run diverges  →  retro entry  →  retro.publish  →  upstream issue
 
 ## What changed, by version
 
-### v1.45.0 — what a run leaves running, what "done" costs to say, what a check is for
+### v1.47.1 — the fixes a reader found, which three releases shipped without
+Three releases went out from another session while this branch was reviewed, and
+each carried the same defects forward: the doc map forbidding the decisions register
+shipping beside it, this file naming a version two releases stale, two absolute rules
+invertible by an appended clause, a probe demanding PyYAML the harness disclaims, and
+a probe that stopped reproducing its own hazard when its target string was quoted —
+which leaves `main`'s own suite red. Every one was found and fixed before the first
+of those releases; this is them arriving.
+
+### v1.47.0 / v1.46.0 — shipped from another session
+A green suite that cannot speak for an agent; and this branch's residue, completion
+honesty, proportionate verification and improvement-iteration doctrine, which reached
+main through the merge commit rather than through this branch's own release.
+
+v1.46.0 was tagged from a commit carrying this branch's doctrine and none of the
+three commits answering its reviewer. The doc map forbade the decisions register
+shipping beside it; this file called itself the version it was written under; two
+absolute rules were guarded by substring alone and inverted by an appended clause;
+and a probe demanded PyYAML the harness disclaims. All four were found and fixed
+before the release went out, and arrived after it.
+
+### v1.46.0 — what a run leaves running, what "done" costs to say, what a check is for
 Four rules the pipeline had been following by disposition rather than by doctrine.
 **Residue**: eight classes of thing a run leaves running or leaves behind, enumerated
 by class and never by one tool — the case that produced it was a task inventory
@@ -304,6 +325,12 @@ the deliverable is the working result and the check is how you know — scaled t
 issue resolves when behaviour changed and its close names the address; unworked ones
 accumulate visibly rather than being triaged into silence. Guards: 275 → 291.
 
+### v1.45.1 / v1.45.0 — shipped from another session while this branch was paused
+The reference routing existed three times over and was cut to one home; the guard
+count the self-test plants into was restored. Neither is this branch's work — they
+are here because they hold the version numbers this branch was originally built
+under, which is why it ships as 1.46.0.
+
 ### v1.44.0 — six lessons from other projects, and the guards that hold them
 Six issues published by runs in other repositories were worked into doctrine:
 seam testing at stage 6, `verified by` names that must resolve, a probe's green

package/SKILL-CARD.md

@@ -12,7 +12,7 @@ harmless.
 |---|---|
 | **Purpose** | Runs a substantial task through ten gated delivery stages — intake grill, docs study, brainstorm, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs+registers, acceptance — refusing to advance until each gate passes |
 | **Owner** | ssheleg ([github.com/ssheleg/task-pipeline](https://github.com/ssheleg/task-pipeline)) |
-| **Version** | 1.46.0 |
+| **Version** | 1.47.1 |
 | **Surface** | Claude Code (filesystem skill + plugin) and the vercel `skills` CLI. **Not** uploaded to the Skills API; custom Skills do not sync across surfaces |
 | **Dependencies** | None required. Optional: `context7` (MCP), `figma` (MCP), super-ux, agent-sync, graphify, obsidian-wiki. Every stage's doctrine ships in-repo; the one conditional requirement is super-ux for the stage-3 UX track on a user-facing task |
 | **Evaluation status** | Suite authored, 5 categories. One recorded run, **self-observed by the author**; **zero blind runs on zero of three models** — the split, and the numbers, live in [`evals/RESULTS.md`](evals/RESULTS.md) and are computed by `evals/run.py` |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "task-pipeline-skill",
-  "version": "1.46.0",
+  "version": "1.47.1",
   "description": "Full-cycle delivery pipeline for coding agents: a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine ships inside the skill — no companion plugin required. This package is the installer CLI.",
   "bin": {
     "task-pipeline": "bin/task-pipeline.js"

package/plugins/task-pipeline/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "name": "task-pipeline",
   "displayName": "Task Pipeline",
   "description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
-  "version": "1.46.0",
+  "version": "1.47.1",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/task-pipeline/skills/evidence-docs/SKILL.md

@@ -45,6 +45,7 @@ thing, the canon is the *why* and the rule is the *how*.
 | audit documentation a project already has | [`setup.md`](../task-pipeline/references/setup.md) | seven passes, cheapest first, output is a fix plan |
 | carry a lesson to the next run | [`retrospective.md`](../task-pipeline/references/retrospective.md) | stamp first (the cold trigger reads it), then prune to a cap of ten; every lesson names its commit |
 | seed a gate into a host project | [`templates/docgate.sh`](../task-pipeline/templates/docgate.sh) | it seeds **green**: dormant where there is no input yet |
+| claim that an **agent** behaves | `tdd.md` → *When the thing under test is an agent* — named rather than linked, because this navigator's out-of-directory links break wherever a packager ships this skill alone | the address is a trace id and the assertion that ran (canon 1); a judge nobody watched disagree is a green nobody watched turn red (canon 5) |
 | take a whole change through to acceptance | [`task-pipeline`](../task-pipeline/SKILL.md) | this skill is the standard; that one is how a change reaches the repository |
 
 ## When this applies

package/plugins/task-pipeline/skills/task-pipeline/references/documentation.md

@@ -42,8 +42,11 @@ exists to prevent.
 **1. A claim carries its address.** Every fact that lands in a document names where it
 can be checked: `file:line`, a command with its output, a test name. A lesson names the
 commit that earned it. *"We verified it" is the sentence that passes every review and
-proves nothing.* → the retro's SHA-resolution guard; the finding shape in
-[`setup.md`](setup.md).
+proves nothing.* Where the subject is non-deterministic — an agent, a model call — the
+address is **a trace id and the assertion that ran against it**, because a rerun is not
+the same run and a description of the behaviour is not the behaviour. → the retro's
+SHA-resolution guard; the finding shape in [`setup.md`](setup.md); [`tdd.md`](tdd.md) →
+*When the thing under test is an agent*.
 
 **2. Numbers are computed, never restated.** A count in prose is a number that was true
 once. Derive it at check time and compare the stated one against the computed one as the
@@ -60,8 +63,11 @@ under every link checker, because the checker resolves from the file's home. →
 
 **5. Green nobody watched turn red is not evidence.** A check must be seen rejecting a
 planted defect before its pass means anything — and the plant must be proven to have
-landed in the text the check actually parses. → [`gates.md`](gates.md) → *Probing*;
-[`learned.md`](learned.md) rules 4 and 5.
+landed in the text the check actually parses. **A model used as a judge is the same
+object**: until it has been seen disagreeing with a human label on a case known to be
+bad, its pass is an opinion with a number attached. → [`gates.md`](gates.md) →
+*Probing*; [`learned.md`](learned.md) rules 4 and 5; [`tdd.md`](tdd.md) → *When the
+thing under test is an agent*.
 
 **6. A check proves its scope and nothing beyond it.** Every gate carries what it does
 **not** cover, and quoting it wider is how "the gate is green" becomes a false statement
@@ -74,8 +80,9 @@ wrong, it is not evidence. → [`gates.md`](gates.md) → *False success*.
 
 **8. An estimate is never announced as a measurement.** A rule that fires on a judgement
 states its **evidence condition** — the observable signal that licenses it. A false
-alarm does not cost one interruption; it costs the alarm. → [`continuity.md`](continuity.md)
-→ *The context budget*.
+alarm does not cost one interruption; it costs the alarm. A score produced by a model
+is an estimate and stays one in every report that quotes it, however many decimal places
+it carries. → [`continuity.md`](continuity.md) → *The context budget*.
 
 **9. What was not checked is printed beside what was.** Absence is a finding with one
 side, so it never surfaces by comparison. Carry it as a named, counted set next to every

package/plugins/task-pipeline/skills/task-pipeline/references/tdd.md

@@ -14,6 +14,7 @@ into this skill; nothing to install.
 - The green from residue
 - Tests that stay honest
 - Stage 6 — consolidation and the suite gate
+- When the thing under test is an agent
 - When stuck
 - What a case consumes, and why a timeout is unclassified
 - Rationalizations
@@ -125,6 +126,51 @@ changed code is covered. No `skip` / `xfail` / commented-out assertion smuggles
 red suite past the gate. A partial or red run never advances to deploy; report it
 honestly instead.
 
+## When the thing under test is an agent
+
+An agent's behaviour is not in its source. The code says what it is *allowed* to do;
+only a run says what it did. So the artifact under test is the **execution record**, and
+the suite above needs one more tier before *green* means anything.
+
+Three tiers, and they do not carry equal authority at the gate:
+
+| Tier | Fixture | Asserts | At the gate |
+|---|---|---|---|
+| **Step** | one serialized model call with its prompt, tools and context | the decision — tool chosen, argument shape | **blocks** |
+| **Turn** | one whole execution | the trajectory, the final response, **and the state change** | **blocks** |
+| **Thread** | a scripted multi-turn session | what carried across turns, checked after **every** turn, failing fast | **reports** |
+
+Two rules the ordinary suite does not need:
+
+- **Assert the side effect, not the sentence.** An agent that says it saved the
+  preference and did not passes trajectory and response and is broken. The memory row,
+  the written file, the created record — inspect the thing, not the prose about it.
+- **A model judging a model is not a check until it has been calibrated.** Label the
+  same traces by hand, measure the agreement, and only then let it score unattended.
+  This is canon 5 with a different subject: a judge nobody has watched disagree is a
+  green nobody has watched turn red. Cheap deterministic checks — schema, exact match,
+  business rule, tool-call correctness — run first and take everything they can decide,
+  because they cost nothing and cannot drift.
+
+**The suite is grown, never authored.** Every production failure and every thumbs-down
+is minimised to the smallest input that still reproduces it, filed into the tier that
+isolates it, and **kept there permanently** — a fixed defect that silently returns is
+the whole reason this rule exists.
+
+**GATE (auto):** step and turn tiers green, and every fixture added by this change
+present in the suite. Thread results and any online, reference-free checks are
+**reported beside the verdict, not folded into it** — canon 9, which is why a
+non-blocking tier still has to print.
+
+**What this gate does not cover:** anything the fixtures do not contain. An offline
+suite proves you did not regress the cases already known; it cannot speak for inputs
+nobody has seen, which is canon 6 and the reason production observation is a stage-8
+concern rather than a stage-6 one.
+
+For building the suite this gate reads — the primitives, the tiers, judge design,
+annotation queues and simulated users — see the `agent-evals` skill in the
+`agent-stack` plugin.
+
 ## When stuck
 
 | Problem | What it means |