task-pipeline-skill 1.79.1 → 1.81.1

package/evals/evidence-docs.evals.json

@@ -0,0 +1,109 @@
+{
+  "_note": "Behavioural evaluations for the evidence-docs skill — the ten-canon navigator that ships in this plugin. Same format and dimensions as task-pipeline.evals.json; the negatives are NEAR MISSES on purpose (a commit message and a code comment are the excluded surfaces closest to the trigger ground), because a control that shares no words with the trigger proves nothing about the boundary.",
+  "skill": "evidence-docs",
+  "models": ["haiku", "sonnet", "opus"],
+  "evals": [
+    {
+      "id": "TRIG-01",
+      "category": "should_trigger",
+      "skills": ["evidence-docs"],
+      "query": "запиши решение: внутренние сервисы переходят с REST на gRPC, обратной дороги нет",
+      "expected_behavior": [
+        "Invokes the evidence-docs skill (or its routed documentation doctrine), not a freehand note",
+        "The decision lands in the project's one decision register, append-only, with an id",
+        "The record carries its evidence and its date rather than only the conclusion"
+      ],
+      "why": "«записать решение» is a declared Russian trigger; the failure it guards is a decision written into chat instead of the register."
+    },
+    {
+      "id": "TRIG-02",
+      "category": "should_trigger",
+      "skills": ["evidence-docs"],
+      "query": "Write the acceptance report for the export migration — and make sure every claim in it is actually backed",
+      "expected_behavior": [
+        "Invokes the evidence-docs skill for the canon layer (a claim carries its address; numbers computed, never restated)",
+        "Each verified claim names the check that was seen deciding it; unverified ones are marked as such rather than dropped"
+      ],
+      "why": "An acceptance report is the archetypal will-be-read-as-true document; the canons are its standard."
+    },
+    {
+      "id": "TRIG-03",
+      "category": "should_trigger",
+      "skills": ["evidence-docs"],
+      "query": "Is this verified? The README claims the importer handles 10k rows per second.",
+      "expected_behavior": [
+        "Invokes the evidence-docs skill — the question is whether a claim is documentation or an assertion",
+        "Answers with the claim's evidence condition: what observable signal would license the number, and whether one exists",
+        "Does not re-assert the number without an address"
+      ],
+      "why": "'is this verified' is a declared trigger; the failure is grading the claim by plausibility instead of by evidence."
+    },
+    {
+      "id": "NOTRIG-01",
+      "category": "should_not_trigger",
+      "skills": ["evidence-docs"],
+      "query": "Write a commit message for this diff — it renames the exporter and fixes the retry backoff",
+      "expected_behavior": [
+        "Writes the commit message directly",
+        "Does NOT invoke the evidence-docs skill or demand file:line receipts for a commit message"
+      ],
+      "why": "Near miss by design: a commit message describes a change truthfully but is an excluded surface — demanding canons here teaches routing around them."
+    },
+    {
+      "id": "NOTRIG-02",
+      "category": "should_not_trigger",
+      "skills": ["evidence-docs"],
+      "query": "добавь комментарий в код, почему в этом месте ретрай с экспоненциальной задержкой",
+      "expected_behavior": [
+        "Writes the code comment directly",
+        "Does NOT invoke the evidence-docs skill — a code comment is an excluded surface by the skill's own Not-for list"
+      ],
+      "why": "Near miss: 'почему' invites an evidence reflex, and the boundary says a code comment is not documentation-read-as-true."
+    },
+    {
+      "id": "NOTRIG-03",
+      "category": "should_not_trigger",
+      "skills": ["evidence-docs"],
+      "query": "Draft some rough notes on how we might approach caching — just thinking out loud, no docs",
+      "expected_behavior": [
+        "Produces the draft directly",
+        "Does NOT apply the canons — 'без доков'/'no docs' is the declared opt-out, and a draft is outside the boundary anyway"
+      ],
+      "why": "The refusal phrase plus the draft exclusion; the failure is refusing exploratory writing for want of receipts."
+    },
+    {
+      "id": "AMB-01",
+      "category": "ambiguous",
+      "skills": ["evidence-docs"],
+      "query": "Update the README",
+      "expected_behavior": [
+        "Names the route: a developer README section is direct work, while a claim a reader will take as true (a benchmark, a guarantee) passes the canons",
+        "Does not silently pick one reading of the request"
+      ],
+      "why": "A README holds both excluded and covered content; the observable is naming the split, not the choice itself."
+    },
+    {
+      "id": "COEX-01",
+      "category": "coexistence",
+      "skills": ["evidence-docs", "task-pipeline"],
+      "query": "прогони миграцию через конвейер и запиши архитектурное решение в реестр",
+      "expected_behavior": [
+        "task-pipeline carries the change; the decision record goes through the documentation doctrine evidence-docs routes",
+        "One register entry with an id — not a second decision home invented for the run"
+      ],
+      "why": "The two skills share a plugin and a boundary: the pipeline owns delivery, evidence-docs owns what is written as true."
+    },
+    {
+      "id": "INSTR-01",
+      "category": "instruction_following",
+      "skills": ["evidence-docs"],
+      "query": "Record the decision that we drop Python 3.8 support, and note it was discussed in Slack",
+      "expected_behavior": [
+        "The record is appended to the existing register, never a new file beside it",
+        "A correction or later reversal would be appended, not edited over",
+        "The Slack discussion is cited as the decision's source, not pasted as its evidence"
+      ],
+      "why": "Append-only and one-home are the canons most often broken by a helpful rewrite."
+    }
+  ]
+}

package/evals/project-audit.evals.json

@@ -0,0 +1,108 @@
+{
+  "_note": "Behavioural evaluations for the project-audit skill. Same format and dimensions as task-pipeline.evals.json. The negatives are NEAR MISSES on purpose — «аудит модуля» against «аудит проекта» is one word apart and routes to a different skill (the pipeline's in-run ladder), which is exactly the boundary the descriptions draw.",
+  "skill": "project-audit",
+  "models": ["haiku", "sonnet", "opus"],
+  "evals": [
+    {
+      "id": "TRIG-01",
+      "category": "should_trigger",
+      "skills": ["project-audit"],
+      "query": "сделай аудит проекта — что реально готово, что наполовину, что сломано?",
+      "expected_behavior": [
+        "Invokes the project-audit skill, not the pipeline's in-run audit ladder",
+        "Starts with discovery (what the project IS) before choosing probes",
+        "Leaves the HTML report and the JSON sidecar; proposes board rows and commits nothing"
+      ],
+      "why": "«аудит проекта» is the declared trigger; the failure is running a fixed checklist or treating it as a change-task."
+    },
+    {
+      "id": "TRIG-02",
+      "category": "should_trigger",
+      "skills": ["project-audit"],
+      "query": "What is actually true of this project right now — what is finished, what is half-built, what has nobody looked at?",
+      "expected_behavior": [
+        "Invokes the project-audit skill — the subject is the whole project, not one change",
+        "Reads production evidence (published artefact vs source, CI history, telemetry presence), not only the working tree"
+      ],
+      "why": "The skill's own opening sentence as a user query; the failure is answering from the README."
+    },
+    {
+      "id": "TRIG-03",
+      "category": "should_trigger",
+      "skills": ["project-audit"],
+      "query": "Run a project health check on this repository and leave me the report",
+      "expected_behavior": [
+        "Invokes the project-audit skill",
+        "Blind probes are rendered as their own section with reasons — a probe that could not look is not a probe that found nothing"
+      ],
+      "why": "'project health check' is a declared trigger; the blind verdict is the design worth probing for."
+    },
+    {
+      "id": "NOTRIG-01",
+      "category": "should_not_trigger",
+      "skills": ["project-audit"],
+      "query": "сделай аудит модуля оплат",
+      "expected_behavior": [
+        "Routes to the pipeline's audit path (a finding that lands in the repository), not to project-audit",
+        "Does NOT start a cold whole-project discovery for a one-module question"
+      ],
+      "why": "Near miss by one word: «аудит проекта» is this skill, «аудит модуля» is one deliverable inside a run — the pipeline's own ladder."
+    },
+    {
+      "id": "NOTRIG-02",
+      "category": "should_not_trigger",
+      "skills": ["project-audit"],
+      "query": "Review PR #24 and tell me what is wrong with it",
+      "expected_behavior": [
+        "Routes to the pipeline's PR-review findings path or answers directly",
+        "Does NOT invoke project-audit — reviewing a diff is its declared Not-for"
+      ],
+      "why": "A diff has a change as its subject; project-audit's subject is the project."
+    },
+    {
+      "id": "NOTRIG-03",
+      "category": "should_not_trigger",
+      "skills": ["project-audit"],
+      "query": "проверь, соответствует ли этот скил стандарту Agent Skills",
+      "expected_behavior": [
+        "Routes to make-skill's audit (/skill-audit), not to project-audit",
+        "Does NOT run whole-project probes over a skill-construction question"
+      ],
+      "why": "The disambiguation table's own row: a skill's construction belongs to make-skill even when the word 'audit' appears."
+    },
+    {
+      "id": "AMB-01",
+      "category": "ambiguous",
+      "skills": ["project-audit"],
+      "query": "проверь проект",
+      "expected_behavior": [
+        "Names the route it is taking in one line — whole-project diagnosis (project-audit) versus a specific check the operator may mean",
+        "Does not silently start either the full audit or a random spot-check"
+      ],
+      "why": "Two words with no object; the observable is the named route, not the choice."
+    },
+    {
+      "id": "COEX-01",
+      "category": "coexistence",
+      "skills": ["project-audit", "task-pipeline"],
+      "query": "Audit the whole project, then fix the three worst things you find",
+      "expected_behavior": [
+        "project-audit produces the findings as proposed board rows, read-only",
+        "The fixes are carried by task-pipeline runs off those rows — the audit itself commits nothing"
+      ],
+      "why": "The seam the two skills share: diagnosis is read-only, delivery is the pipeline's; collapsing them makes the audit unrepeatable."
+    },
+    {
+      "id": "INSTR-01",
+      "category": "instruction_following",
+      "skills": ["project-audit"],
+      "query": "Audit this project",
+      "expected_behavior": [
+        "Discovery runs first and the probes are chosen from the profile, not from a fixed list",
+        "The three-verdict vocabulary is kept: clean, finding, blind — with blind reasons on the page",
+        "Findings leave priced with the board header's declared formula; effort never ranks"
+      ],
+      "why": "The procedure's own load-bearing steps, each of which a helpful shortcut would skip."
+    }
+  ]
+}

package/evals/run.py

@@ -17,12 +17,17 @@ What it does:
 
 Zero dependencies, same as the validator.
 """
+import glob
 import json
 import os
 import re
 import sys
 
 ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+# EVERY suite, discovered — the directory held one suite per skill from
+# 2026-08-31 (evidence-docs and project-audit joined task-pipeline), and a
+# runner pinned to one filename would validate a third of what ships.
+SUITES = sorted(glob.glob(os.path.join(ROOT, "evals", "*.evals.json")))
 SUITE = os.path.join(ROOT, "evals", "task-pipeline.evals.json")
 RESULTS = os.path.join(ROOT, "evals", "RESULTS.md")
 
@@ -34,14 +39,12 @@ REQUIRED = ("should_trigger", "should_not_trigger", "ambiguous",
 MIN_EVALS = 3          # Anthropic: "At least three evaluations created"
 
 
-def main(argv):
+def validate_suite(path):
+    """Every gap in one suite, in a stable order. The rules are the same for
+    every skill's suite — a second rule set would drift."""
     errors = []
-    if not os.path.isfile(SUITE):
-        print(f"FAIL: no suite at {os.path.relpath(SUITE, ROOT)}")
-        return 2
-    suite = json.load(open(SUITE, encoding="utf-8"))
+    suite = json.load(open(path, encoding="utf-8"))
     evals = suite.get("evals") or []
-
     seen = set()
     for e in evals:
         where = e.get("id", "<no id>")
@@ -68,6 +71,18 @@ def main(argv):
     for cat in REQUIRED:
         if cat not in covered:
             errors.append(f"no eval covers {cat!r}")
+    return suite, evals, errors
+
+
+def main(argv):
+    if not os.path.isfile(SUITE):
+        print(f"FAIL: no suite at {os.path.relpath(SUITE, ROOT)}")
+        return 2
+    parsed, errors = [], []
+    for _sp in SUITES:
+        _suite, _evals, _errs = validate_suite(_sp)
+        parsed.append((os.path.basename(_sp), _suite, _evals))
+        errors += [f"{os.path.basename(_sp)}: {e}" for e in _errs]
 
     if errors:
         print("FAIL: evaluation suite invalid")
@@ -75,31 +90,42 @@ def main(argv):
             print("  - " + e)
         return 1
 
+    suite = next(s for n, s, ev in parsed if n == "task-pipeline.evals.json")
+    evals = [e for _, _, ev in parsed for e in ev]
     by_cat = {}
     for e in evals:
         by_cat.setdefault(e["category"], []).append(e)
 
     if "--list" in argv:
-        for cat in REQUIRED:
-            for e in by_cat.get(cat, []):
-                print(f"  {e['id']:<10} {cat:<22} {e['query'][:60]}")
-        print(f"\n{len(evals)} evals across {len(by_cat)} categories")
+        for name, _s, ev in parsed:
+            print(f"{name} — {_s.get('skill', '?')}")
+            for cat in REQUIRED:
+                for e in ev:
+                    if e["category"] == cat:
+                        print(f"  {e['id']:<10} {cat:<22} {e['query'][:60]}")
+        print(f"\n{len(evals)} evals across {len(by_cat)} categories, "
+              f"{len(parsed)} suite(s)")
         return 0
 
     print("=" * 72)
-    print("task-pipeline evaluation protocol")
+    print("task-pipeline plugin evaluation protocol (one section per suite)")
     print("=" * 72)
-    print("Run each query in a FRESH session with the skill installed, once per")
+    print("Run each query in a FRESH session with the pack installed, once per")
     print("model in", suite.get("models", []), "— effectiveness varies by model.")
     print("Record every verdict in evals/RESULTS.md with the date and the model.")
     print("A query you did not run is not a pass; leave it blank and say so.\n")
-    for cat in REQUIRED:
-        print(f"\n--- {cat} ---")
-        for e in by_cat.get(cat, []):
-            print(f"\n[{e['id']}] {e['query']}")
-            print(f"    why: {e['why']}")
-            for b in e["expected_behavior"]:
-                print(f"    [ ] {b}")
+    for name, _s, ev in parsed:
+        print(f"\n=== {name} — {_s.get('skill', '?')} ===")
+        for cat in REQUIRED:
+            group = [e for e in ev if e["category"] == cat]
+            if not group:
+                continue
+            print(f"\n--- {cat} ---")
+            for e in group:
+                print(f"\n[{e['id']}] {e['query']}")
+                print(f"    why: {e['why']}")
+                for b in e["expected_behavior"]:
+                    print(f"    [ ] {b}")
 
     print("\n" + "=" * 72)
     if not os.path.isfile(RESULTS):
@@ -118,6 +144,8 @@ def main(argv):
         if not infence:
             outside.append(ln)
     runs = [l for l in outside if re.match(r"^## 20\d{2}-\d{2}-\d{2}\b", l)]
+    print("suites: " + " · ".join(f"{n.replace('.evals.json', '')} {len(ev)}"
+                                    for n, _s, ev in parsed))
     print(f"suite: {len(evals)} evals · recorded runs: {len(runs)}")
     if not runs:
         print("RESULTS.md carries no dated run — the suite is authored and unexecuted.")

package/package.json

@@ -1,16 +1,18 @@
 {
   "name": "task-pipeline-skill",
-  "version": "1.79.1",
+  "version": "1.81.1",
   "description": "Full-cycle delivery pipeline for coding agents: a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine ships inside the skill — no companion plugin required. This package is the installer CLI.",
   "bin": {
     "task-pipeline": "bin/task-pipeline.js"
   },
   "scripts": {
     "test": "python3 test/validate.py && python3 test/graph_test.py && python3 test/project_audit_test.py",
-    "test:all": "python3 test/validate.py && python3 test/graph_test.py && python3 test/project_audit_test.py && python3 test/negatives.py && npm run test:certify && npm run test:exposure && npm run test:probe && npm run test:hooks && npm run test:artifacts && npm run test:docs",
+    "test:all": "python3 test/validate.py && python3 test/graph_test.py && python3 test/project_audit_test.py && python3 test/negatives.py && npm run test:certify && npm run test:exposure && npm run test:probe && npm run test:anchors && npm run test:runner && npm run test:hooks && npm run test:artifacts && npm run test:docs",
     "test:negatives": "python3 test/negatives.py",
     "test:exposure": "python3 test/exposure_test.py",
     "test:probe": "python3 test/probe.py --self-test",
+    "test:anchors": "python3 test/anchors_test.py && python3 test/anchors.py",
+    "test:runner": "python3 test/runner_test.py",
     "test:hooks": "python3 test/release_gate_test.py",
     "test:artifacts": "python3 test/artifact_root_test.py && python3 test/migrate_artifacts_test.py",
     "test:docs": "bash plugins/task-pipeline/skills/task-pipeline/templates/docgate.sh",

package/plugins/task-pipeline/.claude-plugin/plugin.json

@@ -1,8 +1,9 @@
 {
+  "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "task-pipeline",
   "displayName": "Task Pipeline",
-  "description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
-  "version": "1.79.1",
+  "description": "Runs a substantial task through a mandatory built-in intake grill, then 10 gated stages (docs, brainstorm+decompose, spec, plan, subagent build, tests, lint/deploy, post-deploy, docs/wiki, acceptance). Every stage's doctrine is built into the skill — no companion plugin required — with typed auto/judgment/manual gates, a frozen requirement spine that closes with evidence, a work board and a verification ledger that outlive a run, an exposure line naming what shipped unconfirmed, a progress rail computed from the project's own config, a loop guard whose review ceiling measures rather than stops, and stage-3 tracks for what a product does, how it sounds and how it looks. Two modes need no task: `checkup` (what is unverified) and `setup` (audit existing docs). Retro insights can publish upstream as issues, opt-in and redacted.",
+  "version": "1.81.1",
   "author": {
     "name": "ssheleg",
     "url": "https://x.com/sshlg93"

package/plugins/task-pipeline/commands/task-pipeline.md

@@ -100,7 +100,8 @@ Anything deferred enters the carry-over ledger the moment it is said.
 | 9 | Docs + wiki | **three artifacts, not two** — module docs, the wiki, **and the code graph** |
 | 10 | Acceptance | the ladder walk first, then the table, then the retrospective |
 
-**Honor every gate by its type**: `auto` — verify the check yourself; `manual` — wait
+**Honor every gate by its type**: `auto` — verify the check yourself; `judgment` —
+record the named judge's ruling as judgement, never as a measurement; `manual` — wait
 for an explicit go.
 
 ## Cross-cutting — the three that fire at any stage

package/plugins/task-pipeline/hooks/gate-observer.sh

@@ -73,11 +73,28 @@ if not matches:
 # PostToolUse fires on success; PostToolUseFailure carries the error. Both are
 # wired to this script, and `error` present means the command did not exit 0.
 failed = bool(data.get("error")) or data.get("hook_event_name") == "PostToolUseFailure"
-out = data.get("tool_output") or {}
-if isinstance(out, dict) and out.get("exit_code") is not None:
+# The harness documents the result field as `tool_response`; `tool_output` is the
+# name this script shipped reading, so it stays as a fallback rather than a
+# breaking change. Reading only the wrong name left the exit-code branch dead.
+# The fallback is BY FIELD, not by object: the real Bash `tool_response` is
+# {stdout, stderr, interrupted} with no exit_code, and `resp or legacy` made a
+# legacy exit_code unreachable behind it — found by the R-005 reader, measured,
+# before this shipped. The same reading found `interrupted`: a gate cut short
+# is not a gate that passed, whatever a stale exit_code says, so a failure
+# event or an interruption is never recorded as exit 0.
+out = {}
+interrupted = False
+for cand in (data.get("tool_response"), data.get("tool_output")):
+    if isinstance(cand, dict):
+        interrupted = interrupted or bool(cand.get("interrupted"))
+        if not out and cand.get("exit_code") is not None:
+            out = cand
+if out:
     code = int(out["exit_code"])
 else:
     code = 1 if failed else 0
+if (failed or interrupted) and code == 0:
+    code = 1
 
 stamp = datetime.datetime.now(datetime.timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z")
 

package/plugins/task-pipeline/skills/evidence-docs/SKILL.md

@@ -24,6 +24,7 @@ The full statement of each canon, its rationale and its enforcement live in
 7. **Silence is not a pass** — ask what a mechanism prints when it did not look.
 8. **An estimate is never announced as a measurement** — a rule states its evidence condition.
 9. **What was not checked is printed beside what was.**
+   - **9a. A measured zero and an unmeasured quantity may not print the same** — canon 9 says carry the absence; 9a says refuse the number when nothing measured it.
 10. **The document ships in the change that made it true** — and a correction is appended, never written over.
 
 They are **epistemic**: what makes a claim documentation. The operational layer — what to

package/plugins/task-pipeline/skills/project-audit/SKILL.md

@@ -1,6 +1,7 @@
 ---
 name: project-audit
 description: "Use when someone asks what is actually true of a whole project right now — what is finished, what is half-built, what is broken, and what nobody has looked at. Walks a cold start: discover what the project is, run a registry of probes chosen from that, read production evidence (published artefact against source, CI history, telemetry present or absent), then leave a self-contained HTML report and a JSON sidecar so the next audit can say what moved. Read-only: it proposes board rows and commits nothing. Triggers - 'project audit', 'audit the project', 'codebase audit', 'state of the project', 'what is unfinished', 'project health check', 'аудит проекта', 'проаудируй проект', 'состояние проекта', 'что не доделано', 'аудит кодовой базы'. Not for: auditing one deliverable inside a run (that is the pipeline's own ladder), reviewing a diff, or checking a skill's construction — say 'без диагностики' to opt out."
+compatibility: "The collector (scripts/audit.py) needs python3 and reads committed state, so it needs git. Probes needing gh, npm, network or a browser declare it and report blind when it is absent — degraded, never silent."
 ---
 
 # Project audit — what is true of this project right now
@@ -33,6 +34,7 @@ next audit reads.
 | this skill | the **procedure** — cold start, probes, production, the report | a whole project is the subject |
 | `/skill-audit` (make-skill) | a skill's construction against the standard | the thing audited is a skill or plugin |
 | `/ux-audit` (super-ux) | code against documented scenarios | the question is user-facing behaviour |
+| `/seo-aeo-audit` (seo-aeo-audit) | a public surface's search and answer-engine visibility | the question is whether a machine will find it |
 
 **The method is not restated here.** Phase 4 below hands off to `audit.md` and
 comes back; a second copy of the ladder would be a second rule, and the two
@@ -96,8 +98,11 @@ the same object.
 ### 6. Propose — rows, not edits
 
 **This skill commits nothing.** Findings leave as board rows in the project's
-own vocabulary, priced with the project's own formula —
-`P = blast × (1 + age_runs) / effort` — and the operator accepts them. An audit
+own vocabulary, priced with **the board header's declared formula** — the shipped
+default is `Sev × Blast + age_bonus` (`references/backlog.md`, the pipeline's
+board doctrine) — and the operator accepts them. Effort never ranks inside an
+audit: what a fix costs is the fixer's decision, not the finder's
+(`references/prioritisation.md`). An audit
 that edits while it reads cannot be re-run to check itself.
 
 ## Three verdicts, and why the third one exists