pi-dev 0.2.5 → 0.2.6

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-dev",
-  "version": "0.2.5",
+  "version": "0.2.6",
   "description": "An autonomous engineering skill framework for the pi runtime — built on Matt Pocock's skills.",
   "type": "module",
   "bin": {

package/skills/improve-skill-flow/SKILL.md

@@ -7,9 +7,9 @@ description: MAINTAINER-ONLY. Analyse real pi-runtime session telemetry from any
 
 **Audience: pi-dev maintainers only.** Consumers do not get this skill installed. Consumers improve their own setup by editing `docs/agents/preferences.md` (per project) or `~/.pi/agent/preferences.md` (per machine), not by editing SKILL.md bodies. The framework is fixed for them; only the maintainer changes it.
 
-The pi-dev skills are pure markdown. They get better when the maintainer reads what real sessions did across consumer repos, compares that to what the SKILL.md said *should* happen, and edits the gap closed. This skill is the canonical loop for that.
+The pi-dev workflow is not just markdown. It is a layered control system: SKILL.md prose, repo preferences, pi-runtime lifecycle events, tools, commands, TUI surfaces, session persistence, compaction hooks, and extensions. This skill reads what real sessions did across consumer repos, compares that to what the workflow contract said *should* happen, then chooses the lightest effective intervention.
 
-The point: **never edit a SKILL.md from gut feeling.** Edit because session N showed phase P violated predicate Q on M occasions, and here is the line that would have prevented it.
+The point: **never improve the framework from gut feeling, and never collapse every fix into a "guard".** Edit because session N showed phase P violated predicate Q on M occasions, then decide whether the right response is clearer prose, repo-local preferences, a runtime steer, a custom tool, a command, a TUI affordance, state persistence, compaction/context shaping, or — only when the evidence calls for it — a blocking gate.
 
 ## Pre-flight (hard gate)
 
@@ -57,7 +57,7 @@ Always from inside the pi-dev checkout (see Pre-flight).
 pi-runtime loads three artefact kinds the framework can ship:
 
 - **Skill bodies** — `~/.pi/agent/skills/<name>/SKILL.md` (global) or `<repo>/.pi/skills/<name>/SKILL.md` (local). Pure prose.
-- **Extensions** — `~/.pi/agent/extensions/<name>/` (global) or `<repo>/.pi/extensions/<name>/` (local). TypeScript modules auto-loaded via jiti; can subscribe to `tool_call`, `message_end`, `before_agent_start`, etc., and can block tool calls or inject steer messages. pi-dev ships **one** by default — `pi-flow` — and the bar to add a second is high (see below).
+- **Runtime interventions** — extensions under `~/.pi/agent/extensions/<name>/` (global) or `<repo>/.pi/extensions/<name>/` (local). TypeScript modules auto-loaded via jiti. They can subscribe to lifecycle/session/agent/model/tool events, inject context in `before_agent_start`, reshape model context, observe `message_end`, intercept or modify tool calls/results, register custom tools/commands/shortcuts/flags, prompt via `ctx.ui`, render custom TUI widgets, persist state with `pi.appendEntry()`, and customize compaction/session behavior. pi-dev ships **one** extension by default — `pi-flow` — and the bar to add a second package remains high.
 - **Preferences** — `docs/agents/preferences.md` (per repo) and `~/.pi/agent/preferences.md` (per machine). 3-layer override on prose-level decisions.
 
 A finding lands in exactly one of:
@@ -65,7 +65,7 @@ A finding lands in exactly one of:
 | scope | lands in | reaches | propagation | when to pick |
 | --- | --- | --- | --- | --- |
 | **framework** | this repo's `skills/<name>/SKILL.md` | every consumer after the next `npx pi-dev update` | release-please → npm publish | the SKILL.md wording is wrong; gap shows up generically; prose alone is plausibly enough |
-| **extension** | this repo's `extensions/pi-flow/index.ts` (extend) or a new `extensions/<name>/` (rare) | every consumer after `npx pi-dev update` | release-please → npm publish | a real audit shows prose can't recover the rule, AND the rule is a simple deterministic predicate, AND skipping the guard would silently waste ≥1 user prompt per session |
+| **extension** | this repo's `extensions/pi-flow/index.ts` (extend) or a new `extensions/<name>/` (rare) | every consumer after `npx pi-dev update` | release-please → npm publish | a real audit shows prose / prefs cannot reliably preserve the workflow, AND pi-runtime has an event/tool/UI/state hook that can make the desired path easier, more observable, or safer |
 | **consumer-prefs** | the audited consumer repo's `docs/agents/preferences.md` | only that repo, on every `/do` bootstrap | regular consumer-repo commit | gap is the consumer repo's domain / paths / conventions, not the SKILL.md |
 
 Notes:
@@ -75,12 +75,24 @@ Notes:
 - A `consumer-prefs` apply touches no pi-dev files. It is committed to the consumer repo only.
 - A single audit may produce a mix of all three. Decide scope per finding, not per audit.
 
-**Extension scope is the most expensive option.** Every guard runs on every relevant tool call or turn, slows iteration, and tempts the maintainer to disable the whole plugin. Default to `framework` / `consumer-prefs` and reach for `extension` only when the audit makes the ROI undeniable. The bar:
+**Extension scope is the most expensive option, but it is not synonymous with "guards".** Runtime interventions can be passive observability, progress/status UI, context injection, command shortcuts, structured tools, state checkpoints, compaction shaping, soft steers, confirmations, or hard blocks. Default to `framework` / `consumer-prefs` when prose or repo-local convention plausibly fixes the gap; reach for `extension` when the audit shows the workflow needs runtime support.
 
-- The rule was in-context as SKILL.md prose at audit time **and** was violated on ≥60% of opportunities across two consecutive audits.
-- The predicate is deterministic and short — a tool name plus an arg regex, or a tail-of-turn regex — with no LLM call.
-- The failure cost is concrete — the model wastes one or more user prompts per session re-entering a chain, or commits a destructive artefact.
-- A simpler remedy (gitignore lockout, prefs taboo, prose anti-pattern) does **not** plausibly close the gap. If it does, take the simpler remedy.
+Intervention ladder, from lightest to strongest:
+
+1. **Observe** — collect counters / emit low-noise status so the next audit can see what happened.
+2. **Make the right path easier** — custom command, custom tool, context injection, remembered state, or TUI affordance.
+3. **Steer** — inject a follow-up message or system/context nudge when the model is drifting but no irreversible action happened.
+4. **Confirm** — ask the user only for genuinely risky or preference-silent operations.
+5. **Block** — refuse a tool call only for destructive, costly, or repeatedly proven workflow violations.
+
+The bar for extension work:
+
+- The failure is repeated or high-cost, and the expected workflow was available in-context via SKILL.md / prefs / repo docs.
+- The runtime intervention can be described as a small event-driven mechanism over pi's lifecycle/session/agent/tool/UI/state surfaces; no hidden LLM call unless the finding is explicitly about a summarizer/evaluator extension.
+- The success metric is measurable in the next audit: fewer user nudges, higher `/do` phase completion, fewer corrections, better issue shape, shorter stalled intervals, or clearer live evidence.
+- A simpler remedy (docs wording, prefs, `.gitignore` lockout, issue-template fix) does **not** plausibly close the gap. If it does, take the simpler remedy.
+
+External research posture (DECAY): before proposing a new class of runtime intervention, do a quick current-source pass. 2025–2026 agent-reliability literature emphasizes trajectory monitoring, workflow-level observability, recovery orchestration, and textual feedback from traces — not just pre-execution blocking. Use that as a check against over-fitting on "guards"; cite the sources in the audit notes when they shape the proposal.
 
 ## Session-data location & format
 
@@ -174,12 +186,16 @@ Run a single pass over every targeted `.jsonl` and tally:
 - top Edit / Write targets (hotspot files)
 - bash commands matching domain-specific danger / smell patterns
 
-**Skill-flow specific:**
-- After each `<skill name="do">` injection, did another skill get injected within the same session? **This measures `/do` chain depth.** Single-step chains are a red flag.
+**Workflow-compliance specific:**
+- After each `<skill name="do">` injection, did another skill get injected within the same session? **This measures `/do` chain depth.** Single-step chains are a red flag, but not the whole story.
+- Did `/do` emit `[flow plan]`? Count planned phases and observed `[flow N/M]` status lines. Flag: plan missing, N/M skipped, final summary before all phases, or terminal predicate not evidenced.
 - Count user messages shorter than ~80 chars — these are usually nudges ("진행해", "다음은?", "끝났어?"). High proportion = `/do` is handing the flow back too often.
-- Count user messages containing correction markers (`아니`, `wait`, `stop`, `취소`, `다시`, `그만`, `undo`, `revert`). These mark interventions.
+- Count user messages containing correction markers (`아니`, `wait`, `stop`, `취소`, `다시`, `그만`, `undo`, `revert`, `왜`, `??`, `제대로`). These mark interventions.
+- Detect stalled intervals: long gaps between assistant text / tool calls, repeated identical commands, or repeated failed live probes beyond the repo's wait budget.
+- Count local-live / ops-live evidence when runtime behavior changed: command run, observed output, log/screenshot/evidence embedded in summary.
+- Count side-effect gates: commits, pushes, PRs, issue creates/edits/closes vs. merged prefs (`auto-*`) and tracker docs.
 - Count `<!-- migrated: ... -->` marker date vs. any post-marker `docs/handoff/` / `.scratch/flow/` / `SESSION_*.md` writes. Drift = handoff lockout failed.
-- Count tracker writes (`gh issue create`, `gh pr create`) and inspect the bodies; the bodies are plain spec consumed by future worker agents, so look for issues with shape problems (missing parent, no acceptance criteria, etc.) rather than meta tags.
+- Count tracker writes (`gh issue create`, `gh pr create`) and inspect the bodies; the bodies are plain spec consumed by future worker agents, so look for issues with shape problems (missing parent, no acceptance criteria, wrong state labels, wrong parent topology, etc.) rather than meta tags.
 
 Use a deterministic Python or shell script you write once and check into `/tmp` for the duration of the run. Do not eyeball big JSONLs by hand.
 
@@ -230,11 +246,15 @@ If a finding cannot be backed by an excerpt, it is not actionable yet — demote
 
 For each 🔴 / 🟡 finding, pick a default scope using the heuristic below, then show the table once and let the maintainer flip individual rows before applying.
 
+Before assigning scope, write one sentence answering: **what would have made the correct workflow easier to follow at the moment of failure?** Do not jump straight to a blocking guard.
+
 Default to `extension` if the finding matches **any** of:
 
-- The rule was already in SKILL.md prose at the time of the violating session **and** was violated repeatedly (i.e. prose recall is provably low for this rule).
-- The predicate is mechanically expressible against `event.input` (tool name + arg substring / path glob) or `event.message.content` (text tail shape).
-- The fix is a *refusal* or a *steer*, not a *reminder*.
+- The expected behavior was already in SKILL.md / prefs / repo docs at the time of the violating session **and** the session still drifted repeatedly or expensively.
+- The fix needs runtime affordance rather than prose: observe counters, inject context, add a command/tool, show TUI status, persist/checkpoint state, shape compaction/context, steer after a message, confirm a risky action, or block an unsafe tool call.
+- The success condition can be measured in the next session from telemetry, not merely hoped for from stronger wording.
+
+When defaulting to `extension`, classify the intervention type: `observe`, `affordance`, `context`, `state`, `render`, `steer`, `confirm`, or `block`.
 
 Default to `framework` if the finding matches **any** of:
 
@@ -253,9 +273,9 @@ Present the scope-decision table:
 ```
 | # | finding (short)                          | default scope    | target file                                          | flip? |
 | - | ---------------------------------------- | ---------------- | ---------------------------------------------------- | ----- |
-| 1 | /do hands flow back between phases       | extension        | pi-dev:extensions/pi-flow/index.ts (message_end)     |       |
-| 2 | post-marker handoff writes               | framework        | pi-dev:skills/migrate/SKILL.md (gitignore lockout)   |       |
-| 3 | retro-action-item label still alive      | consumer-prefs   | hugn:docs/agents/preferences.md                      |       |
+| 1 | /do hands flow back between phases       | extension:steer  | pi-dev:extensions/pi-flow/index.ts (message_end)     |       |
+| 2 | long live-smoke silence causes user nudges | extension:render/observe | pi-dev:extensions/pi-flow/index.ts or project extension |       |
+| 3 | post-marker handoff writes               | framework        | pi-dev:skills/migrate/SKILL.md (gitignore lockout)   |       |
 | 4 | smoke command name changed in S058       | consumer-prefs   | hugn:docs/agents/preferences.md                      |       |
 ```
 
@@ -274,12 +294,13 @@ For each 🔴 / 🟡 finding, draft the smallest possible edit that, **if it had
 
 **Extension findings (target: pi-dev `extensions/pi-flow/index.ts`, or rarely a new `extensions/<name>/`):**
 
-- Add the handler to `pi-flow` rather than creating a new extension, unless the new concern is large enough to be independently toggleable.
-- A guard must be **deterministic** — same input → same outcome. No LLM calls, no time-of-day branches.
-- A guard must be **narrowly scoped** — tool name + arg shape, or end-of-turn regex. No free-text classification.
-- A guard must be **toggleable** via `~/.pi/agent/settings.json`. Default on. A user that disables it must still be able to read the prose contract.
-- Trim the corresponding SKILL.md prose to a one-line *why* + a pointer to the plugin. Do not delete the why — the model still needs to know the intent when the plugin is off.
-- Update **at most one handler per run.** Two changes at once destroys the next audit's ability to attribute movement.
+- Start from pi's actual extension surface, not from the word "guard": lifecycle/session/agent/model/tool events, `before_agent_start` context injection, `context` shaping, `tool_call` / `tool_result` interception, registered tools, registered commands, `ctx.ui` notifications/widgets/custom UI, custom renderers, `pi.appendEntry()` state, and compaction/session hooks.
+- Prefer extending `pi-flow` when the concern is generic workflow compliance. Create a new extension only when the concern is large enough to be independently named, toggled, installed, and audited.
+- Pick the weakest intervention that would have changed the session outcome: observe → affordance → context/state/render → steer → confirm → block.
+- Determinism is mandatory for `confirm` and `block`; it is desirable but not sufficient for softer interventions. A progress widget, command shortcut, or state checkpoint can be valuable even when it does not block anything.
+- Runtime behavior must be toggleable via `~/.pi/agent/settings.json` when it can alter turns or tool execution. Default on only if the audit shows broad benefit.
+- Keep the corresponding SKILL.md prose as the human-readable contract: one-line *why*, the expected behavior, and a pointer to the runtime support. Do not delete the why — the model still needs to know the intent when the extension is off.
+- Update **at most one runtime mechanism per run** unless the second is pure observability. Two behavior changes at once destroys the next audit's ability to attribute movement.
 
 **Consumer-prefs findings (target: that repo's `docs/agents/preferences.md`):**
 
@@ -363,10 +384,12 @@ audit complete — framework v<X.Y.Z> released, consumer-prefs commit <sha>, nex
 ## Heuristics
 
 - **One screenful of signals beats a dashboard.** Most of the time three or four numbers tell the whole story.
-- **Watch the gap between `/do` injection and next-skill injection.** That is the single highest-information ratio about whether the chain orchestrator is actually orchestrating.
-- **Short user messages are the cheapest interruption proxy.** Count them.
+- **Judge `/do` by workflow completion, not skill injection alone.** Chain depth is useful, but the stronger metric is: plan emitted → phases accounted for → terminal predicates evidenced → side effects match prefs → user did not need to re-enter the chain.
+- **Short user messages are the cheapest interruption proxy.** Count them, then inspect the preceding assistant turn to classify why the user had to nudge.
 - **A taboo without a `.gitignore` lockout will resurrect.** If the same taboo file shows up across two audits, the fix belongs in `/migrate`, not `/do`.
-- **Disclaimer / terminator / lockout / runway** are the four "tripwire" patterns that catch silent contract drift. Reach for them before inventing new ones.
+- **Do not overfit on guards.** The intervention may be an observability counter, status widget, custom command, context injection, state checkpoint, prompt/compaction shaping, soft steer, confirmation, or block. Pick by evidence and measured failure cost.
+- **Trajectory beats scalar success.** A task that eventually shipped after five corrections is not healthy; use the session trace to find where workflow support was missing.
+- **Refresh external reliability patterns when designing new runtime support.** Current agent-reliability work emphasizes trajectory monitoring, recovery orchestration, and trace-derived textual feedback; use that to challenge pi-flow designs before coding.
 
 ## Why this skill exists