pi-dev 0.1.4 → 0.1.6

package/dist/manifest.js

@@ -11,6 +11,7 @@ export const SKILLS = [
     { name: "do", kind: "human", summary: "Do the engineering work end-to-end." },
     { name: "taste", kind: "human", summary: "View / update / onboard preferences." },
     { name: "where", kind: "human", summary: "Recall prior pi sessions for this cwd." },
+    { name: "improve-skill-flow", kind: "human", summary: "Audit pi session telemetry and propose evidence-based SKILL.md edits." },
     // Auto-invoked support skills
     { name: "migrate", kind: "support", summary: "Strict migration gate before /do can run." },
     { name: "setup", kind: "support", summary: "Scaffold issue-tracker / triage / domain docs." },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-dev",
-  "version": "0.1.4",
+  "version": "0.1.6",
   "description": "An autonomous engineering skill framework for the pi runtime — built on Matt Pocock's skills.",
   "type": "module",
   "bin": {

package/skills/do/SKILL.md

@@ -14,10 +14,16 @@ The point: **one request → one finished outcome**, with as few user interrupti
 1. **Migration gate (strict).** If `docs/agents/preferences.md` does not contain the `<!-- migrated: ... -->` marker, **stop** and invoke `/migrate`. Do not run any engineering work on un-migrated repos. The user makes the migration decision; the skill executes it. No partial-migration shortcuts.
 2. **Read preferences (3 layers).** Merge global (`~/.pi/agent/preferences.md`) → project (`docs/agents/preferences.md`) → package (`packages/<pkg>/preferences.md` if a single package is targeted). If global is missing, warn once and use built-in defaults from `/taste`.
 3. **Treat preferences as decisions.** Any choice the merged prefs answer is **decided**. Do not re-ask.
-4. **Keep going.** When something would normally cause a halt (ambiguous classification, scope creep beyond `change-budget`, missing follow-up), **expand and surface in the final summary** instead of stopping mid-flow. Halt only when:
+4. **Keep going. Never hand the flow back between phases.** When something would normally cause a halt (ambiguous classification, scope creep beyond `change-budget`, missing follow-up), **expand and surface in the final summary** instead of stopping mid-flow. Specifically:
+   - Do **not** ask the user "shall I proceed to the next phase?", "want me to continue?", "should I do X next?", or any equivalent. The chain is decided in Step 3 and runs to completion.
+   - Do **not** end your turn between phases. A phase's terminal predicate passing is the cue to immediately print the next phase's status line and start it, in the same turn.
+   - Confirmation-shaped questions are only legal via the **Ambiguity protocol** (one decision the prefs genuinely cannot answer) or the **Failure protocol** (terminal predicate failed twice).
+
+   Halt only when:
    - migration gate fails, OR
    - GitHub gate fails (when GitHub is the tracker), OR
    - a phase's terminal predicate fails twice and the failure cannot be characterised, OR
+   - the Ambiguity protocol fires (preferences truly silent on a one-shot decision), OR
    - user interrupts.
 5. **No handoff files.** State of work lives in three places only: **code (git), issue tracker, merged preferences**. Do not create `.scratch/flow/`, `docs/handoff/`, or any session-log file. Phase outputs are remembered in-context; persistent decisions are committed to code or filed as issues.
 6. **Side-effect gates respect prefs literally.** `auto-create-issues`, `auto-apply-labels`, `auto-commit-per-slice`, `auto-pr` follow merged prefs without reinterpretation.
@@ -99,15 +105,31 @@ If measured scope exceeds `change-budget` (e.g. `module` budget vs `cross-packag
 
 ### Step 4 — Execute the chain
 
-For each phase:
+Before running any phase, **print the planned chain once** so the user can see the runway:
+
+```
+[flow plan] intent=<x> scope=<y> chain: phase1 → phase2 → … → phaseM
+```
+
+Then for each phase:
 
-1. Print status line.
+1. Print status line `[flow N/M] <phase> — <one-sentence what>`.
 2. Run the phase. Use merged prefs to make every taste decision the phase exposes.
 3. Check the terminal predicate. If fail, retry once with explicit diagnosis. If fail again, halt with reason.
-4. Continue.
+4. **Transition without asking.** The instant the terminal predicate passes, print the *next* phase's status line on the very next line and start executing it, in the same turn. No "next, I'll do X" preamble. No "shall I continue?". No turn end.
 
 **No handoff writes.** Pass state in-context only.
 
+**Anti-patterns to detect in your own output before sending** (if you catch any of these mid-draft while phases remain, delete and replace with the next phase's status line):
+
+- "Want me to proceed with <next phase>?"
+- "Shall I continue to <next phase>?"
+- "Ready to move on to <next phase>?"
+- "Let me know if you want me to <next phase>."
+- Ending the turn after a phase summary when N < M.
+
+The only place a wrap-up belongs is **Step 7 — Final summary**, after the last phase has met its terminal predicate.
+
 ### Step 5 — Live verification
 
 Driven by **two separate prefs keys**:
@@ -139,15 +161,29 @@ If a side-effect was skipped because prefs said so, list it in the final summary
 
 ### Step 7 — Final summary
 
-One screen:
+Reached only after the **last** phase in the chain has met its terminal predicate (or the Failure protocol fired). One screen:
 
-- intent / scope / chain executed
+- intent / scope / chain executed (all N/M status lines accounted for)
 - diffs (files touched)
 - side effects done (issue URLs, commit SHAs, branch name, PR URL)
 - side effects deferred (with reason and the exact command to do them)
 - expansions (scope went beyond budget, ambiguous decisions made with rationale)
 - prefs refresh hint if stale
 
+The summary's **last line** must be one of these two terminators, so the user can tell at a glance whether the flow is done:
+
+```
+chain complete — no further action.
+```
+
+or
+
+```
+follow-up: <one-line description> — run `<exact command>`.
+```
+
+If you emitted anything that looks like a wrap-up ("All set!", "Done.", "Let me know if...") without one of those two terminators, you ended early. Reopen the chain at the missed phase.
+
 ## Phase contracts (terminal predicates)
 
 | phase                              | done when…                                                                              |

package/skills/improve-skill-flow/SKILL.md

@@ -0,0 +1,188 @@
+---
+name: improve-skill-flow
+description: Analyse real pi-runtime session telemetry from a consumer repo to find where the engineering skills (especially /do's chain) drift from their stated contract, then propose evidence-anchored edits to the SKILL.md files. Use when the user wants to improve, audit, debug, or evolve the pi-dev skill framework itself based on what actually happened in real sessions ("스킬 개선하자", "do 가 왜 멈춰", "히스토리 보고 분석해서 스킬 고치자", "메타 스킬 작업", etc).
+---
+
+# /improve-skill-flow — Meta-skill for evidence-based skill improvement
+
+The pi-dev skills are pure markdown. They get better by reading what real sessions did, comparing that to what the SKILL.md said *should* happen, and editing the gap closed. This skill is the canonical loop for that.
+
+The point: **never edit a SKILL.md from gut feeling.** Edit because session N showed phase P violated predicate Q on M occasions, and here is the line that would have prevented it.
+
+## When to run
+
+- A consumer repo has accumulated at least one real day of pi sessions (≈ 3+ `.jsonl` files).
+- A specific skill is suspected of misbehaving ("why does `/do` keep stopping?").
+- After landing a skill change, to verify the next session(s) actually follow the new wording.
+- Periodically (every N releases) as a regression sweep across all human-facing skills.
+
+## What this skill is NOT
+
+- Not for analysing the *codebase* of the consumer repo — that is `improve-codebase-architecture`.
+- Not for shipping engineering work — it does not invoke `/do`. Findings turn into proposed SKILL.md diffs, which are committed via the normal release-please flow on `pi-dev`.
+- Not a real-time monitor — it reads completed session files.
+
+## Inputs
+
+- **Target repo path** (or its sessions directory). Defaults: the user names a repo; you resolve it.
+- Optional: a specific skill name to focus the audit on (`do`, `migrate`, `triage`, …).
+- Optional: a date range.
+
+## Session-data location & format
+
+pi-runtime persists every session as JSON Lines at:
+
+```
+~/.pi/agent/sessions/--<cwd-with-slashes-replaced-by-dashes>--/<ISO-ts>_<uuid>.jsonl
+```
+
+For example `cwd=/Users/jason/dev/sandbox/hugn` → `~/.pi/agent/sessions/--Users-jason-dev-sandbox-hugn--/`.
+
+Each line is one of these record types:
+
+| `type` | meaning |
+| --- | --- |
+| `session` | session header — has `cwd`, `timestamp`, `version`, `id` |
+| `model_change` | provider / modelId switch |
+| `thinking_level_change` | reasoning effort knob |
+| `message` | a user or assistant turn |
+
+`message.content` is a **list of blocks**, each block has a `type`:
+
+| block `type` | shape | what it represents |
+| --- | --- | --- |
+| `text` | `{text}` | plain text from either side |
+| `thinking` | `{thinking, thinkingSignature}` | model scratchpad (assistant only) |
+| `toolCall` | `{id, name, arguments}` | **pi uses this** — NOT Anthropic SDK's `tool_use` |
+| `toolResult` | `{id, output}` | **pi uses this** — NOT `tool_result` |
+
+Tool names are **lower-case** (`bash`, `read`, `edit`, `write`, `glob`, `grep`, etc.). Build any parser around `toolCall` / `toolResult` first, then fall back to Anthropic-shaped blocks for robustness.
+
+**Critical detail:** the pi runtime injects each skill's `SKILL.md` content into the conversation as a `<skill name="..." location="...">…</skill>` block embedded inside a **user-role** message (system-side injection, but the role is user). This is how you tell the classifier loaded a skill. Count these to see what `/do` actually picked.
+
+Timestamps on `message` records are ISO strings; some other record types use int millis. Handle both.
+
+## Process
+
+### 1 — Scope and load
+
+Ask the user (one round, only if not already specified):
+
+- target repo (or "all repos with sessions")
+- a skill to focus on, or "everything"
+- a date range or "all"
+
+Resolve the sessions directory. List the `.jsonl` files with size + line count so the user can see the input scale.
+
+### 2 — Build the raw signal table
+
+Run a single pass over every targeted `.jsonl` and tally:
+
+**Per session:**
+- user messages, assistant messages, tool calls, thinking blocks
+- duration (last ts − first ts)
+
+**Aggregates:**
+- tool-use frequency (`bash`, `read`, `edit`, `write`, …)
+- `<skill name="X">` injections per skill (this is the classifier's actual decision)
+- skills mentioned in user text vs. assistant text (request side vs. recall side)
+- top Edit / Write targets (hotspot files)
+- bash commands matching domain-specific danger / smell patterns
+
+**Skill-flow specific:**
+- After each `<skill name="do">` injection, did another skill get injected within the same session? **This measures `/do` chain depth.** Single-step chains are a red flag.
+- Count user messages shorter than ~80 chars — these are usually nudges ("진행해", "다음은?", "끝났어?"). High proportion = `/do` is handing the flow back too often.
+- Count user messages containing correction markers (`아니`, `wait`, `stop`, `취소`, `다시`, `그만`, `undo`, `revert`). These mark interventions.
+- Count `<!-- migrated: ... -->` marker date vs. any post-marker `docs/handoff/` / `.scratch/flow/` / `SESSION_*.md` writes. Drift = handoff lockout failed.
+- Count tracker writes (`gh issue create`, `gh pr create`) and check whether `generated by AI` (the `/triage` disclaimer) appears in the surrounding text. Missing disclaimer = `/to-issues` / `/triage` predicate violated.
+
+Use a deterministic Python or shell script you write once and check into `/tmp` for the duration of the run. Do not eyeball big JSONLs by hand.
+
+### 3 — Cross-reference with repo state
+
+For the same date range, pull:
+
+- `git log --since=<start> --pretty=format:"%h %ad %s"` — commit cadence vs. the predicate `auto-commit-per-slice`.
+- `gh issue list / pr list` (if GitHub) — slice/PR shape vs. `default-issue-style=vertical-slice`.
+- Existence of forbidden paths from `docs/agents/preferences.md` taboos (`docs/handoff/`, `.scratch/flow/`, etc.).
+- `gh label list` filtered against taboo-marked labels.
+
+Cross-reference each signal against:
+
+- The skill's **terminal predicate** in `do/SKILL.md` → "Phase contracts".
+- The repo's **`docs/agents/preferences.md`** taboos and `auto-*` settings.
+- The hard rules in the skill being audited.
+
+### 4 — Score the gaps
+
+Produce a small table per audited skill. Each row:
+
+```
+| signal                         | observed | expected (predicate / rule)        | severity |
+| ------------------------------ | -------- | ---------------------------------- | -------- |
+| /do → next-skill chain depth   | 1/5      | ≥1 per chain step (M phases)       | 🔴       |
+| AI disclaimer on slice issues  | 0/8      | every issue starts with disclaimer | 🔴       |
+| post-marker handoff writes     | 9        | 0                                  | 🟡       |
+| ...                            |          |                                    |          |
+```
+
+Severity rule of thumb:
+- 🔴 — predicate is violated repeatedly AND the rule is binding (in "Hard rules" or "Phase contracts").
+- 🟡 — taboo or preference violated but the skill's wording is soft / advisory.
+- 🟢 — observed behaviour matches the spec; do not propose a change.
+
+### 5 — Anchor each finding to an evidence excerpt
+
+For every 🔴 / 🟡 row, quote the smallest piece of evidence that makes the gap undeniable:
+
+- a user message timestamp + first 80 chars,
+- a tool-call command that violates a taboo,
+- a missing disclaimer in a created issue body.
+
+If a finding cannot be backed by an excerpt, it is not actionable yet — demote to a TODO and keep digging.
+
+### 6 — Propose SKILL.md edits
+
+For each 🔴 finding, draft the smallest possible edit that, **if it had been in the SKILL.md at session time, would have prevented the gap.** Constraints:
+
+- Edit a **rule** or a **step**, not a flavour sentence. The model must be able to detect the constraint in its own draft output.
+- Prefer **explicit anti-pattern strings** ("Do not say 'shall I continue?'") over abstract injunctions ("be decisive"). The hugn-2026-05 audit showed that named anti-patterns work.
+- Prefer **terminal markers** ("the summary's last line must be one of these two literals: …") over qualitative descriptions of "good wrap-up".
+- Update **at most three skills per run.** More than that means findings aren't anchored well enough.
+
+Show the edits as a unified diff before applying.
+
+### 7 — Apply, release, verify
+
+Standard pi-dev release loop:
+
+1. `git add skills/<name>/SKILL.md && git commit -m "<conventional commit anchoring the evidence>"` — the commit body should cite the signal that motivated each change.
+2. `cp` the edited SKILL.md into `~/.pi/agent/skills/<name>/` so the **next** session in any repo picks up the change immediately (release-please takes a minute and a half).
+3. `git push origin main`; release-please opens the version-bump PR; merge it; npm publish runs automatically.
+4. After the next consumer session lands, re-run **this skill** and confirm the targeted signals moved.
+
+## Terminal predicate
+
+This skill is done when **all three** are true:
+
+1. A signal table with severities and evidence excerpts has been presented.
+2. Either (a) zero 🔴 findings — flow is healthy, recorded as "no change this cycle", OR (b) a unified diff for each 🔴 finding has been applied to `skills/<name>/SKILL.md` AND mirrored into `~/.pi/agent/skills/<name>/`.
+3. If diffs landed, the release loop (commit → push → merge release PR → npm publish) finished and the new version is `npm view pi-dev@latest version`.
+
+## What this skill does not do
+
+- It does not modify a consumer repo's code, issues, or preferences. It only edits **pi-dev's own `skills/`**.
+- It does not invent gaps from first principles. Every finding must come from a session excerpt or a repo-state probe.
+- It does not run faster than the data allows — if there is only one session, run it but say so up front; the signal is noisy.
+
+## Heuristics
+
+- **One screenful of signals beats a dashboard.** Most of the time three or four numbers tell the whole story.
+- **Watch the gap between `/do` injection and next-skill injection.** That is the single highest-information ratio about whether the chain orchestrator is actually orchestrating.
+- **Short user messages are the cheapest interruption proxy.** Count them.
+- **A taboo without a `.gitignore` lockout will resurrect.** If the same taboo file shows up across two audits, the fix belongs in `/migrate`, not `/do`.
+- **Disclaimer / terminator / lockout / runway** are the four "tripwire" patterns that catch silent contract drift. Reach for them before inventing new ones.
+
+## Why this skill exists
+
+Skills are prose. Prose drifts. Without a feedback loop, the SKILL.md files become wishful thinking that the agent ignores in real sessions. This skill is the loop.