@davidorex/pi-behavior-monitors 0.1.2

package/CHANGELOG.md

@@ -0,0 +1,61 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## v0.1.2
+
+[compare changes](https://github.com/davidorex/pi-behavior-monitors/compare/v0.1.1...v0.1.2)
+
+### 🚀 Enhancements
+
+- Unify monitor management under /monitors command with subcommand routing ([dbffaa0](https://github.com/davidorex/pi-behavior-monitors/commit/dbffaa0))
+- Add vitest test suite for pure functions in index.ts ([fed0d75](https://github.com/davidorex/pi-behavior-monitors/commit/fed0d75))
+
+### 🩹 Fixes
+
+- Address conformance audit findings — unused param, session_switch, headless escalate ([f2d3baa](https://github.com/davidorex/pi-behavior-monitors/commit/f2d3baa))
+
+### 🏡 Chore
+
+- Add .claude/ to gitignore, version conformance audit in docs/ ([d6a8395](https://github.com/davidorex/pi-behavior-monitors/commit/d6a8395))
+
+### ❤️ Contributors
+
+- David Ryan <davidryan@gmail.com>
+
+## v0.1.1
+
+[compare changes](https://github.com/davidorex/pi-behavior-monitors/compare/v0.1.0...v0.1.1)
+
+### 📖 Documentation
+
+- Add CLAUDE.md with project conventions ([5f5b427](https://github.com/davidorex/pi-behavior-monitors/commit/5f5b427))
+- Expand SKILL.md to cover full runtime behavior and bundled monitors ([5c9980d](https://github.com/davidorex/pi-behavior-monitors/commit/5c9980d))
+
+### 🏡 Chore
+
+- Add npm publish metadata, files whitelist, and normalize repository URL ([4b3f1f4](https://github.com/davidorex/pi-behavior-monitors/commit/4b3f1f4))
+- Add .gitignore, remove runtime .workflow/ from tracking ([c1d4ae5](https://github.com/davidorex/pi-behavior-monitors/commit/c1d4ae5))
+
+### ❤️ Contributors
+
+- David Ryan <davidryan@gmail.com>
+
+## v0.1.0
+
+Initial release.
+
+### Added
+
+- Monitor extension with event-driven classification (message_end, turn_end, agent_end, command)
+- JSON-based monitor definitions (.monitor.json), pattern libraries (.patterns.json), instructions (.instructions.json)
+- Side-channel LLM classification with CLEAN/FLAG/NEW verdict protocol
+- Auto-learning of new patterns from runtime detection
+- Write action for structured JSON findings output
+- Scope targeting (main, subagent, all, workflow)
+- Bundled monitors: fragility, hedge, work-quality
+- Slash commands: /monitors, /<name>, /<name> <instruction>
+- Status bar integration showing engaged/dismissed monitors
+- Escalation with ceiling + ask/dismiss
+- SKILL.md for LLM-assisted monitor creation
+- JSON schemas for monitor definitions and patterns

package/README.md

@@ -0,0 +1,59 @@
+# pi-behavior-monitors
+
+Behavior monitors for [pi](https://github.com/badlogic/pi-mono) that watch agent activity, classify against pattern libraries, steer corrections, and write structured findings to JSON files.
+
+Monitors are JSON files (`.monitor.json`) with typed blocks: classify (LLM side-channel), patterns (JSON library), actions (steer + write to JSON), and scope (main/subagent/workflow targeting).
+
+## Install
+
+```bash
+pi install npm:pi-behavior-monitors
+```
+
+On first run, if no monitors exist in your project, example monitors are seeded into `.pi/monitors/`. Edit or delete them to customize.
+
+## Bundled Example Monitors
+
+- **fragility** — detects when the agent leaves broken state behind (errors it noticed but didn't fix, TODO comments instead of solutions, empty catch blocks). Writes findings to `.workflow/gaps.json`.
+- **hedge** — detects when the agent deviates from what the user actually said (rephrasing questions, assuming intent, deflecting with counter-questions)
+- **work-quality** — on-demand audit of work quality (trial-and-error, not reading before editing, fixing symptoms instead of root causes). Invoked via `/work-quality`. Writes findings to `.workflow/gaps.json`.
+
+## File Structure
+
+Each monitor is a triad of JSON files:
+
+```
+.pi/monitors/
+├── fragility.monitor.json       # Definition (classify + patterns + actions + scope)
+├── fragility.patterns.json      # Known patterns (grows automatically)
+└── fragility.instructions.json  # User corrections (optional)
+```
+
+## Writing Your Own
+
+Create a `.monitor.json` file in `.pi/monitors/` conforming to `schemas/monitor.schema.json`. Ask the LLM to read the `pi-behavior-monitors` skill for the full schema and examples.
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `/monitors` | List all monitors, scope, and state |
+| `/<name>` | Show monitor patterns and instructions |
+| `/<name> <text>` | Add an instruction to calibrate the monitor |
+
+## How It Works
+
+1. A monitor fires on a configured event (e.g., after each assistant message)
+2. It checks scope (main context, subagent, workflow) and activation conditions
+3. It collects relevant conversation context (tool results, assistant text, etc.)
+4. A side-channel LLM call classifies the context against the JSON pattern library
+5. Based on the verdict, the monitor executes actions:
+   - **steer**: inject a correction message into the conversation (main scope only)
+   - **write**: append structured findings to a JSON file (any scope)
+   - **learn**: add new patterns to the library automatically
+6. Downstream workflows can consume the JSON findings (e.g., gaps.json → verify step → gate)
+
+## Schemas
+
+- `schemas/monitor.schema.json` — monitor definition format
+- `schemas/monitor-pattern.schema.json` — pattern library entry format

package/examples/fragility.instructions.json

@@ -0,0 +1 @@
+[]

package/examples/fragility.monitor.json

@@ -0,0 +1,62 @@
+{
+  "name": "fragility",
+  "description": "Detects unaddressed fragilities after tool use",
+  "event": "message_end",
+  "when": "has_tool_results",
+  "scope": {
+    "target": "main"
+  },
+  "classify": {
+    "model": "claude-sonnet-4-20250514",
+    "context": ["tool_results", "assistant_text"],
+    "excludes": [],
+    "prompt": "An agent just performed actions and responded. Determine if it left known\nfragilities — errors, warnings, or broken state it noticed but chose not\nto fix, expecting someone else to deal with them.\n\nRecent tool outputs the agent saw:\n{tool_results}\n\nThe agent then said:\n\"{assistant_text}\"\n\n{instructions}\n\nFragility patterns to check:\n{patterns}\n\nReply CLEAN if the agent addressed problems it encountered or if no\nproblems were present.\nReply FLAG:<one sentence describing the fragility left behind> if a\nknown pattern was matched.\nReply NEW:<new pattern to add>|<one sentence describing the fragility\nleft behind> if the agent left a fragility not covered by existing patterns."
+  },
+  "patterns": {
+    "path": "fragility.patterns.json",
+    "learn": true
+  },
+  "instructions": {
+    "path": "fragility.instructions.json"
+  },
+  "actions": {
+    "on_flag": {
+      "steer": "Fix the issue you left behind.",
+      "write": {
+        "path": ".workflow/gaps.json",
+        "schema": "schemas/gaps.schema.json",
+        "merge": "append",
+        "array_field": "gaps",
+        "template": {
+          "id": "fragility-{finding_id}",
+          "description": "{description}",
+          "status": "open",
+          "category": "fragility",
+          "priority": "{severity}",
+          "source": "monitor"
+        }
+      }
+    },
+    "on_new": {
+      "steer": "Fix the issue you left behind.",
+      "learn_pattern": true,
+      "write": {
+        "path": ".workflow/gaps.json",
+        "schema": "schemas/gaps.schema.json",
+        "merge": "append",
+        "array_field": "gaps",
+        "template": {
+          "id": "fragility-{finding_id}",
+          "description": "{description}",
+          "status": "open",
+          "category": "fragility",
+          "priority": "warning",
+          "source": "monitor"
+        }
+      }
+    },
+    "on_clean": null
+  },
+  "ceiling": 5,
+  "escalate": "ask"
+}

package/examples/fragility.patterns.json

@@ -0,0 +1,86 @@
+[
+  {
+    "id": "dismiss-preexisting",
+    "description": "Dismissing errors as pre-existing instead of fixing them",
+    "severity": "warning",
+    "category": "avoidance",
+    "source": "bundled"
+  },
+  {
+    "id": "empty-catch",
+    "description": "Silently catching exceptions with empty catch blocks",
+    "severity": "error",
+    "category": "error-handling",
+    "source": "bundled"
+  },
+  {
+    "id": "todo-instead-of-fix",
+    "description": "Adding TODO or FIXME comments instead of solving the problem now",
+    "severity": "warning",
+    "category": "deferral",
+    "source": "bundled"
+  },
+  {
+    "id": "happy-path-only",
+    "description": "Writing code that assumes happy path without handling failure cases",
+    "severity": "warning",
+    "category": "error-handling",
+    "source": "bundled"
+  },
+  {
+    "id": "not-my-change",
+    "description": "Leaving known broken state because 'it's not my change'",
+    "severity": "warning",
+    "category": "avoidance",
+    "source": "bundled"
+  },
+  {
+    "id": "early-return-on-unexpected",
+    "description": "Returning early or skipping logic when an unexpected condition is hit instead of handling it",
+    "severity": "warning",
+    "category": "error-handling",
+    "source": "bundled"
+  },
+  {
+    "id": "undocumented-delegation",
+    "description": "Deferring error handling to the caller without documenting or enforcing it",
+    "severity": "warning",
+    "category": "error-handling",
+    "source": "bundled"
+  },
+  {
+    "id": "silent-fallback",
+    "description": "Using fallback values that mask failures silently (returning empty string, null, undefined on error)",
+    "severity": "warning",
+    "category": "error-handling",
+    "source": "bundled"
+  },
+  {
+    "id": "prose-without-action",
+    "description": "Noting a problem in prose but not acting on it in code",
+    "severity": "warning",
+    "category": "deferral",
+    "source": "bundled"
+  },
+  {
+    "id": "blame-environment",
+    "description": "Blaming the environment or dependencies instead of working around or fixing the issue",
+    "severity": "warning",
+    "category": "avoidance",
+    "source": "bundled"
+  },
+  {
+    "id": "workaround-over-root-cause",
+    "description": "Identifying architectural inefficiencies but implementing workarounds instead of fixing the root cause",
+    "severity": "warning",
+    "category": "avoidance",
+    "source": "bundled"
+  },
+  {
+    "id": "elaborate-workaround-for-fixable",
+    "description": "Documenting a known dangerous state and designing elaborate workarounds instead of fixing the root cause",
+    "severity": "error",
+    "category": "avoidance",
+    "source": "bundled"
+  }
+]

package/examples/hedge.instructions.json

@@ -0,0 +1 @@
+[]

package/examples/hedge.monitor.json

@@ -0,0 +1,34 @@
+{
+  "name": "hedge",
+  "description": "Detects when assistant deviates from what the user said",
+  "event": "turn_end",
+  "when": "always",
+  "scope": {
+    "target": "main"
+  },
+  "classify": {
+    "model": "claude-sonnet-4-20250514",
+    "context": ["user_text", "tool_calls", "custom_messages", "assistant_text"],
+    "excludes": ["fragility"],
+    "prompt": "The user said:\n\"{user_text}\"\n\n{tool_calls}\n{custom_messages}\n\nThe assistant's latest response:\n\"{assistant_text}\"\n\n{instructions}\n\nGiven the full context of what the user asked and what the assistant did,\ndid the assistant deviate from what the user actually said in its latest\nresponse?\n\nIf the user's request has been addressed by the actions taken, the\nassistant summarizing that completed work is not a deviation.\n\nCheck against these patterns:\n{patterns}\n\nReply CLEAN if the assistant stuck to what the user actually said.\nReply FLAG:<one sentence, what was added or substituted> if a known\npattern was matched.\nReply NEW:<new pattern to add>|<one sentence, what was added or\nsubstituted> if the assistant deviated in a way not covered by\nexisting patterns."
+  },
+  "patterns": {
+    "path": "hedge.patterns.json",
+    "learn": true
+  },
+  "instructions": {
+    "path": "hedge.instructions.json"
+  },
+  "actions": {
+    "on_flag": {
+      "steer": "Address what the user actually said."
+    },
+    "on_new": {
+      "steer": "Address what the user actually said.",
+      "learn_pattern": true
+    },
+    "on_clean": null
+  },
+  "ceiling": 3,
+  "escalate": "ask"
+}

package/examples/hedge.patterns.json

@@ -0,0 +1,10 @@
+[
+  { "id": "rephrase-question", "description": "Rephrasing the user's question into a different question and answering that instead", "severity": "warning", "category": "substitution", "source": "bundled" },
+  { "id": "assume-intent", "description": "Assuming intent the user did not express", "severity": "warning", "category": "projection", "source": "bundled" },
+  { "id": "add-questions", "description": "Adding questions the user did not ask", "severity": "warning", "category": "augmentation", "source": "bundled" },
+  { "id": "reinterpret-words", "description": "Interpreting the user's words as meaning something other than what they said", "severity": "warning", "category": "substitution", "source": "bundled" },
+  { "id": "attribute-position", "description": "Attributing a position or preference the user did not state", "severity": "warning", "category": "projection", "source": "bundled" },
+  { "id": "ask-permission", "description": "Asking permission to do something instead of doing it when the user asked a direct question", "severity": "warning", "category": "deflection", "source": "bundled" },
+  { "id": "qualify-yesno", "description": "Answering a yes/no question with qualifiers instead of yes or no", "severity": "info", "category": "deflection", "source": "bundled" },
+  { "id": "counter-question", "description": "Deflecting with a counter-question when the user expected an answer", "severity": "warning", "category": "deflection", "source": "bundled" }
+]

package/examples/work-quality.instructions.json

@@ -0,0 +1 @@
+[]

package/examples/work-quality.monitor.json

@@ -0,0 +1,62 @@
+{
+  "name": "work-quality",
+  "description": "On-demand work quality analysis",
+  "event": "command",
+  "when": "always",
+  "scope": {
+    "target": "main"
+  },
+  "classify": {
+    "model": "claude-sonnet-4-20250514",
+    "context": ["user_text", "tool_calls", "assistant_text"],
+    "excludes": [],
+    "prompt": "An agent was asked:\n\"{user_text}\"\n\nIt performed these actions:\n{tool_calls}\n\nThen it said:\n\"{assistant_text}\"\n\n{instructions}\n\nAnalyze the quality of the work. Check against these patterns:\n{patterns}\n\nReply CLEAN if the work was sound.\nReply FLAG:<one sentence describing the quality issue> if a known\npattern was matched.\nReply NEW:<new pattern to add>|<one sentence describing the quality\nissue> if there's a work quality problem not covered by existing patterns."
+  },
+  "patterns": {
+    "path": "work-quality.patterns.json",
+    "learn": true
+  },
+  "instructions": {
+    "path": "work-quality.instructions.json"
+  },
+  "actions": {
+    "on_flag": {
+      "steer": "Fix the quality issue.",
+      "write": {
+        "path": ".workflow/gaps.json",
+        "schema": "schemas/gaps.schema.json",
+        "merge": "append",
+        "array_field": "gaps",
+        "template": {
+          "id": "quality-{finding_id}",
+          "description": "{description}",
+          "status": "open",
+          "category": "work-quality",
+          "priority": "{severity}",
+          "source": "monitor"
+        }
+      }
+    },
+    "on_new": {
+      "steer": "Fix the quality issue.",
+      "learn_pattern": true,
+      "write": {
+        "path": ".workflow/gaps.json",
+        "schema": "schemas/gaps.schema.json",
+        "merge": "append",
+        "array_field": "gaps",
+        "template": {
+          "id": "quality-{finding_id}",
+          "description": "{description}",
+          "status": "open",
+          "category": "work-quality",
+          "priority": "warning",
+          "source": "monitor"
+        }
+      }
+    },
+    "on_clean": null
+  },
+  "ceiling": 3,
+  "escalate": "ask"
+}

package/examples/work-quality.patterns.json

@@ -0,0 +1,13 @@
+[
+  { "id": "trial-and-error", "description": "Trial-and-error instead of reading code to understand it first", "severity": "warning", "category": "methodology", "source": "bundled" },
+  { "id": "no-verify", "description": "Making changes without verifying them (no check/test run after edits)", "severity": "error", "category": "verification", "source": "bundled" },
+  { "id": "symptom-fix", "description": "Fixing symptoms instead of root causes", "severity": "warning", "category": "methodology", "source": "bundled" },
+  { "id": "excessive-changes", "description": "Changing more files than necessary to solve the problem", "severity": "warning", "category": "scope", "source": "bundled" },
+  { "id": "copy-paste", "description": "Copy-pasting code instead of extracting shared logic", "severity": "warning", "category": "quality", "source": "bundled" },
+  { "id": "debug-artifacts", "description": "Leaving debug artifacts (console.log, commented-out code, temporary files)", "severity": "warning", "category": "cleanup", "source": "bundled" },
+  { "id": "double-edit", "description": "Making an edit then immediately making another edit to the same file to fix the first edit", "severity": "info", "category": "methodology", "source": "bundled" },
+  { "id": "edit-without-read", "description": "Not reading a file before editing it", "severity": "error", "category": "methodology", "source": "bundled" },
+  { "id": "insanity-retry", "description": "Running a command, getting an error, and running the same command again expecting different results", "severity": "warning", "category": "methodology", "source": "bundled" },
+  { "id": "wrong-problem", "description": "Solving a different problem than the one that was asked about", "severity": "error", "category": "scope", "source": "bundled" },
+  { "id": "no-plan", "description": "Did not create a plan before starting work", "severity": "info", "category": "methodology", "source": "bundled" }
+]