@mutagent/helix 0.1.0-alpha.14

package/mutagent-diagnostics/README.md

@@ -0,0 +1,343 @@
+# MUTAGENT Diagnostics
+
+```
+ ███▄ ▄███▓ █    ██ ▄▄▄█████▓ ▄▄▄        ▄████ ▓█████  ███▄    █ ▄▄▄█████▓
+▓██▒▀█▀ ██▒ ██  ▓██▒▓  ██▒ ▓▒▒████▄     ██▒ ▀█▒▓█   ▀  ██ ▀█   █ ▓  ██▒ ▓▒
+▓██    ▓██░▓██  ▒██░▒ ▓██░ ▒░▒██  ▀█▄  ▒██░▄▄▄░▒███   ▓██  ▀█ ██▒▒ ▓██░ ▒░
+▒██    ▒██ ▓▓█  ░██░░ ▓██▓ ░ ░██▄▄▄▄██ ░▓█  ██▓▒▓█  ▄ ▓██▒  ▐▌██▒░ ▓██▓ ░
+▒██▒   ░██▒▒▒█████▓   ▒██▒ ░  ▓█   ▓██▒░▒▓███▀▒░▒████▒▒██░   ▓██░  ▒██▒ ░
+░ ▒░   ░  ░░▒▓▒ ▒ ▒   ▒ ░░    ▒▒   ▓▒█░ ░▒   ▒ ░░ ▒░ ░░ ▒░   ▒ ▒   ▒ ░░
+░  ░      ░░░▒░ ░ ░     ░      ▒   ▒▒ ░  ░   ░  ░ ░  ░░ ░░   ░ ▒░    ░
+░      ░    ░░░ ░ ░   ░        ░   ▒   ░ ░   ░    ░      ░   ░ ░   ░
+       ░      ░                    ░  ░      ░    ░  ░         ░
+```
+
+> **Diagnostics-on-Tap for AI agents.**
+> Pull evidence from your agent traces, translate user feedback into root-cause records,
+> surface ranked remedies, and apply approved fixes — all from within your existing AI coding session.
+
+---
+
+[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](./LICENSE)
+[![Version](https://img.shields.io/badge/version-0.2.0--alpha-blueviolet)](./package.json)
+[![Bun](https://img.shields.io/badge/runtime-Bun%201%2B-orange)](https://bun.sh)
+[![Node](https://img.shields.io/badge/node-%3E%3D18-brightgreen)](https://nodejs.org)
+[![Platforms](https://img.shields.io/badge/platforms-Claude%20Code%20%C2%B7%20Codex%20%C2%B7%20Cursor%20%C2%B7%20OpenCode-a78bfa)](./README.md#platform-support)
+
+---
+
+## Table of Contents
+
+- [What it does](#what-it-does)
+- [Quick Start](#quick-start)
+- [Platform Support](#platform-support)
+- [Architecture](#architecture)
+- [Features](#features)
+- [Configuration](#configuration)
+- [Design Principles](#design-principles)
+- [Project Layout](#project-layout)
+- [License](#license)
+
+---
+
+## What it does
+
+`@mutagent/diagnostics` is an AI coding agent skill — a self-contained bundle that installs
+into your existing coding-agent runtime (Claude Code, Codex, Cursor, OpenCode) and gives it
+the ability to diagnose itself.
+
+Invoke it naturally in chat: `"diagnose my agents"` or `"why did my agent fail last night"`.
+The skill:
+
+1. **Tier 0 static scan** — free, instant pattern match on your traces to bound token cost
+2. **Auto-extracts a rich entity context** at ingest (system prompt · tool inventory · input sample) — deterministically, no LLM
+3. **Slices** trace history into up to 5 parallel analysis clusters
+4. **Dispatches N Analyzers** that run RCA against each cluster
+5. **Classifies each finding** on 3 axes: WHAT failed · WHY it failed · WHERE in the stack
+6. **Builds a deterministic render input** (Step 8.5 enricher) — derives the big-stat row, 24h latency heatmap, and signal census; fail-loud if the input is starved
+7. **Renders a gold-standard HTML report** — Methodology [INTERNAL] · Overview · one tab per finding · Decisions — with copy-back markdown approval workflow
+8. **Applies the approved remedy through the shared `mutagent-cli apply` transport** — at
+   the gate it spawns **ai-engineer** (the WRITE actor) which shells `mutagent-cli apply`
+   on an isolated git worktree (PR for local, non-destructive create-rev+activate for cloud)
+   — without touching your checkout (the bespoke apply-worker is retired)
+
+First invocation triggers interactive onboarding to configure your source + target platform.
+All subsequent invocations run the full diagnostic cycle automatically.
+
+---
+
+## Quick Start
+
+```bash
+# First-time install — bootstraps config + runs onboarding wizard
+$ pnpx @mutagent/diagnostics init
+
+mutagent-diagnostics v0.1.0-alpha
+Checking host runtime...  claude-code detected
+
+? Select trace source platform:
+  > Langfuse (cloud / self-hosted)
+    OpenTelemetry (Jaeger / Tempo / Honeycomb)
+    Local JSONL files
+    Claude Code session transcripts
+    Codex session transcripts
+
+? Select target platform for fix delivery:
+  > Claude Code (local agent config)
+    Cursor (.cursorrules)
+    OpenCode (config)
+    Mastra.ai (TypeScript PR)
+    Cloud REST (idempotent PUT)
+
+Config written to .mutagent/config.yaml
+Agents installed: .claude/agents/diagnostics-analyzer.md
+                  .claude/agents/ai-engineer.md    (WRITE — vendored)
+                  .claude/agents/ai-architect.md   (VERIFY — vendored)
+
+Ready. Invoke: "diagnose my agents" in your coding session.
+```
+
+After init, trigger diagnostics naturally from your AI session:
+
+```
+You: diagnose my agents
+
+> Running Tier 0 static scan on 247 traces...
+> 3 signal clusters detected (errors: 12, low-score: 8, feedback: 4)
+> Dispatching 3 analyzers in parallel...
+> RCA complete. 7 findings ranked by severity.
+> Opening HTML report...
+```
+
+---
+
+## Platform Support
+
+### Source Platforms (trace evidence)
+
+| Platform | Type | Notes |
+|----------|------|-------|
+| **Langfuse** | Cloud / self-hosted | Full filter/search coverage |
+| **OpenTelemetry** | Jaeger · Tempo · Honeycomb | OTLP endpoint |
+| **Local JSONL** | `.jsonl` / `.ndjson` | Filesystem read |
+| **Claude Code** | Session transcripts | Auto-detected from `~/.claude/projects/` |
+| **Codex CLI** | Session transcripts | Auto-detected from `~/.codex/sessions/` |
+
+### Target Platforms (apply fixes to)
+
+| Platform | Class | Delivery |
+|----------|-------|---------|
+| **Claude Code** | local-agent | Edit `~/.claude/agents/*.md` via PR |
+| **Codex CLI** | local-agent | Edit agent configs via PR |
+| **Cursor** | local-agent | Edit `.cursorrules` via PR |
+| **OpenCode** | local-agent | Edit config via PR |
+| **Mastra.ai** | local-code-construct | TypeScript PR to agent graph |
+| **Cloud Agent SDK** | local-code-construct | TypeScript PR to SDK integration |
+| **Cloud REST** | remote | Idempotent PUT mutations |
+
+---
+
+## Architecture
+
+<details>
+<summary>Flow diagram — click to expand</summary>
+
+```mermaid
+%%{init: {'theme':'dark','themeVariables':{'primaryColor':'#a78bfa','primaryTextColor':'#e2e8f0','primaryBorderColor':'#7c3aed','lineColor':'#06b6d4','edgeLabelBackground':'#1e1b4b','background':'#0f0f23','clusterBkg':'#1e1b4b','clusterBorder':'#4c1d95','titleColor':'#a78bfa'}}}%%
+flowchart TD
+    INV["Invoke<br/>'diagnose my agents'"]
+    DETECT{Config<br/>present?}
+    ONB["Onboarding<br/>(8-phase)"]
+    PROTO["Load orchestrator-protocol.md<br/>Parent session = orchestrator"]
+
+    TIER0["Tier 0 static scan<br/>scripts/tier0-scan.ts"]
+    SLICER["Dynamic-cluster slicer<br/>scripts/slicer.ts<br/>cap = 5"]
+    ANL["N Analyzers ≤ 5<br/>assets/agents/diagnostics-analyzer.md"]
+    RCA["RCA Layer<br/>WHAT / WHY / WHERE"]
+    ENRICH["Step 8.5 — Build Render Input<br/>scripts/enrich/build-render-input.ts<br/>deterministic · fail-loud"]
+    REPORT["Gold-standard HTML Report<br/>assets/templates/report.html.tpl<br/>copy-back markdown approval"]
+    APPLY["Shared apply — ai-engineer + mutagent-cli apply<br/>worktree-PR / REST create-rev<br/>audited + revertible"]
+
+    classDef purple fill:#4c1d95,stroke:#a78bfa,color:#e2e8f0
+    classDef cyan   fill:#0e7490,stroke:#06b6d4,color:#e2e8f0
+    classDef dim    fill:#1e293b,stroke:#475569,color:#94a3b8
+
+    INV --> DETECT
+    DETECT -->|missing| ONB
+    DETECT -->|present| PROTO
+    PROTO --> TIER0
+    TIER0 --> SLICER
+    SLICER --> ANL
+    ANL --> RCA
+    RCA --> ENRICH
+    ENRICH --> REPORT
+    REPORT --> APPLY
+
+    class INV,DETECT purple
+    class TIER0,SLICER,ANL,RCA,ENRICH cyan
+    class ONB,PROTO,REPORT,APPLY dim
+```
+
+</details>
+
+Full DAG and component dependency graph: [`references/reference.md`](.claude/skills/mutagent-diagnostics/references/reference.md)
+
+---
+
+## Features
+
+| Feature | Detail |
+|---------|--------|
+| **3-axis failure taxonomy** | Every finding classified on `(WHAT, WHY, WHERE)` — 10 WHAT types · 9 WHY types · 8 WHERE types |
+| **Tier 0 static scan** | Cost gate before any LLM call — pattern-match + signal count in milliseconds |
+| **Dynamic cluster slicing** | Groups traces by error / feedback / latency signal; window-based fallback when no a-priori signal |
+| **Cap-of-5 fan-out** | Never more than 5 parallel analyzers — keeps cost and complexity bounded |
+| **Auto-extracted entity context** | Every normalizer derives an `EntityContext` (system prompt · tool inventory with per-tool stats · input sample) at ingest — deterministic, no LLM; operator never hand-fills it. Fields > 1 KB render as collapsed `ExpandableSection` (system prompt always collapsed for PII) |
+| **Gold-standard HTML report** | 8-tab report — Methodology [INTERNAL] · Overview (entity card · 6-tile big-stat · 24h latency heatmap · signal census · scan funnel) · one tab per finding (taxonomy · evidence · why-chain · assumptions pills · ★-recommended remedies) · Decisions. Built by the deterministic Step 8.5 enricher; renderer is fail-loud on starved input (R1 §9.3) |
+| **HTML report + copy-back** | Human-in-the-loop review via rendered HTML; operator pastes approval markdown back into chat |
+| **Structured contract mode** | When a target ships a `self-diagnosis-contract.yaml`, the report becomes a structured 10-category pass/fail/pending scorecard against the declared success criteria |
+| **Branch-hygiene apply** | All local fixes land in an isolated git worktree → PR; operator's checkout is never touched |
+| **Stale-target detection** | Hash-compare before every write; re-diagnose prompt if drift detected |
+| **Idempotent REST** | Cloud-target PUTs include `Idempotency-Key`; safe to retry on 5xx |
+| **Dual audit trail** | Every apply emits `pr-body.md` + `audit.json` + `audit.md` |
+| **Self-diagnostics** | Post-session self-RCA (opt-in, default OFF); feeds skill's own transcripts through the same pipeline |
+| **Platform-portable UX** | `AskUserQuestion` on Claude Code; numbered chat-choice fallback on Codex / Cursor / OpenCode |
+| **Platform-native install** | `pnpx @mutagent/diagnostics init` auto-detects runtime and installs agent `.md` files |
+
+### Failure Taxonomy
+
+```
+WHAT  →  wrong-output · missing-output · loop · latency-spike · cost-overshoot
+         format-violation · hallucination · user-complaint · low-score · missing-context
+
+WHY   →  prompt-underspec · prompt-overspec · tool-misuse · tool-missing
+         context-overflow · provider-limit · data-staleness · handoff-loss · dependency-failure
+
+WHERE →  system-prompt · tool-definition · agent-config · routing-config
+         upstream-data · provider-side · harness-side · user-input
+```
+
+---
+
+## Configuration
+
+Config lives at `<project>/.mutagent/config.yaml` (generated by `init`).
+Secrets (API keys) live at `<project>/.mutagentrc` — gitignored, never committed.
+
+```yaml
+# .mutagent/config.yaml — generated by pnpx @mutagent/diagnostics init
+
+source:
+  platform: "langfuse"           # langfuse | otel | local-jsonl | claude-code | codex
+  config:
+    host: "https://cloud.langfuse.com"
+    # Keys: set LANGFUSE_SECRET_KEY + LANGFUSE_PUBLIC_KEY in .mutagentrc
+
+target:
+  platform: "local-claude"       # local-claude | local-codex | local-cursor | local-opencode
+                                 # local-mastra | local-cloud-agent-sdk | cloud-rest
+  config: {}
+
+filters:
+  time_window:
+    from: "7daysAgo"
+    to:   "now"
+  score_below: null              # auto-scaled threshold when set
+  limit: 100
+
+ask_tool:
+  runtime: "claude-code"         # claude-code | chat-multi-choice
+
+self_diagnostics:
+  enabled: false                 # default OFF
+```
+
+Full schema with doc strings: [`references/config.md`](.claude/skills/mutagent-diagnostics/references/config.md)
+
+---
+
+## Design Principles
+
+Every change is audited against a locked constitution of design principles — a few
+that shape day-to-day behavior:
+
+- **Tier-0 before LLM** — a free static scan runs before any model call.
+- **RCA layer mandatory** — never a raw score; always WHAT + WHY + WHERE + evidence.
+- **Read-before-write** — for every apply: GET current state, show the diff, then write.
+- **Branch hygiene** — local fixes land in an isolated worktree → PR; your checkout is never touched.
+- **Evidence-grounded** — every finding cites a trace message index or code line; no fabricated paths.
+
+Full text lives in the skill's
+[`references/principles.md`](.claude/skills/mutagent-diagnostics/references/principles.md).
+
+---
+
+## Project Layout
+
+```
+mutagent-diagnostics/
+├── README.md                          ← this file
+├── package.json                       ← @mutagent/diagnostics v0.2.0-alpha.2
+├── tsconfig.json
+├── eslint.config.js
+├── docs/index.html                    ← gh-pages standalone artifact
+└── .claude/skills/mutagent-diagnostics/
+    ├── SKILL.md                       ← agentskills.io-compliant manifest (§0-§9)
+    ├── .npmignore                     ← strips internal/ + *.test.ts on publish
+    │
+    ├── scripts/                       ← Type A: pure scripts (bun runtime)
+    │   ├── tier0-scan.ts              ← static scan (free cost gate)
+    │   ├── slicer.ts                  ← dynamic-cluster slicing
+    │   ├── stale-detector.ts          ← hash compare before apply
+    │   ├── cli/
+    │   │   ├── init.ts                ← pnpx entrypoint
+    │   │   ├── install-agents.ts      ← idempotent agent installer
+    │   │   └── run.sh                 ← bun → pnpm → npm fallback selector
+    │   ├── config/                    ← schema.ts · load.ts · validate.ts
+    │   ├── contract/                  ← types.ts — SelfDiagnosisContract schema (structured-report mode)
+    │   ├── fetch/                     ← langfuse.ts · claude-code-transcripts.sh
+    │   ├── normalize/                 ← trace.ts · per-platform normalizers · platforms/entity-context.ts (Wave-5 R1.7)
+    │   ├── enrich/build-render-input.ts ← Step 8.5 deterministic enricher → RenderInput (Wave-5 R1.4)
+    │   ├── report/                    ← render.ts (gold-standard 8-tab HTML) · persist-selections.ts
+    │   ├── lint/template-inline-js.ts ← R-007-B: reject TS in HTML scripts
+    │   ├── setup/                     ← detect.ts · reconfigure.ts
+    │   ├── tier0/                     ← langfuse.ts · claude-code.ts
+    │   ├── validate/
+    │   └── self-diagnostics/          ← [INTERNAL] probe.ts · dispatch.ts
+    │
+    ├── assets/
+    │   ├── agents/
+    │   │   ├── diagnostics-analyzer.md        ← dispatched per cluster (Step 6)
+    │   │   ├── ai-engineer.md                 ← vendored WRITE actor (apply gate, Step 11)
+    │   │   └── ai-architect.md                ← vendored VERIFY actor (applied-remedy rubric)
+    │   ├── templates/
+    │   │   ├── report.html.tpl        ← runtime report template (shipped)
+    │   │   ├── config.yaml.tpl        ← onboarding config skeleton
+    │   │   ├── pr-body.md.tpl         ← apply PR description
+    │   │   ├── audit.json.tpl         ← structured audit trail
+    │   │   └── audit.md.tpl           ← human-readable audit trail
+    │   └── wireframes/                ← picker-UX prompts (onboarding / diagnostics)
+    │
+    ├── references/                    ← load-on-demand docs (not in SKILL.md)
+    │   ├── reference.md               ← entry point + full DAG
+    │   ├── principles.md              ← design principles reference
+    │   ├── operator-feedback-log.md   ← operator feedback on the report shape (Wave-5 R1.6)
+    │   ├── config.md                  ← schema with doc strings
+    │   ├── workflows/                 ← onboarding · orchestrator-protocol (Step 8.5) · diagnostics · apply · rca · schedule-prep
+    │   ├── source-platforms/          ← per-platform fetch + filter examples
+    │   └── target-platforms/          ← per-target apply recipes
+    │
+    ├── examples/
+    │   └── sample-findings.json       ← example RCA output
+    │
+    └── internal/                      ← [stripped on publish — see .npmignore]
+        └── templates/review/          ← dev review templates (maintainer use only)
+```
+
+---
+
+## License
+
+Apache-2.0. See [`LICENSE`](LICENSE) and [`NOTICE`](NOTICE) for full terms.
+`@mutagent/diagnostics` is published to the public npm registry —
+install with `pnpx @mutagent/diagnostics init`.