@massu/core 1.13.1 → 1.15.0

package/commands/massu-autoresearch/references/scoring-protocol.md

@@ -1,151 +0,0 @@
-# Scoring Protocol
-
-## Score Calculation
-
-```
-score = (passing_checks / total_checks) * 100
-```
-
-Each eval check is binary: pass (true) or fail (false). No partial credit. No weighting.
-
----
-
-## Baseline Measurement
-
-Before any edits, establish the baseline:
-
-1. Run the eval subagent 3 times with the same fixture and current command file
-2. Score each run independently
-3. Take the **median** score as the baseline
-4. Log as iteration 0 in `autoresearch-runs.jsonl`
-
-The median (not mean) accounts for subagent output variability without being skewed by a single outlier run.
-
-**Note**: Only the baseline uses 3 runs. Per-iteration evals use a single run as an intentional cost tradeoff — running 3 evals per iteration would triple token usage. The single-run approach works because the simplicity gate and convergence criterion (3 consecutive passes) provide noise resistance.
-
----
-
-## Accept/Reject Threshold
-
-After each iteration edit:
-
-- `score_after > score_before` → candidate for KEEP (subject to simplicity gate)
-- `score_after == score_before` → DISCARD (no improvement = no noise-driven drift) — **exception**: if `net_delta < 0` (deletion), proceed to simplicity gate instead
-- `score_after < score_before` → DISCARD (regression)
-
-**Strict greater-than** is required for non-deletion edits. Equal scores are rejected to prevent random walk drift where the command changes without improving. The one exception is deletion edits (`net_delta < 0`) which maintain score — these proceed to the simplicity gate where the "deletion bonus" rule applies.
-
----
-
-## Simplicity Gate
-
-After scoring, before accepting a KEEP candidate:
-
-1. Count `checks_improved`: how many checks flipped from false to true
-2. Calculate `net_delta`: lines_added - lines_removed in the edit
-3. Apply the simplicity criterion:
-
-| checks_improved | net_delta | Decision | Log action |
-|----------------|-----------|----------|------------|
-| >= 2 | any | KEEP | `"kept"` |
-| == 1 | > 0 | **REJECT** | `"rejected_complexity"` |
-| == 1 | <= 0 | KEEP | `"kept"` |
-| == 0 | < 0 | KEEP (if score maintained) | `"kept"` (simplification) |
-| == 0 | >= 0 | DISCARD | `"discarded"` |
-
-**Deletion bonus**: If `net_delta < 0` (change removes lines) AND `score_after >= score_before`, ALWAYS KEEP. Karpathy: "Improvement from deleting code = definitely keep."
-
----
-
-## Convergence
-
-The loop converges when score >= target (default 90%) for **3 consecutive iterations** where each iteration includes running an eval and scoring. The 3-consecutive requirement applies to iterations that produce a score (KEEP or DISCARD), not CRASH iterations. Three consecutive readings at target eliminates the possibility of a lucky single run.
-
----
-
-## Score Logging Format
-
-Append one JSONL line per iteration to `.claude/metrics/autoresearch-runs.jsonl`:
-
-```json
-{
-  "command": "[target command name]",
-  "iteration": 1,
-  "timestamp": "2026-03-20T10:30:00Z",
-  "score_before": 62.5,
-  "score_after": 75.0,
-  "action": "kept",
-  "edit_type": "add_worked_example",
-  "edit_summary": "Added before/after example for gap analysis section",
-  "checks": {
-    "has_executive_summary": true,
-    "has_score_table": true,
-    "has_massu_comparison": true,
-    "has_gap_analysis": false,
-    "has_action_items": true,
-    "has_source_credibility": true,
-    "mentions_specific_commands": true,
-    "has_implementation_path": false
-  },
-  "lines_added": 8,
-  "lines_removed": 2,
-  "net_delta": 6,
-  "cumulative_kept": 1,
-  "cumulative_discarded": 0,
-  "cumulative_crashed": 0,
-  "cumulative_rejected": 0,
-  "escalation_level": 0,
-  "consecutive_reverts": 0
-}
-```
-
-### Field Definitions
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `command` | string | Target command name (e.g., "massu-article-review") |
-| `iteration` | number | 0 = baseline, 1+ = edit iterations |
-| `timestamp` | string | ISO 8601 timestamp |
-| `score_before` | number\|null | Score percentage before this iteration's edit. `null` for baseline (iteration 0) since there is no prior score. |
-| `score_after` | number | Score percentage after this iteration's edit |
-| `action` | string | One of: `"baseline"`, `"kept"`, `"discarded"`, `"crashed"`, `"rejected_complexity"` |
-| `edit_type` | string\|null | `null` for baseline/dry-run. Otherwise one of: `"add_rule"`, `"add_example"`, `"add_worked_example"`, `"restructure"`, `"promote"`, `"ban_pattern"`, `"merge_sections"`, `"split_section"`, `"reorder"`, `"simplify"` |
-| `edit_summary` | string | One-line human-readable description of the change |
-| `checks` | object | Map of check_id to boolean pass/fail |
-| `lines_added` | number | Lines added by this edit |
-| `lines_removed` | number | Lines removed by this edit |
-| `net_delta` | number | `lines_added - lines_removed` |
-| `cumulative_kept` | number | Running total of kept changes this run |
-| `cumulative_discarded` | number | Running total of discarded changes this run |
-| `cumulative_crashed` | number | Running total of crashed evals this run |
-| `cumulative_rejected` | number | Running total of complexity-rejected edits this run |
-| `escalation_level` | number | Current escalation level (0-3) |
-| `consecutive_reverts` | number | Current consecutive revert count (resets on KEEP) |
-
----
-
-## Final Summary Format
-
-After the loop exits, produce a summary:
-
-```
-  Score Trajectory:
-    [iteration 0]  62.5%  (baseline)
-    [iteration 1]  75.0%  KEPT    "Added worked example for gap analysis"
-    [iteration 2]  75.0%  DISCARD "Tried adding source credibility template"
-    [iteration 3]  87.5%  KEPT    "Promoted source credibility to mandatory section"
-    ...
-
-  Per-Check Progression:
-    Check                       Baseline    Final
-    has_executive_summary       PASS        PASS
-    has_score_table             PASS        PASS
-    has_massu_comparison        PASS        PASS
-    has_gap_analysis            FAIL        PASS  <-- improved
-    has_action_items            PASS        PASS
-    has_source_credibility      FAIL        PASS  <-- improved
-    mentions_specific_commands  PASS        PASS
-    has_implementation_path     FAIL        FAIL  (hardest check)
-```
-
-This summary shows the optimization journey and identifies which checks responded to edits and which are structurally difficult.

package/commands/massu-autoresearch.md

@@ -1,258 +0,0 @@
----
-name: massu-autoresearch
-description: "When user wants autonomous command optimization, says 'autoresearch', 'optimize this command', 'run overnight improvement loop', or wants Karpathy-style iterative prompt optimization"
-allowed-tools: Bash(*), Read(*), Write(*), Edit(*), Grep(*), Glob(*), Agent(*), Task(*)
----
-name: massu-autoresearch
-
-**Shared rules**: Read `.claude/commands/_shared-preamble.md` for POST-COMPACTION (CR-12), ENTERPRISE-GRADE (CR-14), AWS SECRETS (CR-5) rules.
-
-# Massu Autoresearch: Autonomous Command Optimization Loop
-
-## Objective
-
-Iteratively improve a target command file by scoring output against an eval checklist, keeping improvements (git commit), reverting failures (git checkout), and repeating until convergence. Based on Karpathy's autoresearch pattern (42K GitHub stars).
-
----
-
-## ARGUMENTS
-
-```
-/massu-autoresearch [command-name]                    # Default: 20 iterations, 90% target
-/massu-autoresearch [command-name] --max-iterations N # Custom iteration cap
-/massu-autoresearch [command-name] --target-score N   # Custom target (default 90)
-/massu-autoresearch [command-name] --no-limit         # NEVER STOP mode (cost cap only)
-```
-
-**Arguments from $ARGUMENTS**: {{ARGUMENTS}}
-
----
-
-## THREE-FILE ARCHITECTURE
-
-| Role | File | Mutable? |
-|------|------|----------|
-| **Target** (train.py) | `.claude/commands/[command].md` (or `[command]/[command].md` for folder-based) | YES — agent edits this |
-| **Eval** (prepare.py) | `.claude/evals/[command].md` | **NO — IMMUTABLE** |
-| **Instructions** (program.md) | This file | **NO — agent follows this** |
-
-The target command is the ONLY file the agent may edit. The eval checklist and these instructions are READ-ONLY.
-
----
-
-## NON-NEGOTIABLE RULES
-
-1. **ONE change per iteration** — single-variable perturbation only. Never batch multiple edits.
-2. **NEVER edit the eval file** — `.claude/evals/[command].md` is immutable during runs.
-3. **NEVER edit lines containing CR-*/VR-* references** — these encode incident-learned safety rules. See `references/safety-rails.md` for protected line patterns.
-4. **ALWAYS backup before first iteration** — save original to `.claude/metrics/backups/[command]-autoresearch-[timestamp].md.bak`
-5. **git commit on improvement / git checkout on regression** — tristate handling (see below).
-6. **Tristate results**: KEEP / DISCARD / CRASH — crashes do NOT count toward stagnation.
-7. **NEVER STOP mode** (when `--no-limit` passed): "Once the experiment loop has begun, do NOT pause to ask the human if you should continue." Run until convergence, cost cap (500K tokens), or manual interrupt (Ctrl+C). Default mode uses `--max-iterations` cap.
-
----
-
-## TRISTATE RESULTS
-
-Every iteration produces one of three outcomes:
-
-| Outcome | Condition | Action |
-|---------|-----------|--------|
-| **KEEP** | `score_after > score_before` AND passes simplicity gate | `git add [target] && git commit -m "autoresearch([cmd]): [summary]"` — new baseline |
-| **DISCARD** | `score_after <= score_before` OR fails simplicity gate | `git checkout -- [target]` — revert to baseline |
-| **CRASH** | Eval subagent errored or produced unparseable output | Log the crash. Do NOT count toward stagnation. Retry once with simplified edit. If still failing, count as DISCARD. |
-
-Crash handling details: see `references/safety-rails.md`.
-
----
-
-## SIMPLICITY CRITERION
-
-Apply the simplicity gate per `references/scoring-protocol.md`. Key principle (Karpathy): "Marginal improvement + added complexity = reject. Improvement from deleting code = definitely keep."
-
----
-
-## "THINK HARDER" GRADUATED ESCALATION
-
-Replaces CR-37 immediate bail. When stuck, escalate before giving up. Consecutive revert counter resets to 0 after any KEEP.
-
-| Level | Trigger |
-|-------|---------|
-| **Level 1** | 3 consecutive reverts |
-| **Level 2** | 5 consecutive reverts |
-| **Level 3** | 8 consecutive reverts |
-| **BAIL** | 10 consecutive reverts |
-
-Per-level strategies: see `references/safety-rails.md`.
-
----
-
-## CONVERGENCE CRITERIA
-
-The loop stops when ANY of these conditions is met:
-
-| Condition | Default | Overridable? |
-|-----------|---------|-------------|
-| Score >= target for 3 consecutive iterations | 90% | `--target-score N` |
-| Max iterations reached | 20 | `--max-iterations N` or `--no-limit` |
-| Stagnation bail (10 consecutive reverts after Level 3) | Always active | Not overridable |
-| Cost cap (estimated cumulative token usage) | 500K tokens | Not overridable |
-
----
-
-## LOOP CONTROLLER
-
-### Initialization (runs once)
-
-1. **Parse arguments**: Extract command name, max iterations, target score, no-limit flag
-2. **Validate files exist**:
-   - Target: `.claude/commands/[command].md` (or `.claude/commands/[command]/[command].md` for folder-based)
-   - Eval: `.claude/evals/[command].md`
-   - Fixture: `.claude/evals/fixtures/[command]/input-01.md`
-3. **Resolve target path**: Determine actual file — `.claude/commands/[command].md` (flat) or `.claude/commands/[command]/[command].md` (folder-based). Store as `TARGET_PATH` for all git operations.
-4. **Create git branch**: `git checkout -b autoresearch/[command]-[YYYY-MM-DD]` (if exists, append `-2`, `-3`, etc.)
-5. **Create backup**: Copy target to `.claude/metrics/backups/[command]-autoresearch-[timestamp].md.bak`
-6. **Measure baseline**: Run eval 3 times, take median score. Log as `"iteration":0` in `autoresearch-runs.jsonl`
-7. **Initialize state**: `consecutive_reverts = 0`, `escalation_level = 0`, `cumulative_kept = 0`, `cumulative_discarded = 0`, `cumulative_crashed = 0`, `cumulative_rejected = 0`
-
-### Per-Iteration Loop
-
-For each iteration `i` from 1 to max_iterations (or unlimited if `--no-limit`):
-
-**Step 1: Read current state**
-- Read target command file (current version after any prior keeps)
-- Read eval checklist (immutable)
-- Read fixture input
-- Review failing checks from previous iteration (or baseline)
-
-**Step 2: Hypothesize improvement**
-Based on failing checks and escalation level, choose ONE edit:
-- What check are we targeting?
-- What edit type? (add rule, add example, restructure, promote, ban pattern)
-- What specific change? (exact text to add/modify)
-
-**Step 3: Apply edit**
-- Use Edit tool to make ONE change to the target file
-- Count lines added and removed
-
-**Step 4: Run eval**
-Read `references/eval-runner.md` for the eval subagent protocol.
-
-Spawn an eval subagent with:
-- The CURRENT target command text (after edit)
-- The eval checklist
-- The fixture input
-
-Extract: score percentage, per-check results, failing check names.
-
-**Step 5: Score and decide**
-- Compare `score_after` to `score_before` (baseline or last kept score)
-- Apply simplicity criterion
-- Determine outcome: KEEP, DISCARD, CRASH, or REJECTED_COMPLEXITY
-
-**Step 6: Act on decision**
-
-| Decision | Git Action | State Update |
-|----------|-----------|--------------|
-| KEEP | `git add [target] && git commit` | `score_before = score_after`, `consecutive_reverts = 0`, `cumulative_kept += 1` |
-| DISCARD | `git checkout -- [target]` | `consecutive_reverts += 1`, `cumulative_discarded += 1` |
-| CRASH | Log crash, retry once simplified | `cumulative_crashed += 1` (consecutive_reverts unchanged) |
-| REJECTED_COMPLEXITY | `git checkout -- [target]` | `consecutive_reverts += 1`, `cumulative_rejected += 1` |
-
-**Step 7: Log iteration**
-Append to `.claude/metrics/autoresearch-runs.jsonl`. See `references/scoring-protocol.md` for format.
-
-**Step 8: Check convergence**
-- If score >= target for 3 consecutive scored iterations (KEEP or DISCARD, not CRASH): CONVERGED — exit loop. If baseline already >= target, run 3 verification iterations to confirm before converging.
-- If consecutive_reverts triggers escalation level change: log escalation
-- If consecutive_reverts >= 10 (after Level 3): STAGNATION — exit loop
-- If max iterations reached: MAX_ITERATIONS — exit loop
-
-**Step 9: Update session state**
-Update `session-state/CURRENT.md` with current iteration count, score, and status.
-
-**Step 10: Continue**
-Return to Step 1.
-
----
-
-## FINAL REPORT
-
-After the loop exits (any reason), produce:
-
-```
-AUTORESEARCH COMPLETE — [command]
-
-  Reason:           [CONVERGED / MAX_ITERATIONS / STAGNATION / COST_CAP]
-  Iterations:       [N]
-  Branch:           autoresearch/[command]-[date]
-
-  Score Progression:
-    Baseline:       [X]% ([N/M] checks)
-    Final:          [Y]% ([N/M] checks)
-    Delta:          [+/-Z]%
-
-  Results:
-    Kept:           [N] commits
-    Discarded:      [N] reverts
-    Crashed:        [N] transient failures
-    Rejected:       [N] complexity rejections
-
-  Net Lines:        [+/-N] across all kept changes
-  Highest Escalation: Level [0-3]
-
-  Kept Changes:
-    1. [iteration N] [edit summary] (+X checks)
-    2. [iteration N] [edit summary] (+X checks)
-    ...
-
-  Still Failing:
-    - [check_name]: [brief description of why it's hard]
-    ...
-
-  Next Steps:
-    - Review changes: git log autoresearch/[command]-[date]
-    - Merge if satisfied: git merge autoresearch/[command]-[date]
-    - Or cherry-pick specific commits
-```
-
----
-
-## SESSION STATE
-
-Update `session-state/CURRENT.md` after each iteration:
-
-```
-AUTHORIZED_COMMAND: massu-autoresearch
-Target: [command]
-Branch: autoresearch/[command]-[date]
-Iteration: [N]/[max]
-Current Score: [X]%
-Consecutive Reverts: [N]
-Escalation Level: [0-3]
-```
-
----
-
-## Skill Contents
-
-This skill is a folder. The following files are available for reference:
-
-| File | Purpose | Read When |
-|------|---------|-----------|
-| `references/eval-runner.md` | Eval subagent spawn protocol | Before running any eval |
-| `references/safety-rails.md` | Protected lines, backup, branch, cost cap, crash handling, escalation | Before any edit or on error |
-| `references/scoring-protocol.md` | Score calculation, accept/reject, JSONL format, final summary | Before scoring or logging |
-
----
-
-## START NOW
-
-1. Parse `{{ARGUMENTS}}` for command name, flags
-2. Validate target + eval + fixture exist
-3. Create branch `autoresearch/[command]-[date]`
-4. Backup target file
-5. Measure baseline (3 eval runs, median)
-6. Enter iteration loop
-7. On exit: produce final report
-8. **DO NOT ask "should I continue?" between iterations — just continue.**

package/commands/massu-command-health.md

@@ -1,132 +0,0 @@
----
-name: massu-command-health
-description: "When user asks about command quality, says 'how are my commands', 'command health', or wants to see which slash commands have quality issues"
-allowed-tools: Bash(*), Read(*)
----
-name: massu-command-health
-
-# Massu Command Health: Quality Score Dashboard
-
-## Purpose
-
-Read `.claude/metrics/command-scores.jsonl` and display a summary of command quality over time. This is a READ-ONLY command — it does not modify anything.
-
----
-
-## Data Sources
-
-### Quality Scores
-Each line in `.claude/metrics/command-scores.jsonl` is a JSON object:
-
-```json
-{"command":"massu-create-plan","timestamp":"2026-03-18T14:30:00","scores":{"items_have_acceptance_criteria":true,"references_real_tables":true,"ui_items_have_paths":false,"has_vr_types":true,"explicit_counts":true},"pass_rate":"4/5","input_summary":"knowledge-graph-phase-5"}
-```
-
-### Invocation Frequency
-Each line in `.claude/metrics/command-invocations.jsonl` tracks when a command was used:
-
-### Autoresearch Runs
-Each line in `.claude/metrics/autoresearch-runs.jsonl` tracks autonomous optimization iterations:
-```json
-{"command":"massu-article-review","iteration":5,"timestamp":"ISO8601","score_before":75,"score_after":87.5,"action":"kept","edit_summary":"Added worked example for gap analysis"}
-```
-
-```json
-{"skill":"massu-create-plan","timestamp":"2026-03-18T14:30:00Z"}
-```
-
----
-
-## Output Format
-
-```
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-COMMAND HEALTH — [date]
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-
-OVERVIEW
-  Total scored runs: [N]
-  Commands tracked:  [N]
-  Date range:        [earliest] — [latest]
-
-SCORECARD
-  Command                   Last 5 avg    Trend    Weakest check                    Runs
-  ─────────────────────────────────────────────────────────────────────────────────────
-  massu-create-plan          80%           =        ui_items_have_paths (40%)         12
-  massu-loop                 90%           ^        memory_persisted (60%)             8
-  massu-article-review      100%           =        —                                  5
-  massu-plan                 75%           v        zero_gaps_at_exit (50%)            6
-  massu-debug                85%           =        regression_test_added (60%)        4
-
-  Trend: ^ improving (last 3 > prior 3)  v declining  = stable
-
-USAGE (last 7 days from command-invocations.jsonl)
-  Command                   Invocations/week
-  ─────────────────────────────────────────
-  massu-create-plan          12
-  massu-debug                 8
-  massu-loop                  6
-  massu-bearings              5
-  massu-commit                4
-  (or "No invocation data yet")
-
-ALERTS (commands below 60% on last 3 runs)
-  ! massu-plan — 50% on last 3 runs. Weakest: zero_gaps_at_exit
-  (or "No alerts — all commands above threshold")
-
-CHECK DETAIL (per-command breakdown, last 10 runs)
-  massu-create-plan:
-    items_have_acceptance_criteria   9/10 (90%)
-    references_real_tables           8/10 (80%)
-    ui_items_have_paths              4/10 (40%)  <-- weakest
-    has_vr_types                     7/10 (70%)
-    explicit_counts                  9/10 (90%)
-
-  [repeat for each command with data]
-
-AUTORESEARCH (from autoresearch-runs.jsonl)
-  Command               Last run         Iterations   Score: start -> end
-  ─────────────────────────────────────────────────────────────────────────
-  massu-article-review   2026-03-19       12           62% -> 87%
-  (or "No autoresearch runs yet")
-
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-```
-
----
-
-## Logic
-
-1. Read `.claude/metrics/command-scores.jsonl` AND `.claude/metrics/command-invocations.jsonl`
-2. If both empty: display "No command data recorded yet. Scores and invocations accumulate automatically as you use commands."
-3. Parse each line as JSON
-4. Group scores by `command`, group invocations by `skill`
-5. For each command:
-   - Count total runs
-   - Calculate average pass rate for last 5 runs
-   - Calculate trend (compare last 3 avg vs prior 3 avg)
-   - Find weakest check (lowest pass rate across all runs)
-   - Per-check breakdown for last 10 runs
-6. Sort by average pass rate (lowest first — worst commands at top)
-7. Flag any command below 60% on last 3 runs as an ALERT
-8. Read `.claude/metrics/autoresearch-runs.jsonl` — group by command, find last run's iteration range, extract start/end scores, display in AUTORESEARCH section
-
----
-
-## Arguments
-
-Optional: command name to show detail for just one command.
-
-```
-/massu-command-health                    # Full dashboard
-/massu-command-health massu-create-plan   # Detail for one command
-```
-
----
-
-## START NOW
-
-1. Read `.claude/metrics/command-scores.jsonl`
-2. Parse and aggregate scores
-3. Display dashboard in the format above
-4. Do NOT modify any files