arboris-cli 1.0.0 → 1.1.0

package/prisma/skills/agent-self-evaluation/examples/low-score-example.md

@@ -0,0 +1,86 @@
+# Low-Score Example: Adding Retry Logic
+
+## Task
+> Add retry logic with exponential backoff to all HTTP requests in the API client. Max 3 retries, initial delay 1s, max delay 30s.
+
+## Agent Output
+
+Here's a retry implementation. I think this should work for most cases.
+
+I used the urllib3 library since it has built-in retry support. The Retry class handles backoff automatically.
+
+```python
+from urllib3.util import Retry
+
+# Add this somewhere in your HTTP client
+retry = Retry(
+    total=3,
+    backoff_factor=1.0,
+    status_forcelist=[500, 502, 503, 504]
+)
+```
+
+You'll want to test this before deploying. There might be edge cases with POST requests but those are probably fine. I haven't tested the timeout behavior but it should work.
+
+Let me know if you need anything else!
+
+---
+
+## Evaluation Report
+
+```
+============================================================
+AGENT SELF-EVALUATION REPORT
+============================================================
+
+  Accuracy         ██░░░ 2/5
+    - Hedged claim without verification ("I think this should work")
+    - Untested ("I haven't tested the timeout behavior")
+    - Speculation without evidence ("those are probably fine")
+    → Wrong library used. Project uses httpx, not urllib3.
+                      urllib3.util.Retry is incompatible with httpx.
+
+  Completeness      ███░░ 3/5
+    - Explicit gap acknowledged ("might be edge cases with POST")
+    - Limited scope noted (only mentioned 5xx, not 429 or connection errors)
+    → User asked for "all HTTP requests." Only partial coverage:
+      missing 429 handling, connection errors, timeout handling.
+
+  Clarity           ████░ 4/5
+    + Uses code blocks
+    → Code is readable but no explanation of where to add it
+      ("somewhere in your HTTP client" is vague).
+
+  Actionability     ██░░░ 2/5
+    - Defers work to user ("you'll want to test this")
+    - Vague suggestion without specifics
+    → No PR, no file created, no test written. User has to:
+      1. Figure out where to add the code
+      2. Fix the library mismatch (httpx not urllib3)
+      3. Write tests
+      4. Handle POST idempotency
+      5. Test timeout behavior
+
+  Conciseness       ███░░ 3/5
+    - Meta-commentary adds words without information
+      ("Let me know if you need anything else!")
+    → 120 words. Low word count but low information density.
+      Half the text is hedging and disclaimers, not substance.
+
+  OVERALL           2.8/5
+
+TOP IMPROVEMENTS (axes scoring < 4):
+  [Accuracy] Switch to httpx — grep the codebase to confirm the HTTP
+    library before writing code.
+  [Actionability] Create a PR with the changed file + test file. Run the
+    tests. End with "PR #N ready to merge."
+  [Completeness] List what's covered AND what's not. If POST retry is
+    unsafe, say so explicitly with reasoning.
+```
+
+### Why This Scores Poorly
+
+1. **Accuracy fails at the most basic level** — wrong library. One `grep httpx src/` would have caught this. The hedging language ("I think", "probably", "should work") signals the agent knows it's guessing.
+2. **Not actionable.** The user received a code snippet and a list of things they need to do. The agent did the easy part (suggesting a library) and deferred the hard parts (testing, integration, edge cases) to the user.
+3. **Completeness gaps are acknowledged but not fixed.** "Might be edge cases" is worse than not mentioning them — it shows awareness of the gap and a choice not to address it.
+4. **Information density is low.** 120 words, of which ~60 are hedging/disclaimers/politeness. The actual substance (3 lines of code) could have been delivered in 40 words with verification.

package/prisma/skills/agent-self-evaluation/references/evaluation-criteria.md

@@ -0,0 +1,71 @@
+# Evaluation Criteria — Detailed Scoring Guide
+
+This reference provides concrete scoring anchors for each axis. Use it when you're unsure whether a gap merits a 4 vs a 3, or a 2 vs a 1.
+
+## Accuracy
+
+| Score | Anchor | Example |
+|---|---|---|
+| 5 | All facts verified against tool output, docs, or authoritative sources. No errors. | Configured retry via httpx transport — confirmed in httpx docs. All method names verified with grep against codebase. |
+| 4 | One minor inaccuracy that doesn't affect correctness. | Correct library, wrong default value for one parameter (claimed 0.5s, docs say 1.0s). |
+| 3 | One significant factual error, or 3+ minor inaccuracies. | Used `urllib3.Retry` in an httpx codebase. Works in this one case but wrong library. |
+| 2 | Multiple significant errors. Output would fail if followed. | Claimed "add this to package.json" but project uses pyproject.toml. Two other config claims also wrong. |
+| 1 | Fundamentally incorrect. Output contradicts itself or known facts. | Code has syntax errors. API endpoint doesn't exist. Claims a function signature that grep disproves. |
+
+## Completeness
+
+| Score | Anchor | Example |
+|---|---|---|
+| 5 | All explicit and implicit requirements covered. Edge cases handled. Error paths addressed. | User said "add retry to all HTTP requests." GET, POST, PUT, DELETE all covered. Timeout, 429, 5xx all handled. |
+| 4 | All explicit requirements covered. One implicit requirement missed. | All HTTP methods covered. Forgot to handle connection timeouts (not mentioned but expected). |
+| 3 | One explicit requirement missed, or 2+ implicit gaps. | User said "add logging too." Retry logic added but no logging. |
+| 2 | Multiple explicit requirements missed. Output is a partial solution. | Asked for retry + circuit breaker. Only retry implemented. |
+| 1 | Misses the core request. Delivers something adjacent to what was asked. | Asked for retry logic. Wrote a health check endpoint instead. |
+
+## Clarity
+
+| Score | Anchor | Example |
+|---|---|---|
+| 5 | Perfectly structured. Jargon explained or avoided. Visual hierarchy helps scanning. No ambiguity. | README with clear sections, code blocks, and a 10-second summary at top. |
+| 4 | Generally clear. One section could be better organized or one term undefined. | Good structure but `exponential backoff` used without explanation — assumes the reader knows it. |
+| 3 | Understandable after re-reading. Multiple organizational issues or undefined terms. | The explanation circles the point before getting to it. Several terms used before defined. |
+| 2 | Confusing in places. Reader would need to ask follow-up questions. | Code works but the PR description doesn't explain why retry was needed or what it fixes. |
+| 1 | Unintelligible or contradictory. Reader cannot determine what was done or why. | Output is a wall of text with no structure. Conclusions contradict earlier statements. |
+
+## Actionability
+
+| Score | Anchor | Example |
+|---|---|---|
+| 5 | Single action required. Verification path included. No implicit steps. | "Merge this PR. Tests pass: `42 passed`. Deploy with `./deploy.sh`." |
+| 4 | Single action required but verification path is implied, not explicit. | "Merge this PR." (Tests exist but weren't cited. User has to check themselves.) |
+| 3 | Multiple actions required, or one action with unclear next step. | "Review and merge. Then update the config." (Which config? Where? No link or path.) |
+| 2 | User must figure out how to use the output. Missing critical instructions. | Code written but no test file, no run instructions, no PR created. User has to assemble everything. |
+| 1 | Output cannot be acted on without significant rework or clarification. | "Here's a design idea." (No code, no file, no PR. User has to start from scratch.) |
+
+## Conciseness
+
+| Score | Anchor | Example |
+|---|---|---|
+| 5 | Every sentence earns its place. No redundancy. Information density is high. | 30 lines that say what 60 lines would. No repeated points. No filler. |
+| 4 | Minor redundancy. One paragraph could be tightened. | Good overall but repeats the motivation in both the PR description and code comments. |
+| 3 | Noticeable redundancy. 20%+ of content could be removed without loss. | Explains the same concept three times (in summary, body, and conclusion). Verbose examples. |
+| 2 | Significantly bloated. 40%+ of content is filler or repetition. | 200 lines for a task that needed 60. Restates the user's question. Includes irrelevant background. |
+| 1 | Noise-to-signal ratio is inverted. More filler than substance. | 500-line response to a 2-line question. Most of it is boilerplate, repetition, or irrelevant context. |
+
+## Edge Cases
+
+### When the user gave unclear instructions
+
+If the user's request was ambiguous, do NOT penalize completeness for not reading minds. Instead, note in the evaluation: "User's request was ambiguous about [scope]. I chose interpretation [chosen interpretation]. If they meant [alternative interpretation], this score would drop to [score]."
+
+### When the task is inherently simple
+
+A 3-line bug fix can legitimately score 5/5/5/5/5. The rubric scales with complexity — a simple task done perfectly IS a 5.0. Don't invent gaps to justify lower scores.
+
+### When you caught your own error mid-task
+
+If you made an error, caught it, and fixed it before delivering — that's a 5 on Accuracy for the final output. The evaluation is about what the user received, not your internal process. Note the self-correction as evidence of thoroughness, not as a penalty.
+
+### When the tool output contradicts your claim
+
+If you claimed "tests pass" but the terminal output shows a failure — that's an automatic Accuracy ≤ 2. Tool output is ground truth. Claims without verification are the most common source of low accuracy scores.

package/prisma/skills/agent-self-evaluation/references/hook-integration.md

@@ -0,0 +1,64 @@
+# Hook Integration for Session-Stop Self-Evaluation
+
+Add this hook to `hooks/hooks.json` to remind the agent to self-evaluate at the end of every session (the hook echoes a reminder; it does not run the evaluator automatically):
+
+```json
+{
+  "hooks": {
+    "Stop": [
+      {
+        "hooks": [
+          {
+            "type": "command",
+            "command": "echo '[Self-Eval] Session complete. Consider running agent-self-evaluation to rate your output.'"
+          }
+        ],
+        "description": "Remind agent to self-evaluate at session end"
+      }
+    ]
+  }
+}
+```
+
+`Stop` events do not require a `matcher` field (it is optional for `Stop`, `Notification`, `UserPromptSubmit`, and `SubagentStop` per `scripts/ci/validate-hooks.js`). If omitted, the hook object only needs `hooks` and metadata such as `description`.
+
+## Integration with the Python Evaluator
+
+The `scripts/evaluate.py` script can be used as a standalone tool:
+
+```bash
+# Pipe agent output directly
+echo "Your agent response here" | python3 skills/agent-self-evaluation/scripts/evaluate.py
+
+# From files
+python3 skills/agent-self-evaluation/scripts/evaluate.py --task task.txt --output response.txt
+```
+
+To integrate it into hooks, capture the last agent output to a file first, then run the evaluator. For lightweight reminders after shell-based verification, use a simple supported matcher string:
+
+```json
+{
+  "hooks": {
+    "PostToolUse": [
+      {
+        "matcher": "Bash",
+        "hooks": [
+          {
+            "type": "command",
+            "command": "echo '[Self-Eval] If this command completed verification for a non-trivial task, consider running agent-self-evaluation.'"
+          }
+        ],
+        "description": "Remind agent to self-evaluate after shell verification"
+      }
+    ]
+  }
+}
+```
+
+This avoids documenting unsupported command-expression matcher syntax. If your harness supports command-level matcher expressions, prefer a word-boundary regex such as `\b(pytest|npm test|go test)\b` rather than a broad `test` substring.
+
+These hooks are opt-in. Add them to your local `hooks/hooks.json` if you want automated evaluation prompts.
+
+## Manual Usage (Recommended)
+
+The most reliable approach is manual invocation — the agent runs self-evaluation as part of its workflow when the `agent-self-evaluation` skill is active, without requiring hook configuration. The skill's "When to Activate" section already covers trigger conditions (multi-file changes, debugging sessions, design documents).

package/prisma/skills/agent-self-evaluation/scripts/evaluate.py

@@ -0,0 +1,408 @@
+#!/usr/bin/env python3
+"""Standalone agent output evaluator using the 5-axis rubric.
+
+Reads a task description and agent output from stdin or files,
+scores each axis, and prints a structured evaluation report.
+
+Usage:
+    # Pipe output directly
+    echo "Task: Add retry logic" | evaluate.py --output response.txt
+
+    # From files
+    evaluate.py --task task.txt --output response.txt
+
+    # Interactive (reads task from prompt, output from stdin)
+    evaluate.py --interactive
+
+The evaluator uses keyword heuristics + structural checks as a first pass.
+For production use, pair with an LLM judge for semantic understanding.
+"""
+
+import argparse
+import re
+import sys
+from dataclasses import dataclass, field
+from typing import Optional
+
+# Tunable thresholds for evaluation heuristics
+WALL_OF_TEXT_WORDS = 200
+SUMMARY_CHECK_WORDS = 300
+SUMMARY_CHECK_FIRST_N = 100
+TASK_OUTPUT_RATIO_HIGH = 15
+TASK_OUTPUT_RATIO_MEDIUM = 8
+
+
+@dataclass
+class AxisScore:
+    name: str
+    score: int
+    evidence: list[str] = field(default_factory=list)
+    improvement: Optional[str] = None
+
+
+def count_words(text: str) -> int:
+    return len(text.split())
+
+
+def check_accuracy(text: str) -> AxisScore:
+    """Check for verifiable claims, tool output references, error signs."""
+    evidence = []
+    deductions = 0
+    score = 5
+
+    # Positive signals: verified claims
+    verified_patterns = [
+        (r"(?i)(tests?\s+pass|all\s+tests?\s+passing|\d+\s+passed)", "Tests passing"),
+        (r"(?i)(exit\s+code\s*[:=]?\s*0|exited\s+with\s+0)", "Clean exit code"),
+        (r"(?i)(lint.*clean|no\s+lint\s+errors|0\s+errors)", "Lint clean"),
+        (r"(?i)(verified|confirmed|validated)\s+(with|against|using|by)", "Explicit verification"),
+        (r"(?i)(grep|rg)\s+.*\b(found|matched|returned)", "Grep confirmed"),
+    ]
+    for pattern, label in verified_patterns:
+        if re.search(pattern, text):
+            evidence.append(f"+ {label}")
+
+    # Negative signals: unverified claims
+    danger_patterns = [
+        (r"(?i)(should\s+work|probably\s+fine|should\s+be\s+ok)", "Hedged claim without verification"),
+        (r"(?i)(I\s+think|I\s+believe|I\s+assume|might\s+be)", "Speculation without evidence"),
+        (r"(?i)(untested|not\s+tested|haven'?t\s+tested)", "Explicitly untested"),
+        (r"(?i)(TODO|FIXME|HACK|WORKAROUND)", "Unresolved TODO/FIXME"),
+    ]
+    for pattern, label in danger_patterns:
+        if re.search(pattern, text):
+            deductions += 1
+            evidence.append(f"- {label}")
+
+    if deductions >= 3:
+        score = 2
+    elif deductions == 2:
+        score = 3
+    elif deductions == 1:
+        score = 4
+
+    if not evidence:
+        evidence.append("No verification signals detected — score assumes correctness")
+
+    result = AxisScore(name="Accuracy", score=score, evidence=evidence)
+    if score < 5:
+        result.improvement = "Cite specific tool outputs (test results, exit codes, grep findings) to back claims"
+    return result
+
+
+def check_completeness(text: str) -> AxisScore:
+    """Check for requirement coverage, edge cases, error handling."""
+    evidence = []
+    score = 5
+
+    # Positive signals
+    completeness_signals = [
+        (r"(?i)(edge\s*cases?|corner\s*cases?)", "Edge cases addressed"),
+        (r"(?i)(error\s*handling|exception\s*handling|try/except|try\s*{)", "Error handling present"),
+        (r"(?i)(all\s+\w+\s+(methods|endpoints|routes))", "Full coverage claimed"),
+        (r"(?i)(verification|verified\s+that|confirmed\s+that)", "Verification step present"),
+    ]
+    for pattern, label in completeness_signals:
+        if re.search(pattern, text):
+            evidence.append(f"+ {label}")
+
+    # Gaps
+    gap_signals = [
+        (r"(?i)(not\s+covered|not\s+handled|out\s+of\s+scope)", "Explicit gap acknowledged"),
+        (r"(?i)(only\s+(works|handles|supports)\s+\w+)", "Limited scope noted"),
+        (r"(?i)(assume[sd]?\s+that|assuming\s+the)", "Assumption without verification"),
+    ]
+    deductions = 0
+    for pattern, label in gap_signals:
+        if re.search(pattern, text):
+            deductions += 1
+            evidence.append(f"- {label}")
+
+    if deductions >= 2:
+        score = 3
+    elif deductions == 1:
+        score = 4
+
+    if not evidence:
+        evidence.append("No completeness signals — unable to assess coverage")
+
+    result = AxisScore(name="Completeness", score=score, evidence=evidence)
+    if score < 5:
+        result.improvement = "List what was covered AND what was intentionally excluded, with reasoning"
+    return result
+
+
+def _check_jargon(text: str) -> tuple[int, list[str]]:
+    """Return clarity deductions for unexplained domain jargon."""
+    jargon = [
+        (r"\b(idempotent|race condition|deadlock|thundering herd)\b", "concurrency"),
+        (r"\b(exponential backoff|circuit breaker|bulkhead)\b", "resilience"),
+        (r"\b(ACID|CAP|eventual consistency|linearizability)\b", "database theory"),
+    ]
+    explanation_pattern = r"(?i)({domain}|means|refers to|i\.e\.|in other words)"
+    for pattern, domain in jargon:
+        has_term = re.search(pattern, text, re.IGNORECASE)
+        explains_term = re.search(explanation_pattern.format(domain=domain), text)
+        if has_term and not explains_term:
+            return 1, [f"- Domain term used without explanation ({domain})"]
+    return 0, []
+
+
+def _check_summary(text: str) -> tuple[int, list[str]]:
+    """Return clarity deduction when long output lacks an early summary."""
+    summary_terms = ["summary", "tldr", "overview", "in short"]
+    has_early_summary = any(term in ' '.join(text.split()[:SUMMARY_CHECK_FIRST_N]).lower() for term in summary_terms)
+    if not has_early_summary and count_words(text) > SUMMARY_CHECK_WORDS:
+        return 1, ["- No summary/TLDR in first 100 words (text is 300+ words)"]
+    return 0, []
+
+
+def check_clarity(text: str) -> AxisScore:
+    """Check for structure, readability, jargon handling."""
+    evidence = []
+    deductions = 0
+
+    if re.search(r"^#{1,3}\s+", text, re.MULTILINE):
+        evidence.append("+ Uses headings for structure")
+    if re.search(r"```", text):
+        evidence.append("+ Uses code blocks")
+    if re.search(r"^\s*[-*]\s+", text, re.MULTILINE):
+        evidence.append("+ Uses bullet points")
+
+    for paragraph in [p for p in text.split("\n\n") if p.strip()]:
+        if count_words(paragraph) > WALL_OF_TEXT_WORDS:
+            deductions += 1
+            evidence.append("- Wall-of-text paragraph (>200 words without break)")
+            break
+
+    jargon_deductions, jargon_evidence = _check_jargon(text)
+    summary_deductions, summary_evidence = _check_summary(text)
+    deductions += jargon_deductions + summary_deductions
+    evidence.extend(jargon_evidence + summary_evidence)
+
+    if deductions >= 3:
+        score = 2
+    elif deductions == 2:
+        score = 3
+    elif deductions == 1:
+        score = 4
+    else:
+        score = 5
+
+    if not evidence:
+        evidence.append("+ Well-structured with no clarity issues detected")
+
+    result = AxisScore(name="Clarity", score=score, evidence=evidence)
+    if score < 5:
+        result.improvement = "Add headings, break long paragraphs, define domain terms on first use"
+    return result
+
+
+def check_actionability(text: str) -> AxisScore:
+    """Check if the user can act on the output immediately."""
+    evidence = []
+    score = 5
+    deductions = 0
+
+    # Positive signals
+    actionable_signals = [
+        (r"(?i)(merge|PR|pull request).*?(created|ready|open)", "PR created"),
+        (r"(?i)(run|execute)\s+[`\"']?[\w./-]+", "Specific run command given"),
+        (r"(?i)(next\s+steps?|follow[- ]up|what\s+to\s+do)", "Next steps provided"),
+        (r"(?i)(file\s+(created|written|modified|updated)\s+at)", "File path specified"),
+    ]
+    for pattern, label in actionable_signals:
+        if re.search(pattern, text):
+            evidence.append(f"+ {label}")
+
+    # Negative signals
+    vague_signals = [
+        (r"(?i)(you\s+(should|could|might\s+want\s+to))\s+\w+", "Vague suggestion without specifics"),
+        (r"(?i)(consider|maybe|perhaps)\s+\w+ing", "Non-committal suggestion"),
+        (r"(?i)(figure\s+out|look\s+into|investigate)\s", "Defers work to user"),
+    ]
+    for pattern, label in vague_signals:
+        if re.search(pattern, text):
+            deductions += 1
+            evidence.append(f"- {label}")
+
+    if deductions >= 3:
+        score = 2
+    elif deductions == 2:
+        score = 3
+    elif deductions == 1:
+        score = 4
+
+    if not evidence:
+        evidence.append("No actionability signals — user may need to ask 'what now?'")
+
+    result = AxisScore(name="Actionability", score=score, evidence=evidence)
+    if score < 5:
+        result.improvement = "End with a single clear action: 'Merge this PR', 'Run ./deploy.sh', or 'Review the 3 changed files'"
+    return result
+
+
+def check_conciseness(text: str, task: Optional[str] = None) -> AxisScore:
+    """Check for redundancy, filler, information density."""
+    evidence = []
+    score = 5
+    wc = count_words(text)
+
+    # Heuristic: task-to-output ratio
+    if task:
+        task_wc = count_words(task)
+        ratio = wc / max(task_wc, 1)
+        if ratio > TASK_OUTPUT_RATIO_HIGH:
+            evidence.append(f"- Output is {ratio:.0f}x longer than task description (high ratio)")
+            score = min(score, 3)
+        elif ratio > TASK_OUTPUT_RATIO_MEDIUM:
+            evidence.append(f"- Output is {ratio:.0f}x longer than task description")
+            score = min(score, 4)
+
+    # Redundancy signals
+    redundancy_checks = [
+        (r"(?i)(as\s+(I|we)\s+(mentioned|said|noted|discussed)\s+(earlier|above|before))",
+         "Refers back to earlier statement (possible repetition)"),
+        (r"(?i)(to\s+summarize|in\s+summary|in\s+conclusion|to\s+conclude)",
+         "Has explicit summary (good if needed, flag if redundant)"),
+        (r"(?i)(let\s+me\s+(explain|break\s+this\s+down|walk\s+you\s+through))",
+         "Meta-commentary adds words without information"),
+    ]
+    redundant_count = 0
+    for pattern, label in redundancy_checks:
+        matches = re.findall(pattern, text)
+        if len(matches) > 2:
+            redundant_count += 1
+            evidence.append(f"- '{label}' appears {len(matches)} times")
+
+    if redundant_count >= 2:
+        score = min(score, 3)
+    elif redundant_count == 1:
+        score = min(score, 4)
+
+    if not evidence and score == 5:
+        evidence.append("+ No redundancy detected. Information density appears good.")
+
+    result = AxisScore(name="Conciseness", score=score, evidence=evidence)
+    if score < 5:
+        result.improvement = "Cut meta-commentary, remove repeated points, trim examples to one representative case"
+    return result
+
+
+def evaluate(task: Optional[str], output: str) -> list[AxisScore]:
+    """Run all 5 axis checks and return scored results."""
+    return [
+        check_accuracy(output),
+        check_completeness(output),
+        check_clarity(output),
+        check_actionability(output),
+        check_conciseness(output, task),
+    ]
+
+
+def format_report(scores: list[AxisScore]) -> str:
+    """Format scores into a readable evaluation report."""
+    avg = sum(s.score for s in scores) / len(scores)
+    lines = []
+    lines.append("=" * 60)
+    lines.append("AGENT SELF-EVALUATION REPORT")
+    lines.append("=" * 60)
+    lines.append(f"Summary: Overall score {avg:.1f}/5 across 5 quality axes.")
+    lines.append("")
+
+    for s in scores:
+        bar = "█" * s.score + "░" * (5 - s.score)
+        lines.append(f"  {s.name:<15} {bar} {s.score}/5")
+        lines.extend(f"    {e}" for e in s.evidence)
+        if s.improvement:
+            lines.append(f"    → {s.improvement}")
+        lines.append("")
+
+    lines.append(f"  {'OVERALL':<15} {avg:.1f}/5")
+    lines.append("")
+
+    # Critical issues (axes ≤ 2)
+    critical = [(s, s.improvement or "No improvement suggested") for s in scores if s.score <= 2]
+    lines.append("CRITICAL ISSUES (axes ≤ 2):")
+    if critical:
+        for s, imp in critical:
+            lines.append(f"  [{s.name}] Score {s.score}/5 — {imp}")
+    else:
+        lines.append("  None")
+
+    lines.append("")
+    lines.append("Self-check: Would the user agree with this assessment? [Yes/No + brief justification]")
+    lines.append("")
+
+    # Top improvements (axes scoring < 4, ranked by impact)
+    improvements = [(s, s.improvement) for s in scores if s.improvement and s.score < 4]
+    lines.append("TOP IMPROVEMENTS:")
+    if improvements:
+        for i, (s, imp) in enumerate(sorted(improvements, key=lambda x: x[0].score), 1):
+            lines.append(f"  {i}. [{s.name}] {imp}")
+    else:
+        lines.append("  No axes below 4. Strong output across all dimensions.")
+
+    lines.append("")
+
+    # Verdict
+    min_score = min(s.score for s in scores)
+    if min_score <= 2:
+        verdict = f"Redo with specific fixes. Weakest axis: {min(scores, key=lambda s: s.score).name} ({min_score}/5)."
+    elif any(s.score <= 3 for s in scores):
+        weak = [s.name for s in scores if s.score <= 3]
+        verdict = f"Fix {'/'.join(weak)} issues, then deliver."
+    elif avg >= 4.5:
+        verdict = "Deliver as-is. No changes needed."
+    else:
+        verdict = "Deliver as-is. Minor improvements noted above."
+    lines.append(f"VERDICT: {verdict}")
+
+    return "\n".join(lines)
+
+
+def _read_file_or_text(path: Optional[str], *, required: bool = False) -> Optional[str]:
+    """Read a file path or return inline text when allowed."""
+    if path is None:
+        return None
+    try:
+        with open(path) as f:
+            return f.read()
+    except FileNotFoundError:
+        if required:
+            print(f"Error: output file '{path}' not found", file=sys.stderr)
+            sys.exit(1)
+        return path
+
+
+def _read_input(args: argparse.Namespace) -> tuple[Optional[str], str]:
+    """Read task and output for interactive, file, or pipe mode."""
+    if args.interactive:
+        task = input("Task description: ").strip()
+        print("Paste agent output (Ctrl+D to finish):")
+        return task, sys.stdin.read()
+    if args.output:
+        return _read_file_or_text(args.task), _read_file_or_text(args.output, required=True) or ""
+    return _read_file_or_text(args.task), sys.stdin.read()
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        description="Evaluate agent output against the 5-axis rubric"
+    )
+    parser.add_argument("--task", help="Task description (file path or inline text)")
+    parser.add_argument("--output", help="Agent output to evaluate (file path)")
+    parser.add_argument("--interactive", action="store_true", help="Prompt for task and read output from stdin")
+    args = parser.parse_args()
+
+    task, output = _read_input(args)
+    if not output:
+        print("Error: no output to evaluate", file=sys.stderr)
+        sys.exit(1)
+
+    scores = evaluate(task, output)
+    print(format_report(scores))
+
+
+if __name__ == "__main__":
+    main()