ai-agentreflect 0.1.0__tar.gz

ai_agentreflect-0.1.0/AGENTS.md

@@ -0,0 +1,12 @@
+# AGENTS.md — agentreflect repo
+
+## Rules for AI coding agents working in this repo
+
+- Run `pytest tests/ -v` before committing any changes
+- All public functions must have docstrings
+- Use type annotations on all function signatures
+- Never modify `tests/fixtures/` without updating corresponding tests
+- The basic mode (no `--llm`) MUST work without any API key
+- Keep `anthropic` import inside the function that needs it (lazy import)
+- Confidence scores must be floats between 0.0 and 1.0
+- When adding new error type mappings, add tests in `test_generators.py`

ai_agentreflect-0.1.0/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+All notable changes to `agentreflect` will be documented here.
+
+## [0.1.0] - 2026-03-11
+
+### Added
+- `agentreflect generate` command with three input modes:
+  - `--from-pytest <file>`: parse pytest output for failure patterns
+  - `--from-git`: analyze git log for fix/bug/revert commits
+  - `--from-notes "text"`: parse plain text failure descriptions
+- Pattern-based analysis engine (no API key required)
+  - Maps 15+ exception types to targeted rule suggestions
+  - Confidence scoring for each suggestion
+- LLM-enhanced mode (`--llm`) using `claude-3-5-haiku-latest`
+  - Requires `ANTHROPIC_API_KEY`
+  - Graceful fallback to pattern mode if key not set
+- Three output formats:
+  - Markdown (default): formatted suggestion report
+  - Diff (`--format diff`): unified diff ready to apply
+  - Apply (`--apply <file>`): direct AGENTS.md modification
+- `--yes` flag to skip confirmation on file apply
+- `--min-confidence` threshold filtering
+- 40+ tests covering parsers, generators, output, and CLI integration
+
+[0.1.0]: https://github.com/mikiships/agentreflect/releases/tag/v0.1.0

ai_agentreflect-0.1.0/CLAUDE.md

@@ -0,0 +1,41 @@
+# CLAUDE.md — agentreflect repo
+
+## Project overview
+
+`agentreflect` is a Python CLI tool that analyzes AI coding agent failure logs and generates targeted AGENTS.md rule suggestions. It is part of the agent quality trilogy: coderace → agentmd → agentlint → agentreflect.
+
+## Commands
+
+```bash
+# Install in dev mode
+pip install -e '.[dev]'
+
+# Run tests
+pytest tests/ -v
+
+# Run with coverage
+pytest tests/ --cov=agentreflect --cov-report=term-missing
+
+# Build package
+python -m build
+
+# Test CLI
+agentreflect --version
+agentreflect generate --from-notes "agent forgot to check None"
+agentreflect generate --from-pytest tests/fixtures/sample-failures.txt
+```
+
+## Architecture
+
+- `agentreflect/parsers/` — input parsers (pytest, git, notes)
+- `agentreflect/generators/` — suggestion generators (pattern-based, LLM)
+- `agentreflect/output/` — formatters (markdown, diff, apply)
+- `agentreflect/models.py` — FailureRecord, RuleSuggestion dataclasses
+- `agentreflect/cli.py` — click-based CLI entry point
+
+## Key constraints
+
+1. Basic mode (no `--llm`) must work without any API key
+2. The `anthropic` package is only imported when `--llm` is used
+3. Pattern generator must always return at least one suggestion
+4. Apply mode appends to file, never overwrites existing content

ai_agentreflect-0.1.0/PKG-INFO

@@ -0,0 +1,135 @@
+Metadata-Version: 2.4
+Name: ai-agentreflect
+Version: 0.1.0
+Summary: Generate AGENTS.md rules from agent failure logs
+Project-URL: Homepage, https://github.com/mikiships/agentreflect
+Project-URL: Repository, https://github.com/mikiships/agentreflect
+Author: agentreflect contributors
+License: MIT
+Keywords: AGENTS.md,agent-quality,agents,ai,claude,code-quality,codex
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Requires-Dist: click>=8.0
+Provides-Extra: dev
+Requires-Dist: anthropic>=0.20.0; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: llm
+Requires-Dist: anthropic>=0.20.0; extra == 'llm'
+Description-Content-Type: text/markdown
+
+# agentreflect
+
+Generate `AGENTS.md` rules from AI coding agent failure logs. Closes the feedback loop in the agent quality trilogy.
+
+```
+Measure (coderace) → Generate (agentmd) → Guard (agentlint) → Learn (agentreflect)
+```
+
+## What it does
+
+Every developer using Claude Code or Codex has this problem: their agent makes a mistake, they fix it manually, update `AGENTS.md`, and hope it doesn't happen again. `agentreflect` automates the "update AGENTS.md" step.
+
+Feed it failure logs → get targeted rule suggestions → apply them to your `AGENTS.md`.
+
+## Install
+
+```bash
+pip install ai-agentreflect
+```
+
+For LLM-enhanced mode:
+```bash
+pip install 'ai-agentreflect[llm]'
+```
+
+## Usage
+
+### From pytest output
+
+```bash
+# Capture failures
+pytest --tb=short 2>&1 | tee failures.txt
+
+# Generate suggestions
+agentreflect generate --from-pytest failures.txt
+```
+
+### From git log
+
+```bash
+agentreflect generate --from-git
+```
+
+Analyzes `fix:`, `bug:`, `revert:` commits and agent-related mistake commits.
+
+### From plain text notes
+
+```bash
+agentreflect generate --from-notes "agent forgot to check for None before accessing .value"
+```
+
+## Output formats
+
+### Markdown (default)
+
+```
+## agentreflect suggestions (2026-03-11)
+
+### From: pytest failures (failures.txt)
+- [ ] Always check for None before attribute access: use `if obj is not None` or `hasattr(obj, 'attr')`
+- [ ] When catching AttributeError, log the object type with `type(obj).__name__`
+
+_Source: 3 failures analyzed, 2 suggestions generated_
+```
+
+### Diff format
+
+```bash
+agentreflect generate --from-pytest failures.txt --format diff
+```
+
+Outputs a unified diff ready to apply to `AGENTS.md`.
+
+### Apply directly
+
+```bash
+agentreflect generate --from-notes "agent used wrong variable" --apply AGENTS.md
+# Asks for confirmation
+
+agentreflect generate --from-pytest failures.txt --apply AGENTS.md --yes
+# Applies without confirmation
+```
+
+## LLM-enhanced mode
+
+```bash
+export ANTHROPIC_API_KEY=your_key_here
+agentreflect generate --from-pytest failures.txt --llm
+```
+
+Uses `claude-3-5-haiku-latest` for contextual, specific suggestions tailored to your actual failures. Cost: ~$0.001 per analysis.
+
+Basic pattern mode works without any API key.
+
+## Integration with the trilogy
+
+| Tool | Role |
+|------|------|
+| [coderace](https://pypi.org/project/ai-coderace/) | Measure agent output quality |
+| [agentmd](https://pypi.org/project/ai-agentmd/) | Generate AGENTS.md from scratch |
+| [agentlint](https://pypi.org/project/ai-agentlint/) | Guard/validate AGENTS.md rules |
+| **agentreflect** | **Learn from failures → update rules** |
+
+## License
+
+MIT

ai_agentreflect-0.1.0/README.md

@@ -0,0 +1,106 @@
+# agentreflect
+
+Generate `AGENTS.md` rules from AI coding agent failure logs. Closes the feedback loop in the agent quality trilogy.
+
+```
+Measure (coderace) → Generate (agentmd) → Guard (agentlint) → Learn (agentreflect)
+```
+
+## What it does
+
+Every developer using Claude Code or Codex has this problem: their agent makes a mistake, they fix it manually, update `AGENTS.md`, and hope it doesn't happen again. `agentreflect` automates the "update AGENTS.md" step.
+
+Feed it failure logs → get targeted rule suggestions → apply them to your `AGENTS.md`.
+
+## Install
+
+```bash
+pip install ai-agentreflect
+```
+
+For LLM-enhanced mode:
+```bash
+pip install 'ai-agentreflect[llm]'
+```
+
+## Usage
+
+### From pytest output
+
+```bash
+# Capture failures
+pytest --tb=short 2>&1 | tee failures.txt
+
+# Generate suggestions
+agentreflect generate --from-pytest failures.txt
+```
+
+### From git log
+
+```bash
+agentreflect generate --from-git
+```
+
+Analyzes `fix:`, `bug:`, `revert:` commits and agent-related mistake commits.
+
+### From plain text notes
+
+```bash
+agentreflect generate --from-notes "agent forgot to check for None before accessing .value"
+```
+
+## Output formats
+
+### Markdown (default)
+
+```
+## agentreflect suggestions (2026-03-11)
+
+### From: pytest failures (failures.txt)
+- [ ] Always check for None before attribute access: use `if obj is not None` or `hasattr(obj, 'attr')`
+- [ ] When catching AttributeError, log the object type with `type(obj).__name__`
+
+_Source: 3 failures analyzed, 2 suggestions generated_
+```
+
+### Diff format
+
+```bash
+agentreflect generate --from-pytest failures.txt --format diff
+```
+
+Outputs a unified diff ready to apply to `AGENTS.md`.
+
+### Apply directly
+
+```bash
+agentreflect generate --from-notes "agent used wrong variable" --apply AGENTS.md
+# Asks for confirmation
+
+agentreflect generate --from-pytest failures.txt --apply AGENTS.md --yes
+# Applies without confirmation
+```
+
+## LLM-enhanced mode
+
+```bash
+export ANTHROPIC_API_KEY=your_key_here
+agentreflect generate --from-pytest failures.txt --llm
+```
+
+Uses `claude-3-5-haiku-latest` for contextual, specific suggestions tailored to your actual failures. Cost: ~$0.001 per analysis.
+
+Basic pattern mode works without any API key.
+
+## Integration with the trilogy
+
+| Tool | Role |
+|------|------|
+| [coderace](https://pypi.org/project/ai-coderace/) | Measure agent output quality |
+| [agentmd](https://pypi.org/project/ai-agentmd/) | Generate AGENTS.md from scratch |
+| [agentlint](https://pypi.org/project/ai-agentlint/) | Guard/validate AGENTS.md rules |
+| **agentreflect** | **Learn from failures → update rules** |
+
+## License
+
+MIT

ai_agentreflect-0.1.0/agentreflect/__init__.py

@@ -0,0 +1,3 @@
+"""agentreflect — generate AGENTS.md rules from agent failure logs."""
+
+__version__ = "0.1.0"

ai_agentreflect-0.1.0/agentreflect/cli.py

@@ -0,0 +1,171 @@
+"""CLI entry point for agentreflect."""
+
+import sys
+from pathlib import Path
+from typing import Optional
+
+import click
+
+from . import __version__
+from .parsers import PytestParser, GitLogParser, NotesParser
+from .generators import PatternGenerator
+from .models import FailureRecord
+
+
+@click.group()
+@click.version_option(version=__version__, prog_name="agentreflect")
+def cli():
+    """agentreflect — generate AGENTS.md rules from agent failure logs.
+
+    Analyze AI coding agent failures and get targeted rule suggestions
+    to prevent recurrence. Part of the agent quality trilogy:
+    coderace → agentmd → agentlint → agentreflect.
+    """
+
+
+@cli.command()
+@click.option(
+    "--from-pytest",
+    "pytest_file",
+    type=click.Path(exists=True),
+    default=None,
+    help="Path to pytest output file (e.g. captured with `pytest ... 2>&1 | tee failures.txt`)",
+)
+@click.option(
+    "--from-git",
+    "from_git",
+    is_flag=True,
+    default=False,
+    help="Analyze git log of current directory for fix/bug/revert commits",
+)
+@click.option(
+    "--from-notes",
+    "notes",
+    default=None,
+    help="Plain text description of what went wrong (quoted string)",
+)
+@click.option(
+    "--llm",
+    "use_llm",
+    is_flag=True,
+    default=False,
+    help="Use Anthropic API for enhanced analysis (requires ANTHROPIC_API_KEY)",
+)
+@click.option(
+    "--format",
+    "output_format",
+    type=click.Choice(["markdown", "diff"]),
+    default="markdown",
+    show_default=True,
+    help="Output format",
+)
+@click.option(
+    "--apply",
+    "apply_file",
+    type=click.Path(),
+    default=None,
+    help="Apply suggestions directly to an AGENTS.md file",
+)
+@click.option(
+    "--yes",
+    "-y",
+    is_flag=True,
+    default=False,
+    help="Skip confirmation when applying to file",
+)
+@click.option(
+    "--min-confidence",
+    type=float,
+    default=0.0,
+    help="Minimum confidence threshold for suggestions (0.0–1.0)",
+)
+def generate(
+    pytest_file: Optional[str],
+    from_git: bool,
+    notes: Optional[str],
+    use_llm: bool,
+    output_format: str,
+    apply_file: Optional[str],
+    yes: bool,
+    min_confidence: float,
+):
+    """Generate AGENTS.md rule suggestions from failure logs."""
+
+    if not any([pytest_file, from_git, notes]):
+        raise click.UsageError(
+            "At least one input is required: --from-pytest, --from-git, or --from-notes"
+        )
+
+    all_records: list[FailureRecord] = []
+    source_labels = []
+
+    # Parse inputs
+    if pytest_file:
+        content = Path(pytest_file).read_text(encoding="utf-8")
+        parser = PytestParser()
+        records = parser.parse(content)
+        all_records.extend(records)
+        source_labels.append(f"pytest failures ({pytest_file})")
+        if not records:
+            click.echo(
+                f"[warn] No failures found in {pytest_file}. "
+                "Make sure the file contains pytest output with FAILED/ERROR lines.",
+                err=True,
+            )
+
+    if from_git:
+        records = GitLogParser.from_repo(".")
+        all_records.extend(records)
+        source_labels.append("git log")
+        if not records:
+            click.echo(
+                "[info] No fix/bug/revert commits found in git log.",
+                err=True,
+            )
+
+    if notes:
+        parser = NotesParser()
+        records = parser.parse(notes)
+        all_records.extend(records)
+        source_labels.append("notes")
+
+    # Generate suggestions
+    if use_llm:
+        from .generators.llm_generator import generate_with_llm
+
+        agents_content = None
+        if apply_file and Path(apply_file).exists():
+            agents_content = Path(apply_file).read_text(encoding="utf-8")
+
+        try:
+            suggestions = generate_with_llm(all_records, agents_content)
+        except (ImportError, ValueError) as e:
+            click.echo(f"[error] LLM mode failed: {e}", err=True)
+            click.echo("[info] Falling back to pattern-based analysis.", err=True)
+            suggestions = PatternGenerator().generate(all_records)
+    else:
+        suggestions = PatternGenerator().generate(all_records)
+
+    # Filter by confidence
+    if min_confidence > 0:
+        suggestions = [s for s in suggestions if s.confidence >= min_confidence]
+
+    source_label = " + ".join(source_labels)
+
+    # Output
+    if apply_file:
+        from .output.apply import apply_to_file
+
+        apply_to_file(suggestions, apply_file, yes=yes, echo=click.echo)
+    elif output_format == "diff":
+        from .output.diff_formatter import format_diff
+
+        existing = None
+        if Path("AGENTS.md").exists():
+            existing = Path("AGENTS.md").read_text(encoding="utf-8")
+
+        click.echo(format_diff(suggestions, "AGENTS.md", existing), nl=False)
+    else:
+        from .output.markdown_formatter import format_markdown
+
+        click.echo(format_markdown(suggestions, all_records, source_label), nl=False)

ai_agentreflect-0.1.0/agentreflect/generators/__init__.py

@@ -0,0 +1,5 @@
+"""Generators for agentreflect."""
+
+from .pattern_generator import PatternGenerator
+
+__all__ = ["PatternGenerator"]

ai_agentreflect-0.1.0/agentreflect/generators/llm_generator.py

@@ -0,0 +1,96 @@
+"""LLM-enhanced rule generator using Anthropic API."""
+
+import os
+from typing import List, Optional
+from ..models import FailureRecord, RuleSuggestion
+
+
+def generate_with_llm(
+    records: List[FailureRecord],
+    agents_md_content: Optional[str] = None,
+) -> List[RuleSuggestion]:
+    """Generate rule suggestions using the Anthropic API.
+
+    Requires ANTHROPIC_API_KEY environment variable.
+    """
+    # Import only when needed
+    try:
+        import anthropic
+    except ImportError:
+        raise ImportError(
+            "anthropic package is required for LLM mode. Install with: pip install anthropic"
+        )
+
+    api_key = os.environ.get("ANTHROPIC_API_KEY")
+    if not api_key:
+        raise ValueError(
+            "ANTHROPIC_API_KEY environment variable is not set. "
+            "Set it to use LLM mode, or omit --llm to use pattern-based analysis."
+        )
+
+    client = anthropic.Anthropic(api_key=api_key)
+
+    # Build failure summary
+    failure_lines = []
+    for i, record in enumerate(records[:20], 1):  # Cap at 20 records
+        failure_lines.append(f"{i}. [{record.type}] {record.message[:200]}")
+
+    failures_text = "\n".join(failure_lines) if failure_lines else "No specific failures provided."
+
+    # Build prompt
+    agents_section = ""
+    if agents_md_content:
+        # Truncate to avoid token limits
+        truncated = agents_md_content[:3000]
+        agents_section = f"""
+Current AGENTS.md content (for context, avoid duplicates):
+```
+{truncated}
+```
+"""
+
+    prompt = f"""You are helping a developer improve their AGENTS.md file — a set of rules for AI coding agents like Claude Code or Codex.
+
+The following failures were detected from their agent's recent work:
+
+{failures_text}
+{agents_section}
+Based on these failures, suggest 3-5 specific, actionable rules that would prevent these failures in the future.
+
+Requirements:
+- Each rule should be a single, concrete action item
+- Rules should be specific enough to be immediately actionable
+- Avoid vague advice like "be careful" — give specific patterns, checks, or practices
+- Format each rule as a markdown list item starting with "- "
+- Focus on preventing the exact type of failure shown
+
+Respond ONLY with the markdown list items, nothing else."""
+
+    message = client.messages.create(
+        model="claude-3-5-haiku-latest",
+        max_tokens=1024,
+        messages=[{"role": "user", "content": prompt}],
+    )
+
+    response_text = message.content[0].text.strip()
+
+    # Parse the response into RuleSuggestion objects
+    suggestions = []
+    for line in response_text.splitlines():
+        line = line.strip()
+        if line.startswith("- "):
+            text = line[2:].strip()
+            if text:
+                # Remove markdown checkbox if present
+                if text.startswith("[ ] "):
+                    text = text[4:]
+                suggestions.append(
+                    RuleSuggestion(
+                        category="llm-generated",
+                        text=text,
+                        confidence=0.85,
+                        source="anthropic/claude-3-5-haiku-latest",
+                    )
+                )
+
+    return suggestions