coderace 1.9.0__tar.gz → 2.0.0__tar.gz

coderace-2.0.0/BUILD-REPORT-trend.md

@@ -0,0 +1,68 @@
+# Build Report: coderace v2.0.0 trend command
+
+**Date:** 2026-03-12  
+**Contract:** `/Users/mordecai/.openclaw/workspace/memory/contracts/coderace-v2.0.0-trend.md`  
+**Status:** SUCCESS  
+**Commit:** `7a33d88` — "feat: add trend command, bump to v2.0.0"
+
+---
+
+## Deliverables Completed
+
+- [x] `coderace/commands/trend.py` — full implementation
+- [x] `coderace trend` registered in `coderace/cli.py`
+- [x] `tests/commands/test_trend.py` — 32 new tests
+- [x] `pyproject.toml` version bumped: 1.9.0 → 2.0.0
+- [x] `coderace/__init__.py` version bumped: 1.9.0 → 2.0.0
+- [x] `CHANGELOG.md` entry written
+- [x] `coderace trend --help` confirmed working
+
+---
+
+## Test Results
+
+| Suite | Tests | Passed | Failed |
+|-------|------:|-------:|-------:|
+| `tests/commands/test_trend.py` (new) | 32 | 32 | 0 |
+| Full test suite (`tests/`) | 761 | 761 | 0 |
+
+---
+
+## Command Interface
+
+```
+Usage: coderace trend [OPTIONS]
+
+ Visualize agent score progression over time.
+
+Options:
+  --agent    TEXT     Filter by agent name (also enables detailed per-task view)
+  --task     TEXT     Filter by task name
+  --days     INTEGER  Look back this many days (default: 30)
+  --format   TEXT     Output format: terminal (default) | markdown | json
+  --help              Show this message and exit.
+```
+
+---
+
+## Implementation Notes
+
+- `coderace/commands/trend.py` contains all business logic (TrendPoint, AgentTaskTrend, sparkline, format functions) — CLI command in cli.py is thin glue only, matching the `history` command pattern
+- Sparkline uses Unicode block chars `▁▂▃▄▅▆▇█` with ASCII fallback `_.-*^`
+- Date filtering applied post-query (get_runs doesn't support `since` param directly)
+- `--format json` returns structured `{ trends: [{ agent, task, runs, summary }] }` with improvement_rate as `trend_pct`
+
+---
+
+## Issues Encountered
+
+1. **`CliRunner(mix_stderr=False)` unsupported** in this version of typer's CliRunner. Fixed: removed the argument.
+2. **`coderace` on PATH is brew-installed v1.3.0** at `/opt/homebrew/bin/coderace`. The pipx version at `~/.local/bin/coderace` is v2.0.0 and has the `trend` command. Confirmed working: `~/.local/bin/coderace trend --help`. The PATH ordering issue is outside build scope.
+
+---
+
+## Stop Conditions Honored
+
+- Did NOT push to PyPI
+- Did NOT git push
+- Did NOT tag release

coderace-2.0.0/BUILD-REPORT.md

@@ -0,0 +1,41 @@
+# coderace v1.9.0 Build Report
+
+**Built:** 2026-03-12  
+**Version:** 1.9.0  
+**Commit:** 19d65382e908d292b0e42003f98cdf7832fea49a  
+**Tests:** 729 passing (700 baseline + 29 new)  
+**PyPI:** https://pypi.org/project/coderace/1.9.0/
+
+---
+
+## What was built
+
+**CI Quality Gate** — makes the maintainer rubric enforceable in CI.
+
+METR published research (Mar 2026) showing ~50% of SWE-bench-passing PRs would be rejected by real maintainers. coderace v1.8.0 shipped the rubric as a diagnostic. v1.9.0 makes it a gate.
+
+### D1: `--min-score` on `coderace review --maintainer-mode`
+- New `--min-score N` flag (0-100 int)
+- Exits 1 when composite rubric score < N
+- Prints `✅ Maintainer score 87 ≥ 80 (gate: PASS)` or `❌ Maintainer score 54 < 80 (gate: FAIL)`
+- Without `--min-score`: existing behavior unchanged
+
+### D2: `coderace gate` standalone command
+- `coderace gate --diff <file|-> --min-score 80`
+- Accepts diff via file path or stdin (`--diff -`)
+- Exits 0 (pass) or 1 (fail)
+- `--json` flag for CI log parsing (score, gate, dimensions)
+- Pure static analysis — no LLM, no API keys required
+
+### D3: GitHub Action update
+- New `action.yml` input: `maintainer-min-score` (default: empty = no gate, backward compatible)
+- New `scripts/ci-gate.sh` CI script handling all diff sources
+- Example workflow: `.github/workflows/examples/coderace-quality-gate.yml`
+
+### D4: Tests (29 new)
+- `tests/test_ci_gate.py` — 29 tests covering gate pass/fail, threshold edge cases, empty diff, JSON output, error handling, --min-score on review, action.yml structure
+
+### D5: Docs
+- README: "CI Quality Gate" section with one-liner examples and GitHub Action snippet
+- CHANGELOG: v1.9.0 entry
+- Version bumped: 1.8.0 → 1.9.0

{coderace-1.9.0 → coderace-2.0.0}/CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## [2.0.0] - 2026-03-12
+
+### Added
+- `coderace trend` command: visualize agent score progression over time
+  - Summary table: Agent | Task | Runs | Avg Score | Best Score | Latest Score | Trend
+  - Unicode sparkline in Trend column (`▁▂▃▄▅▆▇█`) with delta arrow (`↑ +8.3` / `↓ -2.1` / `→ 0.0`)
+  - Color-coded output: green (improving), red (regressing), white (flat)
+  - `--agent`: filter to one agent and show detailed per-task score history
+  - `--task`: filter to one task
+  - `--days N`: look back N days (default: 30)
+  - `--format terminal|markdown|json`: multiple output formats
+  - `--format json` returns structured `{ agent, task, runs, summary }` suitable for CI
+  - Graceful empty-DB handling (no error, helpful message)
+  - 32 new tests covering unit logic, edge cases, and CLI integration
+
 ## [1.9.0] - 2026-03-12
 
 ### Added

{coderace-1.9.0 → coderace-2.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: coderace
-Version: 1.9.0
+Version: 2.0.0
 Summary: Race coding agents against each other on real tasks
 Project-URL: Homepage, https://github.com/mikiships/coderace
 Project-URL: Repository, https://github.com/mikiships/coderace

{coderace-1.9.0 → coderace-2.0.0}/coderace/__init__.py

@@ -1,3 +1,3 @@
 """coderace - Race coding agents against each other on real tasks."""
 
-__version__ = "1.9.0"
+__version__ = "2.0.0"

{coderace-1.9.0 → coderace-2.0.0}/coderace/cli.py

@@ -1108,6 +1108,83 @@ def dashboard(
         console.print(f"[dim]Opened in browser[/dim]")
 
 
+@app.command()
+def trend(
+    agent: str | None = typer.Option(
+        None, "--agent", help="Filter by agent name (also enables detailed per-task view)"
+    ),
+    task: str | None = typer.Option(
+        None, "--task", help="Filter by task name"
+    ),
+    days: int = typer.Option(
+        30, "--days", help="Look back this many days (default: 30)"
+    ),
+    fmt: str | None = typer.Option(
+        None,
+        "--format",
+        "-F",
+        help="Output format: terminal (default) | markdown | json",
+    ),
+) -> None:
+    """Visualize agent score progression over time."""
+    import sys
+
+    from coderace.commands.trend import (
+        _build_trends,
+        format_trend_json,
+        format_trend_markdown,
+        format_trend_terminal,
+    )
+    from coderace.store import ResultStore
+
+    try:
+        store = ResultStore()
+    except Exception as exc:
+        console.print(f"[red]Cannot open result store: {exc}[/red]")
+        raise typer.Exit(1)
+
+    try:
+        since = f"{days}d"
+        # Fetch runs with optional filters; use a generous limit
+        from coderace.store import _parse_since
+        from coderace.store import get_db_path
+        import sqlite3
+
+        # Get all runs within the time window, filtered by agent/task
+        db_path = get_db_path()
+        db_path.parent.mkdir(parents=True, exist_ok=True)
+        runs = store.get_runs(
+            task_name=task,
+            agent=agent,
+            limit=10000,
+        )
+        # Apply date filter manually since get_runs doesn't support since
+        if days:
+            cutoff = _parse_since(f"{days}d")
+            if cutoff:
+                runs = [r for r in runs if r.timestamp >= cutoff]
+    finally:
+        store.close()
+
+    trends = _build_trends(runs, agent_filter=agent, task_filter=task)
+
+    if not trends:
+        console.print("[yellow]No trend data found. Run some races first.[/yellow]")
+        return
+
+    if fmt == "markdown":
+        sys.stdout.write(format_trend_markdown(trends, detail_agent=agent))
+    elif fmt == "json":
+        sys.stdout.write(format_trend_json(trends))
+    elif fmt is not None and fmt != "terminal":
+        console.print(
+            f"[red]Unknown --format {fmt!r}. Choose: terminal, markdown, json[/red]"
+        )
+        raise typer.Exit(1)
+    else:
+        format_trend_terminal(trends, detail_agent=agent, console=console)
+
+
 @app.command()
 def version() -> None:
     """Show coderace version."""

coderace-2.0.0/coderace/commands/trend.py

@@ -0,0 +1,330 @@
+"""Trend command — visualize agent score progression over time."""
+
+from __future__ import annotations
+
+import json
+import sys
+from dataclasses import dataclass, field
+from datetime import datetime, timedelta, timezone
+from typing import Optional
+
+from rich.console import Console
+from rich.table import Table
+from rich.text import Text
+
+from coderace.store import RunRecord, AgentRecord
+
+# Unicode sparkline characters (low → high)
+_SPARK_CHARS = "▁▂▃▄▅▆▇█"
+_SPARK_ASCII = "_.-*^"
+
+
+@dataclass
+class TrendPoint:
+    """A single data point in a trend series."""
+
+    run_id: int
+    timestamp: str
+    score: float
+    delta: Optional[float]  # None for first point
+    is_winner: bool
+
+
+@dataclass
+class AgentTaskTrend:
+    """Score trend for an (agent, task) pair."""
+
+    agent: str
+    task: str
+    points: list[TrendPoint] = field(default_factory=list)
+
+    @property
+    def runs(self) -> int:
+        return len(self.points)
+
+    @property
+    def avg_score(self) -> float:
+        if not self.points:
+            return 0.0
+        return sum(p.score for p in self.points) / len(self.points)
+
+    @property
+    def best_score(self) -> float:
+        if not self.points:
+            return 0.0
+        return max(p.score for p in self.points)
+
+    @property
+    def latest_score(self) -> float:
+        if not self.points:
+            return 0.0
+        return self.points[-1].score
+
+    @property
+    def latest_delta(self) -> Optional[float]:
+        if len(self.points) < 2:
+            return None
+        return self.points[-1].delta
+
+    @property
+    def improvement_rate(self) -> Optional[float]:
+        """Pct of runs where score improved vs previous run."""
+        if len(self.points) < 2:
+            return None
+        improvements = sum(
+            1 for p in self.points[1:] if p.delta is not None and p.delta > 0
+        )
+        return improvements / (len(self.points) - 1)
+
+    def sparkline(self, use_unicode: bool = True) -> str:
+        """Generate a sparkline string for the score series."""
+        scores = [p.score for p in self.points]
+        if not scores:
+            return "—"
+        if len(scores) == 1:
+            return "—"
+
+        chars = _SPARK_CHARS if use_unicode else _SPARK_ASCII
+        n = len(chars)
+        lo, hi = min(scores), max(scores)
+        span = hi - lo
+
+        result = []
+        for s in scores:
+            if span == 0:
+                idx = n // 2
+            else:
+                idx = round((s - lo) / span * (n - 1))
+            result.append(chars[idx])
+        return "".join(result)
+
+
+def _trend_direction(delta: Optional[float]) -> tuple[str, str]:
+    """Return (symbol, rich_style) for a delta value."""
+    if delta is None:
+        return "—", "dim"
+    if delta > 0.05:
+        return f"↑ +{delta:.1f}", "green"
+    if delta < -0.05:
+        return f"↓ {delta:.1f}", "red"
+    return f"→ {delta:.1f}", "white"
+
+
+def _build_trends(
+    runs: list[RunRecord],
+    agent_filter: Optional[str],
+    task_filter: Optional[str],
+) -> list[AgentTaskTrend]:
+    """Build per-(agent, task) trend objects from run records."""
+    # Group: (agent, task) -> list of (timestamp, run_id, score, is_winner)
+    groups: dict[tuple[str, str], list[tuple[str, int, float, bool]]] = {}
+
+    for run in runs:
+        if task_filter and run.task_name != task_filter:
+            continue
+        for ar in run.agents:
+            if agent_filter and ar.agent != agent_filter:
+                continue
+            key = (ar.agent, run.task_name)
+            groups.setdefault(key, []).append(
+                (run.timestamp, run.run_id, ar.composite_score, ar.is_winner)
+            )
+
+    trends: list[AgentTaskTrend] = []
+    for (agent, task), entries in sorted(groups.items()):
+        # Sort chronologically (oldest first) so delta makes sense
+        entries.sort(key=lambda x: x[0])
+        trend = AgentTaskTrend(agent=agent, task=task)
+        prev_score: Optional[float] = None
+        for ts, run_id, score, is_winner in entries:
+            delta = score - prev_score if prev_score is not None else None
+            trend.points.append(
+                TrendPoint(
+                    run_id=run_id,
+                    timestamp=ts,
+                    score=score,
+                    delta=delta,
+                    is_winner=is_winner,
+                )
+            )
+            prev_score = score
+        trends.append(trend)
+
+    return trends
+
+
+def format_trend_terminal(
+    trends: list[AgentTaskTrend],
+    detail_agent: Optional[str] = None,
+    console: Optional[Console] = None,
+) -> str:
+    """Render trend data as a Rich terminal table."""
+    console = console or Console()
+
+    if not trends:
+        console.print("[yellow]No trend data found.[/yellow]")
+        return ""
+
+    use_unicode = sys.stdout.encoding.lower().startswith(("utf", "us-ascii")) if hasattr(sys.stdout, "encoding") else True
+
+    if detail_agent:
+        # Detailed per-task view for a single agent
+        table = Table(title=f"coderace trend — {detail_agent}", show_lines=True)
+        table.add_column("Run ID", justify="center", style="bold")
+        table.add_column("Date", style="dim")
+        table.add_column("Task", style="cyan")
+        table.add_column("Score", justify="right")
+        table.add_column("Delta", justify="right")
+        table.add_column("Result")
+
+        for trend in trends:
+            for p in trend.points:
+                ts = p.timestamp
+                if "T" in ts:
+                    ts = ts.split("T")[0] + " " + ts.split("T")[1][:8]
+
+                sym, sty = _trend_direction(p.delta)
+                result_str = "win" if p.is_winner else "loss"
+                result_style = "green" if p.is_winner else "dim"
+
+                table.add_row(
+                    str(p.run_id),
+                    ts,
+                    trend.task,
+                    f"{p.score:.1f}",
+                    Text(sym, style=sty),
+                    Text(result_str, style=result_style),
+                )
+
+        console.print(table)
+
+        # Summary stats
+        total_runs = sum(t.runs for t in trends)
+        all_scores = [p.score for t in trends for p in t.points]
+        avg = sum(all_scores) / len(all_scores) if all_scores else 0.0
+        best = max(all_scores) if all_scores else 0.0
+        improvement_rates = [t.improvement_rate for t in trends if t.improvement_rate is not None]
+        avg_impr = sum(improvement_rates) / len(improvement_rates) if improvement_rates else None
+
+        console.print(f"\n[bold]Summary for {detail_agent}[/bold]")
+        console.print(f"  Total runs:       {total_runs}")
+        console.print(f"  Avg score:        {avg:.1f}")
+        console.print(f"  Best score:       {best:.1f}")
+        if avg_impr is not None:
+            console.print(f"  Improvement rate: {avg_impr:.0%}")
+        else:
+            console.print("  Improvement rate: — (need 2+ runs per task)")
+    else:
+        # Summary table: one row per (agent, task)
+        table = Table(title="coderace trend", show_lines=True)
+        table.add_column("Agent", style="cyan")
+        table.add_column("Task")
+        table.add_column("Runs", justify="right")
+        table.add_column("Avg Score", justify="right")
+        table.add_column("Best Score", justify="right")
+        table.add_column("Latest Score", justify="right")
+        table.add_column("Trend")
+
+        for trend in trends:
+            spark = trend.sparkline(use_unicode=use_unicode)
+            sym, sty = _trend_direction(trend.latest_delta)
+
+            if trend.runs < 2:
+                trend_cell: str | Text = Text("—", style="dim")
+            elif trend.latest_delta is not None and trend.latest_delta > 0.05:
+                trend_cell = Text(f"{spark} {sym}", style="green")
+            elif trend.latest_delta is not None and trend.latest_delta < -0.05:
+                trend_cell = Text(f"{spark} {sym}", style="red")
+            else:
+                trend_cell = Text(f"{spark} {sym}", style="white")
+
+            table.add_row(
+                trend.agent,
+                trend.task,
+                str(trend.runs),
+                f"{trend.avg_score:.1f}",
+                f"{trend.best_score:.1f}",
+                f"{trend.latest_score:.1f}",
+                trend_cell,
+            )
+
+        console.print(table)
+
+    str_console = Console(file=None, force_terminal=False, width=120)
+    with str_console.capture() as capture:
+        str_console.print(table)
+    return capture.get()
+
+
+def format_trend_markdown(
+    trends: list[AgentTaskTrend],
+    detail_agent: Optional[str] = None,
+) -> str:
+    """Render trend data as a markdown table."""
+    if not trends:
+        return "## coderace trend\n\n_No trend data found._\n"
+
+    header = "## coderace trend\n\n"
+
+    if detail_agent:
+        cols = "| Run ID | Date | Task | Score | Delta | Result |\n"
+        sep = "|--------|------|------|------:|------:|--------|\n"
+        rows: list[str] = []
+        for trend in trends:
+            for p in trend.points:
+                ts = p.timestamp
+                if "T" in ts:
+                    ts = ts.split("T")[0] + " " + ts.split("T")[1][:8]
+                delta_str = f"+{p.delta:.1f}" if p.delta is not None and p.delta > 0 else (f"{p.delta:.1f}" if p.delta is not None else "—")
+                result_str = "win" if p.is_winner else "loss"
+                rows.append(f"| {p.run_id} | {ts} | `{trend.task}` | {p.score:.1f} | {delta_str} | {result_str} |")
+        return header + cols + sep + "\n".join(rows) + "\n"
+    else:
+        cols = "| Agent | Task | Runs | Avg Score | Best Score | Latest Score | Trend |\n"
+        sep = "|-------|------|-----:|----------:|-----------:|-------------:|-------|\n"
+        rows = []
+        for trend in trends:
+            delta = trend.latest_delta
+            if delta is None:
+                trend_str = "—"
+            elif delta > 0.05:
+                trend_str = f"↑ +{delta:.1f}"
+            elif delta < -0.05:
+                trend_str = f"↓ {delta:.1f}"
+            else:
+                trend_str = f"→ {delta:.1f}"
+            rows.append(
+                f"| {trend.agent} | `{trend.task}` | {trend.runs} | {trend.avg_score:.1f} | {trend.best_score:.1f} | {trend.latest_score:.1f} | {trend_str} |"
+            )
+        return header + cols + sep + "\n".join(rows) + "\n"
+
+
+def format_trend_json(trends: list[AgentTaskTrend]) -> str:
+    """Render trend data as JSON."""
+    data = {
+        "trends": [
+            {
+                "agent": t.agent,
+                "task": t.task,
+                "runs": [
+                    {
+                        "run_id": p.run_id,
+                        "timestamp": p.timestamp,
+                        "score": round(p.score, 2),
+                        "delta": round(p.delta, 2) if p.delta is not None else None,
+                        "is_winner": p.is_winner,
+                    }
+                    for p in t.points
+                ],
+                "summary": {
+                    "total_runs": t.runs,
+                    "avg_score": round(t.avg_score, 2),
+                    "best_score": round(t.best_score, 2),
+                    "latest_score": round(t.latest_score, 2),
+                    "trend_pct": round(t.improvement_rate * 100, 1) if t.improvement_rate is not None else None,
+                },
+            }
+            for t in trends
+        ]
+    }
+    return json.dumps(data, indent=2) + "\n"

{coderace-1.9.0 → coderace-2.0.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "coderace"
-version = "1.9.0"
+version = "2.0.0"
 description = "Race coding agents against each other on real tasks"
 readme = "README.md"
 license = "MIT"