ur-agent 1.20.0 → 1.22.0

package/docs/AGENT_FEATURES.md

@@ -49,8 +49,30 @@ ur browser-qa validate
 ur browser-qa run home-page-smoke --dry-run
 ur --discover-ollama
 ur --ollama-host http://192.168.1.50:11434
+ur worktree list
+ur worktree clean --dry-run
+ur eval run starter --metrics --json
+ur eval report starter --dashboard
+ur eval dashboard
 ```
 
+## v1.22.0 Additions
+
+| Addition | Surface | What it adds |
+| --- | --- | --- |
+| Eval execution metrics | `ur eval run <suite> --metrics`, `UR_EVAL_METRICS_FILE` | Child-serialized cost, tokens, model, duration, files changed, insertions/deletions, command failures, human-edit heuristics, and per-case `testCommand` pass/fail. Safe for parallel runs because each child writes its own metrics file. |
+| Richer eval dashboard | `ur eval dashboard`, `ur eval report <suite> --dashboard` | Local-first HTML dashboard with summary cards and a per-case timeline showing model, time, cost, tokens, diffs, test result, command failures, and human edits. |
+| Per-case run metrics persistence | `.ur/evals/.runs/<suite>/<case>.json` | `ur eval run <suite> --metrics` writes each case's metrics to a JSON file for downstream analysis. |
+
+## v1.21.0 Additions
+
+| Addition | Surface | What it adds |
+| --- | --- | --- |
+| Agent skill runner | `src/services/agents/agentSkillRunner.ts` | Reusable wrapper around `startBackgroundTask({ worktree: true, pr: true })` that polls to completion and returns a PR-style summary. |
+| Worktree slash skills | `/debug-v2`, `/refactor`, `/paper-implementation`, `/benchmark`, `/security-review`, `/dockerize`, `/latex-paper` | Bundled slash skills that expand into prompts for isolated worktree work with clean commits and PR output. |
+| Agent templates | `ur agent-templates install` | Adds `debug-v2`, `refactor`, `paper-implementation`, `benchmark`, `security-review`, `dockerize`, and `latex-paper` reusable agent templates under `.ur/agents/`. |
+| Worktree command | `ur worktree list\|status\|clean` | Inspect and clean up UR agent worktrees created by `ur bg` or slash skills. |
+
 ## v1.20.0 Additions
 
 | Addition | Surface | What it adds |

package/docs/AGENT_UPGRADE_1.21.0.md

@@ -0,0 +1,58 @@
+# Upgrading to UR Agent v1.21.0
+
+## What's new
+
+v1.21.0 adds a set of worktree-first agent skills and a command to manage the worktrees they create.
+
+- **Agent skill runner** — `src/services/agents/agentSkillRunner.ts` wraps
+  `startBackgroundTask({ worktree: true, pr: true })`, polls the background run to
+  completion, and returns a PR-style summary with branch, commits, PR URL, and
+  diff summary.
+- **New bundled slash skills** — each skill expands into a prompt that instructs
+  the model to work in an isolated git worktree and produce a clean branch,
+  commits, and PR:
+  - `/debug-v2` — reproduce, root-cause, and fix a bug with a regression test
+  - `/refactor` — safe, test-backed refactoring
+  - `/paper-implementation` — implement an algorithm or system from a paper/URL
+  - `/benchmark` — add or run benchmarks and commit results
+  - `/security-review` — audit code for security issues, fix low-risk items
+  - `/dockerize` — add Dockerfile, compose file, health checks, `.dockerignore`
+  - `/latex-paper` — generate or compile a LaTeX paper/report
+- **Matching agent templates** — `debug-v2`, `refactor`, `paper-implementation`,
+  `benchmark`, `security-review`, `dockerize`, and `latex-paper` are now part of
+  `AGENT_TEMPLATES`. Install them with `ur agent-templates install`.
+- **`ur worktree`** — `ur worktree list|status|clean` inspects and cleans up UR
+  agent worktrees.
+
+## Quick examples
+
+```sh
+# Run a bug-fix agent in an isolated worktree
+ur -p /debug-v2 "parser crashes on empty input"
+
+# Same through the CLI
+ur bg run "reproduce and fix parser crash on empty input" --worktree --pr
+
+# List active worktrees and clean up finished ones
+ur worktree list
+ur worktree clean --dry-run
+ur worktree clean
+
+# Install the new agent templates
+ur agent-templates install debug-v2 refactor benchmark security-review dockerize latex-paper paper-implementation
+```
+
+## For users of `/debug`
+
+The existing `/debug` skill still reads the session debug log. `/debug-v2` is a
+separate skill for bug-fix work in a worktree. You can keep using `/debug` for
+diagnosing the current session and use `/debug-v2` (or `ur bg run ... --worktree
+--pr`) when you want a branch + PR.
+
+## Upgrade steps
+
+1. Pull the release and run `bun install`.
+2. Run `bun run typecheck` and `bun test` to confirm the local state.
+3. Install agent templates if you want the new `.ur/agents/*.md` files:
+   `ur agent-templates install`.
+4. No settings or project file migration is required.

package/docs/AGENT_UPGRADE_1.22.0.md

@@ -0,0 +1,48 @@
+# Upgrading to UR Agent v1.22.0
+
+## What's new
+
+v1.22.0 turns `ur eval` into a real agent benchmark tool.
+
+- **Execution metrics** — headless `ur -p` child runs now write a metrics JSON file
+  via `UR_EVAL_METRICS_FILE`. The eval runner reads it and also runs
+  `git diff --stat` and an optional per-case `expect.testCommand`. Captured
+  metrics: cost USD, input/output tokens, model, API duration, files changed,
+  insertions/deletions, command failures, human-edit heuristics, and test
+  pass/fail.
+- **Parallel-safe by design** — each child writes its own metrics file, so
+  future parallel eval runs will not corrupt shared cost state.
+- **Richer dashboard** — `ur eval dashboard` and
+  `ur eval report <suite> --dashboard` generate local HTML dashboards with
+  summary cards (pass rate, test pass rate, cost, tokens, files changed, command
+  failures, human edits, duration) and a per-case timeline showing model, time,
+  cost, tokens, diffs, test result, and output preview.
+- **Per-case metrics persistence** — `ur eval run <suite> --metrics` writes
+  each case's metrics to `.ur/evals/.runs/<suite>/<case>.json`.
+- **Aggregate reporting** — `ur eval report <suite>` now prints totals for cost,
+  tokens, files changed, command failures, human edits, duration, and test pass
+  rate.
+
+## Quick examples
+
+```sh
+# Run the starter suite and print JSON with metrics
+ur eval run starter --metrics --json
+
+# Write a single-suite HTML timeline dashboard
+ur eval report starter --dashboard
+
+# Generate the combined dashboard across all saved reports
+ur eval dashboard
+
+# Import a SWE-bench-style export and run it
+ur eval bench swe-bench --file issues.jsonl --name my-bench
+ur eval run my-bench --metrics
+```
+
+## Upgrade steps
+
+1. Pull the release and run `bun install`.
+2. Run `bun run typecheck` and `bun test` to confirm the local state.
+3. Rebuild the bundled CLI: `bun run bundle`.
+4. No settings or project file migration is required.

package/docs/USAGE.md

@@ -125,6 +125,9 @@ UR includes slash commands and CLI subcommands for common workflows:
 - `ur ide diff ...` to capture editor-readable inline diff bundles
 - `ur acp ...` to start/stop/status the Agent Communication Protocol server for IDE extensions
 - `ur exec ...` to run prompts in non-interactive mode with optional concurrency
+- `ur eval run ...` to run a suite, grade results, and capture execution metrics
+- `ur eval report ...` to show a saved report or write a single-suite dashboard
+- `ur eval dashboard` to generate the local HTML dashboard across all reports
 - `ur eval bench ...` to import local SWE-bench, Terminal-Bench, or Aider Polyglot exports
 - `ur doctor` to inspect CLI health
 - `ur update` or `ur upgrade` to check for updates
@@ -154,12 +157,18 @@ ur exec "add tests for the parser" --concurrency 4 --json
 ur ci-loop --command "bun test" --dry-run
 ur artifacts capture-diff
 ur bg run "fix the flaky parser test" --worktree --dry-run
+ur worktree list
+ur worktree clean --dry-run
 ur repo-edit index
 ur repo-edit plan rename oldName --to newName
 ur repo-edit preview rename oldName --to newName
 ur repo-edit apply rename oldName --to newName --check "bun test"
 ur ide diff capture --title "Working tree review"
 ur eval bench list
+ur eval run starter --dry-run --json
+ur eval run starter --metrics --json
+ur eval report starter --dashboard
+ur eval dashboard
 ```
 
 ## Permissions

package/documentation/app.js

@@ -13,15 +13,15 @@ const featureGroups = [
   },
   {
     title: 'Agent platform',
-    tags: ['spec', 'workflow', 'pattern', 'crew', 'goal'],
+    tags: ['spec', 'workflow', 'pattern', 'crew', 'goal', 'worktree'],
     text: 'Spec-driven development, durable workflows, collaboration patterns, parallel crews, long-horizon goals, live execution boards, and resumable checkpoint state.',
-    commands: ['ur spec', 'ur workflow', 'ur pattern', 'ur crew', 'ur goal'],
+    commands: ['ur spec', 'ur workflow', 'ur pattern', 'ur crew', 'ur goal', 'ur worktree'],
   },
   {
     title: 'Judging, escalation, and repair',
     tags: ['oracle', 'arena', 'CI', 'artifacts'],
     text: 'Capability-aware fast/oracle model routing, best-of-N agent judging, test-first execution loops, self-healing CI loops, and reviewable artifacts for diffs, test runs, plans, and feedback.',
-    commands: ['ur escalate', 'ur arena', 'ur test-first', 'ur ci-loop', 'ur artifacts'],
+    commands: ['ur escalate', 'ur arena', 'ur test-first', 'ur ci-loop', 'ur artifacts', 'ur bg'],
   },
   {
     title: 'Reliable repo editing',
@@ -90,6 +90,20 @@ const commands = [
     summary: 'A2A interoperability utilities: Agent Card, local task server, and delegation tokens.',
     examples: ['ur a2a card', 'ur a2a serve --dry-run', 'ur a2a token mint --secret "$UR_A2A_DELEGATION_SECRET" --scope coding,review'],
   },
+  {
+    name: 'bg',
+    category: 'Agent Platform',
+    aliases: ['background-agent'],
+    summary: 'Run and manage detached local background agents with optional worktrees and PR creation.',
+    examples: ['ur bg run "fix the flaky parser test" --worktree', 'ur bg list --json', 'ur bg status <id> --json'],
+  },
+  {
+    name: 'worktree',
+    category: 'Agent Platform',
+    aliases: ['worktrees'],
+    summary: 'List, inspect, and clean up UR agent worktrees created by background runs.',
+    examples: ['ur worktree list', 'ur worktree status <id>', 'ur worktree clean --dry-run'],
+  },
   {
     name: 'agent-features',
     category: 'Agent Platform',
@@ -115,8 +129,8 @@ const commands = [
     name: 'agent-templates',
     category: 'Agent Platform',
     aliases: ['agent-template'],
-    summary: 'List or install reusable project agent templates such as reviewer, test-runner, security-auditor, and docs-researcher.',
-    examples: ['ur agent-templates list', 'ur agent-templates install reviewer test-runner', 'ur agent-templates install --force'],
+    summary: 'List or install reusable project agent templates such as reviewer, test-runner, security-auditor, debug-v2, refactor, paper-implementation, benchmark, security-review, dockerize, and latex-paper.',
+    examples: ['ur agent-templates list', 'ur agent-templates install reviewer test-runner', 'ur agent-templates install debug-v2 refactor --force'],
   },
   {
     name: 'agent-trends',
@@ -227,8 +241,8 @@ const commands = [
     name: 'eval',
     category: 'Verification',
     aliases: ['evals'],
-    summary: 'Create, validate, run, and report public agent eval suites with deterministic grading.',
-    examples: ['ur eval init', 'ur eval list', 'ur eval validate starter', 'ur eval run starter --dry-run', 'ur eval report starter'],
+    summary: 'Create, validate, run, and report public agent eval suites with deterministic grading plus execution metrics and a local HTML dashboard.',
+    examples: ['ur eval init', 'ur eval list', 'ur eval validate starter', 'ur eval run starter --dry-run', 'ur eval run starter --metrics --json', 'ur eval report starter --dashboard', 'ur eval dashboard'],
   },
   {
     name: 'goal',
@@ -390,9 +404,14 @@ const slashGroups = [
   },
   {
     title: 'Agents and orchestration',
-    items: ['/agents', '/agent-templates', '/spec', '/workflow', '/pattern', '/crew', '/goal', '/arena', '/route', '/role-mode'],
+    items: ['/agents', '/agent-templates', '/spec', '/workflow', '/pattern', '/crew', '/goal', '/arena', '/route', '/role-mode', '/bg'],
     text: 'Manage agents, install role modes, run specs and workflows, and coordinate multi-agent work.',
   },
+  {
+    title: 'Agent skills',
+    items: ['/debug-v2', '/refactor', '/paper-implementation', '/benchmark', '/security-review', '/dockerize', '/latex-paper', '/simplify', '/debug'],
+    text: 'Bundled slash skills that dispatch focused agent work in isolated git worktrees with clean commits and PR output.',
+  },
   {
     title: 'Memory and evidence',
     items: ['/memory', '/remember', '/forget', '/knowledge', '/semantic-memory', '/claim-ledger', '/evidence', '/graph'],
@@ -611,8 +630,8 @@ const examples = [
   },
   {
     title: 'Eval suite',
-    text: 'Create a suite, validate, run offline, then run live.',
-    code: 'ur eval init\nur eval validate starter\nur eval run starter --dry-run\nur eval report starter',
+    text: 'Create a suite, validate, run offline with metrics, then open the dashboard.',
+    code: 'ur eval init\nur eval validate starter\nur eval run starter --dry-run\nur eval run starter --metrics\nur eval report starter --dashboard',
   },
   {
     title: 'Safe automation',

package/documentation/index.html

@@ -43,7 +43,7 @@
     <main id="content" class="content">
       <header class="topbar">
         <div>
-          <p class="eyebrow">Version 1.20.0</p>
+          <p class="eyebrow">Version 1.22.0</p>
           <h1>UR Agent Documentation</h1>
           <p class="lead">A practical, tutorial-style reference for installing, configuring, automating, extending, and operating UR Agent.</p>
         </div>

package/extensions/vscode-ur-inline-diffs/package.json

@@ -2,7 +2,7 @@
   "name": "ur-inline-diffs",
   "displayName": "UR Inline Diffs",
   "description": "Review UR inline diff bundles from .ur/ide/diffs inside VS Code.",
-  "version": "1.20.0",
+  "version": "1.22.0",
   "publisher": "ur-agent",
   "engines": {
     "vscode": "^1.92.0"

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ur-agent",
-  "version": "1.20.0",
+  "version": "1.22.0",
   "description": "UR terminal coding agent CLI",
   "type": "module",
   "packageManager": "bun@1.3.14",