ur-agent 1.21.0 → 1.22.1

package/docs/AGENT_FEATURES.md

@@ -51,8 +51,26 @@ ur --discover-ollama
 ur --ollama-host http://192.168.1.50:11434
 ur worktree list
 ur worktree clean --dry-run
+ur eval run starter --metrics --json
+ur eval report starter --dashboard
+ur eval dashboard
+ur spec init auth-refactor --goal "refactor login without changing behavior"
+ur spec run auth-refactor --all
+ur spec verify auth-refactor
 ```
 
+## v1.22.0 Additions
+
+| Addition | Surface | What it adds |
+| --- | --- | --- |
+| Eval execution metrics | `ur eval run <suite> --metrics`, `UR_EVAL_METRICS_FILE` | Child-serialized cost, tokens, model, duration, files changed, insertions/deletions, command failures, human-edit heuristics, and per-case `testCommand` pass/fail. Safe for parallel runs because each child writes its own metrics file. |
+| Richer eval dashboard | `ur eval dashboard`, `ur eval report <suite> --dashboard` | Local-first HTML dashboard with summary cards and a per-case timeline showing model, time, cost, tokens, diffs, test result, command failures, and human edits. |
+| Per-case run metrics persistence | `.ur/evals/.runs/<suite>/<case>.json` | `ur eval run <suite> --metrics` writes each case's metrics to a JSON file for downstream analysis. |
+| Spec verification / verifier kernel role | `ur spec verify <name>`, `src/services/agents/specVerifier.ts` | Deterministic project gates first, then a read-only deep verification subagent that must prove compile/test/lint/diff/runtime before PASS. Writes `.ur/specs/<name>/verification.md` and a `verification` record in `spec.json`. First concrete kernel role: verifier is stricter than the generator. |
+| AgentKernel abstraction | `ur spec run|verify <name> --kernel`, `src/services/agents/kernel.ts` | Pure orchestrator separating planner, executor, verifier, critic, memory, router, and guard. Routes spec run/verify through kernel stages while keeping the legacy loop as default. Foundation for applying the same orchestration to workflows, crew, and CI loop. |
+| Rich task decomposition | `ur crew create|run|plan ... --decompose`, `src/services/agents/decomposer.ts` | Splits large goals into atomic subtasks with goal, files touched, risk level (low/medium/high), tests required, and rollback point. Deterministic fallback + optional LLM-driven JSON decomposition. |
+| Parallel specialized subagents | `ur pattern parallel "<task>" --execute`, `src/services/agents/patterns.ts` | Bug finder, patch writer, test writer, security auditor, and style reviewer run in parallel via the workflow executor, then a synthesizer merges results into one plan. |
+
 ## v1.21.0 Additions
 
 | Addition | Surface | What it adds |

package/docs/AGENT_UPGRADE_1.22.0.md

@@ -0,0 +1,48 @@
+# Upgrading to UR Agent v1.22.0
+
+## What's new
+
+v1.22.0 turns `ur eval` into a real agent benchmark tool.
+
+- **Execution metrics** — headless `ur -p` child runs now write a metrics JSON file
+  via `UR_EVAL_METRICS_FILE`. The eval runner reads it and also runs
+  `git diff --stat` and an optional per-case `expect.testCommand`. Captured
+  metrics: cost USD, input/output tokens, model, API duration, files changed,
+  insertions/deletions, command failures, human-edit heuristics, and test
+  pass/fail.
+- **Parallel-safe by design** — each child writes its own metrics file, so
+  future parallel eval runs will not corrupt shared cost state.
+- **Richer dashboard** — `ur eval dashboard` and
+  `ur eval report <suite> --dashboard` generate local HTML dashboards with
+  summary cards (pass rate, test pass rate, cost, tokens, files changed, command
+  failures, human edits, duration) and a per-case timeline showing model, time,
+  cost, tokens, diffs, test result, and output preview.
+- **Per-case metrics persistence** — `ur eval run <suite> --metrics` writes
+  each case's metrics to `.ur/evals/.runs/<suite>/<case>.json`.
+- **Aggregate reporting** — `ur eval report <suite>` now prints totals for cost,
+  tokens, files changed, command failures, human edits, duration, and test pass
+  rate.
+
+## Quick examples
+
+```sh
+# Run the starter suite and print JSON with metrics
+ur eval run starter --metrics --json
+
+# Write a single-suite HTML timeline dashboard
+ur eval report starter --dashboard
+
+# Generate the combined dashboard across all saved reports
+ur eval dashboard
+
+# Import a SWE-bench-style export and run it
+ur eval bench swe-bench --file issues.jsonl --name my-bench
+ur eval run my-bench --metrics
+```
+
+## Upgrade steps
+
+1. Pull the release and run `bun install`.
+2. Run `bun run typecheck` and `bun test` to confirm the local state.
+3. Rebuild the bundled CLI: `bun run bundle`.
+4. No settings or project file migration is required.

package/docs/USAGE.md

@@ -116,7 +116,7 @@ UR includes slash commands and CLI subcommands for common workflows:
 - `ur context-pack ...` to summarize architecture and persist task decisions, constraints, commands, and diffs
 - `ur code-index watch` to keep the local semantic code index fresh
 - `ur memory retention ...` to prune project-local memory by TTL, max entries, and decay
-- `ur spec ...` to scaffold requirements, design, and tasks, then run a spec task list
+- `ur spec ...` to scaffold requirements, design, and tasks, run a spec task list, and verify with strict proof gates
 - `ur escalate ...` to plan, run, or ask an oracle model for hard tasks
 - `ur arena ...` to run multiple agents on the same task and select a winner
 - `ur test-first ...` to detect compile/test/lint commands, store failure traces, and install after-edit gates
@@ -125,7 +125,12 @@ UR includes slash commands and CLI subcommands for common workflows:
 - `ur ide diff ...` to capture editor-readable inline diff bundles
 - `ur acp ...` to start/stop/status the Agent Communication Protocol server for IDE extensions
 - `ur exec ...` to run prompts in non-interactive mode with optional concurrency
+- `ur eval run ...` to run a suite, grade results, and capture execution metrics
+- `ur eval report ...` to show a saved report or write a single-suite dashboard
+- `ur eval dashboard` to generate the local HTML dashboard across all reports
 - `ur eval bench ...` to import local SWE-bench, Terminal-Bench, or Aider Polyglot exports
+- `ur crew ...` to run lead+worker agent crews with optional automatic task decomposition
+- `ur pattern ...` to run multi-agent collaboration patterns (PEER, DOE, concurrent, handoff, debate, parallel)
 - `ur doctor` to inspect CLI health
 - `ur update` or `ur upgrade` to check for updates
 
@@ -139,6 +144,8 @@ Agent platform examples:
 ```sh
 ur spec init demo --goal "1. add a utils.add function 2. add a test"
 ur spec run demo --all --dry-run
+ur spec run demo --all --kernel
+ur spec verify demo --kernel
 ur arena "implement a debounce helper" --agents 2 --dry-run
 ur escalate run "refactor the cache layer" --force-oracle --dry-run
 ur test-first detect
@@ -162,6 +169,13 @@ ur repo-edit preview rename oldName --to newName
 ur repo-edit apply rename oldName --to newName --check "bun test"
 ur ide diff capture --title "Working tree review"
 ur eval bench list
+ur eval run starter --dry-run --json
+ur eval run starter --metrics --json
+ur eval report starter --dashboard
+ur eval dashboard
+ur crew create parser-crew --goal "fix the flaky parser test" --decompose --dry-run
+ur crew run parser-crew --workers 3 --decompose --dry-run
+ur pattern parallel "refactor login without changing behavior" --execute --dry-run
 ```
 
 ## Permissions

package/documentation/app.js

@@ -241,8 +241,8 @@ const commands = [
     name: 'eval',
     category: 'Verification',
     aliases: ['evals'],
-    summary: 'Create, validate, run, and report public agent eval suites with deterministic grading.',
-    examples: ['ur eval init', 'ur eval list', 'ur eval validate starter', 'ur eval run starter --dry-run', 'ur eval report starter'],
+    summary: 'Create, validate, run, and report public agent eval suites with deterministic grading plus execution metrics and a local HTML dashboard.',
+    examples: ['ur eval init', 'ur eval list', 'ur eval validate starter', 'ur eval run starter --dry-run', 'ur eval run starter --metrics --json', 'ur eval report starter --dashboard', 'ur eval dashboard'],
   },
   {
     name: 'goal',
@@ -630,8 +630,8 @@ const examples = [
   },
   {
     title: 'Eval suite',
-    text: 'Create a suite, validate, run offline, then run live.',
-    code: 'ur eval init\nur eval validate starter\nur eval run starter --dry-run\nur eval report starter',
+    text: 'Create a suite, validate, run offline with metrics, then open the dashboard.',
+    code: 'ur eval init\nur eval validate starter\nur eval run starter --dry-run\nur eval run starter --metrics\nur eval report starter --dashboard',
   },
   {
     title: 'Safe automation',

package/documentation/index.html

@@ -43,7 +43,7 @@
     <main id="content" class="content">
       <header class="topbar">
         <div>
-          <p class="eyebrow">Version 1.21.0</p>
+          <p class="eyebrow">Version 1.22.1</p>
           <h1>UR Agent Documentation</h1>
           <p class="lead">A practical, tutorial-style reference for installing, configuring, automating, extending, and operating UR Agent.</p>
         </div>

package/extensions/vscode-ur-inline-diffs/package.json

@@ -2,7 +2,7 @@
   "name": "ur-inline-diffs",
   "displayName": "UR Inline Diffs",
   "description": "Review UR inline diff bundles from .ur/ide/diffs inside VS Code.",
-  "version": "1.21.0",
+  "version": "1.22.1",
   "publisher": "ur-agent",
   "engines": {
     "vscode": "^1.92.0"

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ur-agent",
-  "version": "1.21.0",
+  "version": "1.22.1",
   "description": "UR terminal coding agent CLI",
   "type": "module",
   "packageManager": "bun@1.3.14",