ur-agent 1.22.7 → 1.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/dist/cli.js +31084 -30479
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,7 +1,39 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.22.8
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
-
|
|
7
|
+
|
|
8
|
+
### Changed
|
|
9
|
+
-
|
|
10
|
+
|
|
11
|
+
### Fixed
|
|
12
|
+
-
|
|
13
|
+
|
|
3
14
|
## 1.22.7
|
|
4
15
|
|
|
16
|
+
### Added
|
|
17
|
+
- **Benchmark mode (`ur eval`).**
|
|
18
|
+
- `ur eval run <suite> [--model <m>] [--metrics]` runs eval suites with explicit model overrides and per-case metrics files.
|
|
19
|
+
- `ur eval report <suite>` now surfaces rollbacks, test pass rate, files changed, command failures, human edits, cost, time, and tokens.
|
|
20
|
+
- `ur eval compare <suite> <label1> <label2>...` runs the same suite against multiple model/runner labels and prints a comparison matrix.
|
|
21
|
+
- `ur eval route "<task>" [--strategy auto|cheap|strong|default]` recommends a model using the capability router.
|
|
22
|
+
- **Public leaderboard and built-in benchmark suites.**
|
|
23
|
+
- `ur eval builtin list` shows six small built-in suites: `bug-fix`, `refactor`, `test-gen`, `docker-repair`, `ts-migrate`, `py-package-repair`.
|
|
24
|
+
- `ur eval builtin <id>` installs a suite under `.ur/evals/`.
|
|
25
|
+
- `ur eval leaderboard [--format html|json|md] [<suite>]` writes a public leaderboard from saved reports.
|
|
26
|
+
- **Model routing integration.**
|
|
27
|
+
- Added `RouteStrategy`, `ModelPool`, and `resolveModelForTask` in `src/services/agents/modelRouter.ts` with `src/services/agents/modelPool.ts` config.
|
|
28
|
+
- `makeCliEvalRunner` accepts a per-run `model` override.
|
|
29
|
+
- Background tasks (`startBackgroundTask`) support `routeStrategy` and resolve the model from the local pool before spawning.
|
|
30
|
+
- Bundled skills (`debug-v2`, `security-review`, `refactor`, `benchmark`, `dockerize`) now prompt the agent to use `route: strong/auto` hints.
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
- `ur eval` CLI now accepts `[action] [name] [rest...]` and `--model` / `--strategy` flags to support compare and route subcommands.
|
|
34
|
+
- `EvalRunMetrics` gained `rollbacks`; `EvalReport` gained `totalRollbacks`; report and dashboard now display them.
|
|
35
|
+
- `startBackgroundTask` and `fanoutBackgroundTasks` are now async so they can resolve model routing from local capabilities.
|
|
36
|
+
|
|
5
37
|
### Fixed
|
|
6
38
|
- Fixed `ur task run <id>` so it starts the queued worktree task created by
|
|
7
39
|
`ur task start` instead of creating a second background task whose prompt is
|
|
@@ -19,6 +51,8 @@
|
|
|
19
51
|
### Verified
|
|
20
52
|
- Added focused coverage for approval-level mapping, `ur task start` to
|
|
21
53
|
`ur task run`, PR-quality output sections, and CI-loop failure memory.
|
|
54
|
+
- Added `test/evalCompare.test.ts` and `test/evalBenchmarkSuites.test.ts` for compare matrices, built-in suites, and leaderboard rendering.
|
|
55
|
+
- Updated `test/execCommand.test.ts` for the async `runExecPool` signature.
|
|
22
56
|
- Verified source and production bundle release checks for the `1.22.7` build.
|
|
23
57
|
|
|
24
58
|
## 1.22.6
|