ur-agent 1.65.6 → 1.65.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +52 -0
- package/README.md +5 -4
- package/dist/cli.js +5432 -3076
- package/dist/sdk/index.cjs +189 -0
- package/dist/sdk/index.d.ts +68 -0
- package/dist/sdk/index.js +149 -0
- package/docs/VALIDATION.md +1 -1
- package/documentation/app.js +91 -0
- package/documentation/index.html +1 -1
- package/extensions/jetbrains-ur/build.gradle.kts +1 -1
- package/extensions/vscode-ur-inline-diffs/package.json +1 -1
- package/package.json +13 -2
- package/technical/01-architecture.md +146 -0
- package/technical/02-cli-reference.md +227 -0
- package/technical/03-slash-commands.md +318 -0
- package/technical/04-tools.md +119 -0
- package/technical/05-providers-and-models.md +192 -0
- package/technical/06-configuration.md +407 -0
- package/technical/07-memory-and-context.md +147 -0
- package/technical/08-skills-plugins-workflows.md +211 -0
- package/technical/09-multi-agent.md +249 -0
- package/technical/10-headless-automation-eval.md +441 -0
- package/technical/11-integrations.md +156 -0
- package/technical/12-security-sandbox-stability.md +271 -0
- package/technical/13-research.md +129 -0
- package/technical/14-sessions.md +177 -0
- package/technical/README.md +43 -0
|
@@ -0,0 +1,441 @@
|
|
|
1
|
+
# 10 — Headless Use, Automation & Evaluation
|
|
2
|
+
|
|
3
|
+
Source of truth: `src/main.tsx` (print mode);
|
|
4
|
+
`src/commands/{exec,sdk,eval,ci-loop,test-first,trigger,automation,cloud,bg,learn,context-pack,agent-ci,desktop-qa,workspace,arena}`;
|
|
5
|
+
`src/services/agents/{headlessAgent,ciLoop,testFirstLoop,evals,trajectory,benchmarkSuites,triggerBridge,scheduler,cloudTasks,cloudManagedRunner,agentControl,learnedPlaybooks,agenticCi,workspaceCoordinator,arena}.ts`;
|
|
6
|
+
`src/services/{context/memoryCitations,qa,sideChats}/`; `src/sdk/`;
|
|
7
|
+
`src/entrypoints/sdk/`; and `scripts/benchmark-*.mjs`.
|
|
8
|
+
|
|
9
|
+
## Print mode (`-p`)
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
ur -p "summarize the failing tests"
|
|
13
|
+
ur -p "…" --output-format json # structured result
|
|
14
|
+
ur -p "…" --output-format stream-json --verbose # add --include-partial-messages for chunks
|
|
15
|
+
cat prompts.jsonl | ur -p --input-format stream-json --output-format stream-json --verbose --replay-user-messages
|
|
16
|
+
ur -p "…" --max-turns 5 --fallback-model llama3.3 --no-session-persistence
|
|
17
|
+
```
|
|
18
|
+
The trust dialog is skipped in `-p` — only run it in directories you trust.
|
|
19
|
+
Hook lifecycle events can be included with `--include-hook-events`.
|
|
20
|
+
|
|
21
|
+
## Batch execution (`/exec`, `ur exec`)
|
|
22
|
+
|
|
23
|
+
Multiple prompts, optional concurrency and worktree isolation
|
|
24
|
+
(`src/commands/exec/index.ts`):
|
|
25
|
+
```
|
|
26
|
+
ur exec "fix lint errors" "update snapshots" --concurrency 2
|
|
27
|
+
ur exec --file prompts.jsonl --max-turns 20 --model qwen2.5-coder:7b \
|
|
28
|
+
--output-dir ./runs --worktree --json
|
|
29
|
+
ur exec "risky idea" --dry-run
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Task planning is enabled by default. Each top-level prompt becomes an ordered
|
|
33
|
+
plan/DAG with a bounded agent budget, live task board, approval state, observed
|
|
34
|
+
file/command evidence, and strict claim verification. Relevant controls are
|
|
35
|
+
`--max-agents`, `--no-task-planning`, `--no-parallel-agents`,
|
|
36
|
+
`--no-task-board`, `--no-strict-verification`, and `--quiet`; all are
|
|
37
|
+
registered and forwarded by the shipped top-level CLI.
|
|
38
|
+
|
|
39
|
+
With `--worktree`, one worktree is created per top-level prompt, not per child
|
|
40
|
+
step. Dependencies in that plan therefore see prerequisite changes. The result
|
|
41
|
+
reports the worktree path/branch and states explicitly that `/exec` does not
|
|
42
|
+
merge, push, or publish it. Without worktrees, read-only tasks from different
|
|
43
|
+
plans may overlap, but workspace-mutating tasks take a global exclusive gate so
|
|
44
|
+
two prompt plans cannot write the shared checkout concurrently.
|
|
45
|
+
|
|
46
|
+
Planner roles (`planner`, `executor`, `verifier`, `reporter`) are placed in the
|
|
47
|
+
bounded child prompt as an assigned role; `/exec` does not pass those labels as
|
|
48
|
+
`--agent` definitions. Approval-required tasks are held before execution and
|
|
49
|
+
make that invocation incomplete/nonzero. `/exec` has no persisted
|
|
50
|
+
approve/resume subcommand, so grant the required authority and rerun the
|
|
51
|
+
top-level prompt.
|
|
52
|
+
|
|
53
|
+
File evidence comes from recursive pre/post workspace metadata snapshots
|
|
54
|
+
(excluding `.git`, dependency, and common build-cache directories),
|
|
55
|
+
supplemented by the runner's dirty-path delta. Command evidence from the
|
|
56
|
+
default runner proves the outer `ur -p` child invocation; commands the child
|
|
57
|
+
model runs inside that session are not independently observed unless an
|
|
58
|
+
executor surfaces them. Natural-language success still begins with the child
|
|
59
|
+
process result; strict verification separately rejects unsupported
|
|
60
|
+
file/command claims it can parse.
|
|
61
|
+
|
|
62
|
+
`--output-dir` uses sequence, prompt, and task identity and creates files
|
|
63
|
+
exclusively with collision suffixes, so duplicate prompts and concurrent
|
|
64
|
+
processes do not overwrite prior output. JSON results include `outputFile`.
|
|
65
|
+
Usage or invalid numeric options exit 2; any failed, canceled, blocked,
|
|
66
|
+
approval-held, or otherwise incomplete planned run exits 1.
|
|
67
|
+
|
|
68
|
+
## SDK / programmatic use (`/sdk`, `src/sdk/`)
|
|
69
|
+
|
|
70
|
+
```ts
|
|
71
|
+
import { query, queryJSON, UrClient } from 'ur-agent/sdk'
|
|
72
|
+
|
|
73
|
+
const run = await query('Summarize the README', { maxTurns: 4 })
|
|
74
|
+
const client = new UrClient({ cwd: process.cwd(), model: 'qwen2.5-coder:7b' })
|
|
75
|
+
const data = await client.queryJSON<{ files: string[] }>(
|
|
76
|
+
'Return JSON with the relevant files',
|
|
77
|
+
)
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
/sdk info # show headless patterns (spawn `ur -p`, stream-json protocol, MCP serve)
|
|
82
|
+
/sdk init # scaffold TypeScript + Python SDK example projects
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The npm package publishes a typed `ur-agent/sdk` subpath for ESM and CommonJS.
|
|
86
|
+
It is a dependency-free **subprocess wrapper**: each `query` launches the
|
|
87
|
+
installed `ur -p`, buffers stdout, and returns
|
|
88
|
+
`{ok,text,raw,exitCode,stderr}`. It therefore inherits the child CLI's
|
|
89
|
+
permissions, configuration, MCP setup, and model routing; it is not an
|
|
90
|
+
in-process model API. `queryJSON` returns `null` on a nonzero child result or
|
|
91
|
+
invalid final JSON. `UrClient` merges default and per-call environment maps,
|
|
92
|
+
and an explicit `model` wins over an `UR_MODEL` entry in either map.
|
|
93
|
+
|
|
94
|
+
`outputFormat: 'stream-json'` automatically supplies the CLI-required
|
|
95
|
+
`--verbose`; `parseResultText` selects the terminal result from NDJSON even
|
|
96
|
+
when lifecycle events follow it. The current API still buffers the stream and
|
|
97
|
+
preserves the full NDJSON in `raw`; it does not expose an event iterator.
|
|
98
|
+
Empty prompts, non-positive/non-integer `maxTurns` or `timeoutMs`, and unknown
|
|
99
|
+
output formats reject before spawning.
|
|
100
|
+
|
|
101
|
+
`src/entrypoints/agentSdkTypes.ts` remains an internal structured-protocol type
|
|
102
|
+
barrel, not this runtime wrapper. `/sdk init` scaffolds runnable TypeScript and
|
|
103
|
+
Python subprocess examples. `ur mcp serve` exposes UR over MCP; `ur a2a serve`
|
|
104
|
+
and `ur acp serve` are the shipped HTTP-facing agent protocols. The separate
|
|
105
|
+
direct-connect `ur server` implementation is behind the compile-time
|
|
106
|
+
`DIRECT_CONNECT` feature and is absent from the standard npm bundle.
|
|
107
|
+
|
|
108
|
+
## CI loop (`/ci-loop`, alias `/heal`)
|
|
109
|
+
|
|
110
|
+
Run a command, let the agent fix failures, rerun until green — or prove cannot-fix with
|
|
111
|
+
command evidence (`src/services/agents/ciLoop.ts`):
|
|
112
|
+
```
|
|
113
|
+
/ci-loop --command "bun test" --max-attempts 3
|
|
114
|
+
/ci-loop --command "bun test" --cwd ./packages/app
|
|
115
|
+
/ci-loop --command "npm run build" --commit --push
|
|
116
|
+
/ci-loop --from-log ci-output.log # start from an existing failure log
|
|
117
|
+
/ci-loop --dry-run --json
|
|
118
|
+
```
|
|
119
|
+
Flags `--allow-generated`, `--allow-delete` widen what the fixer may touch.
|
|
120
|
+
`--commit` and `--push` are explicit opt-ins; the default run publishes
|
|
121
|
+
nothing. The result always prints the actual working directory. Failure
|
|
122
|
+
summaries retain nearby assertion and stack context while excluding passing
|
|
123
|
+
test names that merely contain words such as "failed". A "No tests found"
|
|
124
|
+
failure stops after the first attempt without starting a fix agent; run from
|
|
125
|
+
the test root or pass `--cwd <path>`.
|
|
126
|
+
Runs inside `/devcontainer` target when configured (doc 12).
|
|
127
|
+
Script-facing status is fail-closed: a passing run exits 0;
|
|
128
|
+
failed/blocked/exhausted/cannot-fix (including dry-run's non-completed preview
|
|
129
|
+
result) exits 1; and invalid arguments, working directories, or seed-log paths
|
|
130
|
+
exit 2.
|
|
131
|
+
|
|
132
|
+
## Test-first loop (`/test-first`, aliases `/quality-loop`, `/tf-loop`)
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
/test-first detect # detect stack: compiler, test runner, linter
|
|
136
|
+
/test-first run --max-attempts 3
|
|
137
|
+
/test-first install --install-gates # edit-time verify gates (verifier projectGates)
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
## Webhook triggers (`/trigger`, alias `/mention`)
|
|
141
|
+
|
|
142
|
+
Parse a GitHub or Slack webhook payload and optionally launch a headless run
|
|
143
|
+
(`triggerBridge.ts`):
|
|
144
|
+
```
|
|
145
|
+
/trigger parse --file payload.json --source github --keyword /ur
|
|
146
|
+
/trigger run --file payload.json --dry-run --json
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Scheduled automations (`/automation`)
|
|
150
|
+
|
|
151
|
+
Cron-style project automations with host-scheduler installation (doc 08):
|
|
152
|
+
```
|
|
153
|
+
/automation create nightly --schedule "0 3 * * *" --prompt "run tests and summarize"
|
|
154
|
+
ur automation install --platform systemd --interval 300
|
|
155
|
+
ur automation run-due --now 2026-07-09T03:00:00Z
|
|
156
|
+
```
|
|
157
|
+
The source tree also contains `/loop` plus cron tools behind the compile-time
|
|
158
|
+
`AGENT_TRIGGERS` feature, and `/schedule` plus `RemoteTrigger` behind
|
|
159
|
+
`AGENT_TRIGGERS_REMOTE`. Neither feature is compiled into the standard npm
|
|
160
|
+
bundle, so environment variables alone cannot enable those two in-session
|
|
161
|
+
surfaces there.
|
|
162
|
+
|
|
163
|
+
## Eval harness (`/eval`, aliases `/evals`)
|
|
164
|
+
|
|
165
|
+
Public eval harness (`src/services/agents/evals.ts`) with project suites under
|
|
166
|
+
`.ur/evals/`:
|
|
167
|
+
```
|
|
168
|
+
/eval init # scaffold a suite
|
|
169
|
+
/eval list · /eval validate my-suite
|
|
170
|
+
/eval run my-suite --model llama3.3 --repeat 3
|
|
171
|
+
/eval report my-suite --dashboard
|
|
172
|
+
/eval compare my-suite model-a model-b
|
|
173
|
+
/eval route "which strategy for this suite?"
|
|
174
|
+
/eval leaderboard
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Cases run in fresh detached worktrees by default. `--no-isolate` is an
|
|
178
|
+
explicit opt-out. A case can add `expect.trajectory` rules for required,
|
|
179
|
+
forbidden, ordered, and successfully completed tools, plus tool-call,
|
|
180
|
+
failure, repetition, permission-denial, and turn limits. The
|
|
181
|
+
`EvalTrajectory` object stores only control-flow metadata: normalized tool
|
|
182
|
+
names, hashed/opaque call IDs, success flags, and counts. It does not retain
|
|
183
|
+
prompts, assistant prose, paths, tool inputs, or tool outputs.
|
|
184
|
+
|
|
185
|
+
The saved eval report is broader than the trajectory object: it keeps a
|
|
186
|
+
bounded terminal-output preview and may keep bounded, redacted test
|
|
187
|
+
stdout/stderr in metrics. Treat reports as potentially sensitive artifacts
|
|
188
|
+
despite trajectory redaction.
|
|
189
|
+
|
|
190
|
+
Use a saved report as a CI gate:
|
|
191
|
+
|
|
192
|
+
```
|
|
193
|
+
ur eval run starter
|
|
194
|
+
ur eval gate starter \
|
|
195
|
+
--min-pass-rate 1 \
|
|
196
|
+
--min-trajectory-score 0.9 \
|
|
197
|
+
--min-test-pass-rate 1
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Cost, duration, and baseline-regression ceilings are also supported. A
|
|
201
|
+
requested metric that is absent fails closed rather than being silently
|
|
202
|
+
skipped.
|
|
203
|
+
|
|
204
|
+
### Built-in benchmark suites (`benchmarkSuites/`)
|
|
205
|
+
`builtin-bug-fix` (off-by-one, null-guard, missing-await),
|
|
206
|
+
`builtin-refactor` (extract-function, rename-fields, remove-duplication),
|
|
207
|
+
`builtin-test-gen` (calc, string-utils, async),
|
|
208
|
+
`builtin-docker-repair` (base-image-typo, missing-cmd, cache-layer-order),
|
|
209
|
+
`builtin-ts-migrate` (add-types, null-types, module-types),
|
|
210
|
+
`builtin-py-package-repair` (missing-dep, missing-pyproject, entrypoint).
|
|
211
|
+
```
|
|
212
|
+
/eval builtin bug-fix --json
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### External benchmark adapters
|
|
216
|
+
`/eval bench <adapter>` plus npm scripts:
|
|
217
|
+
```
|
|
218
|
+
npm run benchmark:smoke | benchmark:local | benchmark:compare | benchmark:report
|
|
219
|
+
npm run benchmark:swe-bench-lite | benchmark:terminal-bench | benchmark:aider-polyglot
|
|
220
|
+
```
|
|
221
|
+
Results are stored under `benchmarks/results/<version>/` against
|
|
222
|
+
`benchmarks/result.schema.json`.
|
|
223
|
+
|
|
224
|
+
## Learning loop
|
|
225
|
+
|
|
226
|
+
`/learn run --reflect` mines `.ur/artifacts` + CI outcomes into
|
|
227
|
+
per-category/per-model success-rate stats and lessons. Learned statistics are
|
|
228
|
+
consulted automatically by eligible model-routing and escalation paths once
|
|
229
|
+
their evidence thresholds are met. `/learn apply` has a narrower effect: it
|
|
230
|
+
persists the best sufficiently sampled overall model as the escalation
|
|
231
|
+
policy's `oracle`; it does not directly rewrite arena or model-route policy
|
|
232
|
+
(doc 05).
|
|
233
|
+
|
|
234
|
+
## Frontier automation contracts
|
|
235
|
+
|
|
236
|
+
The following ten capabilities share a fail-closed design: untrusted task text
|
|
237
|
+
never becomes policy, publishing remains explicit, and outputs are bounded and
|
|
238
|
+
reviewable.
|
|
239
|
+
|
|
240
|
+
### 1. Managed cloud fan-out
|
|
241
|
+
|
|
242
|
+
```
|
|
243
|
+
ur cloud environments
|
|
244
|
+
ur cloud run "repair the parser race" \
|
|
245
|
+
--runner managed --environment <id> --attempts 3
|
|
246
|
+
ur cloud sync
|
|
247
|
+
ur cloud show <task-id>
|
|
248
|
+
ur cloud logs <task-id> --tail 200
|
|
249
|
+
ur cloud cancel <task-id>
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
Each candidate has an isolated managed session. UR persists bounded,
|
|
253
|
+
secret-redacted lifecycle state, cursors, branches, and logs under
|
|
254
|
+
`.ur/cloud/`. Managed selection is eligibility ordering, not comparative
|
|
255
|
+
quality judging: a candidate must terminate successfully, explicitly return
|
|
256
|
+
`PASS`, and expose a safe non-empty review branch. Eligible candidates are
|
|
257
|
+
ordered deterministically. UR does not fetch, merge, or apply a managed branch.
|
|
258
|
+
Cancellation is terminal and also cancels a remote session that completes its
|
|
259
|
+
start concurrently with the cancel request.
|
|
260
|
+
|
|
261
|
+
### 2. Live steering
|
|
262
|
+
|
|
263
|
+
```
|
|
264
|
+
ur cloud steer <task-id> \
|
|
265
|
+
--message "preserve the public API" --request-id review-1
|
|
266
|
+
ur bg steer <task-id> \
|
|
267
|
+
--message "also cover timeouts" --request-id timeout-1
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
Managed steering is accepted only while at least one candidate session is
|
|
271
|
+
active. Messages are bounded to 64 KiB. The request ID is reserved before
|
|
272
|
+
network delivery, persisted with a message digest, and deduplicated so retries
|
|
273
|
+
cannot deliver the same request twice; reusing an ID for different text is
|
|
274
|
+
rejected. Local background agents use a bounded inbox. Authenticated A2A
|
|
275
|
+
steering additionally requires task ownership and a running background task.
|
|
276
|
+
|
|
277
|
+
### 3. Evidence-backed learned playbooks
|
|
278
|
+
|
|
279
|
+
```
|
|
280
|
+
ur learn playbooks mine --min-runs 3
|
|
281
|
+
ur learn playbooks list --status candidate
|
|
282
|
+
ur learn playbooks show <id>
|
|
283
|
+
ur learn playbooks approve <id> --name parser-repair
|
|
284
|
+
ur learn playbooks run <id> --max-concurrency 2
|
|
285
|
+
ur learn playbooks reject <id> --reason "insufficient evidence"
|
|
286
|
+
ur learn playbooks disable <id>
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
Mining groups repeated successful run trajectories and requires command proof
|
|
290
|
+
plus a confidence floor. Secret-like, destructive, publishing, deployment,
|
|
291
|
+
and unsafe traces are excluded. A candidate cannot execute until explicit
|
|
292
|
+
approval materializes a validated normal workflow under `.ur/workflows/`.
|
|
293
|
+
Approval revalidates every evidence digest. Rejection is terminal. Disabling
|
|
294
|
+
an approved playbook verifies that the materialized workflow was not changed,
|
|
295
|
+
moves it to a private `.ur/learning/disabled/*.yaml.disabled` archive, and
|
|
296
|
+
prevents future runs.
|
|
297
|
+
|
|
298
|
+
### 4. Citation-validated task memory
|
|
299
|
+
|
|
300
|
+
```
|
|
301
|
+
ur context-pack remember --decision "Keep the parser streaming" \
|
|
302
|
+
--cite-file src/parser.ts --lines 20:48
|
|
303
|
+
ur context-pack remember --note "Regression passed" \
|
|
304
|
+
--cite-run <run-id>:manifest.json
|
|
305
|
+
ur context-pack remember --constraint "Keep the public API" \
|
|
306
|
+
--cite-user <session-id>:<message-id>
|
|
307
|
+
ur context-pack remember --note "Protocol requirement" \
|
|
308
|
+
--cite-web https://example.com/spec
|
|
309
|
+
ur context-pack memory revalidate
|
|
310
|
+
ur context-pack memory search --query "parser streaming"
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
File excerpts and run artifacts are captured with SHA-256 digests and safe
|
|
314
|
+
path/size checks. Resolution excludes rejected, superseded, missing, and stale
|
|
315
|
+
entries by default. User-message and web citations remain `unverifiable`
|
|
316
|
+
until their source is explicitly reopened; searching memory never performs a
|
|
317
|
+
network request. Prompt-facing memory is source-labelled and byte-bounded.
|
|
318
|
+
|
|
319
|
+
### 5. Patch-only Agentic CI
|
|
320
|
+
|
|
321
|
+
```
|
|
322
|
+
ur agent-ci init default
|
|
323
|
+
ur agent-ci validate default
|
|
324
|
+
ur agent-ci workflow default --force
|
|
325
|
+
ur agent-ci run default \
|
|
326
|
+
--event "$GITHUB_EVENT_PATH" \
|
|
327
|
+
--event-name "$GITHUB_EVENT_NAME" \
|
|
328
|
+
--output-dir "$RUNNER_TEMP/ur-agentic-ci"
|
|
329
|
+
```
|
|
330
|
+
|
|
331
|
+
The generated GitHub job has read-only permissions, uses commit-pinned actions,
|
|
332
|
+
checks out a trusted base with credentials disabled, and accepts issue-comment
|
|
333
|
+
tasks only from configured repository associations. Event JSON is read from a
|
|
334
|
+
bounded file and treated as untrusted data. The agent works in a detached
|
|
335
|
+
worktree and is instructed not to commit or publish. It receives no platform
|
|
336
|
+
write token, and the only deliverable is a reviewable patch artifact.
|
|
337
|
+
|
|
338
|
+
Path allow/deny rules, deletion policy (including rename sources), generated
|
|
339
|
+
files, self-review, guardrails, and the patch-size limit run before repository
|
|
340
|
+
verification commands. Changed paths come from exact NUL-delimited,
|
|
341
|
+
no-rename Git records, so tabs/newlines are data rather than delimiters;
|
|
342
|
+
malformed statuses, unsafe paths, and Git/parser failures block the run.
|
|
343
|
+
Checks receive an allow-listed environment, isolated home/temp directories,
|
|
344
|
+
and no provider, platform, package-manager, proxy, or user-config credentials.
|
|
345
|
+
UR snapshots the staged patch, unstaged diff, tracked/untracked status, and
|
|
346
|
+
index visibility flags immediately before and after checks. Any verifier
|
|
347
|
+
mutation blocks the run and suppresses the patch. On an unchanged tree, the
|
|
348
|
+
emitted hash-addressed patch is recaptured after verification and bound to
|
|
349
|
+
`verificationStateSha256`. The manifest and redacted check tails are the only
|
|
350
|
+
other outputs. Publishing requires a separate trusted job or human review.
|
|
351
|
+
|
|
352
|
+
### 6. Trajectory-aware eval gates
|
|
353
|
+
|
|
354
|
+
Trajectory constraints and the fail-closed `ur eval gate` behavior are
|
|
355
|
+
described in [Eval harness](#eval-harness-eval-aliases-evals). Keep trajectory
|
|
356
|
+
rules about observable control flow; never encode secrets or expected prompt
|
|
357
|
+
text in them.
|
|
358
|
+
|
|
359
|
+
### 7. Electron desktop QA
|
|
360
|
+
|
|
361
|
+
```
|
|
362
|
+
ur desktop-qa init
|
|
363
|
+
ur desktop-qa validate .ur/desktop-qa/fixtures/smoke.json
|
|
364
|
+
ur desktop-qa doctor
|
|
365
|
+
ur desktop-qa run .ur/desktop-qa/fixtures/smoke.json
|
|
366
|
+
```
|
|
367
|
+
|
|
368
|
+
Fixtures provide bounded click, fill, key, selection, checkbox, wait,
|
|
369
|
+
text/visibility assertion, and screenshot steps. The driver closes the
|
|
370
|
+
application on every path, redacts secret-like diagnostics, hashes evidence,
|
|
371
|
+
and exits non-zero on a failed assertion. Screenshot `redactSelectors` are
|
|
372
|
+
rendered as opaque masks. Raw video and trace data cannot guarantee those
|
|
373
|
+
masks, so validation refuses `recording.video` or `recording.trace` whenever
|
|
374
|
+
selector redaction is configured. To record raw video/trace, remove selector
|
|
375
|
+
redaction deliberately and treat the resulting artifact as sensitive.
|
|
376
|
+
Evidence persistence copies only bounded regular non-symlink files into the
|
|
377
|
+
artifact store and records their hashes. The loopback artifact server resolves
|
|
378
|
+
downloads inside that store, sends `private, no-store` and sandbox CSP headers,
|
|
379
|
+
serves only a small image/video MIME allow-list inline, and forces every other
|
|
380
|
+
declared type to `application/octet-stream` attachment delivery.
|
|
381
|
+
|
|
382
|
+
### 8. Durable side chats
|
|
383
|
+
|
|
384
|
+
```
|
|
385
|
+
/btw Why does this parser use a sentinel?
|
|
386
|
+
/btw continue <chat-id> What invariant does it protect?
|
|
387
|
+
/btw list
|
|
388
|
+
/btw show <chat-id>
|
|
389
|
+
/btw rename <chat-id> Parser notes
|
|
390
|
+
/btw close <chat-id>
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
Each question remains a one-turn, tool-free fork, so it does not block or alter
|
|
394
|
+
the main task. Exchanges are persisted atomically in private per-project
|
|
395
|
+
session storage, linked to the parent session/message, and protected by a
|
|
396
|
+
per-turn SHA-256 chain. Chats survive CLI restarts and support continuation,
|
|
397
|
+
rename, inspection, and close. Chat count, turn count, individual content, and
|
|
398
|
+
store size are bounded; closed chats cannot accept new turns, and cancellation
|
|
399
|
+
aborts the forked request.
|
|
400
|
+
|
|
401
|
+
### 9. Multi-repository workspace coordination
|
|
402
|
+
|
|
403
|
+
```
|
|
404
|
+
ur workspace init checkout
|
|
405
|
+
ur workspace add checkout api ../api --base main --verify "bun test"
|
|
406
|
+
ur workspace add checkout web ../web --base main --verify "bun test"
|
|
407
|
+
ur workspace task checkout api-contract --repo api \
|
|
408
|
+
--prompt "add the response field"
|
|
409
|
+
ur workspace task checkout web-client --repo web \
|
|
410
|
+
--prompt "consume the response field" --depends-on api-contract
|
|
411
|
+
ur workspace validate checkout
|
|
412
|
+
ur workspace run checkout --max-concurrency 4
|
|
413
|
+
ur workspace verify checkout
|
|
414
|
+
ur workspace pr-plan checkout
|
|
415
|
+
ur workspace rollback-plan checkout
|
|
416
|
+
```
|
|
417
|
+
|
|
418
|
+
Enrollment records the canonical repository root and a digest of its remote
|
|
419
|
+
identity. A validated dependency DAG controls execution order; dependencies
|
|
420
|
+
wait, while tasks that target the same repository serialize behind one writer.
|
|
421
|
+
Repositories keep independent base refs and isolated worktrees. Durable state
|
|
422
|
+
under `.ur/workspaces/` refuses resume after the spec changes. Verification
|
|
423
|
+
uses each repository's declared commands. PR and rollback operations only
|
|
424
|
+
print dependency-ordered plans; they execute no GitHub or destructive command.
|
|
425
|
+
|
|
426
|
+
### 10. Verified model-judged arena
|
|
427
|
+
|
|
428
|
+
```
|
|
429
|
+
ur arena "repair the cache race" --agents 3 \
|
|
430
|
+
--judge hybrid --judge-model <model> \
|
|
431
|
+
--verify "bun run typecheck" --verify "bun test"
|
|
432
|
+
```
|
|
433
|
+
|
|
434
|
+
Candidates run in detached worktrees and are eligible only with an explicit
|
|
435
|
+
`PASS`, a non-empty bounded patch, no blocking safety finding, and successful
|
|
436
|
+
verification. `deterministic`, `model`, and `hybrid` judging are available.
|
|
437
|
+
The one-turn model judge has no tools or session persistence and sees only
|
|
438
|
+
bounded, secret-redacted, anonymous eligible candidates. Oversized diffs and
|
|
439
|
+
invalid or out-of-set schema results produce no winner. `--apply` writes a
|
|
440
|
+
hash-addressed patch only after confirming that the original worktree is
|
|
441
|
+
still clean and at the exact base commit.
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# 11 — Integrations
|
|
2
|
+
|
|
3
|
+
Source of truth: `src/services/mcp/`, `src/entrypoints/{mcp,mcp2026,agUi}.ts`,
|
|
4
|
+
`src/services/agents/{acpStdio,acpServer,a2aProtocol,a2aServer,agUi}.ts`,
|
|
5
|
+
`src/services/agents/ideConfig.ts`, `extensions/{vscode-ur-inline-diffs,jetbrains-ur}/`,
|
|
6
|
+
`src/tools/BrowserTool/BrowserTool.ts`, `src/commands/{browser,browser-qa,chrome,desktop,voice}/`,
|
|
7
|
+
and `scripts/bundle.mjs`.
|
|
8
|
+
|
|
9
|
+
Availability terms used here:
|
|
10
|
+
|
|
11
|
+
- **Shipped** — present in the normal external npm build.
|
|
12
|
+
- **Conditional** — shipped, but hidden or inactive until its documented platform,
|
|
13
|
+
authentication, setting, dependency, or environment condition is satisfied.
|
|
14
|
+
- **Source-only** — implemented behind a Bun build feature that
|
|
15
|
+
`scripts/bundle.mjs` does not enable for the normal external build. Source
|
|
16
|
+
presence alone does not make that command available to npm users.
|
|
17
|
+
|
|
18
|
+
## MCP client
|
|
19
|
+
|
|
20
|
+
UR can consume MCP servers over `stdio`, SSE, streamable HTTP, and WebSocket.
|
|
21
|
+
The `mcp add` CLI supports `stdio`, `sse`, and `http`; WebSocket entries can be
|
|
22
|
+
loaded from a validated settings/config object.
|
|
23
|
+
|
|
24
|
+
Configuration can come from the `mcpServers` setting, project `.mcp.json`, or
|
|
25
|
+
one or more `--mcp-config` JSON files/strings. `--strict-mcp-config` ignores the
|
|
26
|
+
ordinary user/project MCP sources, but managed enterprise policy still applies.
|
|
27
|
+
|
|
28
|
+
```text
|
|
29
|
+
ur mcp add fs -- npx -y @modelcontextprotocol/server-filesystem /tmp
|
|
30
|
+
ur mcp add --transport http sentry https://mcp.sentry.dev/mcp
|
|
31
|
+
ur mcp add --transport http corridor https://app.corridor.dev/api/mcp \
|
|
32
|
+
--header "Authorization: Bearer …"
|
|
33
|
+
ur mcp add-json db '{"command":"pg-mcp","args":["--dsn","…"]}'
|
|
34
|
+
ur mcp list
|
|
35
|
+
ur mcp get fs
|
|
36
|
+
ur mcp remove fs
|
|
37
|
+
ur mcp add-from-ur-desktop
|
|
38
|
+
ur mcp reset-project-choices
|
|
39
|
+
/mcp
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Runtime behavior:
|
|
43
|
+
|
|
44
|
+
- Tool names use `mcp__<server>__<tool>`. Resources use
|
|
45
|
+
`ListMcpResources`/`ReadMcpResource`; server prompts may be registered as
|
|
46
|
+
slash commands.
|
|
47
|
+
- `allowedMcpServers`, `deniedMcpServers`, managed MCP policy, per-project
|
|
48
|
+
approval, and session enable/disable state filter which servers can connect.
|
|
49
|
+
- HTTP/SSE OAuth state is handled by `src/services/mcp/auth.ts`. `--client-secret`
|
|
50
|
+
prompts for a secret, while `MCP_CLIENT_SECRET` supplies it non-interactively.
|
|
51
|
+
XAA commands are registered only when the XAA runtime gate is enabled.
|
|
52
|
+
- Environment expansion and header-helper execution are supported by
|
|
53
|
+
`envExpansion.ts` and `headersHelper.ts`; those helpers do not bypass normal
|
|
54
|
+
MCP policy.
|
|
55
|
+
|
|
56
|
+
## UR server surfaces
|
|
57
|
+
|
|
58
|
+
| Surface | Availability and start command | Actual contract |
|
|
59
|
+
|---|---|---|
|
|
60
|
+
| MCP stdio | Shipped: `ur mcp serve` | Lists enabled built-in UR tools only. Input is schema-validated, normal permissions are rechecked, calls are bounded, and any operation needing an unavailable interactive approval fails closed. |
|
|
61
|
+
| MCP 2026 HTTP | Shipped: `ur mcp serve-http` | Bun HTTP `/mcp` adapter with negotiated Tasks/Apps metadata. Loopback may run without a token; an off-loopback bind requires `UR_MCP_HTTP_TOKEN`. `--allow-origin` entries are exact HTTP(S) origins. |
|
|
62
|
+
| ACP stdio | Shipped: `ur acp stdio` | Official-SDK-backed ACP v1 agent with persisted ACP sessions, new/load/list/delete/resume/close, prompt streaming, modes/config updates, MCP input, cancellation, and native `session/request_permission` requests. |
|
|
63
|
+
| UR HTTP JSON-RPC | Shipped: `ur acp serve` | UR-specific JSON-RPC at `/acp`; it is not the ACP wire protocol. Supports UR sessions, direct tool calls, and task methods. Off-loopback requires `--token` or `UR_ACP_TOKEN`. |
|
|
64
|
+
| A2A | Shipped: `ur a2a serve` | Negotiated A2A v1 routes, stable v0.3 JSON-RPC at `/a2a/jsonrpc`, and separate UR compatibility task routes. Off-loopback requires a static token or delegation secret. |
|
|
65
|
+
| AG-UI | Shipped: `ur ag-ui serve` | HTTP/SSE `/ag-ui` adapter with `/ag-ui/capabilities`. Loopback is the default; off-loopback requires `UR_AG_UI_TOKEN`. Browser origins are exact allow-list entries. |
|
|
66
|
+
| Direct-connect session server | Source-only (`DIRECT_CONNECT`) | `ur server` and `ur open` are not in the normal external bundle. |
|
|
67
|
+
| Remote-control bridge | Source-only (`BRIDGE_MODE`) | `ur remote-control`/`rc` and the bridge fast paths are not in the normal external bundle. |
|
|
68
|
+
| SSH remote runner | Source-only (`SSH_REMOTE`) | `ur ssh` is not in the normal external bundle. |
|
|
69
|
+
|
|
70
|
+
The stdio MCP limits are
|
|
71
|
+
`UR_MCP_MAX_CALLS_PER_MINUTE`, `UR_MCP_MAX_CONCURRENT_CALLS`,
|
|
72
|
+
`UR_MCP_TOOL_TIMEOUT_MS`, `UR_MCP_MAX_INPUT_CHARS`, and
|
|
73
|
+
`UR_MCP_MAX_OUTPUT_CHARS`. The HTTP adapter has request/rate/concurrency limits
|
|
74
|
+
under `UR_MCP_HTTP_*` and uses `UR_MCP_TOOL_TIMEOUT_MS` for underlying tool
|
|
75
|
+
calls.
|
|
76
|
+
|
|
77
|
+
The network agent adapters also enforce bounded requests and work:
|
|
78
|
+
|
|
79
|
+
- UR HTTP JSON-RPC uses `UR_ACP_*`.
|
|
80
|
+
- A2A uses `UR_A2A_*`; compatibility-route `skipPermissions` additionally
|
|
81
|
+
requires the static server token or a delegation token scoped to
|
|
82
|
+
`permissions:bypass`. The standard A2A protocol runner uses `dontAsk`, so an
|
|
83
|
+
unavailable interactive approval is denied.
|
|
84
|
+
- AG-UI uses `UR_AG_UI_*`, disables session persistence for adapter runs,
|
|
85
|
+
denies permission requests because it advertises no approval UI, and aborts
|
|
86
|
+
the child run when the stream is cancelled.
|
|
87
|
+
- ACP stdio is the exception: it relays UR permission decisions to the ACP
|
|
88
|
+
client's native request-permission channel and denies on cancellation/client
|
|
89
|
+
failure.
|
|
90
|
+
|
|
91
|
+
## IDE integration
|
|
92
|
+
|
|
93
|
+
`ur ide status|doctor|config` reports integration state. `/ide` provides the
|
|
94
|
+
interactive connection UI and inline-diff commands:
|
|
95
|
+
|
|
96
|
+
```text
|
|
97
|
+
/ide diff capture
|
|
98
|
+
/ide diff list
|
|
99
|
+
/ide diff show <id>
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
The shipped editor paths are deliberately different:
|
|
103
|
+
|
|
104
|
+
- The VS Code extension in `extensions/vscode-ur-inline-diffs/` spawns
|
|
105
|
+
`ur -p --output-format stream-json --verbose --permission-prompt-tool stdio`
|
|
106
|
+
for chat turns. It does **not** use ACP stdio.
|
|
107
|
+
- The experimental JetBrains plugin uses the loopback, UR-specific HTTP
|
|
108
|
+
`/acp` JSON-RPC service (`ur acp serve`), not ACP stdio.
|
|
109
|
+
- Editors with their own ACP client can launch `ur acp stdio`; this is a
|
|
110
|
+
supported generic ACP surface, but it is not the transport used by the two
|
|
111
|
+
bundled plugins above.
|
|
112
|
+
- `--ide` auto-connects to the legacy detected-IDE channel when exactly one
|
|
113
|
+
valid IDE is detected. LSP diagnostics are a separate integration under
|
|
114
|
+
`src/services/lsp/`.
|
|
115
|
+
|
|
116
|
+
## Browser surfaces
|
|
117
|
+
|
|
118
|
+
These three surfaces are not interchangeable:
|
|
119
|
+
|
|
120
|
+
1. `/browser <url|task>` is a **shipped advisory command**. It detects a
|
|
121
|
+
workspace Playwright installation and tells the user/model which path is
|
|
122
|
+
available; it does not navigate, click, type, or take a screenshot itself.
|
|
123
|
+
2. The `Browser` model tool is **conditional** on `UR_BROWSER_TOOL=1` or
|
|
124
|
+
`WEB_BROWSER_TOOL=1`. Its `fetch` action uses bounded plain HTTP. Interactive
|
|
125
|
+
`goto`, `click`, `type`, `evaluate`, and `screenshot` actions dynamically
|
|
126
|
+
require `playwright-core` plus an installed Chromium-compatible browser.
|
|
127
|
+
Every action asks through the normal permission engine.
|
|
128
|
+
3. `/chrome` is an interactive settings/onboarding UI for the Chrome extension
|
|
129
|
+
and its MCP/native-host bridge. It is unavailable in print/offline mode and
|
|
130
|
+
the current UI requires a UR subscription. `--chrome` and `--no-chrome`
|
|
131
|
+
select the integration for a session.
|
|
132
|
+
|
|
133
|
+
`/browser-qa` validates `.ur/browser-qa/*.json` fixtures. Its `run` action is a
|
|
134
|
+
five-second HTTP fetch smoke test that reports status/body size; it does not
|
|
135
|
+
launch Playwright, evaluate fixture assertions, or replay browser interactions.
|
|
136
|
+
|
|
137
|
+
## GitHub, Slack, desktop, and voice
|
|
138
|
+
|
|
139
|
+
- The `GitHub` tool, `/pr-comments`, `/review`, and `--from-pr` cover GitHub
|
|
140
|
+
workflows. `/trigger` parses GitHub/Slack webhook JSON and can explicitly
|
|
141
|
+
launch a headless `ur -p` run; it is not a resident webhook listener.
|
|
142
|
+
- `/install-slack-app` only opens the Slack Marketplace installation page and
|
|
143
|
+
records the click locally.
|
|
144
|
+
- `/desktop` (alias `/app`) is conditional on macOS or x64 Windows. It flushes
|
|
145
|
+
the current transcript and hands the session to an installed compatible UR
|
|
146
|
+
Desktop app; otherwise it offers the platform download.
|
|
147
|
+
- `/session` (alias `/remote`) is visible only when the current runtime is
|
|
148
|
+
already in remote mode. It displays the existing remote-session URL; it does
|
|
149
|
+
not create a remote session. `/remote-env` additionally requires a
|
|
150
|
+
subscription, the `allow_remote_sessions` policy, and network access.
|
|
151
|
+
- `/voice` is shipped because the external bundle enables `VOICE_MODE`, but is
|
|
152
|
+
conditional on the GrowthBook kill switch, a valid UR OAuth login, microphone
|
|
153
|
+
access, a recording utility, and interactive mode. It toggles streaming voice
|
|
154
|
+
input. `/speak` is a separate local OS text-to-speech command and does not
|
|
155
|
+
require voice input.
|
|
156
|
+
- `/buddy` is source-only because the normal bundle does not enable `BUDDY`.
|