@aryacode/dsh-plugin-arya-computer-use 1.0.5 → 1.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -8,7 +8,7 @@ Replaces [dsh-computer-use-win](https://github.com/Yu-tao-Li/dsh-computer-use-wi
8
8
 
9
9
  | wincu complaint | Arya Computer Use |
10
10
  | --- | --- |
11
- | Slow (PowerShell + UIA backend) | Native Rust engine: snapshot ~117 ms, ref action ~197 ms on a Windows devbox |
11
+ | Slow (PowerShell + UIA backend) | Native Rust engine: ~90 ms per call measured on this devbox (snapshot 90 ms, ref action 85 ms, `list-windows` 56 ms). Every CLI call costs ~85 ms of process start plus UIA init, so the number of calls, not the tool, sets the latency — `batch` runs up to 20 steps in one call (6 steps: 332 ms engine vs ~510 ms step by step) |
12
12
  | Pixel alignment off | Ref-based semantic UIA — coordinates are never the targeting mechanism; identical twins are refused, not mis-clicked |
13
13
  | Chinese UI/docs | English throughout |
14
14
  | (kept) DSH integration + setup UI | Settings page, chat step rows, takeover overlay, Esc kill, tool presets |
@@ -31,12 +31,12 @@ Replaces [dsh-computer-use-win](https://github.com/Yu-tao-Li/dsh-computer-use-wi
31
31
  | Phase | Deliverable | Status |
32
32
  | --- | --- | --- |
33
33
  | 0 | Evaluation spike: raw `agent-desktop mcp` row, drive Notepad/Explorer/Chrome — speed ✓, readable trees ✓, refs-not-pixels ✓ | ✅ accepted, spike removed |
34
- | 1 | Adapter MCP server: Lite 16 tools, per-conversation session lifecycle, headed toggle + foreground promotion, screenshot→file path, WinRT OCR, skill | ✅ built, installed, driven in-host |
34
+ | 1 | Adapter MCP server: Lite 22 tools, per-conversation session lifecycle, headed toggle + foreground promotion, screenshot→file path, WinRT OCR, skill | ✅ built, installed, driven in-host |
35
35
  | 2 | Takeover UI: window rim + banner + Esc kill, configurable ghost cursor, settings page, chat step rows, driver lock, English UI | ✅ built + verified live |
36
36
  | 3 | Web lane (`arya_browser` row), sign-in profiles, attach-to-your-Chrome, **wincu retirement** | planned |
37
37
  | 4 | Lazy preset, notifications/clipboard/trace, npm publish | planned |
38
38
 
39
- Lite 16 tool surface: `snapshot, find, get, is, click, set_value, type, press, scroll, list_windows, launch, wait, screenshot, ocr, batch, tools` (superset of wincu Lite 12; `set_value` fills a field through the accessibility value pattern without touching the pointer — **the headless text path, use it for text entry**; `type` synthesizes real keystrokes, which on Windows is physical input, so it moves the pointer and focuses the window; `tools` manages headed/overlay/session state).
39
+ Lite 22 tool surface: `snapshot, find, get, is, click, focus, double_click, right_click, expand, collapse, toggle, set_value, type, press, scroll, list_windows, launch, wait, screenshot, ocr, batch, tools` (superset of wincu Lite 12; `set_value` fills a field through the accessibility value pattern without touching the pointer — **the headless text path, use it for text entry**; `expand`/`collapse`/`toggle` are semantic UIA patterns and stay headless; `focus`, `double_click` and `right-click` have no headless path on Windows, so each one drives the real pointer for that single call and never latches the conversation; `type` likewise synthesizes real keystrokes, which on Windows is physical input; `tools` manages headed/overlay/session state).
40
40
 
41
41
  Full plan: [docs/PLAN.md](docs/PLAN.md).
42
42
 
@@ -47,7 +47,7 @@ package.json Cordis bundle manifest (dsh.bundle.patch) + pinned engine
47
47
  cordis.patch.yml rows: arya-computer-use (Host), mcp-arya-desktop, mcp-arya-browser
48
48
  index.js Host plugin entry (settings, driver lock, takeover mgmt, status route)
49
49
  client.js Settings page + chat step rows (client plugin)
50
- mcp/adapter-server.mjs adapter MCP server (Lite 16 tool surface)
50
+ mcp/adapter-server.mjs adapter MCP server (Lite 22 tool surface)
51
51
  scripts/windows-takeover.cs window rim + banner + progress + ghost cursor + global Esc
52
52
  scripts/ocr.ps1 WinRT OCR helper (Phase 1)
53
53
  skills/arya-desktop/ observe-act loop guide (Phase 1)
@@ -79,8 +79,9 @@ The published package is `@aryacode/dsh-plugin-arya-computer-use` (npm, public).
79
79
  | Banner text | `Arya is controlling your computer` | One line, 1–120 characters |
80
80
  | Real mouse and keyboard | off | Opted-in headed input for one conversation: **every** action then drives the real pointer and keyboard. It releases on `session_end` and expires after 5 minutes. Text entry never needs it — `set_value` is headless; `type` is physical on its own. |
81
81
  | One conversation drives at a time | on | Refuses desktop tools in other conversations while a drive is live |
82
- | Stop shortcut | Esc | Global kill switch; synthetic keystrokes are ignored, so it needs a real keypress |
83
- | Tool preset | `lite` | `lite` lists all 16 desktop tools; `lazy` starts with tools/snapshot/click and reveals the rest on demand |
82
+ | Progress panel detail | `summary` | What the on-screen step panel prints: `summary` = short action lines, `source` = the assistant's raw text stream (the old default — it paints model text on your desktop) |
83
+ | Stop shortcut | Esc | Global kill switch; synthetic keystrokes are ignored, so it needs a real keypress. It now also gates the driver: desktop input tools return `COMPUTER_USE_STOPPED_BY_USER` for that turn (reads keep working) |
84
+ | Tool preset | `lite` | `lite` lists all 22 desktop tools; `lazy` starts with tools/snapshot/click and reveals the rest on demand |
84
85
  | Tool surface (web) | `core` | agent-browser tool profile: core 29, network 9, state 27, debug 40, tabs 13, react 8, mobile 15, webmcp 4, all 156 |
85
86
  | Chrome profile / Session | – / `arya` | Reuse a Chrome login state and name the browser session |
86
87
  | Connect to my browser | – | `--cdp` port or URL to attach to a Chrome you already have open |
@@ -110,7 +111,7 @@ repair them itself at start-up - the status route reports the outcome under `mcp
110
111
 
111
112
  ## Tool presets
112
113
 
113
- - `lite` (default) — all 16 desktop tools listed up front.
114
+ - `lite` (default) — all 22 desktop tools listed up front.
114
115
  - `lazy` — starts with `tools`, `snapshot`, `click`; everything else appears after `tools {operation:"load", names:[...]}`. The adapter advertises `tools.listChanged` and emits `notifications/tools/list_changed`, which DSH's MCP client re-syncs (`listChanged.onChanged → refreshTools`), so the new tools become callable immediately.
115
116
 
116
117
  ## License
@@ -65,10 +65,71 @@ const TARGET_FILE = join(PLUGIN_DIR, 'control', 'target.json');
65
65
  // The host takeover lane persists its paused/stopped series here. The adapter
66
66
  // watches it so the user's Esc stop also drops the physical-input latch.
67
67
  const TAKEOVER_STATE_FILE = join(PLUGIN_DIR, 'control', 'state.json');
68
+ // The host publishes the current turn's activity lease here. The adapter reads it
69
+ // to recompute the same series key the takeover lane blocks a stopped series on.
70
+ const ACTIVITY_FILE = join(PLUGIN_DIR, 'control', 'activity.json');
68
71
 
69
72
  const VERSION = '0.1.0';
70
73
  const MAX_BATCH = 20;
71
74
 
75
+ // The engine's own batch runs entries in one process, through the same policy
76
+ // checks and dispatch as the CLI. Entry keys are the command's own field names: a
77
+ // positional is named after itself, a flag drops `--` and writes `-` as `_`
78
+ // (skills/agent-desktop/references/commands-system.md:170). Unknown keys are
79
+ // rejected, so each action declares the keys it forwards.
80
+ const NATIVE_BATCH_COMMANDS = new Map([
81
+ ['snapshot', 'snapshot'], ['find', 'find'], ['get', 'get'], ['is', 'is'],
82
+ ['click', 'click'], ['set_value', 'set-value'], ['press', 'press'],
83
+ ['scroll', 'scroll'], ['wait', 'wait'], ['focus', 'focus'],
84
+ ['double_click', 'double-click'], ['right_click', 'right-click'],
85
+ ['expand', 'expand'], ['collapse', 'collapse'], ['toggle', 'toggle'],
86
+ ]);
87
+ // The engine's --headed is global and applies to every batch entry, so a step
88
+ // that needs physical input cannot share one native batch with semantic steps:
89
+ // those run through the adapter one at a time, each with its own scoped flag.
90
+ const LOOP_BATCH_ACTIONS = new Set(['type', 'screenshot', 'ocr', 'focus', 'double_click', 'right_click']);
91
+ const BATCH_ARG_KEYS = {
92
+ snapshot: ['app', 'window_id', 'compact', 'skeleton', 'interactive_only', 'max_depth', 'root', 'timeout_ms'],
93
+ find: ['app', 'window_id', 'name', 'role', 'text', 'value', 'native_id', 'description', 'first', 'last', 'nth', 'limit', 'count', 'exact', 'root', 'timeout_ms'],
94
+ get: ['ref_id', 'property'],
95
+ is: ['ref_id', 'property'],
96
+ click: ['ref_id', 'timeout_ms'],
97
+ focus: ['ref_id', 'timeout_ms'],
98
+ double_click: ['ref_id', 'timeout_ms'],
99
+ right_click: ['ref_id', 'timeout_ms'],
100
+ expand: ['ref_id', 'timeout_ms'],
101
+ collapse: ['ref_id', 'timeout_ms'],
102
+ toggle: ['ref_id', 'timeout_ms'],
103
+ set_value: ['ref_id', 'value', 'timeout_ms'],
104
+ press: ['combo', 'app', 'window_id', 'force'],
105
+ scroll: ['ref_id', 'direction', 'amount', 'timeout_ms'],
106
+ wait: ['ms', 'app', 'window', 'element', 'text', 'timeout'],
107
+ };
108
+
109
+ function batchEntryArgs(action, args) {
110
+ const entry = {};
111
+ for (const key of BATCH_ARG_KEYS[action] ?? []) {
112
+ const value = action === 'wait' && key === 'timeout' ? (args.timeout ?? args.timeout_ms) : args[key];
113
+ if (value === undefined || value === null) continue;
114
+ entry[key] = value;
115
+ }
116
+ return entry;
117
+ }
118
+
119
+ // A batch entry keeps its payload — a find step must yield refs — but a snapshot
120
+ // tree is the largest thing the engine can return, and the batch cap allows 4 MB.
121
+ function trimBatchData(data) {
122
+ if (!data || !Array.isArray(data.results)) return data;
123
+ return {
124
+ ...data,
125
+ results: data.results.map((entry) => (
126
+ entry?.command === 'snapshot' && entry.data?.tree
127
+ ? { ...entry, data: { ...entry.data, tree: undefined, tree_omitted: true } }
128
+ : entry
129
+ )),
130
+ };
131
+ }
132
+
72
133
  const state = {
73
134
  sessionId: `arya-${randomBytes(4).toString('hex')}`,
74
135
  sessionStarted: false,
@@ -139,6 +200,46 @@ setInterval(() => {
139
200
  }
140
201
  }, 1000).unref();
141
202
 
203
+ /* ---------------------------------------------------------- user stop */
204
+
205
+ // Esc lives in the host takeover lane: it blocks the series it was acting for,
206
+ // keyed by [sessionId, turn] from the activity lease (mcp/takeover.mjs:111,184).
207
+ // The adapter is a separate process, so it recomputes that key and refuses every
208
+ // input action for a blocked series. The banner says "Esc Stop"; until now the
209
+ // driver never honoured it. Reads stay available — the user asked to stop being
210
+ // driven, not to be observed. A new user turn is a new key, so the block releases
211
+ // exactly as it does in the host lane, with no settings change.
212
+ const STOPPED_BY_USER = 'COMPUTER_USE_STOPPED_BY_USER: The user stopped this operation series with Esc or their custom shortcut. Do not retry desktop input in this turn. A new user turn, including a request to continue, can use the desktop automatically; no settings change is required.';
213
+ const INPUT_TOOLS = new Set(['click', 'set_value', 'type', 'press', 'scroll', 'launch', 'focus', 'double_click', 'right_click', 'expand', 'collapse', 'toggle', 'batch']);
214
+
215
+ function stoppedSeriesMessage() {
216
+ try {
217
+ const activity = JSON.parse(readFileSync(ACTIVITY_FILE, 'utf8'));
218
+ if (!(activity.active === true && Number.isFinite(activity.expiresAt) && activity.expiresAt > Date.now())) return null;
219
+ const saved = JSON.parse(readFileSync(TAKEOVER_STATE_FILE, 'utf8'));
220
+ const key = JSON.stringify([activity.sessionId, activity.turn]);
221
+ if (saved?.blockedSeries && Object.prototype.hasOwnProperty.call(saved.blockedSeries, key)) return saved.blockedSeries[key];
222
+ return saved?.paused === true && saved?.stoppedSeriesKey === key ? (saved.pauseMessage || STOPPED_BY_USER) : null;
223
+ } catch {
224
+ return null; // no lease published yet: nothing has been stopped
225
+ }
226
+ }
227
+
228
+ function stoppedResult(name, message) {
229
+ return envelopeResult({
230
+ version: '2.6',
231
+ ok: false,
232
+ command: name,
233
+ error: {
234
+ code: 'POLICY_DENIED',
235
+ message,
236
+ suggestion: 'Do not retry desktop input in this turn. Report that the user stopped the takeover; a new user turn can drive the desktop again.',
237
+ details: { kind: 'stopped_by_user' },
238
+ disposition: { delivery: 'not_delivered', retry: 'safe' },
239
+ },
240
+ });
241
+ }
242
+
142
243
  /* ------------------------------------------------------- overlay target */
143
244
 
144
245
  // The overlay frames the app window that is being driven and puts the ghost
@@ -209,7 +310,10 @@ async function noteTarget(name, args, result) {
209
310
  if (name === 'list_windows') rememberWindows(data);
210
311
  if (name === 'snapshot') rememberWindows(data);
211
312
  if (args?.window_id) state.windowId = args.window_id;
212
- if (!state.windowId) return;
313
+ // target.json feeds the rim and the ghost cursor only. With the overlay off,
314
+ // skip the CLI calls they cost (element bounds, sometimes a window list):
315
+ // ~85-140 ms per action spent on presentation nobody is looking at.
316
+ if (!effectiveOverlay() || !state.windowId) return;
213
317
  const window = await windowRectFor(state.windowId);
214
318
  const ref = args?.ref_id ?? data?.match?.ref_id ?? null;
215
319
  let pointer = ref ? await elementPoint(ref) : null;
@@ -351,6 +455,12 @@ const TOOLS = [
351
455
  tool('get', 'Read one control: property text|value|title|bounds|role|states.', { ref_id: str, property: str }),
352
456
  tool('is', 'Check one control: visible|enabled|checked|focused|expanded|selected.', { ref_id: str, property: str }),
353
457
  tool('click', 'Press a control via its semantic action (no pixels). Verify by re-snapshot.', { ref_id: str, timeout_ms: int }),
458
+ tool('focus', 'Focus a control through UIA SetFocus. On Windows SetFocus takes the desktop foreground window, so the engine refuses it headless: the adapter runs it as physical input for this one call (same rule as type) without latching the conversation. For text entry prefer set_value, which focuses the field itself.', { ref_id: str, timeout_ms: int }),
459
+ tool('double_click', 'Double-click a control. Windows has no headless double-click, so this drives the real pointer for this one call (same rule as type). Verify by re-snapshot.', { ref_id: str, timeout_ms: int }),
460
+ tool('right_click', 'Open a control\u2019s context menu. Windows has no headless right-click, so this drives the real pointer for this one call (same rule as type). The menu is a separate surface: snapshot it after.', { ref_id: str, timeout_ms: int }),
461
+ tool('expand', 'Expand an expander, tree item, or dropdown via ExpandCollapsePattern (no pixels).', { ref_id: str, timeout_ms: int }),
462
+ tool('collapse', 'Collapse an expander or tree item via ExpandCollapsePattern (no pixels).', { ref_id: str, timeout_ms: int }),
463
+ tool('toggle', 'Flip a checkbox or toggle button via TogglePattern. Prefer it over click: it reports the resulting state, and click on a checked control can be refused.', { ref_id: str, timeout_ms: int }),
354
464
  tool('set_value', 'Set a control value via UIA SetValue (no keystrokes, no pointer movement). This is the headless text path — use it first for any text entry. Some edits reject it — only then use type.', { ref_id: str, value: str, timeout_ms: int }),
355
465
  tool('type', 'Type real keystrokes into a text target. On Windows keystroke synthesis is physical: this moves the pointer and briefly focuses the window. Prefer set_value (headless UIA write) for text entry.', { ref_id: str, text: str, interval_ms: int, timeout_ms: int }),
356
466
  tool('press', 'Send a key chord (e.g. ctrl+s) to an app/window. Foreground queue: for non-frontmost apps expect delivered_unverified.', { combo: str, app: str, window_id: str, force: bool }),
@@ -360,7 +470,7 @@ const TOOLS = [
360
470
  tool('wait', 'Pause (ms) or wait for a condition: window/app/text/element before acting. Prevents acting on half-settled windows.', { ms: int, app: str, window: str, element: str, text: str, timeout_ms: int }),
361
471
  tool('screenshot', 'Capture a window/screen PNG to a file; returns the file path. Use for visual evidence only — trees are the primary signal.', { app: str, window_id: str, screen: int }),
362
472
  tool('ocr', 'Read text from a screenshot via Windows OCR (for canvas/games/no-a11y-tree UIs). Returns lines with SCREEN coordinates (not refs). query filters lines.', { app: str, window_id: str, screen: int, query: str }),
363
- tool('batch', `Run 1-${MAX_BATCH} planned actions on one target; stops on first error, then observes once. Each step: {action, arguments}. Actions: snapshot,find,get,is,click,set_value,type,press,scroll,wait.`, { commands_json: str, stop_on_error: bool, timeout_ms: int }, ['commands_json']),
473
+ tool('batch', `Run 1-${MAX_BATCH} planned actions on one target; stops on first error, then observes once. Each step: {action, arguments}. Native single-process actions (fast path): snapshot,find,get,is,click,set_value,press,scroll,wait,focus,double_click,right_click,expand,collapse,toggle. type,screenshot,ocr run step by step.`, { commands_json: str, stop_on_error: bool, timeout_ms: int }, ['commands_json']),
364
474
  tool('tools',
365
475
  'Manage the adapter: operation list|describe (tool surface + state), load {names} (reveal tools when the lazy preset is on), headed {on} (per-conversation headed toggle — use when headless is not an option: cursor-moving input, notifications, shell surfaces), overlay {on} (presentation cursor overlay), session_end {operation} (end this conversation\'s driver session).',
366
476
  { operation: str, name: str, names: { type: 'array', items: str }, on: bool }),
@@ -388,6 +498,10 @@ function flagArgs(pairs) {
388
498
  }
389
499
 
390
500
  async function callTool(name, args = {}) {
501
+ if (INPUT_TOOLS.has(name)) {
502
+ const stopped = stoppedSeriesMessage();
503
+ if (stopped) return stoppedResult(name, stopped);
504
+ }
391
505
  switch (name) {
392
506
  case 'snapshot':
393
507
  return enqueue(async () => {
@@ -419,6 +533,24 @@ async function callTool(name, args = {}) {
419
533
  await ensureSession();
420
534
  return parseEnvelope(await runCli(['click', args.ref_id, ...flagArgs([['--timeout-ms', args.timeout_ms]])]));
421
535
  });
536
+ case 'expand':
537
+ case 'collapse':
538
+ case 'toggle':
539
+ return enqueue(async () => {
540
+ await ensureSession();
541
+ return parseEnvelope(await runCli([NATIVE_BATCH_COMMANDS.get(name), args.ref_id, ...flagArgs([['--timeout-ms', args.timeout_ms]])]));
542
+ });
543
+ case 'focus':
544
+ case 'double_click':
545
+ case 'right_click':
546
+ return enqueue(async () => {
547
+ await ensureSession();
548
+ // Engine fact on Windows (platform skill): right-click, double-click and
549
+ // focus are physical or take the desktop foreground, so they return
550
+ // POLICY_DENIED headless. --headed is scoped to THIS call, exactly like
551
+ // `type`; the conversation is never latched into physical input.
552
+ return parseEnvelope(await runCli([NATIVE_BATCH_COMMANDS.get(name), args.ref_id, ...flagArgs([['--timeout-ms', args.timeout_ms]])], { headed: true }));
553
+ });
422
554
  case 'set_value':
423
555
  return enqueue(async () => parseEnvelope(await runCli(['set-value', args.ref_id, String(args.value ?? ''), ...flagArgs([['--timeout-ms', args.timeout_ms]])])));
424
556
  case 'type':
@@ -467,13 +599,35 @@ async function callTool(name, args = {}) {
467
599
  if (!Array.isArray(commands) || commands.length < 1 || commands.length > MAX_BATCH) {
468
600
  return { content: [{ type: 'text', text: `batch needs 1-${MAX_BATCH} steps (got ${Array.isArray(commands) ? commands.length : 'non-array'}).` }], isError: true };
469
601
  }
602
+ const actions = commands.map((step) => String(step?.action || ''));
603
+ const unknown = actions.filter((action) => !NATIVE_BATCH_COMMANDS.has(action) && !LOOP_BATCH_ACTIONS.has(action));
604
+ if (unknown.length > 0) {
605
+ return { content: [{ type: 'text', text: JSON.stringify({ ok: false, unknown, supported: [...NATIVE_BATCH_COMMANDS.keys(), ...LOOP_BATCH_ACTIONS] }) }], isError: true };
606
+ }
607
+ if (!actions.some((action) => LOOP_BATCH_ACTIONS.has(action))) {
608
+ // One engine process for the whole batch. Every CLI call costs ~85 ms of
609
+ // spawn plus UIA cold start, and the engine runs the same policy checks and
610
+ // dispatch per entry, so this is ~1.7x faster than the loop and keeps each
611
+ // entry's payload — a find step yields refs instead of "ok".
612
+ const entries = commands.map((step) => {
613
+ const action = String(step?.action || '');
614
+ return { command: NATIVE_BATCH_COMMANDS.get(action), args: batchEntryArgs(action, step?.arguments || {}) };
615
+ });
616
+ return enqueue(async () => {
617
+ await ensureSession();
618
+ const cli = ['batch', JSON.stringify(entries)];
619
+ if (args.stop_on_error !== false) cli.push('--stop-on-error');
620
+ if (args.timeout_ms !== undefined && args.timeout_ms !== null) cli.push('--timeout-ms', String(args.timeout_ms));
621
+ const envelope = parseEnvelope(await runCli(cli));
622
+ if (envelope.ok === true) envelope.data = trimBatchData(envelope.data);
623
+ return envelopeResult(envelope);
624
+ });
625
+ }
626
+ // type promotes the window to the foreground first, and screenshot/ocr own an
627
+ // adapter-side file path, so a batch containing them runs step by step.
470
628
  const results = [];
471
629
  for (const step of commands) {
472
630
  const action = String(step?.action || '');
473
- if (!TOOLS_BY_NAME.has(action) || action === 'batch' || action === 'tools') {
474
- results.push({ action, error: `unsupported step action: ${action}` });
475
- break;
476
- }
477
631
  const out = await callTool(action, step?.arguments || {});
478
632
  const envelope = out.structuredContent ?? out;
479
633
  results.push({ action, ok: envelope.ok === true, summary: summarizeEnvelope(envelope) });
@@ -565,6 +719,13 @@ function summarizeEnvelope(envelope) {
565
719
  const d = envelope.data ?? {};
566
720
  const bits = [];
567
721
  if (d.ref_count !== undefined) bits.push(`refs ${d.ref_count}`);
722
+ // A step that locates something is useless without the ref it found, and a
723
+ // read step without its value is the same. Report the payload, not just "ok".
724
+ if (d.match?.ref_id) bits.push(`ref ${d.match.ref_id}${d.match.name ? ` (${d.match.name})` : ''}`);
725
+ if (Array.isArray(d.matches) && d.matches.length) bits.push(`refs ${d.matches.slice(0, 8).map((m) => m.ref_id).join(',')}`);
726
+ if (d.snapshot_id) bits.push(`snapshot ${d.snapshot_id}`);
727
+ if (d.value !== undefined) bits.push(`value ${JSON.stringify(d.value)}`);
728
+ if (d.result !== undefined) bits.push(`result ${JSON.stringify(d.result)}`);
568
729
  if (d.window?.id) bits.push(`window ${d.window.id}`);
569
730
  if (d.target?.process?.pid) bits.push(`pid ${d.target.process.pid}`);
570
731
  if (d.disposition?.delivery) bits.push(`delivery ${d.disposition.delivery}`);
@@ -623,6 +784,7 @@ const INSTRUCTIONS = [
623
784
  'Verify stateful actions by re-read; delivered_verified = confirmed, delivered_unverified retry:unsafe = do not blind-repeat.',
624
785
  'On ACTION_FAILED post_action_verification the action landed: read data.post_state, do NOT retry.',
625
786
  'This host must run elevated (Administrator); lease_directory_untrusted means restart AryaCode as Administrator.',
787
+ 'Esc (or the stop shortcut) stops the series: input tools then return COMPUTER_USE_STOPPED_BY_USER for this turn. Reads still work; do not retry input until a new user turn.',
626
788
  'Destructive actions (delete, send, pay, close unsaved) require user confirmation first.',
627
789
  'No accessibility tree (games/canvas): use ocr for text and headed mouse coordinates for input.',
628
790
  ].join('\n');
package/mcp/settings.mjs CHANGED
@@ -3,7 +3,10 @@
3
3
  // switches for the adapter (headedDefault, overlayDefault, driverLockEnabled).
4
4
 
5
5
  export const DEFAULT_SETTINGS = Object.freeze({
6
- progressOverlayEnabled: true, progressOverlayMode: 'source',
6
+ // 'source' paints the assistant's raw text stream on the user's desktop. The
7
+ // default is the condensed lane; 'source' stays available for anyone who wants
8
+ // the verbatim panel.
9
+ progressOverlayEnabled: true, progressOverlayMode: 'summary',
7
10
  progressOverlayOpacity: 60, progressOverlayWidth: 360, progressOverlayHeight: 240, progressOverlayPosition: 'top-right', progressOverlayX: 80, progressOverlayY: 80, progressOverlayLocked: true, progressLockShortcut: '', progressPreviewToken: 0,
8
11
  indicatorEnabled: true, cursorIndicatorEnabled: true, remoteIndicatorVisible: true, indicatorText: 'Arya is controlling your computer', pointerLabel: 'Arya',
9
12
  indicatorPosition: 'top', indicatorIdleMs: 800, stopShortcut: '',
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@aryacode/dsh-plugin-arya-computer-use",
3
- "version": "1.0.5",
3
+ "version": "1.0.6",
4
4
  "description": "Computer use for DSH/AryaCode: agent-desktop UIA engine, agent-browser web lane, takeover UI with a ghost cursor",
5
5
  "license": "MIT",
6
6
  "type": "module",