agent-device 0.2.4 → 0.2.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/README.md +41 -4
  2. package/dist/src/bin.js +26 -21
  3. package/dist/src/daemon.js +9 -8
  4. package/ios-runner/AgentDeviceRunner/AgentDeviceRunner.xcodeproj/project.pbxproj +2 -0
  5. package/ios-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift +16 -0
  6. package/package.json +3 -2
  7. package/skills/agent-device/SKILL.md +22 -6
  8. package/skills/agent-device/references/session-management.md +9 -0
  9. package/skills/agent-device/references/snapshot-refs.md +18 -5
  10. package/skills/agent-device/references/video-recording.md +2 -2
  11. package/src/cli.ts +6 -0
  12. package/src/core/__tests__/capabilities.test.ts +67 -0
  13. package/src/core/capabilities.ts +49 -0
  14. package/src/core/dispatch.ts +29 -118
  15. package/src/daemon/__tests__/is-predicates.test.ts +68 -0
  16. package/src/daemon/__tests__/selectors.test.ts +128 -0
  17. package/src/daemon/__tests__/session-routing.test.ts +108 -0
  18. package/src/daemon/__tests__/session-selector.test.ts +64 -0
  19. package/src/daemon/__tests__/session-store.test.ts +95 -0
  20. package/src/daemon/__tests__/snapshot-processing.test.ts +47 -0
  21. package/src/daemon/action-utils.ts +29 -0
  22. package/src/daemon/app-state.ts +66 -0
  23. package/src/daemon/context.ts +36 -0
  24. package/src/daemon/device-ready.ts +13 -0
  25. package/src/daemon/handlers/__tests__/find.test.ts +99 -0
  26. package/src/daemon/handlers/__tests__/replay-heal.test.ts +364 -0
  27. package/src/daemon/handlers/__tests__/snapshot.test.ts +128 -0
  28. package/src/daemon/handlers/find.ts +304 -0
  29. package/src/daemon/handlers/interaction.ts +510 -0
  30. package/src/daemon/handlers/parse-utils.ts +8 -0
  31. package/src/daemon/handlers/record-trace.ts +154 -0
  32. package/src/daemon/handlers/session.ts +732 -0
  33. package/src/daemon/handlers/snapshot.ts +396 -0
  34. package/src/daemon/is-predicates.ts +46 -0
  35. package/src/daemon/selectors.ts +423 -0
  36. package/src/daemon/session-routing.ts +22 -0
  37. package/src/daemon/session-selector.ts +39 -0
  38. package/src/daemon/session-store.ts +275 -0
  39. package/src/daemon/snapshot-processing.ts +127 -0
  40. package/src/daemon/types.ts +55 -0
  41. package/src/daemon.ts +66 -1592
  42. package/src/platforms/ios/index.ts +0 -62
  43. package/src/platforms/ios/runner-client.ts +2 -0
  44. package/src/utils/args.ts +19 -10
  45. package/src/utils/interactors.ts +102 -16
  46. package/src/utils/snapshot.ts +1 -0
package/README.md CHANGED
@@ -33,9 +33,11 @@ npx agent-device open SampleApp
33
33
 
34
34
  ## Quick Start
35
35
 
36
+ Use refs for agent-driven exploration and normal automation flows.
37
+
36
38
  ```bash
37
39
  agent-device open Contacts --platform ios # creates session on iOS Simulator
38
- agent-device snapshot
40
+ agent-device snapshot
39
41
  agent-device click @e5
40
42
  agent-device fill @e6 "John"
41
43
  agent-device fill @e7 "Doe"
@@ -75,7 +77,7 @@ Coordinates:
75
77
  ## Command Index
76
78
  - `open`, `close`, `home`, `back`, `app-switcher`
77
79
  - `snapshot`, `find`, `get`
78
- - `click`, `focus`, `type`, `fill`, `press`, `long-press`, `scroll`, `scrollintoview`
80
+ - `click`, `focus`, `type`, `fill`, `press`, `long-press`, `scroll`, `scrollintoview`, `is`
79
81
  - `alert`, `wait`, `screenshot`
80
82
  - `trace start`, `trace stop`
81
83
  - `settings wifi|airplane|location on|off`
@@ -107,20 +109,55 @@ Flags:
107
109
  ## Skills
108
110
  Install the automation skills listed in [SKILL.md](skills/agent-device/SKILL.md).
109
111
 
112
+ ```bash
113
+ npx skills add https://github.com/callstackincubator/agent-device --skill agent-device
114
+ ```
115
+
110
116
  Sessions:
111
117
  - `open` starts a session. Without args boots/activates the target device/simulator without launching an app.
112
118
  - All interaction commands require an open session.
113
119
  - If a session is already open, `open <app>` switches the active app and updates the session app bundle.
114
120
  - `close` stops the session and releases device resources. Pass an app to close it explicitly, or omit to just close the session.
115
121
  - Use `--session <name>` to manage multiple sessions.
116
- - Session logs are written to `~/.agent-device/sessions/<session>-<timestamp>.ad`.
117
- - With `--record-json`, JSON logs are written to `~/.agent-device/sessions/<session>-<timestamp>.json` by default.
122
+ - Session scripts are written to `~/.agent-device/sessions/<session>-<timestamp>.ad` when recording is enabled with `--save-script`.
123
+ - Deterministic replay is `.ad`-based; use `replay --update` (`-u`) to update selector drift and rewrite the replay file in place.
118
124
 
119
125
  Find (semantic):
120
126
  - `find <text> <action> [value]` finds by any text (label/value/identifier) using a scoped snapshot.
121
127
  - `find text|label|value|role|id <value> <action> [value]` for specific locators.
122
128
  - Actions: `click` (default), `fill`, `type`, `focus`, `get text`, `get attrs`, `wait [timeout]`, `exists`.
123
129
 
130
+ Assertions:
131
+ - `is` predicates: `visible`, `hidden`, `exists`, `editable`, `selected`, `text`.
132
+ - `is text` uses exact equality.
133
+
134
+ Replay update:
135
+ - `replay <path>` runs deterministic replay from `.ad` scripts.
136
+ - `replay -u <path>` attempts selector updates on failures and atomically rewrites the same file.
137
+ - Refs are the default/core mechanism for interactive agent flows.
138
+ - Update targets: `click`, `fill`, `get`, `is`, `wait`.
139
+ - Selector matching is a replay-update internal: replay parses `.ad` lines into actions, tries them, snapshots on failure, resolves a better selector, then rewrites that failing line.
140
+
141
+ Update examples:
142
+
143
+ ```sh
144
+ # Before (stale selector)
145
+ click "id=\"old_continue\" || label=\"Continue\""
146
+
147
+ # After replay -u (rewritten in place)
148
+ click "id=\"auth_continue\" || label=\"Continue\""
149
+ ```
150
+
151
+ ```sh
152
+ # Before (ref-based action from discovery)
153
+ snapshot -i -c -s "Continue"
154
+ click @e13 "Continue"
155
+
156
+ # After replay -u (upgraded to selector-based action)
157
+ snapshot -i -c -s "Continue"
158
+ click "id=\"auth_continue\" || label=\"Continue\""
159
+ ```
160
+
124
161
  Android fill reliability:
125
162
  - `fill` clears the current value, then enters text.
126
163
  - `type` enters text into the focused field without clearing.
package/dist/src/bin.js CHANGED
@@ -1,8 +1,8 @@
1
- import{node_path as e,fileURLToPath as t,asAppError as r,pathToFileURL as n,runCmdDetached as i,node_fs as a,node_os as s,node_net as o,errors_AppError as l}from"./861.js";function c(e){process.stdout.write(`${JSON.stringify(e,null,2)}
1
+ import{node_path as e,fileURLToPath as t,asAppError as r,pathToFileURL as n,runCmdDetached as a,node_fs as i,node_os as s,node_net as o,errors_AppError as l}from"./861.js";function c(e){process.stdout.write(`${JSON.stringify(e,null,2)}
2
2
  `)}function d(e){let t=e.details?`
3
3
  ${JSON.stringify(e.details,null,2)}`:"";process.stderr.write(`Error (${e.code}): ${e.message}${t}
4
- `)}function u(e,t,r){let n=p(e.type??"Element"),i=function(e,t){var r,n;let i=e.label?.trim(),a=e.value?.trim();if("text-field"===(r=t)||"text-view"===r||"search"===r){if(a)return a;if(i)return i}else if(i)return i;if(a)return a;let s=e.identifier?.trim();return!s||(n=s,/^[\w.]+:id\/[\w.-]+$/i.test(n)&&("group"===t||"image"===t||"list"===t||"collection"===t))?"":s}(e,n),a=" ".repeat(t),s=e.ref?`@${e.ref}`:"",o=[!1===e.enabled?"disabled":null].filter(Boolean).join(", "),l=o?` [${o}]`:"",c=i?` "${i}"`:"";return r?`${a}${s} [${n}]${l}`.trimEnd():`${a}${s} [${n}]${c}${l}`.trimEnd()}function p(e){let t=e.replace(/XCUIElementType/gi,"").toLowerCase(),r=e.includes(".")&&(e.startsWith("android.")||e.startsWith("androidx.")||e.startsWith("com."));switch(t.startsWith("ax")&&(t=t.replace(/^ax/,"")),t.includes(".")&&(t=t.replace(/^android\.widget\./,"").replace(/^android\.view\./,"").replace(/^android\.webkit\./,"").replace(/^androidx\./,"").replace(/^com\.google\.android\./,"").replace(/^com\.android\./,"")),t){case"application":return"application";case"navigationbar":return"navigation-bar";case"tabbar":return"tab-bar";case"button":case"imagebutton":return"button";case"link":return"link";case"cell":return"cell";case"statictext":case"checkedtextview":return"text";case"textfield":case"edittext":return"text-field";case"textview":return r?"text":"text-view";case"textarea":return"text-view";case"switch":return"switch";case"slider":return"slider";case"image":case"imageview":return"image";case"webview":return"webview";case"framelayout":case"linearlayout":case"relativelayout":case"constraintlayout":case"viewgroup":case"view":case"group":return"group";case"listview":case"recyclerview":return"list";case"collectionview":return"collection";case"searchfield":return"search";case"segmentedcontrol":return"segmented-control";case"window":return"window";case"checkbox":return"checkbox";case"radio":return"radio";case"menuitem":return"menu-item";case"toolbar":return"toolbar";case"scrollarea":case"scrollview":case"nestedscrollview":return"scroll-area";case"table":return"table";default:return t||"element"}}let f=e.join(s.homedir(),".agent-device"),m=e.join(f,"daemon.json"),h=function(){let e=process.env.AGENT_DEVICE_DAEMON_TIMEOUT_MS;if(!e)return 6e4;let t=Number(e);return Number.isFinite(t)?Math.max(1e3,Math.floor(t)):6e4}();async function w(e){let t=await y(),r={...e,token:t.token};return await x(t,r)}async function y(){let t=g(),r=function(){try{let t=$();return JSON.parse(a.readFileSync(e.join(t,"package.json"),"utf8")).version??"0.0.0"}catch{return"0.0.0"}}();if(t&&t.version===r&&await v(t))return t;t&&(t.version!==r||!await v(t))&&a.existsSync(m)&&a.unlinkSync(m),await b();let n=Date.now();for(;Date.now()-n<5e3;){let e=g();if(e&&await v(e))return e;await new Promise(e=>setTimeout(e,100))}throw new l("COMMAND_FAILED","Failed to start daemon",{infoPath:m,hint:"Run pnpm build, or delete ~/.agent-device/daemon.json if stale."})}function g(){if(!a.existsSync(m))return null;try{let e=JSON.parse(a.readFileSync(m,"utf8"));if(!e.port||!e.token)return null;return e}catch{return null}}async function v(e){return new Promise(t=>{let r=o.createConnection({host:"127.0.0.1",port:e.port},()=>{r.destroy(),t(!0)});r.on("error",()=>{t(!1)})})}async function b(){let t=$(),r=e.join(t,"dist","src","daemon.js"),n=e.join(t,"src","daemon.ts"),s=a.existsSync(r);if(!s&&!a.existsSync(n))throw new l("COMMAND_FAILED","Daemon entry not found",{distPath:r,srcPath:n});let o=s?[r]:["--experimental-strip-types",n];i(process.execPath,o)}async function x(e,t){return new Promise((r,n)=>{let i=o.createConnection({host:"127.0.0.1",port:e.port},()=>{i.write(`${JSON.stringify(t)}
5
- `)}),a=setTimeout(()=>{i.destroy(),n(new l("COMMAND_FAILED","Daemon request timed out",{timeoutMs:h}))},h),s="";i.setEncoding("utf8"),i.on("data",e=>{let t=(s+=e).indexOf("\n");if(-1===t)return;let o=s.slice(0,t).trim();if(o)try{let e=JSON.parse(o);i.end(),clearTimeout(a),r(e)}catch(e){clearTimeout(a),n(e)}}),i.on("error",e=>{clearTimeout(a),n(e)})})}function $(){let r=e.dirname(t(import.meta.url)),n=r;for(let t=0;t<6;t+=1){let t=e.join(n,"package.json");if(a.existsSync(t))return n;n=e.dirname(n)}return r}async function S(t){let n=function(e){let t={json:!1,help:!1},r=[];for(let n=0;n<e.length;n+=1){let i=e[n];if("--json"===i){t.json=!0;continue}if("--help"===i||"-h"===i){t.help=!0;continue}if("--verbose"===i||"-v"===i){t.verbose=!0;continue}if("-i"===i){t.snapshotInteractiveOnly=!0;continue}if("-c"===i){t.snapshotCompact=!0;continue}if("--raw"===i){t.snapshotRaw=!0;continue}if("--no-record"===i){t.noRecord=!0;continue}if("--record-json"===i){t.recordJson=!0;continue}if("--user-installed"===i){t.appsFilter="user-installed";continue}if("--all"===i){t.appsFilter="all";continue}if("--metadata"===i){t.appsMetadata=!0;continue}if(i.startsWith("--backend")){let r=i.includes("=")?i.split("=")[1]:e[n+1];if(i.includes("=")||(n+=1),"ax"!==r&&"xctest"!==r)throw new l("INVALID_ARGS",`Invalid backend: ${r}`);t.snapshotBackend=r;continue}if(i.startsWith("--")){let[r,a]=i.split("="),s=a??e[n+1];switch(!a&&(n+=1),r){case"--platform":if("ios"!==s&&"android"!==s)throw new l("INVALID_ARGS",`Invalid platform: ${s}`);t.platform=s;break;case"--depth":{let e=Number(s);if(!Number.isFinite(e)||e<0)throw new l("INVALID_ARGS",`Invalid depth: ${s}`);t.snapshotDepth=Math.floor(e);break}case"--scope":t.snapshotScope=s;break;case"--device":t.device=s;break;case"--udid":t.udid=s;break;case"--serial":t.serial=s;break;case"--out":t.out=s;break;case"--session":t.session=s;break;case"--activity":t.activity=s;break;default:throw new l("INVALID_ARGS",`Unknown flag: ${r}`)}continue}if("-d"===i){let r=e[n+1];n+=1;let i=Number(r);if(!Number.isFinite(i)||i<0)throw new l("INVALID_ARGS",`Invalid depth: ${r}`);t.snapshotDepth=Math.floor(i);continue}if("-s"===i){let r=e[n+1];n+=1,t.snapshotScope=r;continue}r.push(i)}return{command:r.shift()??null,positionals:r,flags:t}}(t);(n.flags.help||!n.command)&&(process.stdout.write(`agent-device <command> [args] [--json]
4
+ `)}function u(e,t,r){let n=p(e.type??"Element"),a=function(e,t){var r,n;let a=e.label?.trim(),i=e.value?.trim();if("text-field"===(r=t)||"text-view"===r||"search"===r){if(i)return i;if(a)return a}else if(a)return a;if(i)return i;let s=e.identifier?.trim();return!s||(n=s,/^[\w.]+:id\/[\w.-]+$/i.test(n)&&("group"===t||"image"===t||"list"===t||"collection"===t))?"":s}(e,n),i=" ".repeat(t),s=e.ref?`@${e.ref}`:"",o=[!1===e.enabled?"disabled":null].filter(Boolean).join(", "),l=o?` [${o}]`:"",c=a?` "${a}"`:"";return r?`${i}${s} [${n}]${l}`.trimEnd():`${i}${s} [${n}]${c}${l}`.trimEnd()}function p(e){let t=e.replace(/XCUIElementType/gi,"").toLowerCase(),r=e.includes(".")&&(e.startsWith("android.")||e.startsWith("androidx.")||e.startsWith("com."));switch(t.startsWith("ax")&&(t=t.replace(/^ax/,"")),t.includes(".")&&(t=t.replace(/^android\.widget\./,"").replace(/^android\.view\./,"").replace(/^android\.webkit\./,"").replace(/^androidx\./,"").replace(/^com\.google\.android\./,"").replace(/^com\.android\./,"")),t){case"application":return"application";case"navigationbar":return"navigation-bar";case"tabbar":return"tab-bar";case"button":case"imagebutton":return"button";case"link":return"link";case"cell":return"cell";case"statictext":case"checkedtextview":return"text";case"textfield":case"edittext":return"text-field";case"textview":return r?"text":"text-view";case"textarea":return"text-view";case"switch":return"switch";case"slider":return"slider";case"image":case"imageview":return"image";case"webview":return"webview";case"framelayout":case"linearlayout":case"relativelayout":case"constraintlayout":case"viewgroup":case"view":case"group":return"group";case"listview":case"recyclerview":return"list";case"collectionview":return"collection";case"searchfield":return"search";case"segmentedcontrol":return"segmented-control";case"window":return"window";case"checkbox":return"checkbox";case"radio":return"radio";case"menuitem":return"menu-item";case"toolbar":return"toolbar";case"scrollarea":case"scrollview":case"nestedscrollview":return"scroll-area";case"table":return"table";default:return t||"element"}}let f=e.join(s.homedir(),".agent-device"),m=e.join(f,"daemon.json"),h=function(){let e=process.env.AGENT_DEVICE_DAEMON_TIMEOUT_MS;if(!e)return 6e4;let t=Number(e);return Number.isFinite(t)?Math.max(1e3,Math.floor(t)):6e4}();async function w(e){let t=await y(),r={...e,token:t.token};return await x(t,r)}async function y(){let t=g(),r=function(){try{let t=$();return JSON.parse(i.readFileSync(e.join(t,"package.json"),"utf8")).version??"0.0.0"}catch{return"0.0.0"}}();if(t&&t.version===r&&await v(t))return t;t&&(t.version!==r||!await v(t))&&i.existsSync(m)&&i.unlinkSync(m),await b();let n=Date.now();for(;Date.now()-n<5e3;){let e=g();if(e&&await v(e))return e;await new Promise(e=>setTimeout(e,100))}throw new l("COMMAND_FAILED","Failed to start daemon",{infoPath:m,hint:"Run pnpm build, or delete ~/.agent-device/daemon.json if stale."})}function g(){if(!i.existsSync(m))return null;try{let e=JSON.parse(i.readFileSync(m,"utf8"));if(!e.port||!e.token)return null;return e}catch{return null}}async function v(e){return new Promise(t=>{let r=o.createConnection({host:"127.0.0.1",port:e.port},()=>{r.destroy(),t(!0)});r.on("error",()=>{t(!1)})})}async function b(){let t=$(),r=e.join(t,"dist","src","daemon.js"),n=e.join(t,"src","daemon.ts"),s=i.existsSync(r);if(!s&&!i.existsSync(n))throw new l("COMMAND_FAILED","Daemon entry not found",{distPath:r,srcPath:n});let o=s?[r]:["--experimental-strip-types",n];a(process.execPath,o)}async function x(e,t){return new Promise((r,n)=>{let a=o.createConnection({host:"127.0.0.1",port:e.port},()=>{a.write(`${JSON.stringify(t)}
5
+ `)}),i=setTimeout(()=>{a.destroy(),n(new l("COMMAND_FAILED","Daemon request timed out",{timeoutMs:h}))},h),s="";a.setEncoding("utf8"),a.on("data",e=>{let t=(s+=e).indexOf("\n");if(-1===t)return;let o=s.slice(0,t).trim();if(o)try{let e=JSON.parse(o);a.end(),clearTimeout(i),r(e)}catch(e){clearTimeout(i),n(e)}}),a.on("error",e=>{clearTimeout(i),n(e)})})}function $(){let r=e.dirname(t(import.meta.url)),n=r;for(let t=0;t<6;t+=1){let t=e.join(n,"package.json");if(i.existsSync(t))return n;n=e.dirname(n)}return r}async function S(t){let n=function(e){let t={json:!1,help:!1},r=[];for(let n=0;n<e.length;n+=1){let a=e[n];if("--json"===a){t.json=!0;continue}if("--help"===a||"-h"===a){t.help=!0;continue}if("--verbose"===a||"-v"===a){t.verbose=!0;continue}if("-i"===a){t.snapshotInteractiveOnly=!0;continue}if("-c"===a){t.snapshotCompact=!0;continue}if("--raw"===a){t.snapshotRaw=!0;continue}if("--no-record"===a){t.noRecord=!0;continue}if("--save-script"===a){t.saveScript=!0;continue}if("--update"===a||"-u"===a){t.replayUpdate=!0;continue}if("--user-installed"===a){t.appsFilter="user-installed";continue}if("--all"===a){t.appsFilter="all";continue}if("--metadata"===a){t.appsMetadata=!0;continue}if(a.startsWith("--backend")){let r=a.includes("=")?a.split("=")[1]:e[n+1];if(a.includes("=")||(n+=1),"ax"!==r&&"xctest"!==r)throw new l("INVALID_ARGS",`Invalid backend: ${r}`);t.snapshotBackend=r;continue}if(a.startsWith("--")){let[r,i]=a.split("="),s=i??e[n+1];switch(!i&&(n+=1),r){case"--platform":if("ios"!==s&&"android"!==s)throw new l("INVALID_ARGS",`Invalid platform: ${s}`);t.platform=s;break;case"--depth":{let e=Number(s);if(!Number.isFinite(e)||e<0)throw new l("INVALID_ARGS",`Invalid depth: ${s}`);t.snapshotDepth=Math.floor(e);break}case"--scope":t.snapshotScope=s;break;case"--device":t.device=s;break;case"--udid":t.udid=s;break;case"--serial":t.serial=s;break;case"--out":t.out=s;break;case"--session":t.session=s;break;case"--activity":t.activity=s;break;default:throw new l("INVALID_ARGS",`Unknown flag: ${r}`)}continue}if("-d"===a){let r=e[n+1];n+=1;let a=Number(r);if(!Number.isFinite(a)||a<0)throw new l("INVALID_ARGS",`Invalid depth: ${r}`);t.snapshotDepth=Math.floor(a);continue}if("-s"===a){let r=e[n+1];n+=1,t.snapshotScope=r;continue}r.push(a)}return{command:r.shift()??null,positionals:r,flags:t}}(t);(n.flags.help||!n.command)&&(process.stdout.write(`agent-device <command> [args] [--json]
6
6
 
7
7
  CLI to control iOS and Android devices for AI agents.
8
8
 
@@ -24,17 +24,19 @@ Commands:
24
24
  back Navigate back (where supported)
25
25
  home Go to home screen (where supported)
26
26
  app-switcher Open app switcher (where supported)
27
- wait <ms>|text <text>|@ref [timeoutMs] Wait for duration or text to appear
27
+ wait <ms>|text <text>|@ref|<selector> [timeoutMs]
28
+ Wait for duration, text, ref, or selector to appear
28
29
  alert [get|accept|dismiss|wait] [timeout] Inspect or handle alert (iOS simulator)
29
- click <@ref> Click element by snapshot ref
30
- get text <@ref> Return element text by ref
31
- get attrs <@ref> Return element attributes by ref
32
- replay <path> Replay a recorded session
30
+ click <@ref|selector> Click element by snapshot ref or selector
31
+ get text <@ref|selector> Return element text by ref or selector
32
+ get attrs <@ref|selector> Return element attributes by ref or selector
33
+ replay <path> [--update|-u] Replay a recorded session
33
34
  press <x> <y> Tap at coordinates
34
35
  long-press <x> <y> [durationMs] Long press (where supported)
35
36
  focus <x> <y> Focus input at coordinates
36
37
  type <text> Type text in focused field
37
- fill <x> <y> <text> | fill <@ref> <text> Tap then type
38
+ fill <x> <y> <text> | fill <@ref|selector> <text>
39
+ Tap then type
38
40
  scroll <direction> [amount] Scroll in direction (0-1 amount)
39
41
  scrollintoview <text> Scroll until text appears (Android only)
40
42
  screenshot [path] Capture screenshot
@@ -48,6 +50,7 @@ Commands:
48
50
  find value <value> <action> [value] Find by value
49
51
  find role <role> <action> [value] Find by role/type
50
52
  find id <id> <action> [value] Find by identifier/resource-id
53
+ is <predicate> <selector> [value] Assert UI state (visible|hidden|exists|editable|selected|text)
51
54
  settings <wifi|airplane|location> <on|off> Toggle OS settings (simulators)
52
55
  session list List active sessions
53
56
 
@@ -60,32 +63,34 @@ Flags:
60
63
  --session <name> Named session
61
64
  --verbose Stream daemon/runner logs
62
65
  --json JSON output
66
+ --save-script Save session script (.ad) on close
63
67
  --no-record Do not record this action
64
- --record-json Record JSON session log
68
+ --update, -u Replay: update selectors and rewrite replay file in place
65
69
  --user-installed Apps: list user-installed packages (Android only)
66
70
  --all Apps: list all packages (Android only)
67
71
 
68
- `),process.exit(+!n.flags.help));let{command:i,positionals:o,flags:f}=n,m=f.session??process.env.AGENT_DEVICE_SESSION??"default",h=f.verbose&&!f.json?function(){try{let t=e.join(s.homedir(),".agent-device","daemon.log"),r=0,n=!1,i=setInterval(()=>{if(n||!a.existsSync(t))return;let e=a.statSync(t);if(e.size<=r)return;let i=a.openSync(t,"r"),s=Buffer.alloc(e.size-r);a.readSync(i,s,0,s.length,r),a.closeSync(i),r=e.size,s.length>0&&process.stdout.write(s.toString("utf8"))},200);return()=>{n=!0,clearInterval(i)}}catch{return null}}():null;try{if("session"===i){let e=o[0]??"list";if("list"!==e)throw new l("INVALID_ARGS","session only supports list");let t=await w({session:m,command:"session_list",positionals:[],flags:{}});if(!t.ok)throw new l(t.error.code,t.error.message);f.json?c({success:!0,data:t.data??{}}):process.stdout.write(`${JSON.stringify(t.data??{},null,2)}
69
- `),h&&h();return}let e=await w({session:m,command:i,positionals:o,flags:f});if(e.ok){if(f.json){c({success:!0,data:e.data??{}}),h&&h();return}if("snapshot"===i){process.stdout.write(function(e,t={}){let r=e.nodes,n=Array.isArray(r)?r:[],i=!!e.truncated,a="string"==typeof e.appName?e.appName:void 0,s="string"==typeof e.appBundleId?e.appBundleId:void 0,o=[];a&&o.push(`Page: ${a}`),s&&o.push(`App: ${s}`);let l=`Snapshot: ${n.length} nodes${i?" (truncated)":""}`,c=o.length>0?`${o.join("\n")}
72
+ `),process.exit(+!n.flags.help));let{command:a,positionals:o,flags:f}=n,m=f.session??process.env.AGENT_DEVICE_SESSION??"default",h=f.verbose&&!f.json?function(){try{let t=e.join(s.homedir(),".agent-device","daemon.log"),r=0,n=!1,a=setInterval(()=>{if(n||!i.existsSync(t))return;let e=i.statSync(t);if(e.size<=r)return;let a=i.openSync(t,"r"),s=Buffer.alloc(e.size-r);i.readSync(a,s,0,s.length,r),i.closeSync(a),r=e.size,s.length>0&&process.stdout.write(s.toString("utf8"))},200);return()=>{n=!0,clearInterval(a)}}catch{return null}}():null;try{if("session"===a){let e=o[0]??"list";if("list"!==e)throw new l("INVALID_ARGS","session only supports list");let t=await w({session:m,command:"session_list",positionals:[],flags:{}});if(!t.ok)throw new l(t.error.code,t.error.message);f.json?c({success:!0,data:t.data??{}}):process.stdout.write(`${JSON.stringify(t.data??{},null,2)}
73
+ `),h&&h();return}let e=await w({session:m,command:a,positionals:o,flags:f});if(e.ok){if(f.json){c({success:!0,data:e.data??{}}),h&&h();return}if("snapshot"===a){process.stdout.write(function(e,t={}){let r=e.nodes,n=Array.isArray(r)?r:[],a=!!e.truncated,i="string"==typeof e.appName?e.appName:void 0,s="string"==typeof e.appBundleId?e.appBundleId:void 0,o=[];i&&o.push(`Page: ${i}`),s&&o.push(`App: ${s}`);let l=`Snapshot: ${n.length} nodes${a?" (truncated)":""}`,c=o.length>0?`${o.join("\n")}
70
74
  `:"";if(0===n.length)return`${c}${l}
71
75
  `;if(t.raw){let e=n.map(e=>JSON.stringify(e));return`${c}${l}
72
76
  ${e.join("\n")}
73
77
  `}if(t.flatten){let e=n.map(e=>u(e,0,!1));return`${c}${l}
74
78
  ${e.join("\n")}
75
- `}let d=[],f=[];for(let e of n){let t=e.depth??0;for(;d.length>0&&t<=d[d.length-1];)d.pop();let r=e.label?.trim()||e.value?.trim()||e.identifier?.trim()||"",n="group"===p(e.type??"Element")&&!r;n&&d.push(t);let i=n?t:Math.max(0,t-d.length);f.push(u(e,i,n))}return`${c}${l}
79
+ `}let d=[],f=[];for(let e of n){let t=e.depth??0;for(;d.length>0&&t<=d[d.length-1];)d.pop();let r=e.label?.trim()||e.value?.trim()||e.identifier?.trim()||"",n="group"===p(e.type??"Element")&&!r;n&&d.push(t);let a=n?t:Math.max(0,t-d.length);f.push(u(e,a,n))}return`${c}${l}
76
80
  ${f.join("\n")}
77
- `}(e.data??{},{raw:f.snapshotRaw,flatten:f.snapshotInteractiveOnly})),h&&h();return}if("get"===i){let t=o[0];if("text"===t){let t=e.data?.text??"";process.stdout.write(`${t}
81
+ `}(e.data??{},{raw:f.snapshotRaw,flatten:f.snapshotInteractiveOnly})),h&&h();return}if("get"===a){let t=o[0];if("text"===t){let t=e.data?.text??"";process.stdout.write(`${t}
78
82
  `),h&&h();return}if("attrs"===t){let t=e.data?.node??{};process.stdout.write(`${JSON.stringify(t,null,2)}
79
- `),h&&h();return}}if("find"===i){let t=e.data;if("string"==typeof t?.text){process.stdout.write(`${t.text}
83
+ `),h&&h();return}}if("find"===a){let t=e.data;if("string"==typeof t?.text){process.stdout.write(`${t.text}
80
84
  `),h&&h();return}if("boolean"==typeof t?.found){process.stdout.write(`Found: ${t.found}
81
85
  `),h&&h();return}if(t?.node){process.stdout.write(`${JSON.stringify(t.node,null,2)}
82
- `),h&&h();return}}if("click"===i){let t=e.data?.ref??"",r=e.data?.x,n=e.data?.y;t&&"number"==typeof r&&"number"==typeof n&&process.stdout.write(`Clicked @${t} (${r}, ${n})
83
- `),h&&h();return}if(e.data&&"object"==typeof e.data){let t=e.data;if("devices"===i){let e=(Array.isArray(t.devices)?t.devices:[]).map(e=>{let t=e?.name??e?.id??"unknown",r=e?.platform??"unknown",n=e?.kind?` ${e.kind}`:"",i="boolean"==typeof e?.booted?` booted=${e.booted}`:"";return`${t} (${r}${n})${i}`});process.stdout.write(`${e.join("\n")}
84
- `),h&&h();return}if("apps"===i){let e=(Array.isArray(t.apps)?t.apps:[]).map(e=>{if("string"==typeof e)return e;if(e&&"object"==typeof e){let t=e.bundleId??e.package,r=e.name??e.label;return r&&t?`${r} (${t})`:t&&"boolean"==typeof e.launchable?`${t} (launchable=${e.launchable})`:t?String(t):JSON.stringify(e)}return String(e)});process.stdout.write(`${e.join("\n")}
85
- `),h&&h();return}if("appstate"===i){let e=t?.platform,r=t?.appBundleId,n=t?.appName,i=t?.source,a=t?.package,s=t?.activity;if("ios"===e){process.stdout.write(`Foreground app: ${n??r}
86
+ `),h&&h();return}}if("is"===a){let t=e.data?.predicate??"assertion";process.stdout.write(`Passed: is ${t}
87
+ `),h&&h();return}if("click"===a){let t=e.data?.ref??"",r=e.data?.x,n=e.data?.y;t&&"number"==typeof r&&"number"==typeof n&&process.stdout.write(`Clicked @${t} (${r}, ${n})
88
+ `),h&&h();return}if(e.data&&"object"==typeof e.data){let t=e.data;if("devices"===a){let e=(Array.isArray(t.devices)?t.devices:[]).map(e=>{let t=e?.name??e?.id??"unknown",r=e?.platform??"unknown",n=e?.kind?` ${e.kind}`:"",a="boolean"==typeof e?.booted?` booted=${e.booted}`:"";return`${t} (${r}${n})${a}`});process.stdout.write(`${e.join("\n")}
89
+ `),h&&h();return}if("apps"===a){let e=(Array.isArray(t.apps)?t.apps:[]).map(e=>{if("string"==typeof e)return e;if(e&&"object"==typeof e){let t=e.bundleId??e.package,r=e.name??e.label;return r&&t?`${r} (${t})`:t&&"boolean"==typeof e.launchable?`${t} (launchable=${e.launchable})`:t?String(t):JSON.stringify(e)}return String(e)});process.stdout.write(`${e.join("\n")}
90
+ `),h&&h();return}if("appstate"===a){let e=t?.platform,r=t?.appBundleId,n=t?.appName,a=t?.source,i=t?.package,s=t?.activity;if("ios"===e){process.stdout.write(`Foreground app: ${n??r}
86
91
  `),r&&process.stdout.write(`Bundle: ${r}
87
- `),i&&process.stdout.write(`Source: ${i}
88
- `),h&&h();return}if("android"===e){process.stdout.write(`Foreground app: ${a??"unknown"}
92
+ `),a&&process.stdout.write(`Source: ${a}
93
+ `),h&&h();return}if("android"===e){process.stdout.write(`Foreground app: ${i??"unknown"}
89
94
  `),s&&process.stdout.write(`Activity: ${s}
90
95
  `),h&&h();return}}}h&&h();return}throw new l(e.error.code,e.error.message,e.error.details)}catch(t){let e=r(t);if(f.json)c({success:!1,error:{code:e.code,message:e.message,details:e.details}});else if(d(e),f.verbose)try{let e=await import("node:fs"),t=await import("node:os"),r=(await import("node:path")).join(t.homedir(),".agent-device","daemon.log");if(e.existsSync(r)){let t=e.readFileSync(r,"utf8").split("\n"),n=t.slice(Math.max(0,t.length-200)).join("\n");n.trim().length>0&&process.stderr.write(`
91
96
  [daemon log]