@swmansion/argent 0.29.1-next.21 → 0.29.1-next.23
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/dist/cli-cmds.mjs +808 -574
- package/dist/cli.js +3 -2
- package/dist/cli.js.map +1 -1
- package/dist/client.js +389 -226
- package/dist/flow-script-runner.mjs +1 -1
- package/dist/installer.mjs +606 -496
- package/dist/mcp-server.mjs +467 -298
- package/dist/tool-args.generated.d.ts +51 -37
- package/dist/tool-server.cjs +1994 -1738
- package/package.json +1 -1
- package/rules/argent.md +10 -10
- package/skills/{argent-create-flow → argent-create-e2e}/SKILL.md +16 -16
- package/skills/{argent-create-flow/references/flow-yaml.md → argent-create-e2e/references/e2e-yaml.md} +30 -30
- package/skills/{argent-create-flow → argent-create-e2e}/references/live-authoring.md +62 -62
- package/skills/{argent-create-flow → argent-create-e2e}/references/reliability-and-recovery.md +18 -18
- package/skills/argent-device-interact/SKILL.md +2 -2
- package/skills/argent-device-interact/references/chromium.md +1 -1
- package/skills/argent-ios-device-interact/SKILL.md +1 -1
- package/skills/argent-native-profiler/SKILL.md +1 -1
- package/skills/{argent-qa-flows → argent-qa-e2e}/SKILL.md +23 -23
- package/skills/argent-react-native-optimization/SKILL.md +2 -2
- package/skills/argent-react-native-profiler/SKILL.md +3 -3
- package/skills/argent-screen-recording/SKILL.md +1 -1
- package/skills/argent-test-ui-flow/SKILL.md +4 -4
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
# Live authoring
|
|
2
2
|
|
|
3
|
-
Read this file before creating or changing
|
|
3
|
+
Read this file before creating or changing an e2e test. Exercise the saved path through the recorder. Perform syntax cleanup only after finishing.
|
|
4
4
|
|
|
5
5
|
- [Live authoring](#live-authoring)
|
|
6
6
|
- [Recorder contract](#recorder-contract)
|
|
7
7
|
- [Start in the correct order](#start-in-the-correct-order)
|
|
8
|
-
- [iOS, Android, and Vega
|
|
9
|
-
- [Chromium
|
|
8
|
+
- [iOS, Android, and Vega standalone tests](#ios-android-and-vega-standalone-tests)
|
|
9
|
+
- [Chromium standalone tests](#chromium-standalone-tests)
|
|
10
10
|
- [Fragments](#fragments)
|
|
11
11
|
- [Record the first walkthrough](#record-the-first-walkthrough)
|
|
12
12
|
- [Record identity, then readiness, after every navigation](#record-identity-then-readiness-after-every-navigation)
|
|
@@ -23,34 +23,34 @@ Read this file before creating or changing a flow. Exercise the saved path throu
|
|
|
23
23
|
|
|
24
24
|
## Recorder contract
|
|
25
25
|
|
|
26
|
-
`
|
|
26
|
+
`e2e-add-step.command` is an MCP tool name. `args` is a JSON string, not an object. Omit `args` for a no-argument tool.
|
|
27
27
|
|
|
28
28
|
```text
|
|
29
29
|
command: "gesture-tap"
|
|
30
30
|
args: "{\"udid\":\"DEVICE\",\"x\":0.5,\"y\":0.35}"
|
|
31
31
|
```
|
|
32
32
|
|
|
33
|
-
A recorded `
|
|
33
|
+
A recorded `e2e-execute` has two names. The top-level `name` identifies the recording. `args.name` identifies the sibling test captured as `run:`.
|
|
34
34
|
|
|
35
|
-
When the user requests a script, call `
|
|
35
|
+
When the user requests a script, call `e2e-add-script` at the point where it must run. Read [E2E YAML: Local scripts](e2e-yaml.md#local-scripts) first. If the call fails, check its changes before you retry.
|
|
36
36
|
|
|
37
|
-
Over a link, `
|
|
37
|
+
Over a link, `e2e-add-step` refuses a call whose step a replay over the same link rejects, and runs nothing. It also refuses an `e2e-execute` call whose test does not pass. That test can have changed the device before it stopped. In both cases the recorder records no step. See [Replay](#replay) for the steps that a replay over a link rejects.
|
|
38
38
|
|
|
39
39
|
Obey these lifecycle rules:
|
|
40
40
|
|
|
41
41
|
1. Pass the same `name` and absolute `project_root` to every recording tool.
|
|
42
|
-
2. Choose a name unique to the task. Another caller can take over the same pair without an ownership check. The pair is keyed by the file the filesystem resolves to, not the spelling you passed, so a differently-cased name or a symlinked `.argent/
|
|
42
|
+
2. Choose a name unique to the task. Another caller can take over the same pair without an ownership check. The pair is keyed by the file the filesystem resolves to, not the spelling you passed, so a differently-cased name or a symlinked `.argent/e2e` collides too. That collision is reported: the second start says `restarted`, and the first recording's next call fails naming both spellings.
|
|
43
43
|
3. Give concurrent recordings separate devices. Their files are isolated, but their live device actions are not.
|
|
44
|
-
4. Treat `
|
|
44
|
+
4. Treat `e2e-start-recording` as destructive. It always truncates the named YAML, including a finished or committed test. `restarted` reports only a displaced live take.
|
|
45
45
|
5. If a call says the recording is inactive, do not restart under that name. The completed take can still be on disk. Copy it aside or record under a fresh name.
|
|
46
|
-
6. Inspect `toolResult`, `message`, and `recorded` after each call. A call that errors records nothing, but a call that returns normally while reporting an unmet condition **does** append the step, and `message` says the step was added either way. A failed `
|
|
47
|
-
7. Edit or reorder the YAML only after `
|
|
46
|
+
6. Inspect `toolResult`, `message`, and `recorded` after each call. A call that errors records nothing, but a call that returns normally while reporting an unmet condition **does** append the step, and `message` says the step was added either way. A failed `e2e-add-script` call appends nothing. `await-ui-element` is the case that turns up in practice (see [Live waits and checks](#live-waits-and-checks)). Only `e2e-start-recording` and `e2e-finish-recording` return the whole YAML as `flowFile`. A step call returns `recorded` (one summary line for the step it appended) plus a running `stepCount`. Read `recorded`: the recorder does not always store the tool call you made, and that line is where a rewrite shows up. To see the whole file mid-recording, read it at `savedTo`. A `savedTo` that comes back `null` means the write failed on your side. The step is still in the recording, so continue: the next step rewrites the whole file, and `e2e-finish-recording` returns `flowFile` regardless.
|
|
47
|
+
7. Edit or reorder the YAML only after `e2e-finish-recording`. A recording over a link, also to 127.0.0.1, overwrites edits made during the recording.
|
|
48
48
|
|
|
49
49
|
## Start in the correct order
|
|
50
50
|
|
|
51
|
-
### iOS, Android, and Vega
|
|
51
|
+
### iOS, Android, and Vega standalone tests
|
|
52
52
|
|
|
53
|
-
1. Call `
|
|
53
|
+
1. Call `e2e-start-recording` before launching or touching the app.
|
|
54
54
|
2. Record a plain `restart-app` as the first action that is neither an echo nor a script. Pass only the device id and app id. The recorder converts it to `launch:`.
|
|
55
55
|
3. Record `await-ui-element` for the real first screen immediately after restart.
|
|
56
56
|
|
|
@@ -60,11 +60,11 @@ Do not use splash content as a selector or landmark. Wait for the first real scr
|
|
|
60
60
|
|
|
61
61
|
On iOS, only `restart-app` guarantees an instrumented launch. `launch-app` can foreground an uninstrumented process. Use [iOS selector recovery](reliability-and-recovery.md#ios-selector-recovery) when the tree is missing.
|
|
62
62
|
|
|
63
|
-
### Chromium
|
|
63
|
+
### Chromium standalone tests
|
|
64
64
|
|
|
65
|
-
**The app sets the window size; no boot argument overrides it.** Electron ignores `--window-size`: the size comes from the app's own `BrowserWindow` options, and Argent never resizes the window. Boot a fresh target with `boot-device` and `electronAppPath`, then take one `screenshot` and record its pixel dimensions: that capture size is the snapshot device class ([
|
|
65
|
+
**The app sets the window size; no boot argument overrides it.** Electron ignores `--window-size`: the size comes from the app's own `BrowserWindow` options, and Argent never resizes the window. Boot a fresh target with `boot-device` and `electronAppPath`, then take one `screenshot` and record its pixel dimensions: that capture size is the snapshot device class ([E2E YAML: Snapshots](e2e-yaml.md#snapshots-and-cli-runs)) and must match when CI replays. If the app sizes its window from host or session state, say so in the report.
|
|
66
66
|
|
|
67
|
-
After boot, start the recorder before the first in-app action. Record the first-screen wait. `restart-app` has no Chromium support, so the call errors and records nothing, and a recorded Chromium
|
|
67
|
+
After boot, start the recorder before the first in-app action. Record the first-screen wait. `restart-app` has no Chromium support, so the call errors and records nothing, and a recorded Chromium test is always a fragment: its launch is written in during polish rather than captured, and any `executionPrerequisite` the recording declared must go with it. A launch-first test must not carry one. During polish, add the matching launch:
|
|
68
68
|
|
|
69
69
|
```yaml
|
|
70
70
|
steps:
|
|
@@ -74,7 +74,7 @@ steps:
|
|
|
74
74
|
args: ["--enable-feature-flag"]
|
|
75
75
|
```
|
|
76
76
|
|
|
77
|
-
The path is relative to `.argent/
|
|
77
|
+
The path is relative to `.argent/e2e/`. Copy the live boot arguments verbatim, and omit `args` when the boot passed none. This packaging exception represents the boot already exercised live. It does not permit a rehearsed UI path.
|
|
78
78
|
|
|
79
79
|
### Fragments
|
|
80
80
|
|
|
@@ -82,7 +82,7 @@ Stage the entry state before recording. Then start with a precise `executionPrer
|
|
|
82
82
|
|
|
83
83
|
## Record the first walkthrough
|
|
84
84
|
|
|
85
|
-
For Vega, first read [
|
|
85
|
+
For Vega, first read [E2E YAML: Composition and platform limits](e2e-yaml.md#composition-and-platform-limits). Vega is remote-driven and does not use touch gestures.
|
|
86
86
|
|
|
87
87
|
Reach each screen through the app's UI. Do not replace tested navigation with `open-url`. Starting the app is not navigation.
|
|
88
88
|
|
|
@@ -91,7 +91,7 @@ For every action:
|
|
|
91
91
|
1. **Discover without mutation.** Use `describe`, iOS native discovery, `debugger-component-tree`, or `screenshot`. Do not record discovery or `debugger-*` calls: `port` is not a device-bind key, so a recorded one replays against whatever Metro owns that port.
|
|
92
92
|
2. **Choose a durable target.** Prefer a stable id, then stable text or an accessibility label. On iOS, use native discovery for ids that trimmed accessibility output omits.
|
|
93
93
|
3. **Add an echo.** Name the current state, action, and expected outcome before the action can fail.
|
|
94
|
-
4. **Execute through `
|
|
94
|
+
4. **Execute through `e2e-add-step`.** Inspect the result and the `recorded` line immediately.
|
|
95
95
|
5. **Verify immediately.** Record outcome checks when their states first appear. After navigation, prove identity then readiness: record the identity check live, and add the readiness gate during polish.
|
|
96
96
|
|
|
97
97
|
### Record identity, then readiness, after every navigation
|
|
@@ -100,7 +100,7 @@ For every action:
|
|
|
100
100
|
2. If a specific control marks application readiness, record another visible wait on that control before the next action.
|
|
101
101
|
3. During polish, add `await: { idle: true }` after the identity check.
|
|
102
102
|
|
|
103
|
-
A shared tab bar, source element, or positional id does not prove navigation. `idle` cannot identify the screen or prove application data is ready. Keep the control wait because asynchronous loading can continue after stillness. If the screen intentionally moves, disclose it and gate the next action on a stable element. Read [
|
|
103
|
+
A shared tab bar, source element, or positional id does not prove navigation. `idle` cannot identify the screen or prove application data is ready. Keep the control wait because asynchronous loading can continue after stillness. If the screen intentionally moves, disclose it and gate the next action on a stable element. Read [E2E YAML: Prove a navigation](e2e-yaml.md#prove-a-navigation-identity-then-readiness) for the directive semantics.
|
|
104
104
|
|
|
105
105
|
### Record absence in three steps
|
|
106
106
|
|
|
@@ -114,10 +114,10 @@ Without step 1, `hidden` also passes for a typo or an element that never existed
|
|
|
114
114
|
|
|
115
115
|
### Taps
|
|
116
116
|
|
|
117
|
-
`
|
|
117
|
+
`e2e-add-step` cannot receive a YAML selector directly. Discover the element first, then record `gesture-tap` at its frame center; the live coordinates are transport for the gesture, not a final locator. The recorder reads the pre-tap tree and derives the selector in a fixed order (`id`, then `text`, then `role`), giving three outcomes. Read the `recorded` line after every tap, because only two of them warn. It names the derived form (a selector map, or the kept point):
|
|
118
118
|
|
|
119
119
|
1. **`tap: { id: ... }` or `tap: { text: ... }`**: the good case.
|
|
120
|
-
2. **`tap: { role: ... }`, appended with no warning.** An icon-only button with neither id nor visible label lands here. `role` matches as a case-insensitive substring, so a replay screen holding a second control of that role can win the [ranking](
|
|
120
|
+
2. **`tap: { role: ... }`, appended with no warning.** An icon-only button with neither id nor visible label lands here. `role` matches as a case-insensitive substring, so a replay screen holding a second control of that role can win the [ranking](e2e-yaml.md#the-runner-tree-is-not-the-discovery-tree) and the tap reports a pass on the wrong control.
|
|
121
121
|
3. **A kept raw point**, with a warning naming the reason and the retarget.
|
|
122
122
|
|
|
123
123
|
Treat outcomes 2 and 3 alike. Restore the source screen with direct MCP calls, record a corrected tap, then remove the weak step after finishing. Keep a point or a bare role only through the [coordinate fallback gate](reliability-and-recovery.md#coordinate-fallback-gate).
|
|
@@ -146,9 +146,9 @@ For every retained raw gesture, add an echo and a recorded result check.
|
|
|
146
146
|
|
|
147
147
|
### Live waits and checks
|
|
148
148
|
|
|
149
|
-
Record `await-ui-element` through `
|
|
149
|
+
Record `await-ui-element` through `e2e-add-step`. The recorder writes the step even when `toolResult.success` is false. Read `success` and `cause` after each check:
|
|
150
150
|
|
|
151
|
-
- `unmet`: The tree was readable, but the condition was false. Restore the expected state or correct the selector or timeout. Record the check again, then delete the failed step after `
|
|
151
|
+
- `unmet`: The tree was readable, but the condition was false. Restore the expected state or correct the selector or timeout. Record the check again, then delete the failed step after `e2e-finish-recording`.
|
|
152
152
|
- `unreadable`: The wait ended without a trustworthy read. Restore the tree source and record the check again. Keep the failed step: the condition is unknown, not false.
|
|
153
153
|
- `cancelled`: The caller stopped the wait. Record the check again. Keep the failed step: the condition is unknown, not false.
|
|
154
154
|
|
|
@@ -158,23 +158,23 @@ A stale `hidden` whose selector matches nothing replays as a silent pass: the un
|
|
|
158
158
|
|
|
159
159
|
A wait inside `run-sequence` gets no recorder warning. Inspect the nested result. Any `success: false` fails the sequence during replay.
|
|
160
160
|
|
|
161
|
-
The live tool and
|
|
161
|
+
The live tool and e2e runner use [different trees](e2e-yaml.md#the-runner-tree-is-not-the-discovery-tree). After a successful wait, the recorder checks the same condition on the runner tree:
|
|
162
162
|
|
|
163
163
|
- No warning: The condition holds on both trees.
|
|
164
164
|
- Mismatch: For `text`, first rule out a selector that matches more than one element. Then rule out a changed screen. If the trees really differ, keep the raw `await-ui-element` step: it replays on the tree it passed on.
|
|
165
165
|
- Unreadable, slow, or cancelled check: The conversion is unknown. Restore the source or re-record before conversion.
|
|
166
166
|
|
|
167
|
-
A warning does not reject the step. `
|
|
167
|
+
A warning does not reject the step. `e2e-finish-recording` repeats each warning below its step and reports dropped warnings.
|
|
168
168
|
|
|
169
169
|
Do not edit YAML before finishing because edits can drop recorded verdicts. If the finish reports drops, record the waits again. Replay every conversion. Keep a raw tool only for `pollIntervalMs`, `bundleId`, or a cross-tree mismatch.
|
|
170
170
|
|
|
171
171
|
### Wrong turns
|
|
172
172
|
|
|
173
|
-
Stop immediately. Restore the last valid screen with direct MCP calls, not `
|
|
173
|
+
Stop immediately. Restore the last valid screen with direct MCP calls, not `e2e-add-step`. Continue only from verified state. Remove the bad step after finishing. If recovery changed or skipped meaningful behavior, re-record that portion live.
|
|
174
174
|
|
|
175
175
|
## Finish and polish
|
|
176
176
|
|
|
177
|
-
Call `
|
|
177
|
+
Call `e2e-finish-recording`, then read the saved YAML. Apply only meaning-preserving conversions:
|
|
178
178
|
|
|
179
179
|
| Recorded form | Finished form |
|
|
180
180
|
| ------------------------------------------ | ------------------------------------------------------------------ |
|
|
@@ -186,9 +186,9 @@ Call `flow-finish-recording`, then read the saved YAML. Apply only meaning-prese
|
|
|
186
186
|
| coordinate tap or long-press | strict selector after the fallback gate |
|
|
187
187
|
| `tool: gesture-pinch` | selector-based `pinch:` with `scale = endDistance / startDistance` |
|
|
188
188
|
| `tool: gesture-rotate` | selector-based `rotate:` with `by = endAngle - startAngle` |
|
|
189
|
-
| sibling `tool:
|
|
189
|
+
| sibling `tool: e2e-execute` | recorder-captured `run:` |
|
|
190
190
|
|
|
191
|
-
Copy the recorded `selector:` map when you convert a wait. Do not use the loose bare-string form.
|
|
191
|
+
Copy the recorded `selector:` map when you convert a wait. Do not use the loose bare-string form. E2E YAML accepts `identifier`; rename it to `id` only for style. Convert `textMatch: equals` to `equals:` and other text checks to `contains:`.
|
|
192
192
|
|
|
193
193
|
Only these unrecorded insertions are allowed, at states observed live:
|
|
194
194
|
|
|
@@ -196,25 +196,25 @@ Only these unrecorded insertions are allowed, at states observed live:
|
|
|
196
196
|
- `await: { idle: true }` after a navigation identity check.
|
|
197
197
|
- The Chromium launch that packages the live boot.
|
|
198
198
|
|
|
199
|
-
Keep raw forms only when conversion changes behavior. Examples include point-anchored or panning pinch, an edge swipe or one with exotic velocity control, or rotation with a tested start angle, radius, pivot, duration, or speed. Keep screenshots for human evidence. Use `snapshot:` for automated visual comparison. Read [
|
|
199
|
+
Keep raw forms only when conversion changes behavior. Examples include point-anchored or panning pinch, an edge swipe or one with exotic velocity control, or rotation with a tested start angle, radius, pivot, duration, or speed. Keep screenshots for human evidence. Use `snapshot:` for automated visual comparison. Read [E2E YAML](e2e-yaml.md) for syntax.
|
|
200
200
|
|
|
201
201
|
If polish reveals a missing action or structural check, restore its preceding state and record it. Do not add remembered behavior directly to YAML.
|
|
202
202
|
|
|
203
203
|
## Worked example
|
|
204
204
|
|
|
205
|
-
`
|
|
205
|
+
`TEST` below abbreviates `name: "open-settings", project_root: "/Users/dev/AcmeNotes"`. Repeat both fields in every call.
|
|
206
206
|
|
|
207
207
|
```text
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
208
|
+
e2e-start-recording { TEST }
|
|
209
|
+
e2e-add-echo { TEST, message: "Restart Acme Notes; expect Home" }
|
|
210
|
+
e2e-add-step { TEST, command: "restart-app", args: "{\"udid\":\"ABC\",\"bundleId\":\"com.acme.notes\"}" }
|
|
211
211
|
# captured as: - launch: com.acme.notes
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
212
|
+
e2e-add-step { TEST, command: "await-ui-element", args: "{\"udid\":\"ABC\",\"condition\":\"visible\",\"selector\":{\"identifier\":\"home-screen\"}}" }
|
|
213
|
+
e2e-add-echo { TEST, message: "On Home; open Settings" }
|
|
214
|
+
e2e-add-step { TEST, command: "gesture-tap", args: "{\"udid\":\"ABC\",\"x\":0.91,\"y\":0.94}" }
|
|
215
215
|
# pre-tap capture resolves to: - tap: { id: settings-tab }
|
|
216
|
-
|
|
217
|
-
|
|
216
|
+
e2e-add-step { TEST, command: "await-ui-element", args: "{\"udid\":\"ABC\",\"condition\":\"visible\",\"selector\":{\"identifier\":\"settings-screen\"}}" }
|
|
217
|
+
e2e-finish-recording { TEST }
|
|
218
218
|
```
|
|
219
219
|
|
|
220
220
|
After meaning-preserving conversion:
|
|
@@ -237,15 +237,15 @@ Run these checks before replay:
|
|
|
237
237
|
|
|
238
238
|
```text
|
|
239
239
|
# Weak targets: coordinates, raw gestures, role-only selectors
|
|
240
|
-
rg -n '(\{ *x:|^ +(x|centerX|fromX|toX):|gesture-(tap|swipe|scroll|drag|pinch|rotate|custom))' .argent/
|
|
241
|
-
rg -n -B2 '^ +role:' .argent/
|
|
240
|
+
rg -n '(\{ *x:|^ +(x|centerX|fromX|toX):|gesture-(tap|swipe|scroll|drag|pinch|rotate|custom))' .argent/e2e/<name>.yaml
|
|
241
|
+
rg -n -B2 '^ +role:' .argent/e2e/<name>.yaml
|
|
242
242
|
# Stored device ids
|
|
243
|
-
rg -n '(udid|device_id)' .argent/
|
|
243
|
+
rg -n '(udid|device_id)' .argent/e2e/<name>.yaml
|
|
244
244
|
# Positional ids and loose condition selectors
|
|
245
|
-
rg -n '(-selector-\d+|selector-\d+\b)' .argent/
|
|
246
|
-
rg -n '(visible|hidden|exists) *: *["'"'"'A-Za-z0-9]' .argent/
|
|
245
|
+
rg -n '(-selector-\d+|selector-\d+\b)' .argent/e2e/<name>.yaml
|
|
246
|
+
rg -n '(visible|hidden|exists) *: *["'"'"'A-Za-z0-9]' .argent/e2e/<name>.yaml
|
|
247
247
|
# Fixed waits and skipped navigation
|
|
248
|
-
rg -n '^\s*- wait:|open-url' .argent/
|
|
248
|
+
rg -n '^\s*- wait:|open-url' .argent/e2e/<name>.yaml
|
|
249
249
|
```
|
|
250
250
|
|
|
251
251
|
The condition grep matches a condition key with a scalar after it (`visible: Save`) and not `visible:` opening a map. Recorder output is always block style, so both forms are one line below their `await:`/`assert:`. It covers `when:` guards too, which take the same loose fallback.
|
|
@@ -260,38 +260,38 @@ Resolve every hit and confirm:
|
|
|
260
260
|
- Every fixed wait has an echo and a following hard check. Prefer a condition or `idle`.
|
|
261
261
|
- No `open-url` replaces tested navigation.
|
|
262
262
|
- Every snapshot is intentional, deterministic, non-mutating, and ready for reviewed baseline creation.
|
|
263
|
-
- The
|
|
263
|
+
- The standalone launch and real first-screen gate are present. Only Chromium permits an inserted launch.
|
|
264
264
|
- Every screen change has a destination-only identity check and an `idle` readiness check. The two are repaired differently: record a missing identity check live on the restored screen, but add a missing readiness gate in YAML, because `await: { idle: true }` has no recorder form.
|
|
265
265
|
- Every `hidden` check follows a `visible` check on the same stable selector and the removing action. A proven containing screen is not a substitute, because it is no evidence the selector itself ever resolved.
|
|
266
266
|
|
|
267
267
|
## Replay
|
|
268
268
|
|
|
269
|
-
Run `
|
|
269
|
+
Run `e2e-execute` on the complete YAML with the absolute project root. For a fragment, verify its prerequisite before setting `prerequisiteAcknowledged: true`.
|
|
270
270
|
|
|
271
|
-
`
|
|
271
|
+
`e2e-execute` takes exactly one test source: `name`, for a test saved under `.argent/e2e/` (or the legacy `.argent/flows/`), or `flow_path`, an absolute path to any test `.yaml`.
|
|
272
272
|
|
|
273
|
-
Over `argent link` or `ARGENT_TOOLS_URL`, the client sends the
|
|
273
|
+
Over `argent link` or `ARGENT_TOOLS_URL`, the client sends the test file, its `run:` fragments, the tests that its `tool: e2e-execute` steps name, the snapshot baselines of each run and the file arguments of its `tool:` steps with the call. These steps work over a link:
|
|
274
274
|
|
|
275
|
-
- `run:` fragments and the
|
|
276
|
-
- `snapshot:` steps in the
|
|
277
|
-
- A `tool:` step whose file argument is an absolute path to a `.png` or `.yaml` file, such as `baselinePath` of `screenshot-diff`. Keep the real file under the project root or beside the
|
|
278
|
-
- A `tool:
|
|
275
|
+
- `run:` fragments and the tests that `tool: e2e-execute` steps name, when they are under `project_root`, under its `.argent/e2e/` or `.argent/flows/`, or beside the test file. Keep them there.
|
|
276
|
+
- `snapshot:` steps in the test and in its fragments. The baselines stay in the project. With `updateBaselines: true`, the client writes each new baseline when the run ends. A `baseline not written` line in the report means that the client did not save that baseline: fix the reason in the line and run again.
|
|
277
|
+
- A `tool:` step whose file argument is an absolute path to a `.png` or `.yaml` file, such as `baselinePath` of `screenshot-diff`. Keep the real file under the project root or beside the test file, not behind a symlink to another place. Otherwise the step reports `error` when it runs, after the earlier steps acted.
|
|
278
|
+
- A `tool: e2e-execute` step that names its test with `name` and an absolute `project_root` without a `..` segment. Keep the test that the step names under the project on the client, and spell `name` in the letter case of its file. That test uses its own baselines. A replay with `updateBaselines` also updates them, unless the step sets `updateBaselines` itself.
|
|
279
279
|
|
|
280
|
-
These steps do not work over a link, in the
|
|
280
|
+
These steps do not work over a link, in the test or in a fragment or test that it runs:
|
|
281
281
|
|
|
282
282
|
- `script:`
|
|
283
283
|
- a `tool:` step that takes a directory, an app or an output directory, such as `reinstall-app`, `gather-workspace-data`, or `screenshot-diff` with `outputDir`
|
|
284
284
|
- a `tool:` step with a relative file path, or a file other than `.png` or `.yaml`
|
|
285
|
-
- a `tool:` step whose tool builds the file path from several arguments, such as `
|
|
286
|
-
- a `tool:
|
|
287
|
-
- a `tool:` step that records
|
|
285
|
+
- a `tool:` step whose tool builds the file path from several arguments, such as `e2e-read-prerequisite` with `name`
|
|
286
|
+
- a `tool: e2e-execute` step with `flow_path`, or with `name` and a relative `project_root` or one with a `..` segment
|
|
287
|
+
- a `tool:` step that records an e2e test
|
|
288
288
|
|
|
289
|
-
The tool-server checks the
|
|
289
|
+
The tool-server checks the test, its fragments and the tests that it runs before the first step. The error lists each step that does not work over a link, and each `run:` or `tool: e2e-execute` step whose test the client could not send, with the reason. A fragment or a test that does not exist fails its step when the step runs. Run a test that needs these steps with no link and no `ARGENT_TOOLS_URL`, on the computer that runs the tool-server.
|
|
290
290
|
|
|
291
291
|
Manual rescue invalidates the pass. An `errored` step was never evaluated: an `idle` wait whose tree source could not be read, a step that threw, an unresolvable `run:` target, or a `launch:` that did not start the app. Read the reason: most name the environment, but a failed `launch:` is a verdict about the app. Unconfirmed focus is not in this class at all: the replay focus poll has no failure return, so a `type:` step whose focus was never confirmed is scored a **pass**, and only the value check after typing catches it.
|
|
292
292
|
|
|
293
|
-
**A passing step that carries a `warning` is a finding, not noise.** `await: { idle: true }` raises [six different warnings](
|
|
293
|
+
**A passing step that carries a `warning` is a finding, not noise.** `await: { idle: true }` raises [six different warnings](e2e-yaml.md#idle-readiness) and they do not share one meaning. Two say the screen was moving; one says the wait ran out mid-hold and is repaired by raising the step's `timeout:`; one says the tree stayed empty; one says the tree did hold still and only the screenshot pairs were missing, so the capture path is what to check; one says the step ended with no evidence either way. No report separates intended motion from a load that never finished. Read which one it is, look at that screen, disclose what you found, and confirm the following step targets a stable element rather than stillness.
|
|
294
294
|
|
|
295
|
-
A [selector-less gesture](
|
|
295
|
+
A [selector-less gesture](e2e-yaml.md#directives) raises a warning of a different shape, not one of those six: a tree-source outage left it unsettled, so it dispatched blind and the green says only that the gesture was sent. Restore the source, usually by relaunching the app so the instrumentation loads. Accept it only where the app serves no tree at all, such as the [injection-free iOS form](reliability-and-recovery.md#terminally-non-injectable-ios-apps).
|
|
296
296
|
|
|
297
|
-
One uninterrupted full pass completes a normal
|
|
297
|
+
One uninterrupted full pass completes a normal e2e test. `argent-qa-e2e` requires two consecutive passes of unchanged YAML. For CI, use `argent e2e run <name> [--platform ...]`; it exits non-zero on failure.
|
package/skills/{argent-create-flow → argent-create-e2e}/references/reliability-and-recovery.md
RENAMED
|
@@ -26,49 +26,49 @@ Convert element-seeking swipes to `scroll-to`. Keep a coordinate swipe only when
|
|
|
26
26
|
|
|
27
27
|
Work this gate as soon as capture warns that it kept a raw point, and equally when it silently recorded a role-only selector, which warns about nothing. Keep the source screen available and do these checks:
|
|
28
28
|
|
|
29
|
-
1. **iOS:** query plausible ids or labels with `native-find-views`. If no term is useful, call `native-full-hierarchy` with narrow fields and `maxDepth: 100`. `describe` and `native-describe-screen` are accessibility projections. They cannot prove that no
|
|
29
|
+
1. **iOS:** query plausible ids or labels with `native-find-views`. If no term is useful, call `native-full-hierarchy` with narrow fields and `maxDepth: 100`. `describe` and `native-describe-screen` are accessibility projections. They cannot prove that no YAML selector exists.
|
|
30
30
|
2. **Other platforms:** use `debugger-component-tree` for React Native; otherwise, use `describe`. Verify Android and Chromium candidates in step 3. Their discovery trees can omit runner elements.
|
|
31
31
|
3. Test each candidate in a scratch fragment with `assert: { visible: <candidate> }` on the valid screen. Inspect every failure before trying a better id, label, app, or container.
|
|
32
32
|
4. If source is available, inspect its `testID`, `accessibilityIdentifier`, or `resource-id`. If none exists, report the missing stable id as the real fix.
|
|
33
33
|
|
|
34
34
|
An unavailable tree makes the candidate test void. It proves the tree was absent, not that the selector failed, so it never authorizes coordinates. Relevant failures include `native devtools is unavailable` or `No native-devtools-connected apps are available` on iOS, an unreachable Android helper, an unreachable Chromium CDP session, or missing Vega page source. The recorder quotes the same reason back in its `selector capture failed` warning, so read that warning before treating it as a verdict about the element. Restore the tree and repeat the test.
|
|
35
35
|
|
|
36
|
-
Keep coordinates only for a genuinely unlabeled target or after all plausible labeled candidates fail against a working
|
|
36
|
+
Keep coordinates only for a genuinely unlabeled target or after all plausible labeled candidates fail against a working runner tree. Add an echo naming the target and a hard check on the action's outcome. Report the point, discovery results, and candidate failures. Re-record every uncleared point.
|
|
37
37
|
|
|
38
|
-
QA
|
|
38
|
+
QA e2e tests are stricter. They can keep a coordinate only for a genuinely unlabeled target. Failed selector candidates alone are insufficient.
|
|
39
39
|
|
|
40
40
|
## iOS selector recovery
|
|
41
41
|
|
|
42
|
-
The full iOS
|
|
42
|
+
The full iOS runner tree exists only for an app launched by Argent with instrumentation.
|
|
43
43
|
|
|
44
44
|
1. If Metro, Expo, Xcode, an icon, or a prior process launched the app, call `restart-app`. Restore the source screen and retry capture. `launch-app` can only foreground the existing process.
|
|
45
45
|
2. Tap capture does **not** wait for that connection. It makes one tree read and turns any failure straight into the kept-coordinates warning. A recording-time `restart-app` returns before the devtools connection opens, so the first tap after a restart can warn transiently. Re-record that tap once before escalating; only a warning that survives the retry is evidence of a real fault.
|
|
46
46
|
3. If the warning survives, call `native-devtools-status` with the same UDID and bundle id and follow its `message`, which names the one action that helps and says when to stop. `requiresRestart` covers only the states a fresh process fixes: an `unregistered` or `connecting` app reports it false, because it already launched under the terms a restart would recreate.
|
|
47
47
|
4. If an injectable app remains disconnected, call `stop-all-simulator-servers` once, **scoped to `devices: [<this simulator's UDID>]`**. One tool-server serves every agent on this Argent install, so an unscoped call tears down their devices too. This does not change app or account data. Then restart and check status again.
|
|
48
|
-
5. If it still fails, report an instrumentation blocker. Do not replace selectors with coordinates in a QA
|
|
48
|
+
5. If it still fails, report an instrumentation blocker. Do not replace selectors with coordinates in a QA e2e test.
|
|
49
49
|
|
|
50
|
-
Use the same explicit UDID throughout. Multiple booted simulators are not an injection fault. Pass `--device <udid>` when
|
|
50
|
+
Use the same explicit UDID throughout. Multiple booted simulators are not an injection fault. Pass `--device <udid>` when CLI device selection is ambiguous.
|
|
51
51
|
|
|
52
52
|
### Terminally non-injectable iOS apps
|
|
53
53
|
|
|
54
54
|
This fallback applies only to `com.apple.*` system apps. A connection failure in another app never authorizes it.
|
|
55
55
|
|
|
56
|
-
Argent refuses `com.apple.*` bundle ids at every native-devtools read that names one, because a system app is never the app under test. The instrumentation has been seen both loading and not loading into one, depending on the simulator runtime; either way it is no basis for a selector. `restart-app`, `launch-app`, and `describe` still work on one; it just never gets a
|
|
56
|
+
Argent refuses `com.apple.*` bundle ids at every native-devtools read that names one, because a system app is never the app under test. The instrumentation has been seen both loading and not loading into one, depending on the simulator runtime; either way it is no basis for a selector. `restart-app`, `launch-app`, and `describe` still work on one; it just never gets a runner tree.
|
|
57
57
|
|
|
58
|
-
Give the
|
|
58
|
+
Give the test a `launch:` step as usual. On iOS the launch waits the full devtools budget out, then passes for one of these bundle ids: starting the app is all that step is for, and a coordinate-driven test needs nothing more. The test stays standalone; it just pays roughly sixteen seconds at the launch. Where the refusal bites is selector resolution, and the first selector step reports it there (terminally, naming the coordinate remedy) rather than as a launch failure. The rest of the tree-free form:
|
|
59
59
|
|
|
60
60
|
- Raw `tool: await-ui-element` accessibility checks.
|
|
61
61
|
- Point taps or long-presses derived from `describe`, each named by an echo.
|
|
62
62
|
- A point focus tap plus a raw text-only `keyboard` with `delayMs: 500`, and a second raw `keyboard` with `key: "enter"` to submit.
|
|
63
|
-
- Raw swipes with `momentum: false` because `scroll-to` needs the missing
|
|
63
|
+
- Raw swipes with `momentum: false` because `scroll-to` needs the missing runner tree. Momentum-free scrolling keeps later coordinate taps valid. `momentum: false` needs `durationMs` of at least 150 and is rejected below it, so keep the 300 default or raise it.
|
|
64
64
|
|
|
65
|
-
Every point tap, long-press or coordinate swipe in such a
|
|
65
|
+
Every point tap, long-press or coordinate swipe in such a test passes **carrying a warning** for as long as the app serves no tree: each [selector-less gesture](e2e-yaml.md#directives) dispatches unsettled. Nothing here repairs it. Accept the warnings, read each green as "the gesture was sent, not that it landed", and put an explicit `wait:` or a raw `tool: await-ui-element` before a gesture that follows a transition. Raw `tool:` steps take no settle, so they never carry that warning.
|
|
66
66
|
|
|
67
67
|
A recorded wait carries a different warning: it adds about one second and reports that the runner tree is unavailable. That warning is expected too. Keep the wait as a raw `tool:` step.
|
|
68
68
|
|
|
69
|
-
Report that the
|
|
69
|
+
Report that the test has no runner tree and its coordinates are not portable. It cannot satisfy the QA contract. Report the artifact and platform blocker instead.
|
|
70
70
|
|
|
71
|
-
A normally injectable app that is broken in the environment gets the same coordinate-only treatment, but not the same launch: there the `launch:` step fails, since the gate withholds its verdict only for a bundle id argent refuses outright. Start such a
|
|
71
|
+
A normally injectable app that is broken in the environment gets the same coordinate-only treatment, but not the same launch: there the `launch:` step fails, since the gate withholds its verdict only for a bundle id argent refuses outright. Start such a test with a raw `tool: restart-app`, which terminates and relaunches without the readiness gate, and accept that the result is a **fragment**: its first step that is neither `echo:` nor `script:` is not `launch:`, so the runner never classifies it as standalone, and it cannot complete `argent-qa-e2e`, which requires a leading `launch:`. Report the blocker rather than labeling that fallback a completed QA test.
|
|
72
72
|
|
|
73
73
|
## Tree source recovery on Android, Chromium, and Vega
|
|
74
74
|
|
|
@@ -80,7 +80,7 @@ While the required source is down, selector failures and raw-point capture are v
|
|
|
80
80
|
| Chromium | No reachable CDP session | Boot again with `electronAppPath` and remote debugging |
|
|
81
81
|
| Vega | Toolkit returns no page source | Relaunch an app built with automation support |
|
|
82
82
|
|
|
83
|
-
On Android, healthy `describe` output does not prove the
|
|
83
|
+
On Android, healthy `describe` output does not prove the runner tree is available. It can fall back to legacy `uiautomator`, while the runner refuses that trimmed fallback.
|
|
84
84
|
|
|
85
85
|
## Strong transition gates
|
|
86
86
|
|
|
@@ -114,8 +114,8 @@ Classify before editing:
|
|
|
114
114
|
| Silent misfire | The run passes but final state is wrong | Restore the first wrong screen and record a stronger gate |
|
|
115
115
|
| Partial divergence | An intermediate result disagrees with its echo | Find the first divergent transition |
|
|
116
116
|
| Acceptance failure | Actions pass but a requested check fails | Preserve the check and investigate behavior |
|
|
117
|
-
| Idle warning | A readiness step passes without settling | Read [which of the six warnings](
|
|
118
|
-
| Unsettled gesture | A selector-less gesture passes unsettled | Restore the tree source, usually by relaunching the app; the green says [only that the gesture was sent](
|
|
117
|
+
| Idle warning | A readiness step passes without settling | Read [which of the six warnings](e2e-yaml.md#idle-readiness) it is, then gate the next action on a stable element |
|
|
118
|
+
| Unsettled gesture | A selector-less gesture passes unsettled | Restore the tree source, usually by relaunching the app; the green says [only that the gesture was sent](e2e-yaml.md#directives) |
|
|
119
119
|
|
|
120
120
|
Then:
|
|
121
121
|
|
|
@@ -130,7 +130,7 @@ Then:
|
|
|
130
130
|
- For one parameter or selector error, edit the YAML and prefer a stable selector.
|
|
131
131
|
- For readiness or identity failure, repair that gate and audit every transition with the same shape.
|
|
132
132
|
- For one missing transition or two to three structural steps, copy the working prefix and re-record the affected span live.
|
|
133
|
-
- For four or more broken steps, unclear state, or a comparison or profiling
|
|
133
|
+
- For four or more broken steps, unclear state, or a comparison or profiling test, fully re-record.
|
|
134
134
|
- Treat manual recovery as diagnosis only. It never counts as a replay pass.
|
|
135
135
|
|
|
136
136
|
Starting again under the same name truncates the YAML. Copy any working prefix before re-recording.
|
|
@@ -149,6 +149,6 @@ State the added proof before rerunning.
|
|
|
149
149
|
|
|
150
150
|
### Correction limit
|
|
151
151
|
|
|
152
|
-
After each correction, audit and replay from the declared start. Stop after two unsuccessful correction cycles and report the remaining blocker. If failures move while the
|
|
152
|
+
After each correction, audit and replay from the declared start. Stop after two unsuccessful correction cycles and report the remaining blocker. If failures move while the test grows, re-record the affected span instead of adding more patches.
|
|
153
153
|
|
|
154
|
-
Never weaken, remove, or hide a requested check to obtain a pass. Keep a failing product check and report the
|
|
154
|
+
Never weaken, remove, or hide a requested check to obtain a pass. Keep a failing product check and report the test as an unproven regression artifact. QA remains incomplete until its two-pass gate succeeds.
|
|
@@ -171,7 +171,7 @@ One call does one action. `text` and `key` are mutually exclusive, and a call th
|
|
|
171
171
|
|
|
172
172
|
Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12`. Optional: `"delayMs": 100` between keystrokes (default 50ms); applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on Vega, and on TV targets.
|
|
173
173
|
|
|
174
|
-
**Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and
|
|
174
|
+
**Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and e2e `type` steps):
|
|
175
175
|
|
|
176
176
|
```json
|
|
177
177
|
{ "udid": "<UDID>", "text": "{{secret:APP_PASSWORD}}" }
|
|
@@ -249,7 +249,7 @@ Use after launch/navigation and before a raw tap, when an early-painted element
|
|
|
249
249
|
|
|
250
250
|
On local iOS, Android, and Chromium, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen.
|
|
251
251
|
|
|
252
|
-
Use it only for live diagnosis. Do not record it or put it in `run-sequence`.
|
|
252
|
+
Use it only for live diagnosis. Do not record it or put it in `run-sequence`. E2E tests use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation.
|
|
253
253
|
|
|
254
254
|
---
|
|
255
255
|
|
|
@@ -21,4 +21,4 @@ Note: a cross-process navigation (some redirects) can swap a tab's underlying CD
|
|
|
21
21
|
|
|
22
22
|
`chromium-storage` reads/writes Web Storage for the active page: `store=local|session`, `action=get` (one `key` or all entries), `set`, `remove`, `clear`.
|
|
23
23
|
|
|
24
|
-
Both are per-origin / active-tab. Handy for seeding auth before
|
|
24
|
+
Both are per-origin / active-tab. Handy for seeding auth before an e2e test or asserting app state after one.
|
|
@@ -15,6 +15,6 @@ Read this only for a physical iPhone. `argent-device-interact` still applies for
|
|
|
15
15
|
|
|
16
16
|
## Tools on hardware
|
|
17
17
|
|
|
18
|
-
- Only these exist: `list-devices`, `launch-app`, `restart-app`, `reinstall-app`, `open-url`, `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `gesture-custom`, `button` (`home`, `volumeUp`, `volumeDown`, `actionButton` on models that have one), `keyboard`, `await-ui-element`, `await-screen-idle`, `run-sequence`, the
|
|
18
|
+
- Only these exist: `list-devices`, `launch-app`, `restart-app`, `reinstall-app`, `open-url`, `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `gesture-custom`, `button` (`home`, `volumeUp`, `volumeDown`, `actionButton` on models that have one), `keyboard`, `await-ui-element`, `await-screen-idle`, `run-sequence`, the e2e tools, `stop-simulator-server` and `stop-all-simulator-servers` (session end). Every other device tool fails with `not supported on ios device`.
|
|
19
19
|
- No two-finger gestures, `rotate`, `shake`, `paste`, `settings-permissions`, screen recording, `debugger-*`, `react-profiler-*`, `native-profiler-*`, `native-*`, `boot-device`: drive the app's own zoom and rotate UI with taps and drags, move the phone by hand, type with `keyboard`, change permissions in the phone's Settings, and debug, profile or record on a simulator.
|
|
20
20
|
- `gesture-swipe`: `durationMs` sets drag speed, not time; `momentum:false` only rests 300 ms at the end, no damping. `open-url` https lands in Safari, never in the app that owns the link: pass `bundleId`.
|
|
@@ -32,7 +32,7 @@ After `native-profiler-analyze` surfaces findings, use `profiler-stack-query` to
|
|
|
32
32
|
|
|
33
33
|
After presenting findings, ask the user whether to investigate further, implement fixes, or stop. After applying fixes, always re-profile the same scenario and compare with `profiler-load`. Report honestly whether the target metric improved, regressed, or stayed flat. If the fix showed no net benefit or introduced regressions elsewhere, say so and reconsider.
|
|
34
34
|
|
|
35
|
-
**Tip:** For reproducible before/after comparisons, record the interaction sequence as
|
|
35
|
+
**Tip:** For reproducible before/after comparisons, record the interaction sequence as an e2e test using the `argent-create-e2e` skill before the first profiling run. Replay with `e2e-execute` on subsequent runs to eliminate interaction variance.
|
|
36
36
|
|
|
37
37
|
> **Note:** The `argent-react-native-profiler` instructs to start native profiling automatically alongside React profiling. This skill's workflow and investigation patterns apply in both cases.
|
|
38
38
|
|