flowstride 1.0.20-rc.6 → 1.0.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/NATIVE_BROWSER.md +341 -0
- package/NATIVE_CI.md +88 -169
- package/README.md +46 -214
- package/dist/adapters/api/undici.d.ts +8 -0
- package/dist/adapters/api/undici.js +11 -0
- package/dist/adapters/web/playwright.d.ts +25 -2
- package/dist/adapters/web/playwright.js +285 -21
- package/dist/ci/config.js +14 -1
- package/dist/ci/distribution.d.ts +38 -0
- package/dist/ci/distribution.js +134 -0
- package/dist/ci/init.d.ts +4 -1
- package/dist/ci/init.js +13 -2
- package/dist/ci/runtime.js +24 -5
- package/dist/cli/commands/run.js +3 -0
- package/dist/cli/index.js +24 -5
- package/dist/manual/security.d.ts +2 -2
- package/dist/native/accessibility.d.ts +3 -0
- package/dist/native/accessibility.js +68 -0
- package/dist/native/assertions.d.ts +2 -0
- package/dist/native/assertions.js +140 -0
- package/dist/native/compatibility.d.ts +3 -0
- package/dist/native/compatibility.js +16 -0
- package/dist/native/components.d.ts +12 -0
- package/dist/native/components.js +122 -0
- package/dist/native/dashboard.d.ts +7 -0
- package/dist/native/dashboard.js +32 -0
- package/dist/native/events.d.ts +34 -0
- package/dist/native/events.js +281 -0
- package/dist/native/evidence.d.ts +3 -0
- package/dist/native/evidence.js +31 -0
- package/dist/native/functions.d.ts +9 -0
- package/dist/native/functions.js +103 -0
- package/dist/native/runtime.d.ts +14 -0
- package/dist/native/runtime.js +154 -0
- package/dist/native/validation.d.ts +107 -0
- package/dist/native/validation.js +84 -0
- package/dist/parser/lexer.js +15 -4
- package/dist/parser/parser.js +1 -1
- package/dist/runner/flow_orchestrator.d.ts +4 -0
- package/dist/runner/flow_orchestrator.js +22 -5
- package/dist/runner/flow_worker.d.ts +7 -0
- package/dist/runner/flow_worker.js +141 -37
- package/dist/security/redaction.js +3 -1
- package/dist/sessions/manager.js +3 -1
- package/dist/types/index.d.ts +12 -0
- package/dist/utils/replayRequest.d.ts +8 -0
- package/dist/utils/replayRequest.js +77 -0
- package/package.json +22 -9
- package/src/ui/dist/assets/index-DglsL_J2.js +636 -0
- package/src/ui/dist/index.html +1 -1
- package/src/ui/dist/assets/index-DVwp0K-J.js +0 -462
|
@@ -0,0 +1,341 @@
|
|
|
1
|
+
# Native browser workflows
|
|
2
|
+
|
|
3
|
+
OS and Cloud support these commands directly in `.flow` files. Existing commands, plugins and saved sessions continue to work. Use `flow.<area>.<action>` followed by a JSON object. Unknown actions and options fail explicitly.
|
|
4
|
+
|
|
5
|
+
Keep both engine installations current when using these features. If an installed Cloud engine lacks the required capability protocol, the public runner warns and uses OS fallback for the run. Existing commands can still use an older compatible Cloud engine; Cloud-only services require a compatible Cloud installation.
|
|
6
|
+
|
|
7
|
+
## Separate users in one test
|
|
8
|
+
|
|
9
|
+
A **profile** is an independent browser context: its cookies, local storage, session storage, pages and permissions are separate. The original profile is named `default`. Creating a profile selects it; switching profiles returns to the page that user was using.
|
|
10
|
+
|
|
11
|
+
```flow
|
|
12
|
+
flow.browser.create {
|
|
13
|
+
"name": "customer",
|
|
14
|
+
"options": {
|
|
15
|
+
"locale": "en-GB",
|
|
16
|
+
"colorScheme": "dark"
|
|
17
|
+
}
|
|
18
|
+
};
|
|
19
|
+
flow.open "/login";
|
|
20
|
+
flow.type input "Email" "{{env.CUSTOMER_EMAIL}}";
|
|
21
|
+
flow.type input "Password" "{{env.CUSTOMER_PASSWORD}}";
|
|
22
|
+
flow.click button "Sign in";
|
|
23
|
+
flow.browser.create {
|
|
24
|
+
"name": "admin"
|
|
25
|
+
};
|
|
26
|
+
flow.open "/admin/login";
|
|
27
|
+
flow.browser.use {
|
|
28
|
+
"name": "customer"
|
|
29
|
+
};
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
`flow.save session "customer" persist;` still saves a snapshot of the **currently selected** profile. `flow.use session "customer";` still injects a saved snapshot into the current profile. Saving a snapshot does not create another live user. Sequential customer/admin scenarios remain valid without profiles; use profiles when the users must stay independently logged in.
|
|
33
|
+
|
|
34
|
+
`flow.browser.close` closes a named profile. The `default` profile remains available until file completion. Profiles are reset between files. `flow.switchTo` still selects tabs within the current profile.
|
|
35
|
+
|
|
36
|
+
## Browser selection and settings
|
|
37
|
+
|
|
38
|
+
```sh
|
|
39
|
+
npx flowstride install chromium firefox webkit
|
|
40
|
+
npx flowstride run flows --browser firefox --ci
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Use `--browser firefox` or `--browser webkit` to select the test engine. The terminal prints the verified engine and version after launch; the dashboard shows **Test browser**, and CI's `summary.json` includes the actual engine for each file. `install` downloads browser binaries and does not open a browser window.
|
|
44
|
+
|
|
45
|
+
Interactive runs display the dashboard in a separate Firefox or WebKit window when either engine is selected. The dashboard remains in the system default browser for Chromium runs. Test pages run headlessly by default; add `--headed` to see the application's own browser window:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
npx flowstride run flows --browser firefox --headed
|
|
49
|
+
npx flowstride run flows --browser webkit --headed
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
CI forces headless execution and opens no dashboard. Firefox and WebKit keep their native user-agent unless you explicitly configure `browserOptions.userAgent`. WebKit is Safari's engine, rather than the installed Safari application.
|
|
53
|
+
|
|
54
|
+
Chromium remains the default. Browser installation must match the installed Flowstride package. In Linux CI, add `--with-deps` to the install command. Browser support does not mean every operating-system/browser/device combination is qualified. Firefox does not support Playwright's mobile emulation setting; phone profiles use its viewport/touch settings without that flag. Chromium-specific game touch and audio-file injection still require Chromium.
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"browser": "chromium",
|
|
59
|
+
"browserOptions": {
|
|
60
|
+
"locale": "en-GB",
|
|
61
|
+
"timezoneId": "Europe/London",
|
|
62
|
+
"colorScheme": "dark",
|
|
63
|
+
"permissions": ["geolocation"],
|
|
64
|
+
"geolocation": {
|
|
65
|
+
"latitude": 6.5244,
|
|
66
|
+
"longitude": 3.3792
|
|
67
|
+
},
|
|
68
|
+
"serviceWorkers": "block"
|
|
69
|
+
}
|
|
70
|
+
}
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Validated settings include viewport/screen, device scale, touch/mobile emulation, locale/timezone, color/contrast/motion, JavaScript, HTTPS handling, CSP bypass, permissions/geolocation, offline mode, headers, HTTP credentials, proxy, downloads, strict selectors, storage state and project-local client certificate paths. Use `browserOptions` for the default profile and `options` in `flow.browser.create` for a separate profile. Secrets belong in environment variables or private local files.
|
|
74
|
+
|
|
75
|
+
Runtime commands include `flow.browser.offline` with `value`, `flow.browser.permissions` with `permissions` and optional `origin`, `flow.browser.clearPermissions`, and `flow.browser.geolocation` with latitude/longitude. Recording paths are controlled by evidence settings, rather than arbitrary browser path options. JavaScript logger callbacks are not JSON DSL settings.
|
|
76
|
+
|
|
77
|
+
## Assertions that wait for a condition
|
|
78
|
+
|
|
79
|
+
```flow
|
|
80
|
+
flow.assert.that {
|
|
81
|
+
"selector": "#order-status",
|
|
82
|
+
"condition": "text",
|
|
83
|
+
"value": "Approved",
|
|
84
|
+
"timeout": 10000
|
|
85
|
+
};
|
|
86
|
+
flow.assert.that {
|
|
87
|
+
"role": "button",
|
|
88
|
+
"name": "Submit",
|
|
89
|
+
"condition": "enabled"
|
|
90
|
+
};
|
|
91
|
+
flow.assert.that {
|
|
92
|
+
"selector": ".loading",
|
|
93
|
+
"condition": "visible",
|
|
94
|
+
"not": true
|
|
95
|
+
};
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Choose one CSS/Playwright `selector` or accessible `role` with an optional exact `name`. An optional `frame` selects an iframe. Conditions include:
|
|
99
|
+
|
|
100
|
+
| Conditions | Expected input |
|
|
101
|
+
| --- | --- |
|
|
102
|
+
| `visible`, `hidden`, `attached`, `enabled`, `disabled`, `editable`, `checked`, `empty`, `focused`, `inViewport` | No value |
|
|
103
|
+
| `text`, `containsText`, `value`, `id`, `accessibleName`, `accessibleDescription` | `value`, or `matches` and optional regex `flags` |
|
|
104
|
+
| `values` | Array of selected option values |
|
|
105
|
+
| `count` | Nonnegative integer `value` |
|
|
106
|
+
| `attribute` | `attribute`; optional expected `value`; omit it to check presence |
|
|
107
|
+
| `css`, `property` | `property` and expected `value` |
|
|
108
|
+
| `url`, `title` | Expected `value` or regex `matches`; no selector needed |
|
|
109
|
+
| `localStorage`, `sessionStorage` | `key` and expected string `value`, or `null` for absence |
|
|
110
|
+
|
|
111
|
+
Text comparisons normalize whitespace. Assertions retry until `timeout` or the configured default; they do not repeatedly click or repeat requests. Existing `flow.expect` syntax remains supported.
|
|
112
|
+
|
|
113
|
+
## Mock requests and wait for network events
|
|
114
|
+
|
|
115
|
+
Use `serviceWorkers: "block"` when mocking an application that registers service workers. Mocking applies to browser traffic in the current profile; it does not intercept `flow.get` or other direct API commands.
|
|
116
|
+
|
|
117
|
+
```flow
|
|
118
|
+
flow.network.route {
|
|
119
|
+
"name": "customer-response",
|
|
120
|
+
"url": "**/api/customer",
|
|
121
|
+
"method": "GET",
|
|
122
|
+
"json": {
|
|
123
|
+
"name": "Documentation Reader"
|
|
124
|
+
},
|
|
125
|
+
"times": 1
|
|
126
|
+
};
|
|
127
|
+
flow.network.arm {
|
|
128
|
+
"name": "customer-request",
|
|
129
|
+
"url": "**/api/customer",
|
|
130
|
+
"kind": "response"
|
|
131
|
+
};
|
|
132
|
+
flow.click button "Load customer";
|
|
133
|
+
flow.network.wait {
|
|
134
|
+
"name": "customer-request"
|
|
135
|
+
};
|
|
136
|
+
flow.network.expect {
|
|
137
|
+
"name": "customer-request",
|
|
138
|
+
"status": 200,
|
|
139
|
+
"path": "name",
|
|
140
|
+
"equals": "Documentation Reader"
|
|
141
|
+
};
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
**Arm before the trigger, then wait.** This captures fast responses and prevents an unobserved timeout while a click is still finishing. Waiters are cleaned up on retry, file completion and shutdown.
|
|
145
|
+
|
|
146
|
+
Routes accept one of `json`, text `body`, or a project-local `fixture`. Optional `status`, `headers`, `contentType`, `method` and positive `times` control the response. URL patterns support `*` within a path segment and `**` across segments.
|
|
147
|
+
|
|
148
|
+
| Route `mode` | Behavior |
|
|
149
|
+
| --- | --- |
|
|
150
|
+
| `fulfill` (default) | Supply the configured response |
|
|
151
|
+
| `abort` | Block the matching request |
|
|
152
|
+
| `continue` | Send the real request; optional `request` changes URL, method, headers or body |
|
|
153
|
+
| `patch` | Fetch the real response and merge configured JSON fields or replace its text body |
|
|
154
|
+
|
|
155
|
+
Remove a route with `flow.network.unroute` and its `name`. Rules and captured events belong to the selected profile. `network.arm` also supports `kind: "request"`, a method, response status filter and timeout. `network.expect` checks status/method and captured body paths or a response/request `header`, using `equals` or `contains`.
|
|
156
|
+
|
|
157
|
+
## Downloads
|
|
158
|
+
|
|
159
|
+
```flow
|
|
160
|
+
flow.download.arm {
|
|
161
|
+
"name": "invoice"
|
|
162
|
+
};
|
|
163
|
+
flow.click link "Download invoice";
|
|
164
|
+
flow.download.save {
|
|
165
|
+
"name": "invoice",
|
|
166
|
+
"file": "downloads/invoice.json"
|
|
167
|
+
};
|
|
168
|
+
flow.download.expect {
|
|
169
|
+
"name": "invoice",
|
|
170
|
+
"filename": "invoice.json",
|
|
171
|
+
"minSize": 1,
|
|
172
|
+
"json": {
|
|
173
|
+
"paid": true
|
|
174
|
+
}
|
|
175
|
+
};
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
The destination must stay inside the project and must not already exist. Assertions support the suggested `filename`, exact `size`, `minSize`, text `contains`, exact `json`, and a SHA-256 `sha256`. Content assertions have a size limit; use size/hash checks for larger downloads. Downloads do not need a custom plugin.
|
|
179
|
+
|
|
180
|
+
## Control time and observe functions
|
|
181
|
+
|
|
182
|
+
```flow
|
|
183
|
+
flow.clock.install {
|
|
184
|
+
"time": "2026-01-01T00:00:00Z"
|
|
185
|
+
};
|
|
186
|
+
flow.open "/timer";
|
|
187
|
+
flow.clock.runFor {
|
|
188
|
+
"milliseconds": 5000
|
|
189
|
+
};
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Clock commands include `pauseAt`, `resume`, `runFor`, `fastForward`, `setFixedTime` and `setSystemTime`. Date operations accept `time`; advancing accepts `milliseconds`. Install the clock before the application installs its timers. Time control affects the browser page, not the backend or operating-system clock.
|
|
193
|
+
|
|
194
|
+
```flow
|
|
195
|
+
flow.stub.install {
|
|
196
|
+
"name": "analytics",
|
|
197
|
+
"target": "analytics.track",
|
|
198
|
+
"mode": "spy"
|
|
199
|
+
};
|
|
200
|
+
flow.click button "Purchase";
|
|
201
|
+
flow.stub.expect {
|
|
202
|
+
"name": "analytics",
|
|
203
|
+
"count": 1,
|
|
204
|
+
"args": ["purchase"]
|
|
205
|
+
};
|
|
206
|
+
flow.stub.restore {
|
|
207
|
+
"name": "analytics"
|
|
208
|
+
};
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
A spy records calls and invokes the original function. Modes `return` and `throw` replace it with a configured `value` or `error`; `async: true` returns a promise. `beforeLoad: true` installs on the next document, for functions available before application initialization. It cannot replace a function that does not yet exist. `stub.expect` accepts `count`, `minCount`, arguments and timeout. Doubles belong to a page and are restored on cleanup.
|
|
212
|
+
|
|
213
|
+
## Traces, HAR and DOM evidence
|
|
214
|
+
|
|
215
|
+
```json
|
|
216
|
+
{
|
|
217
|
+
"trace": true,
|
|
218
|
+
"har": true,
|
|
219
|
+
"domSnapshots": true,
|
|
220
|
+
"browserOptions": {
|
|
221
|
+
"serviceWorkers": "block"
|
|
222
|
+
}
|
|
223
|
+
}
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Trace ZIPs include browser actions, DOM snapshots and screenshots. HAR files capture HTTP requests and responses and finalize when their browser context closes. Evidence files include flow/profile names; retries retain earlier attempt videos/traces. A trace records browser operations; Flowstride assertion results and attempts are in `summary.json`.
|
|
227
|
+
|
|
228
|
+
Use `flow.trace.start` (optional `name`) and `flow.trace.stop` for an explicitly bounded trace instead of automatic `trace: true`. Open a local ZIP with:
|
|
229
|
+
|
|
230
|
+
```sh
|
|
231
|
+
npx flowstride trace flowstride-artifacts/example-trace.zip
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
HAR replay uses a local recorded file:
|
|
235
|
+
|
|
236
|
+
```flow
|
|
237
|
+
flow.har.replay {
|
|
238
|
+
"file": "fixtures/customer.har",
|
|
239
|
+
"notFound": "abort"
|
|
240
|
+
};
|
|
241
|
+
flow.open "/customer";
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
An optional `url` limits matching; `notFound: "fallback"` allows unmatched real requests. Replay does not restore application databases or a WebSocket session.
|
|
245
|
+
|
|
246
|
+
The dashboard preserves each step's profile and attempt, lets you choose earlier attempt screenshots, and links to optional DOM snapshots. DOM files are served as inert text. They do not rewind the live application or execute an earlier page.
|
|
247
|
+
|
|
248
|
+
Traces, HAR, downloaded files and DOM/accessibility snapshots can contain application secrets or personal data. They remain local artifacts; these new formats are not automatically uploaded to Cloud. Choose private CI artifacts and an appropriate retention period.
|
|
249
|
+
|
|
250
|
+
## Accessibility and ARIA baselines
|
|
251
|
+
|
|
252
|
+
```flow
|
|
253
|
+
flow.a11y.scan {
|
|
254
|
+
"name": "checkout",
|
|
255
|
+
"include": ["main"],
|
|
256
|
+
"tags": ["wcag2a", "wcag2aa"]
|
|
257
|
+
};
|
|
258
|
+
flow.a11y.snapshot {
|
|
259
|
+
"name": "checkout",
|
|
260
|
+
"selector": "main",
|
|
261
|
+
"baseline": "baselines/checkout.aria.yml"
|
|
262
|
+
};
|
|
263
|
+
```
|
|
264
|
+
|
|
265
|
+
Scans use axe, write a structured report, and fail on violations by default. Options include `exclude`, explicit `rules`, `disableRules`, selected `impacts`, and `maxViolations`. Automated scans cover only part of accessibility; keyboard and assistive-technology review remain necessary.
|
|
266
|
+
|
|
267
|
+
Snapshots compare the complete selected accessibility tree with a reviewed YAML baseline. `flow.a11y.match` compares an inline `yaml` string with the complete captured tree and retries until timeout. These are exact tree comparisons, not Playwright Test's partial snapshot matcher.
|
|
268
|
+
|
|
269
|
+
Create or deliberately update a baseline locally with `--update-baselines`; the snapshot command must specify `baseline`. CI refuses updates. Review baseline changes before accepting them. To capture evidence without comparing or writing a baseline, omit `baseline`.
|
|
270
|
+
|
|
271
|
+
## Component tests
|
|
272
|
+
|
|
273
|
+
Flowstride starts and cleans up a local Vite harness. Keep your normal application framework packages installed in the project. Supported integrations are vanilla JavaScript factories, React components and Vue single-file components. Other frameworks need a dedicated integration and qualification.
|
|
274
|
+
|
|
275
|
+
```flow
|
|
276
|
+
flow.component.mount {
|
|
277
|
+
"framework": "react",
|
|
278
|
+
"module": "src/components/AccountButton.tsx",
|
|
279
|
+
"export": "default",
|
|
280
|
+
"props": {
|
|
281
|
+
"label": "Open account"
|
|
282
|
+
}
|
|
283
|
+
};
|
|
284
|
+
flow.click button "Open account";
|
|
285
|
+
flow.assert.that {
|
|
286
|
+
"selector": "button",
|
|
287
|
+
"condition": "text",
|
|
288
|
+
"value": "Account opened"
|
|
289
|
+
};
|
|
290
|
+
flow.component.update {
|
|
291
|
+
"props": {
|
|
292
|
+
"label": "Updated account"
|
|
293
|
+
}
|
|
294
|
+
};
|
|
295
|
+
flow.component.unmount {};
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
An optional `css` array imports project styles. Supply required application providers through a small test component wrapper. The harness does not load your application Vite configuration, framework router, backend or production build automatically. Vanilla exports receive `(target, props)` and can return a DOM node or an object with `unmount()`.
|
|
299
|
+
|
|
300
|
+
## Scenario retries and flaky outcomes
|
|
301
|
+
|
|
302
|
+
```json
|
|
303
|
+
{
|
|
304
|
+
"retries": 1,
|
|
305
|
+
"failOnFlaky": true
|
|
306
|
+
}
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
Retries are disabled by default. A retry reruns the entire failed scenario, restores its starting browser profiles/storage and local variables, and retains previous attempt diagnostics. A later pass is marked `flaky`; `failOnFlaky` turns it into a failing CI outcome. Bail takes effect after the final failed attempt.
|
|
310
|
+
|
|
311
|
+
Retries cannot undo backend writes, emails, payments, persistent files, or other external effects. Use idempotent test data and explicit application fixtures. Keep retries low while investigating unstable tests.
|
|
312
|
+
|
|
313
|
+
## Browser matrices and multiple CI machines
|
|
314
|
+
|
|
315
|
+
```sh
|
|
316
|
+
npx flowstride ci init github --browsers chromium,firefox,webkit --shards 3
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
The generated workflow creates separate CI jobs with unique evidence names. It uses normal CI runners. Review project dependency/secrets setup as usual. Cancellation stops that job and records an interrupted result; it does not delete other machines' evidence.
|
|
320
|
+
|
|
321
|
+
```sh
|
|
322
|
+
npx flowstride run flows --ci --shard 1/3
|
|
323
|
+
npx flowstride run flows --ci --shard 2/3
|
|
324
|
+
npx flowstride run flows --ci --shard 3/3
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
Run each command on its assigned machine. Preserve its `summary.json` beside its `run-*` evidence folder. Collect all shards for **one browser configuration** into separate project directories and merge:
|
|
328
|
+
|
|
329
|
+
```sh
|
|
330
|
+
npx flowstride merge shard-1/summary.json shard-2/summary.json shard-3/summary.json
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
Merging rejects missing/duplicate shards, mixed source suites and mixed browser configurations. A failed or interrupted shard cannot become a passing merge. Setup `.first.flow` files execute independently on every machine, so setup must be safe to repeat. Metrics count actual shard executions, including repeated setup work.
|
|
334
|
+
|
|
335
|
+
For later runs, use `--shard-history path/to/timings.json` from a previous complete merge. Every machine must use the same file: history assigns longer files first while preserving deterministic, disjoint standard-file coverage. There is no hosted scheduling service or shared in-memory state between machines.
|
|
336
|
+
|
|
337
|
+
## Browser compatibility notes
|
|
338
|
+
|
|
339
|
+
Chromium's current stealth behavior can override an explicitly requested locale. Use Firefox or WebKit for locale-sensitive checks and verify the observed language. For HAR playback in Firefox/WebKit, leave forced browser-offline mode unset and stop the real server to verify the recording; combining forced offline mode and HAR can reject navigation before fulfillment.
|
|
340
|
+
|
|
341
|
+
The maintained command reference, browser compatibility guide and API-to-UI examples are available in the [Flowstride documentation](https://docs.flowstride.io/guide/native-browser-workflows).
|
package/NATIVE_CI.md
CHANGED
|
@@ -1,179 +1,98 @@
|
|
|
1
|
-
# Native CI and browser-game
|
|
1
|
+
# Native CI and browser-game testing
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
execution remains retired. Existing npm OSS / private Pro archive distribution
|
|
5
|
-
and fallback remain supported. Consumer project migrations are managed separately. Native CI uses protocol 1.
|
|
6
|
-
The public CLI selects compatible Pro engines or logs the reason and falls back
|
|
7
|
-
to OSS. Older Pro CLIs may own the npm binary and do not understand the new
|
|
8
|
-
commands; install both candidates when testing the Pro path.
|
|
3
|
+
Flowstride runs `.flow` tests locally and in CI without a custom runner wrapper.
|
|
9
4
|
|
|
10
|
-
|
|
11
|
-
installs that engine's matching Chromium.
|
|
5
|
+
## Installation and execution
|
|
12
6
|
|
|
13
|
-
|
|
7
|
+
Use Node.js 22.12 or newer. Install Flowstride as a project dependency and commit the lockfile:
|
|
14
8
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
9
|
+
```sh
|
|
10
|
+
npm install --save-dev flowstride
|
|
11
|
+
npx flowstride install
|
|
12
|
+
npx flowstride run flows --ci
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
In CI, use `npm ci` before browser installation and test execution. Browser installation uses the Chromium version matching the installed Flowstride engine by default. See [native browser workflows](NATIVE_BROWSER.md) for Firefox, WebKit and browser configuration. Linux runners may require system browser dependencies; the generated GitHub workflow handles installation for its runner.
|
|
16
|
+
|
|
17
|
+
Cloud users authenticate using `npx flowstride login`. In CI, supply the developer token through the `FLOWSTRIDE_TOKEN` secret, never in committed files. Install project dependencies before login, then install the matching browser and run the tests. The OS CLI uses compatible, authorized Cloud engine installations and falls back to OS when the Cloud engine is unavailable.
|
|
18
|
+
|
|
19
|
+
Generate a GitHub Actions workflow explicitly:
|
|
19
20
|
|
|
20
21
|
```sh
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
22
|
+
npx flowstride ci init github
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
The command requires an npm project with a committed lockfile and refuses to overwrite an existing workflow. It does not run automatically during installation.
|
|
26
|
+
|
|
27
|
+
## Configuration and reports
|
|
28
|
+
|
|
29
|
+
`flowstride.config.json` supports `testDir`, `baseUrl`, `timeout`, `device` (`desktop`, `phone`, `landscape`), optional `viewport`, `pluginsDir`, `artifactsDir`, `recordVideo` (boolean), `browserErrors` (`fail` or `record`), and `webServer` (`command`, local readiness `url`, optional `cwd` and `timeout`).
|
|
30
|
+
|
|
31
|
+
Malformed configuration and unsafe output paths fail before execution. Configuration, plugin, environment, upload, session and output paths must remain inside the project, including resolved symlinks.
|
|
32
|
+
|
|
33
|
+
Native CI is selected by `--ci` or a recognized CI environment. It runs headlessly without a dashboard server and returns nonzero for failed, incomplete, crashed or interrupted execution. JSON and JUnit reports are written at the configured artifact root, which defaults to `flowstride-artifacts`. Media and attachments use a unique run directory referenced by the summary.
|
|
34
|
+
|
|
35
|
+
Flow discovery is recursive. `.first.flow` files run before ordinary files. A failed scenario leaves its remaining steps untested and execution continues to later scenarios. `--bail` stops later work; the Cloud engine executes bail runs sequentially.
|
|
36
|
+
|
|
37
|
+
An optional configured local web server is started and stopped with the run. Flowstride rejects a readiness URL that is already serving.
|
|
38
|
+
|
|
39
|
+
Cloud reporting is disabled by default in native CI. Set `cloudSync: true` to enable authenticated reporting. Reporting and video status are recorded separately in the summary and `.flowstride/cloud-sync.json`; a reporting failure does not change a passed test into a failed test.
|
|
40
|
+
|
|
41
|
+
## Browser-game actions
|
|
42
|
+
|
|
43
|
+
Register the built-in alias:
|
|
44
|
+
|
|
45
|
+
```text
|
|
46
|
+
let game = "flowstride:game";
|
|
26
47
|
```
|
|
27
48
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
entry point must be covered when checking fallback.
|
|
33
|
-
|
|
34
|
-
The sample in `examples/native-game` uses a tiny local canvas game. Copy its
|
|
35
|
-
files into a disposable project containing the installed candidates, then run
|
|
36
|
-
the command above. `server.mjs` is the game fixture's HTTP server; it contains no
|
|
37
|
-
Flowstride runner, adapter, worker, or CI orchestration code. A real application
|
|
38
|
-
uses its normal development/start command instead.
|
|
39
|
-
|
|
40
|
-
## Configuration
|
|
41
|
-
|
|
42
|
-
`flowstride.config.json` supports `testDir`, `baseUrl`, `timeout`, `device`
|
|
43
|
-
(`desktop`, `phone`, `landscape`), optional `viewport`, `pluginsDir`,
|
|
44
|
-
`artifactsDir`, `recordVideo` (boolean), `browserErrors` (`fail` or `record`), and
|
|
45
|
-
`webServer` (`command`, local readiness `url`, optional `cwd` and `timeout`).
|
|
46
|
-
Malformed configuration and unsafe output paths fail before execution.
|
|
47
|
-
|
|
48
|
-
Native CI is selected by `--ci` or a recognized CI environment. It runs without
|
|
49
|
-
a dashboard server, returns nonzero for failed/incomplete/crashed execution,
|
|
50
|
-
and writes latest `summary.json`/`junit.xml` reports at the configured artifact
|
|
51
|
-
root. Media and attachments go into a unique run directory referenced by the
|
|
52
|
-
summary, so local reruns retain history without mixing evidence.
|
|
53
|
-
SIGINT/SIGTERM produce interrupted results. The run owns its optional local
|
|
54
|
-
server and refuses an already-serving readiness URL.
|
|
55
|
-
|
|
56
|
-
Cloud reporting is off by default in native CI. `cloudSync: true` explicitly
|
|
57
|
-
allows existing authenticated reporting; matching Cloud deployment and configured media storage are required. Reporting and
|
|
58
|
-
video status are written separately in the summary and `.flowstride/cloud-sync.json`;
|
|
59
|
-
a reporting failure does not turn a passed test into a failed test. Normal interactive reporting and the restored
|
|
60
|
-
Cloud URL overrides remain separate from retired managed execution.
|
|
61
|
-
|
|
62
|
-
`npx flowstride ci init github` generates a workflow explicitly, never during
|
|
63
|
-
installation, and refuses to overwrite it. It assumes an npm project with a
|
|
64
|
-
committed lockfile. Other package managers and monorepos can use the composite
|
|
65
|
-
action's install-command and working-directory inputs. The composite action is
|
|
66
|
-
in the OSS repository's `action/` directory and becomes remotely usable only
|
|
67
|
-
once that source is pushed. Pin its commit when configuring a consumer workflow.
|
|
68
|
-
Pro jobs pass a developer token as a secret; dependency installation precedes
|
|
69
|
-
Pro login/install, matching-browser installation, execution, and artifact upload.
|
|
70
|
-
|
|
71
|
-
## Native game actions
|
|
72
|
-
|
|
73
|
-
Register the built-in alias with `let game = "flowstride:game"`. It requires no
|
|
74
|
-
custom plugin file. Available actions:
|
|
75
|
-
|
|
76
|
-
- `tap`: canvas selector, x/y; relative coordinates by default, or `unit: "pixels"`.
|
|
77
|
-
- `drag`/`swipe`: canvas, from/to coordinates, optional duration and input type.
|
|
49
|
+
The built-in game API provides:
|
|
50
|
+
|
|
51
|
+
- `tap`: canvas selector and coordinates; relative coordinates by default, or `unit: "pixels"`.
|
|
52
|
+
- `drag` and `swipe`: canvas, start/end coordinates, optional duration and input type.
|
|
78
53
|
- `key`: key name and optional hold duration.
|
|
79
54
|
- `checkpoint`: named screenshot, optionally of one canvas.
|
|
80
|
-
- `snapshot` and `expectVisual`: canvas comparison with explicit
|
|
81
|
-
- `state
|
|
82
|
-
- `snapshotState
|
|
83
|
-
- `fixture`: a named fixture registered by the application in
|
|
84
|
-
- `select`: a stable object ID resolved by
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
physical
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
`
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
missing/broken Pro and revoked-token fallback. Validate GitHub's Linux execution,
|
|
125
|
-
real licensed download after local acceptance.
|
|
126
|
-
No push, npm publication, Cloud archive replacement or Naija Throne migration
|
|
127
|
-
is performed by building these candidates.
|
|
128
|
-
|
|
129
|
-
## Combined candidate behavior
|
|
130
|
-
|
|
131
|
-
Node.js 22.12 or newer is required. Flow discovery is recursive; `.first.flow`
|
|
132
|
-
files run before ordinary files. Reports identify nested flows by their relative
|
|
133
|
-
paths, and same-named videos receive distinct filenames. A failed scenario skips
|
|
134
|
-
its remaining steps and continues to later scenarios. `--bail` stops later work;
|
|
135
|
-
Pro executes bail runs sequentially. `only.Scenario:` selects that file's focus
|
|
136
|
-
scenario and report planning uses the same selection.
|
|
137
|
-
|
|
138
|
-
Configuration, plugin, environment, upload, vault and output paths must stay
|
|
139
|
-
inside the project, including resolved symlinks. Move shared fixtures into the
|
|
140
|
-
project rather than referring to external files. `FLOWSTRIDE_API_URL` takes
|
|
141
|
-
precedence over `NEXTJS_WEBHOOK_URL`; the legacy completion suffix is normalized.
|
|
142
|
-
Managed runner commands remain retired.
|
|
143
|
-
|
|
144
|
-
Logs redact credential fields, known generated secrets and signed capabilities.
|
|
145
|
-
Redaction cannot guarantee removal of arbitrary application data or visible
|
|
146
|
-
secrets in screenshots/videos. Review artifact access and test data accordingly.
|
|
147
|
-
|
|
148
|
-
The Pro installer rejects unsafe or incomplete archives, downloads above 32 MiB,
|
|
149
|
-
and checksum mismatches. Older Cloud servers without checksum headers remain
|
|
150
|
-
compatible and emit a warning; `FLOWSTRIDE_PRO_SHA256` can pin an expected digest.
|
|
151
|
-
On failed npm installation the previous Pro directory and CLI launchers are
|
|
152
|
-
restored. This is not a transaction over every project dependency npm may alter;
|
|
153
|
-
use a disposable project for candidate acceptance.
|
|
154
|
-
|
|
155
|
-
## Dashboard tab history
|
|
156
|
-
|
|
157
|
-
Completed test tabs restore that file's last captured step by default. Selecting
|
|
158
|
-
an earlier step remembers its history cursor separately for each file. A sidebar
|
|
159
|
-
step also selects the owning tab. Return to Live applies only to the selected
|
|
160
|
-
file, and a new suite/rerun clears old cursors and screens. Runner console and
|
|
161
|
-
network panels show the selected file's records; the full Logs view retains
|
|
162
|
-
suite-wide history. Flows/steps without screenshots show an explicit empty
|
|
163
|
-
state rather than another flow's screen.
|
|
164
|
-
|
|
165
|
-
Run `npm run test:dashboard:unit` for the Zustand store/history unit tests and
|
|
166
|
-
`npm run test:dashboard` for those tests plus the real Chromium dashboard
|
|
167
|
-
regression. The browser regression runs real local browser flows, then replays
|
|
168
|
-
their worker events over WebSocket into the built dashboard. It also supplies
|
|
169
|
-
explicit API-only/background-event fixtures. No live Cloud is required.
|
|
170
|
-
|
|
171
|
-
### Local cURL debugging
|
|
172
|
-
|
|
173
|
-
The interactive local dashboard keeps its cURL preview [REDACTED]. Copy raw cURL
|
|
174
|
-
uses the original request held separately in local memory, including its headers
|
|
175
|
-
and body. Other log fields stay redacted. Native CI and Cloud reporting continue redacting cURL and
|
|
176
|
-
credentials. Raw cURL is not restored for already-redacted historical runs.
|
|
177
|
-
The interactive runner binds to 127.0.0.1 and restricts HTTP Host/Origin and
|
|
178
|
-
WebSocket Origin to its own loopback address and port. Raw commands remain in
|
|
179
|
-
local debugger memory and IPC; they are not written to CI or Cloud reports.
|
|
55
|
+
- `snapshot` and `expectVisual`: canvas comparison with explicit change thresholds.
|
|
56
|
+
- `state` and `expectState`: exposed application state with optional dotted paths.
|
|
57
|
+
- `snapshotState` and `expectPreserved`: compare selected state across actions or reloads.
|
|
58
|
+
- `fixture`: a named fixture explicitly registered by the application in test mode.
|
|
59
|
+
- `select`: a stable object ID resolved by a game bridge, followed by real pointer input.
|
|
60
|
+
|
|
61
|
+
The package includes a small runnable example in `examples/native-game`. Its HTTP server serves the fixture application; it is not a custom test runner. Use your application's normal start command for your own game.
|
|
62
|
+
|
|
63
|
+
Visual checks measure pixel differences. Pair them with state or UI assertions to verify gameplay behavior. Mobile profiles emulate Chromium devices; physical device testing is a separate qualification. Touch dragging currently uses Chromium CDP.
|
|
64
|
+
|
|
65
|
+
## Application bridges
|
|
66
|
+
|
|
67
|
+
For a generic browser game, import `registerGame` from `flowstride/dist/game/bridge.js`. Supply `readState` and optional test-only fixtures.
|
|
68
|
+
|
|
69
|
+
For Three.js, import `registerThreeGame` from `flowstride/dist/game/three-bridge.js`. Register the live scene, camera, canvas, your application's `Vector3` and `Raycaster` constructors, and an optional map of stable object IDs. Alternatively, assign `userData.flowstrideId` to scene objects. Optional local pick points use `userData.flowstridePickPoint: [x, y, z]`.
|
|
70
|
+
|
|
71
|
+
The bridge projects and raycasts the live scene. Hidden, occluded, off-screen, covered or non-pickable objects fail explicitly. It creates no renderer or animation loop. Dispose the bridge when replacing the scene.
|
|
72
|
+
|
|
73
|
+
Application fixtures are available only when explicitly registered with `testMode: true`. Enable this in dedicated test builds, never public production builds. Your application still defines its own rules, observable state and fixtures.
|
|
74
|
+
|
|
75
|
+
Custom plugins receive `sdk.game`, `sdk.browser.page()` and `sdk.run`, including artifact attachment support, alongside the existing SDK.
|
|
76
|
+
|
|
77
|
+
## Assertions and browser sessions
|
|
78
|
+
|
|
79
|
+
```text
|
|
80
|
+
flow.expect attribute "#order" "data-state" "approved";
|
|
81
|
+
flow.expect attribute "#order" "class" contains "selected";
|
|
82
|
+
flow.save session "browser" persist;
|
|
83
|
+
flow.use session "browser";
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Attribute comparisons wait up to the configured timeout. Exact matching is case-sensitive; `contains` checks a substring. Missing attributes fail even when the expected value is empty.
|
|
87
|
+
|
|
88
|
+
Sessions capture cookies, local storage, session storage and the current HTTP(S) storage origin. Session storage is restored immediately on the matching origin or before its first matching navigation. It is not seeded into unrelated origins. The initial seed is then removed so later application changes and reloads are preserved.
|
|
89
|
+
|
|
90
|
+
Existing cookie/local-storage snapshots remain compatible. Older snapshots with session storage but no origin use the current application origin or configured `baseUrl`; open the application first or set `baseUrl` if neither is available. Restoration merges saved keys into storage. This is a single-origin storage snapshot.
|
|
91
|
+
|
|
92
|
+
## Interactive history and debugging
|
|
93
|
+
|
|
94
|
+
Each completed flow tab restores its own last captured screen. Selecting an earlier step remembers a separate history cursor for that file. Flows without screenshots show an empty state. New runs clear previous screens and cursors.
|
|
95
|
+
|
|
96
|
+
The local cURL preview remains `[REDACTED]`. Copy uses the original captured request held in interactive debugger memory. Already-redacted historical data cannot be recovered. CI and Cloud reports retain redaction.
|
|
97
|
+
|
|
98
|
+
Review artifact access and test data: credential redaction does not remove every possible application secret, and screenshots or videos can show sensitive information.
|