@milkstraw/traceframe 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -4,61 +4,65 @@ Traceframe records a scripted Chromium flow and renders an MP4 with a smooth cur
4
4
 
5
5
  https://github.com/user-attachments/assets/38e02cd5-a9ba-4f3c-938c-159955353141
6
6
 
7
- ## Install
7
+ ## Prerequisites
8
8
 
9
- ```sh
10
- npm install @milkstraw/traceframe
11
- ```
12
-
13
- Traceframe requires Node.js 22+, FFmpeg 5+ with `libx264` and `ffprobe` (for example, `brew install ffmpeg` or `sudo apt install ffmpeg`), and Playwright's Chromium:
9
+ Traceframe requires Node.js 22+ and system FFmpeg 5+ with `libx264` and `ffprobe` to render the MP4 (for example, `brew install ffmpeg` or `sudo apt install ffmpeg`). Playwright uses its own Chromium and FFmpeg builds to record the page. Install them with the exact Playwright version Traceframe uses; other cached versions are not used:
14
10
 
15
11
  ```sh
16
- npx playwright@1.64.1 install --only-shell chromium
12
+ npx playwright@1.64.1 install --only-shell chromium ffmpeg
17
13
  ```
18
14
 
19
- To use a system Chrome instead, pass `--browser` or set `CHROME_PATH`. Playwright still needs its own FFmpeg build to record the page, so install it with `npx playwright@1.64.1 install ffmpeg`.
15
+ To use system Chrome 150 or later, pass `--browser` or set `CHROME_PATH`, and install Playwright's recording FFmpeg with `npx playwright@1.64.1 install ffmpeg`. Older Chrome versions can fail to scroll targets inside panels.
16
+
17
+ In a sandbox, such as a coding agent's, allow Playwright's browser download and let Chromium start.
20
18
 
21
19
  To record pages behind a login, pass a Playwright storage-state file with `--storage-state`. Keep it in `.auth/` or another ignored location.
22
20
 
23
21
  ## Record a video
24
22
 
25
- The included fixture is available in a repository clone. It is not included in the installed package.
23
+ Run Traceframe with `npx` without adding it to your project's dependencies. Pass your application URL and your own [flow file](#write-a-flow):
26
24
 
27
25
  ```sh
28
- git clone https://github.com/milkstrawai/traceframe.git
29
- cd traceframe
30
- npm ci
31
- npx traceframe record \
32
- --url "file://$PWD/examples/fixture.html" \
33
- --flow ./examples/fixture-flow.mjs \
34
- --out ./sample.mp4 \
35
- --preset draft
36
- ```
37
-
38
- For an installed package, pass your application URL and your own flow file:
39
-
40
- ```sh
41
- npx traceframe record \
26
+ npx @milkstraw/traceframe record \
42
27
  --url http://localhost:3000 \
43
28
  --flow ./flow.mjs \
44
29
  --out ./recording.mp4
45
30
  ```
46
31
 
47
- The `draft` preset uses a 1280×800 CSS viewport and produces 1280×800 at 30 FPS. The `standard` preset uses the same CSS viewport and encodes it at 2560×1600 at 60 FPS. The cursor and click pulses are always drawn at the full frame rate, but Chromium may capture page changes, such as scrolling and animations, at a lower rate at 2560×1600, depending on the machine. When that happens, the video repeats the last page frame until the next one arrives. The video is always the viewport size times `--scale`, so `--preset draft --scale 2` gives 2560×1600 at 30 FPS. The command prints a JSON summary when it finishes.
32
+ For regular use, you can install the command globally:
48
33
 
49
- Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and the FFmpeg filter file.
34
+ ```sh
35
+ npm install -g @milkstraw/traceframe
36
+ traceframe record --url http://localhost:3000 --flow ./flow.mjs --out ./recording.mp4
37
+ ```
50
38
 
51
- ## Have an agent create the flow
39
+ To check installation before touching your application, save this as `smoke.mjs`:
52
40
 
53
- For a real application, ask your coding agent to inspect the running UI and create the flow from the user journey. This keeps selectors tied to the product and avoids manually guessing coordinates. For example:
41
+ ```js
42
+ export default async function ({ wait }) {
43
+ await wait(0.2)
44
+ }
45
+ ```
54
46
 
55
- ```text
56
- Create a Traceframe flow for this user journey: open Settings, change the theme, and save it.
57
- Inspect the running application, use stable role or test-id locators, add a ready hook,
58
- and use Traceframe's click, hover, move, and wait helpers for visible interactions.
59
- Write the result to flow.mjs and explain how to run it.
47
+ ```sh
48
+ npx @milkstraw/traceframe record \
49
+ --url 'data:text/html,<h1>Traceframe smoke</h1>' \
50
+ --flow ./smoke.mjs --out ./smoke.mp4 \
51
+ --preset draft
60
52
  ```
61
53
 
54
+ The [recording guide](docs/recording-flows.md) includes a runnable example and recipes for form input, asynchronous results, browser settings, and editing or sharing a recording.
55
+
56
+ Both presets use a 1280×800 CSS viewport. `draft` produces 1280×800 at 30 FPS; `standard` (the default) produces 2560×1600 at 60 FPS. The output dimensions are the viewport times `--scale`, so `--preset draft --scale 2` gives 2560×1600 at 30 FPS. The command prints a JSON summary when it finishes.
57
+
58
+ The cursor and click pulses are always drawn at the full frame rate. Chromium may capture page changes, such as scrolling and animations, at a lower rate at 2560×1600, depending on the machine. The video repeats the last page frame until the next one arrives.
59
+
60
+ Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and FFmpeg filter file, including a partial capture if the flow fails during recording. On a preparation or recording failure, Traceframe also tries to save a viewport screenshot as `failure.png`. Keep that directory out of Git, because captures and screenshots can contain private page data.
61
+
62
+ When available, failure output includes the error code, flow-file locations, elapsed recording time, and paths to artifacts saved for that attempt. Without `--run-dir`, temporary captures are removed and no failure screenshot is saved. See [diagnosing a failed take](docs/recording-flows.md#diagnose-a-failed-take).
63
+
64
+ ## Write a flow
65
+
62
66
  The flow module exports a default function. Timing values are seconds:
63
67
 
64
68
  ```js
@@ -72,14 +76,64 @@ export default async function ({ page, click, wait }) {
72
76
  }
73
77
  ```
74
78
 
75
- `ready({ page, signal })` runs before recording starts. `--ready-timeout` (30 seconds by default) is Playwright's timeout for the page load and for each Playwright call in `ready`; it does not limit other work in the hook. A flow can also export `initialPointer = { x, y }` to choose where the cursor starts. By default it starts near the lower left.
79
+ The optional `ready({ page, signal })` hook runs after initial navigation and before recording starts. Export `initialPointer = { x, y }` to choose where the cursor starts; by default it starts near the lower left.
80
+
81
+ ### Helpers
82
+
83
+ The default function receives a Playwright `page`, an `AbortSignal` named `signal`, and five helpers:
76
84
 
77
- The default function receives `page`, `signal`, and four helpers: `move({ x, y }, { duration })`, `hover(locator, { duration })`, `click(locator, { duration, dwell })`, and `wait(seconds)`. A move takes 0.5 seconds unless you pass `duration`, and a click rests on the target for 0.25 seconds (`dwell`) before it presses. If the target is disabled, the click waits for it to become enabled. A flow may contain at most 100 moves, 100 clicks, and 120 seconds of work, and each `duration`, `dwell`, and `wait` is at most 30 seconds. Error messages name the locator; give it a clearer name with Playwright's `locator.describe('Save settings')`.
85
+ | Helper | Use |
86
+ | --- | --- |
87
+ | `move({ x, y }, { duration })` | Move the pointer to viewport coordinates. |
88
+ | `hover(locator, { duration })` | Move the pointer to a locator. |
89
+ | `click(locator, { duration, dwell })` | Move, pause, and click a locator. |
90
+ | `scroll(locator)` | Center a target without moving the pointer. |
91
+ | `wait(seconds)` | Pause for reading time. |
78
92
 
79
- `click` and `hover` use real Playwright mouse events and add their movement to the overlay timeline. If the pointer is already inside the target, it stays where it is, so a hover effect that moves or resizes the target does not pull the cursor along. A click holds the mouse button for about 100 ms, as a person would, so press styles such as `:active` show in the video. Headless Chromium draws pages without a GPU, so on heavy pages, such as ones with blurred backgrounds, hover effects can appear a little after the cursor passes. A longer `duration`, or a `move` around hover-reactive elements, keeps this from showing. Typing, scrolling, dragging, multiple tabs, and direct `page.mouse` or `locator.click()` calls are not currently represented in that timeline. A flow receives an `AbortSignal`; helper calls stop promptly when the duration limit is reached.
93
+ A move takes 0.5 seconds by default. A click pauses on the target for 0.25 seconds (`dwell`) before pressing, and waits for a disabled target to become enabled. `click` and `hover` leave the pointer in place if it is already inside the target's box.
94
+
95
+ Error messages name the locator; give it a clearer name with Playwright's `locator.describe('Save settings')`.
96
+
97
+ ### Scrolling
98
+
99
+ `click` and `hover` scroll automatically only when their aim point starts outside the viewport. They scroll smoothly through the page, scrollable containers, and iframes, and wait for scrolling to stop before moving the pointer.
100
+
101
+ `scroll(target)` smoothly centers the target vertically, even if it is already visible, and waits for scrolling to stop without moving the pointer. Use it to show a result or before clicking a target clipped by a scrollable panel or covered by a sticky header. Wait for a dialog or toast to close instead.
102
+
103
+ Centering can leave a target partly visible or covered, for example by a sticky bar taller than half the height of the viewport or its scrolling panel. It cannot reveal content hidden by `overflow: clip`. `click` fails with `FLOW_TARGET_OBSCURED` if the target is still covered when it presses.
104
+
105
+ ### Timing and limits
106
+
107
+ `--ready-timeout` (default: 30 seconds) sets Playwright's default timeout during initial navigation and `ready`, for operations that accept a timeout. It does not set a deadline for the whole hook.
108
+
109
+ The default function may run for 120 seconds, including waits for your backend. For longer flows, pass `--flow-timeout 300` (`flowTimeout: 300` in the API). This does not change Playwright's action timeouts. Helper calls stop promptly when the flow timeout is reached.
110
+
111
+ A flow may contain at most 100 moves and 100 clicks, and each `duration`, `dwell`, and `wait` is at most 30 seconds.
112
+
113
+ ### What appears in the video
114
+
115
+ Playwright typing and scrolling appear in the captured page. Direct `page.mouse`, `locator.click()`, and dragging calls bypass Traceframe's cursor trace and click pulses, so use its pointer helpers for visible interactions. Only the starting page is recorded; new tabs and popups do not appear in the video.
116
+
117
+ ### Have an agent create the flow
118
+
119
+ You can ask a coding agent to inspect your running UI and write the flow from a journey:
120
+
121
+ ```text
122
+ Create a Traceframe flow for this user journey: open Settings, change the theme, and save it.
123
+ Follow the Traceframe recording guide: https://github.com/milkstrawai/traceframe/blob/main/docs/recording-flows.md
124
+ Inspect the running application, use stable role or test-id locators, add a ready hook,
125
+ and use Traceframe's click, hover, move, scroll, and wait helpers for visible interactions.
126
+ Write the result to flow.mjs and explain how to run it.
127
+ ```
80
128
 
81
129
  ## Programmatic usage
82
130
 
131
+ To call Traceframe from JavaScript, install it in your project:
132
+
133
+ ```sh
134
+ npm install @milkstraw/traceframe
135
+ ```
136
+
83
137
  ```js
84
138
  import { record } from '@milkstraw/traceframe'
85
139
 
@@ -91,19 +145,32 @@ await record({
91
145
  })
92
146
  ```
93
147
 
94
- Traceframe's own errors are `TraceframeError` instances (exported by the package) with a `code`, such as `READY_FAILED` or `FLOW_TARGET_OBSCURED`.
148
+ Traceframe's own errors are `TraceframeError` instances (exported by the package) with a `code`, such as `READY_FAILED` or `FLOW_TARGET_OBSCURED`. Other exceptions thrown by the flow become `FLOW_FAILED`, with the original thrown value in `cause`; existing Traceframe error codes and causes are preserved.
149
+
150
+ When available, `error.diagnostics` contains:
151
+
152
+ | Field | Meaning |
153
+ | --- | --- |
154
+ | `flowLocations` | Locations in the supplied flow file, as `{ file, line, column }` objects with an absolute path and 1-based line/column. Empty when the error stack has no matching frame. |
155
+ | `recordingTime` | Elapsed seconds from the first captured frame for a capture-stage failure, measured before collecting failure artifacts. Omitted before the first frame and for later rendering failures. |
156
+ | `capturePath` | Absolute path to the retained raw capture, only when `runDirectory` was supplied and saving succeeded. |
157
+ | `screenshotPath` | Absolute path to the saved failure screenshot, only when `runDirectory` was supplied and the screenshot succeeded. |
158
+
159
+ A flow timeout may have no flow location because it is raised outside the blocked operation. Screenshots are best effort and have a five-second collection budget. Partial captures are saved before browser shutdown, with no fixed time limit so slower encoders can finish. Diagnostic failures do not replace the original error.
95
160
 
96
161
  Pass `signal` to stop a recording early. Traceframe closes the browser, stops FFmpeg, and removes its temporary files before rejecting with the signal's reason. When you pass a signal, Traceframe leaves Ctrl-C and other termination signals to you. The CLI does this for you: the first Ctrl-C stops and cleans up, and a second one exits at once.
97
162
 
163
+ A stuck recording encoder can delay saving and cancellation indefinitely. A second Ctrl-C forces the CLI to exit and can discard the unfinished capture. API callers that need a hard shutdown deadline should run recording in a separate process they can terminate.
164
+
98
165
  ## Sample output
99
166
 
100
- The committed sample is generated from the included fixture flow with the `standard` preset (2560×1600 at 60 FPS) and is about 12 seconds long:
167
+ The committed sample is generated from the [repository's fixture flow](examples/fixture-flow.mjs) with the `standard` preset (2560×1600 at 60 FPS) and is about 12 seconds long:
101
168
 
102
169
  [Download the sample video](https://github.com/milkstrawai/traceframe/blob/main/examples/sample-output.mp4)
103
170
 
104
- The cursor and click-pulse artwork and the example page's icons were made for Traceframe and are distributed under this repository's MIT license.
171
+ The cursor and click-pulse artwork and the fixture page's icons were made for Traceframe and are distributed under this repository's MIT license.
105
172
 
106
- The example page uses the [Inter](https://github.com/rsms/inter) font, included under the SIL Open Font License in `examples/fonts/OFL.txt`.
173
+ The fixture page uses the [Inter](https://github.com/rsms/inter) font, included under the SIL Open Font License in `examples/fonts/OFL.txt`.
107
174
 
108
175
  ## Development
109
176
 
@@ -115,6 +182,8 @@ TRACEFRAME_PRESET=standard npm run test:smoke
115
182
  npm run test:package
116
183
  ```
117
184
 
185
+ To release, set the version in `package.json` and `package-lock.json`, push all release changes to `main`, and publish a GitHub Release with the tag `v<version>` on the final tested commit containing those changes. The Release workflow reruns CI and stages the package on npm. A maintainer downloads it with `npm stage download <stage-id>`, checks that its files match `npm pack` at the tag, and approves it on npmjs.com with 2FA.
186
+
118
187
  After editing an SVG in `src/assets`, run `npm run build:assets` to rebuild its 1× and 2× PNGs.
119
188
 
120
189
  To re-record the committed sample:
@@ -0,0 +1,175 @@
1
+ # Write a recording flow
2
+
3
+ Use this guide yourself or give it to a coding agent with the journey you want to show. Traceframe provides pointer and scrolling helpers; the supplied Playwright page handles typing, file selection, and application checks.
4
+
5
+ ## Run the example
6
+
7
+ The package includes [a complete flow](../examples/recipes-flow.mjs) and [its local page](../examples/recipes.html). It types English and Arabic, waits for two simulated replies, selects a fake file, and clicks a final action in a panel below the fold, which `click` scrolls into view first. After a global installation and the [prerequisites](../README.md#prerequisites), run:
8
+
9
+ ```sh
10
+ TRACEFRAME_EXAMPLES="$(npm root -g)/@milkstraw/traceframe/examples"
11
+ traceframe record \
12
+ --url "file://$TRACEFRAME_EXAMPLES/recipes.html" \
13
+ --flow "$TRACEFRAME_EXAMPLES/recipes-flow.mjs" \
14
+ --preset draft --out ./recipes.mp4
15
+ ```
16
+
17
+ In a repository clone, run `npx traceframe record --url "file://$PWD/examples/recipes.html" --flow ./examples/recipes-flow.mjs --preset draft --out ./recipes.mp4` instead. These shell commands use macOS/Linux path syntax. The flow itself has no application dependencies, external services, or credentials.
18
+
19
+ ## Prepare the journey
20
+
21
+ Inspect the running application at the recording viewport. Pick a short journey with a visible result and known demo data. Use role, label, or test-id locators; `locator.describe()` can make errors clearer.
22
+
23
+ Put setup and page readiness in `ready({ page, signal })`. This runs after initial navigation and before recording. Use `--storage-state` for an existing login, and keep that file in an ignored directory. For headers, locale, or themes, see [optional startup settings](#optional-startup-settings).
24
+
25
+ ## Choose the right actions
26
+
27
+ Use Traceframe's `click`, `hover`, and `move` for visible pointer actions. Direct `locator.click()` and `page.mouse` calls move the browser pointer without updating Traceframe's overlay or click pulses. Use Playwright for the other actions below; their page changes appear in the video.
28
+
29
+ For a target clipped by a scrollable panel or covered by a sticky header, try `await scroll(target)` before the click. For a tooltip, a toast, or a closing dialog, wait until it is gone, or give the click a longer `dwell`. A covered click fails with an error that it "is covered at the click point" (`FLOW_TARGET_OBSCURED`); `hover` does not check for covers. See the [scrolling rules and limits](../README.md#scrolling).
30
+
31
+ Both automatic scrolling and `scroll(target)` can move the outer page, which closes some lists. For those lists, use their own scrolling container through Playwright, for example `await list.evaluate((element) => element.scrollTo({ top: element.scrollHeight, behavior: 'smooth' }))` to show the final options, then click the option. Wait for any opening animation before scrolling.
32
+
33
+ To show a result without a pointer action, use `await scroll(result)`. Centering a result taller than the view hides its start, so scroll to its heading or another element near its start instead.
34
+
35
+ Focus a field visibly, then type with a delay in milliseconds:
36
+
37
+ ```js
38
+ await click(message)
39
+ await message.pressSequentially('A short question', { delay: 45 })
40
+ ```
41
+
42
+ For an upload, hover the visible control, set the actual file input, and wait for the application's selected-file feedback. The visible label or drop zone and the input are often different elements:
43
+
44
+ ```js
45
+ const dropZone = page.getByText('Choose a file', { exact: true })
46
+ const fileInput = page.getByLabel('Choose a file')
47
+ await hover(dropZone)
48
+ await fileInput.setInputFiles('./sample.txt')
49
+ await page.getByText('sample.txt').waitFor()
50
+ await wait(0.6)
51
+ ```
52
+
53
+ A relative path is resolved from the directory where you run Traceframe. File selection does not show an operating-system file picker. The complete example supplies an in-memory fake file so it can run on its own.
54
+
55
+ ## Wait for the current result
56
+
57
+ Before a follow-up, wait for the application to show that the current step is complete. For a conversation that appends completed replies, the example waits for the next reply by index. For streamed answers, wait for that request's completion indicator instead; the first visible text or a previous reply is not completion.
58
+
59
+ Set a Playwright timeout explicitly when an individual operation may exceed its usual 30 seconds, such as `await newReply.waitFor({ timeout: 60_000 })`. Use Traceframe's `wait(seconds)` for reading time. Each helper wait is limited to 30 seconds, and the whole flow has a 120-second timeout that `--flow-timeout` (`flowTimeout` in the API) raises.
60
+
61
+ ## Add captions and reading time
62
+
63
+ For a review video, explain what the viewer should notice. Show the caption, pause before moving the pointer, perform the action, then hold the result. Start with about 12 characters per second and a 2.5-second minimum for reading, followed by a 2.5–4 second result hold.
64
+
65
+ Use Playwright's [`page.screencast.showOverlay()`](https://playwright.dev/docs/api/class-screencast#screencast-show-overlay) inside the flow. Keep the caption compact, centered, and clear of the controls and results being shown. Traceframe's covered-click check ignores this pointer-transparent caption; use `top: 32px` instead of `bottom: 32px` when it would hide controls or results near the bottom. Keep its handle and dispose of it after the result hold:
66
+
67
+ ```js
68
+ const text = 'Open Actions to inspect the available options.'
69
+ const caption = await page.screencast.showOverlay(`
70
+ <div style="position: fixed; left: 24px; right: 24px; bottom: 32px;
71
+ display: flex; justify-content: center; pointer-events: none;">
72
+ <div dir="auto" style="max-width: 720px; padding: 12px 20px; border-radius: 10px;
73
+ background: #18181b; color: #fff;
74
+ box-shadow: 0 4px 16px rgba(0, 0, 0, 0.16);
75
+ font: 500 20px/1.45 system-ui, sans-serif;
76
+ text-align: center; text-wrap: balance;">
77
+ ${text}
78
+ </div>
79
+ </div>
80
+ `)
81
+ try {
82
+ await wait(Math.max(2.5, Array.from(text).length / 12))
83
+ await click(page.getByRole('button', { name: 'Actions', exact: true }))
84
+ await page.getByRole('menu').waitFor()
85
+ await wait(3)
86
+ } finally {
87
+ await caption.dispose()
88
+ }
89
+ ```
90
+
91
+ Omit `duration` when actions should happen while the caption is visible. Playwright uses milliseconds for `duration`: awaiting `showOverlay(html, { duration: 3000 })` waits until the overlay is removed. The example uses fixed text; escape dynamic text before inserting it into HTML.
92
+
93
+ Playwright restores overlays after a full-page navigation, but they can briefly disappear while the new document loads. Opening a native modal `<dialog>` can also briefly cover an existing caption. For a clean caption transition, finish the reading pause and dispose of the current caption before the action that navigates or opens the dialog, then show the next caption after the destination or dialog is ready.
94
+
95
+ Keep captions short. Split longer text into separate captions so each reading pause stays within the helper's 30-second limit. Reading pauses and result holds count toward the flow timeout; for a longer video, raise it with, for example, `--flow-timeout 240` (seconds).
96
+
97
+ ## Optional startup settings
98
+
99
+ `page.emulateMedia({ colorScheme: 'dark' })` in `ready` updates CSS themes that follow the browser's color scheme. If the app reads the setting only at startup, set it before opening the app, as below.
100
+
101
+ If the application needs request headers, a browser locale, or a color scheme from startup, record `--url about:blank` and open the application yourself in `ready`. Keep only the lines you need:
102
+
103
+ ```js
104
+ export async function ready({ page }) {
105
+ await page.emulateMedia({ colorScheme: 'dark' })
106
+ await page.setExtraHTTPHeaders({ 'x-demo-access': 'demo' })
107
+ const cdp = await page.context().newCDPSession(page)
108
+ await cdp.send('Emulation.setLocaleOverride', { locale: 'ar-EG' })
109
+ await cdp.send('Emulation.setUserAgentOverride', { userAgent: '', acceptLanguage: 'ar-EG' })
110
+ await page.goto('http://localhost:3000')
111
+ await page.getByRole('heading', { name: 'Example workspace' }).waitFor()
112
+ await page.evaluate(() => document.fonts.ready)
113
+ }
114
+ ```
115
+
116
+ The two `Emulation` calls set the locale that `navigator.language`, `Intl`, and the `Accept-Language` header use; the empty `userAgent` keeps Chromium's own user agent and client hints. Traceframe waits for web fonts only on the `--url` page, so the last line waits for the application's fonts.
117
+
118
+ ## Record, inspect, and refine
119
+
120
+ Use `--preset draft` while writing the flow and the default preset for the final video.
121
+
122
+ While writing the flow, pass `--run-dir` with an ignored folder, such as `.traceframe/runs/demo`. Use a separate directory for each take. A failed take keeps its partial `capture.webm` when saving succeeds and attempts a viewport screenshot, `failure.png`. The raw capture has no added cursor overlay and may include repeated final frames after the failure time.
123
+
124
+ A failed take may already have submitted real jobs, so reset demo data before recording again. Check the important transitions and the final result. You can also check the file for decode errors:
125
+
126
+ ```sh
127
+ ffmpeg -v error -xerror -i ./recording.mp4 -f null -
128
+ ```
129
+
130
+ ## Diagnose a failed take
131
+
132
+ The CLI keeps the original error message and adds available context on standard error. For example:
133
+
134
+ ```text
135
+ traceframe: [FLOW_FAILED] Expected the Saved status.
136
+ Flow: /project/review.mjs:18:9
137
+ Recording time: 12.35 s
138
+ Capture: /project/.traceframe/runs/demo/capture.webm
139
+ Screenshot: /project/.traceframe/runs/demo/failure.png
140
+ ```
141
+
142
+ Inspect the reported screenshot, then the capture near the reported time if needed. The time is measured from the first captured frame before diagnostic collection; it is not the raw file's duration. Correct the locator or application check and reset demo data before retrying.
143
+
144
+ Only paths saved for the current attempt are reported. A closed or unresponsive page may prevent a screenshot or capture from being saved. Errors before the first frame have no recording time, and a flow timeout may have no source location. Without `--run-dir`, flow locations and timing are still reported when available, but temporary captures are removed and no failure screenshot is saved. The API exposes the same context through [`error.diagnostics`](../README.md#programmatic-usage).
145
+
146
+ ## Edit the finished video
147
+
148
+ Edit the finished MP4 with FFmpeg rather than changing the flow. Editing finished frames keeps the page and cursor together. Keep the original, and say when a clip's pacing was edited.
149
+
150
+ To speed up an idle stretch, such as a progress bar, you need its start and end times. Read them in a video player, or log them from the flow; logged times run up to about 0.1 seconds earlier than in the video:
151
+
152
+ ```js
153
+ export default async function ({ page, click, wait }) {
154
+ const started = Date.now()
155
+ const mark = (label) => console.error(`${label}: ${((Date.now() - started) / 1000).toFixed(2)} s`)
156
+ // ...
157
+ mark('progress started')
158
+ }
159
+ ```
160
+
161
+ Log to `console.error`, because Traceframe prints its JSON summary on standard output. This command plays seconds 2 to 9 at 4× speed. Set `fps` to the video's frame rate: 60 for the default preset, 30 for draft.
162
+
163
+ ```sh
164
+ ffmpeg -i recording.mp4 -filter_complex "[0:v]trim=0:2,setpts=PTS-STARTPTS[a];[0:v]trim=2:9,setpts=(PTS-STARTPTS)/4[b];[0:v]trim=9,setpts=PTS-STARTPTS[c];[a][b][c]concat=n=3,fps=60[v]" -map "[v]" -c:v libx264 -crf 18 -movflags +faststart faster.mp4
165
+ ```
166
+
167
+ For an animated preview where an MP4 does not play inline, make a WebP. Not every FFmpeg build includes the WebP encoder.
168
+
169
+ ```sh
170
+ ffmpeg -i recording.mp4 -vf "fps=12,scale=1280:-1" -c:v libwebp_anim -quality 75 -loop 0 preview.webp
171
+ ```
172
+
173
+ ## Optional sharing
174
+
175
+ With push access to a repository, current [GitHub CLI](https://docs.github.com/en/github-cli/github-cli/attaching-files-with-github-cli) can attach the MP4 to a comment so it plays there; check `gh pr comment --help` for your version. An MP4 hosted elsewhere shows as a link, not a player, so use the WebP preview there. Upload only when asked and after checking that the recording suits the destination; recording itself uploads nothing.
@@ -0,0 +1,27 @@
1
+ export async function ready({ page }) {
2
+ await page.getByRole('heading', { name: 'Conversation demo' }).waitFor()
3
+ }
4
+
5
+ export default async function ({ page, click, hover, wait }) {
6
+ const message = page.getByRole('textbox', { name: 'Message' })
7
+ const replies = page.getByRole('list', { name: 'Replies' }).getByRole('listitem')
8
+ for (const [index, text] of ['Give me a short summary.', 'هل يمكنك التوضيح؟'].entries()) {
9
+ await click(message)
10
+ await message.pressSequentially(text, { delay: 45 })
11
+ await click(page.getByRole('button', { name: 'Send', exact: true }))
12
+ await replies.nth(index).waitFor({ timeout: 60_000 })
13
+ await wait(0.6)
14
+ }
15
+
16
+ const upload = page.getByText('Choose a sample file', { exact: true })
17
+ await hover(upload)
18
+ await page.getByLabel('Choose a sample file').setInputFiles({
19
+ name: 'sample.txt', mimeType: 'text/plain', buffer: Buffer.from('Example file for a local demo.\n'),
20
+ })
21
+ await page.getByText('Selected: sample.txt', { exact: true }).waitFor()
22
+ await wait(0.6)
23
+
24
+ await click(page.getByRole('button', { name: 'Finish demo', exact: true }))
25
+ await page.getByRole('button', { name: 'Demo complete', exact: true }).waitFor()
26
+ await wait(0.8)
27
+ }
@@ -0,0 +1,74 @@
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>Traceframe recording recipes</title>
7
+ <style>
8
+ :root { color-scheme: light; font: 18px/1.5 system-ui, sans-serif; color: #172033; background: #f4f6fa; }
9
+ * { box-sizing: border-box; }
10
+ main { max-width: 900px; margin: 40px auto; padding: 32px; border: 1px solid #dce1eb; border-radius: 16px; background: #fff; }
11
+ h1 { margin: 0; font-size: 28px; }
12
+ h2 { margin: 20px 0 8px; font-size: 18px; }
13
+ p { margin: 8px 0 20px; }
14
+ form > label { display: block; margin-bottom: 8px; }
15
+ .compose { display: flex; gap: 12px; }
16
+ input, button { font: inherit; padding: 10px 16px; border: 1px solid #aeb7c8; border-radius: 8px; }
17
+ input { min-width: 0; flex: 1; }
18
+ button { color: #fff; background: #3454b5; cursor: pointer; }
19
+ button:hover { background: #284393; }
20
+ button:disabled { opacity: 0.6; }
21
+ #status { min-height: 27px; margin: 8px 0; }
22
+ #replies { min-height: 80px; padding-left: 28px; }
23
+ #replies li { margin: 8px 0; }
24
+ .upload { display: inline-block; padding: 12px 16px; border: 1px dashed #aeb7c8; border-radius: 8px; cursor: pointer; }
25
+ .upload:hover { background: #f4f6fa; }
26
+ .scroll-area { height: 160px; overflow: auto; border: 1px solid #dce1eb; border-radius: 8px; }
27
+ .scroll-content { height: 360px; display: flex; align-items: end; padding: 20px; }
28
+ .below-fold { margin-top: 480px; }
29
+ </style>
30
+ </head>
31
+ <body>
32
+ <main>
33
+ <h1>Conversation demo</h1>
34
+ <p>Replies are simulated locally. No messages or files leave this page.</p>
35
+ <form id="conversation">
36
+ <label for="message">Message</label>
37
+ <div class="compose"><input id="message" dir="auto" required autocomplete="off"><button id="send">Send</button></div>
38
+ </form>
39
+ <p id="status" role="status">Ready for your question.</p>
40
+ <ol id="replies" aria-label="Replies"></ol>
41
+ <h2>Attach a file</h2>
42
+ <label class="upload" for="file">Choose a sample file</label><input id="file" type="file" hidden>
43
+ <p id="selected-file">No file selected.</p>
44
+ <h2 class="below-fold">Scroll to the last action</h2>
45
+ <section class="scroll-area" aria-label="More actions" tabindex="0">
46
+ <div class="scroll-content"><button id="finish" type="button">Finish demo</button></div>
47
+ </section>
48
+ </main>
49
+ <script>
50
+ const message = document.querySelector('#message')
51
+ const send = document.querySelector('#send')
52
+ const status = document.querySelector('#status')
53
+ const replies = document.querySelector('#replies')
54
+ document.querySelector('#conversation').addEventListener('submit', async (event) => {
55
+ event.preventDefault()
56
+ const text = message.value
57
+ message.value = ''
58
+ send.disabled = true
59
+ status.textContent = 'Preparing a reply…'
60
+ await new Promise((resolve) => setTimeout(resolve, 1000))
61
+ const reply = document.createElement('li')
62
+ reply.dir = 'auto'
63
+ reply.textContent = `Reply ${replies.children.length + 1}: ${text}`
64
+ replies.append(reply)
65
+ status.textContent = 'Reply complete.'
66
+ send.disabled = false
67
+ })
68
+ document.querySelector('#file').addEventListener('change', (event) => {
69
+ document.querySelector('#selected-file').textContent = `Selected: ${event.target.files[0]?.name ?? 'none'}`
70
+ })
71
+ document.querySelector('#finish').addEventListener('click', (event) => { event.currentTarget.textContent = 'Demo complete' })
72
+ </script>
73
+ </body>
74
+ </html>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@milkstraw/traceframe",
3
- "version": "0.1.0",
3
+ "version": "0.3.0",
4
4
  "description": "Record scripted browser flows and render smooth cursor overlays into video.",
5
5
  "keywords": [
6
6
  "playwright",
@@ -24,7 +24,10 @@
24
24
  ".": "./src/index.mjs"
25
25
  },
26
26
  "files": [
27
- "src"
27
+ "src",
28
+ "docs/recording-flows.md",
29
+ "examples/recipes.html",
30
+ "examples/recipes-flow.mjs"
28
31
  ],
29
32
  "engines": {
30
33
  "node": ">=22"
package/src/capture.mjs CHANGED
@@ -1,13 +1,16 @@
1
1
  import { addAbortListener } from 'node:events'
2
+ import { writeFile } from 'node:fs/promises'
2
3
  import { join } from 'node:path'
3
4
  import { setTimeout as delay } from 'node:timers/promises'
4
5
  import { chromium } from 'playwright'
5
6
 
6
7
  import { TraceframeError } from './errors.mjs'
7
- import { createFlowApi, LIMITS } from './flow.mjs'
8
+ import { addDiagnostics, retainedFile } from './diagnostics.mjs'
9
+ import { createFlowApi } from './flow.mjs'
8
10
 
9
11
  const PLAYWRIGHT_DEFAULT_TIMEOUT = 30_000
10
12
  const FIRST_FRAME_TIMEOUT = 10_000
13
+ const FAILURE_SCREENSHOT_TIMEOUT = 5_000
11
14
 
12
15
  export async function captureFlow({ config, flow, directory, signal: callerSignal }) {
13
16
  const signal = callerSignal ?? new AbortController().signal
@@ -17,6 +20,9 @@ export async function captureFlow({ config, flow, directory, signal: callerSigna
17
20
  signal.throwIfAborted()
18
21
  const browser = await launchBrowser(config, { handleSignals: !callerSignal })
19
22
  let page
23
+ let clock
24
+ let recording = false
25
+ let failed = false
20
26
  try {
21
27
  page = await browser.newPage({ viewport: config.viewport, deviceScaleFactor: config.scale, storageState: config.storageState })
22
28
  await raceAbort(waitForReady(page, flow, config, signal), signal)
@@ -24,28 +30,60 @@ export async function captureFlow({ config, flow, directory, signal: callerSigna
24
30
 
25
31
  const firstFrame = Promise.withResolvers()
26
32
  await startScreencast(page, rawPath, config, ({ timestamp }) => firstFrame.resolve(timestamp))
27
- const clock = await withDeadline(
33
+ recording = true
34
+ clock = await withDeadline(
28
35
  () => firstFrame.promise,
29
36
  FIRST_FRAME_TIMEOUT,
30
37
  new TraceframeError(`No screencast frame arrived within ${FIRST_FRAME_TIMEOUT / 1000} seconds.`, { code: 'FIRST_FRAME_TIMEOUT' }),
31
38
  signal,
32
39
  )
33
- await withDeadline(
34
- (flowSignal) => flow.run({ page, signal: flowSignal, ...createFlowApi({ page, trace, clock, viewport: config.viewport, signal: flowSignal }) }),
35
- LIMITS.maxDuration * 1000,
36
- new TraceframeError(`Flow exceeded the ${LIMITS.maxDuration}-second duration limit.`, { code: 'FLOW_TOO_LONG' }),
37
- signal,
38
- )
40
+ try {
41
+ await withDeadline(
42
+ (flowSignal) => flow.run({ page, signal: flowSignal, ...createFlowApi({ page, trace, clock, viewport: config.viewport, signal: flowSignal }) }),
43
+ config.flowTimeout * 1000,
44
+ new TraceframeError(`Flow exceeded the ${config.flowTimeout}-second flow timeout. Raise it with the flowTimeout option, or --flow-timeout.`, { code: 'FLOW_TOO_LONG' }),
45
+ signal,
46
+ )
47
+ } catch (error) {
48
+ if (signal.aborted || error instanceof TraceframeError) throw error
49
+ throw new TraceframeError(error instanceof Error ? error.message : String(error), { code: 'FLOW_FAILED', cause: error })
50
+ }
39
51
  trace.end = (Date.now() - clock) / 1000
40
52
  await delay(config.postRoll * 1000, undefined, { signal })
53
+ // A failed stop must not be retried: Playwright may still be waiting to finish its artifact.
54
+ recording = false
41
55
  await page.screencast.stop()
42
56
  return { rawPath, trace }
43
57
  } catch (error) {
44
- // Closing the browser alone discards the screencast. Stop it so a run directory keeps what was recorded.
45
- await page?.screencast.stop().catch(() => {})
46
- throw error
58
+ failed = true
59
+ const recordingTime = clock === undefined ? undefined : Math.max(0, (Date.now() - clock) / 1000)
60
+ // Let the encoder finish saving before closing the browser, including on slower machines.
61
+ const [capturePath, screenshotPath] = await Promise.all([
62
+ recording ? page.screencast.stop()
63
+ .then(() => config.runDirectory ? retainedFile(rawPath) : undefined)
64
+ .catch(() => undefined) : undefined,
65
+ config.runDirectory && page && !signal.aborted ? saveFailureScreenshot(page, directory, signal) : undefined,
66
+ ])
67
+ if (signal.aborted) throw error
68
+ throw await addDiagnostics(error, { flow: config.flow, recordingTime, capturePath, screenshotPath })
47
69
  } finally {
48
- await browser.close()
70
+ // A shutdown error must not hide the flow failure.
71
+ if (failed) await browser.close().catch(() => {})
72
+ else await browser.close()
73
+ }
74
+ }
75
+
76
+ async function saveFailureScreenshot(page, directory, signal) {
77
+ try {
78
+ return await withDeadline(async (diagnosticSignal) => {
79
+ const screenshot = await page.screenshot({ timeout: FAILURE_SCREENSHOT_TIMEOUT })
80
+ diagnosticSignal.throwIfAborted()
81
+ const path = join(directory, 'failure.png')
82
+ await writeFile(path, screenshot, { signal: diagnosticSignal })
83
+ return path
84
+ }, FAILURE_SCREENSHOT_TIMEOUT, new Error('Failure screenshot timed out.'), signal)
85
+ } catch {
86
+ return undefined
49
87
  }
50
88
  }
51
89
 
@@ -62,7 +100,7 @@ async function launchBrowser(config, { handleSignals }) {
62
100
  if (!/Executable doesn't exist/i.test(error.message)) throw error
63
101
  const message = config.browser
64
102
  ? `Configured browser executable is unavailable: ${config.browser}`
65
- : 'Playwright Chromium is not installed. Run `npx playwright@1.64.1 install --only-shell chromium` or pass --browser to a Chrome executable.'
103
+ : 'Playwright Chromium is not installed. Run `npx playwright@1.64.1 install --only-shell chromium`, or pass --browser to a Chrome executable and run `npx playwright@1.64.1 install ffmpeg`.'
66
104
  throw new TraceframeError(message, { code: 'PLAYWRIGHT_RUNTIME_MISSING', cause: error })
67
105
  }
68
106
  }
package/src/cli.mjs CHANGED
@@ -4,6 +4,7 @@ import { createRequire } from 'node:module'
4
4
  import { parseArgs } from 'node:util'
5
5
 
6
6
  import { LIMITS } from './flow.mjs'
7
+ import { formatFailure } from './diagnostics.mjs'
7
8
  import { record } from './record.mjs'
8
9
 
9
10
  const { version: VERSION } = createRequire(import.meta.url)('../package.json')
@@ -21,9 +22,10 @@ const OPTION_KEYS = {
21
22
  fps: 'fps',
22
23
  'post-roll': 'postRoll',
23
24
  'ready-timeout': 'readyTimeout',
25
+ 'flow-timeout': 'flowTimeout',
24
26
  'run-dir': 'runDirectory',
25
27
  }
26
- const NUMBER_KEYS = new Set(['viewportWidth', 'viewportHeight', 'scale', 'fps', 'postRoll', 'readyTimeout'])
28
+ const NUMBER_KEYS = new Set(['viewportWidth', 'viewportHeight', 'scale', 'fps', 'postRoll', 'readyTimeout', 'flowTimeout'])
27
29
  const HELP = `traceframe ${VERSION}
28
30
 
29
31
  Record a scripted Chromium flow and render a smooth cursor and click pulse.
@@ -41,16 +43,17 @@ Options:
41
43
  --scale <1|2> Device scale factor; the video is the viewport times this
42
44
  --fps <30|60> Encoded frame rate
43
45
  --post-roll <seconds> Tail after the flow (default: 0.3)
44
- --ready-timeout <seconds> Playwright timeout for the page load and calls in ready() (default: 30)
45
- --run-dir <path> Keep the raw capture and FFmpeg filter in this directory
46
+ --ready-timeout <seconds> Playwright default timeout during navigation and ready() (default: 30)
47
+ --flow-timeout <seconds> Time limit for the flow, including waits (default: 120)
48
+ --run-dir <path> Keep the raw capture, failure screenshot and FFmpeg filter
46
49
  -h, --help Show this help
47
50
  -v, --version Show the version
48
51
 
49
52
  The flow module must export a default function. It may also export
50
53
  ready({ page, signal }) to establish state before capture and initialPointer
51
54
  { x, y } to set where the cursor starts.
52
- The flow helpers are move(), hover(), click(), and wait(). A flow may contain
53
- at most ${LIMITS.maxMoves} move events, ${LIMITS.maxClicks} click events, and ${LIMITS.maxDuration} seconds of work.
55
+ The flow helpers are move(), hover(), click(), scroll(), and wait(). A flow
56
+ may contain at most ${LIMITS.maxMoves} move events and ${LIMITS.maxClicks} click events.
54
57
  Each duration, dwell, and wait is at most ${LIMITS.maxActionDuration} seconds.
55
58
  `
56
59
 
@@ -72,7 +75,7 @@ try {
72
75
  process.stderr.write('traceframe: Recording was interrupted.\n')
73
76
  process.exitCode = SIGNAL_EXIT_CODES[interruptedBy]
74
77
  } else {
75
- process.stderr.write(`traceframe: ${error instanceof Error ? error.message : String(error)}\n`)
78
+ process.stderr.write(formatFailure(error))
76
79
  process.exitCode = 1
77
80
  }
78
81
  }
package/src/config.mjs CHANGED
@@ -36,6 +36,8 @@ export function createConfig(options = {}) {
36
36
  check(Number.isFinite(postRoll) && postRoll >= 0, 'Post-roll must be 0 or more seconds.')
37
37
  const readyTimeout = options.readyTimeout ?? 30
38
38
  check(Number.isFinite(readyTimeout) && readyTimeout > 0, 'Ready timeout must be more than 0 seconds.')
39
+ const flowTimeout = options.flowTimeout ?? 120
40
+ check(Number.isFinite(flowTimeout) && flowTimeout > 0, 'Flow timeout must be more than 0 seconds.')
39
41
 
40
42
  return {
41
43
  url: options.url,
@@ -48,6 +50,7 @@ export function createConfig(options = {}) {
48
50
  scale,
49
51
  postRoll,
50
52
  readyTimeout,
53
+ flowTimeout,
51
54
  runDirectory: options.runDirectory ? resolve(options.runDirectory) : undefined,
52
55
  }
53
56
  }
@@ -0,0 +1,50 @@
1
+ import { realpath, stat } from 'node:fs/promises'
2
+ import { pathToFileURL } from 'node:url'
3
+
4
+ import { TraceframeError } from './errors.mjs'
5
+
6
+ // Only report the supplied flow's locations, never the rest of a stack or error object.
7
+ export async function addDiagnostics(error, { flow, ...details }) {
8
+ if (!(error instanceof Error)) return error
9
+ try {
10
+ const paths = new Set([flow, await realpath(flow).catch(() => flow)])
11
+ const names = [...paths].flatMap((path) => [path, pathToFileURL(path).href])
12
+ const escaped = names.map((name) => name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'))
13
+ const location = new RegExp(`^\\s+at (?:async )?(?:.+ \\()?(${escaped.join('|')}):(\\d+):(\\d+)\\)?$`)
14
+ const flowLocations = []
15
+ const seenErrors = new Set()
16
+ const seenLocations = new Set()
17
+ for (let current = error; current instanceof Error && !seenErrors.has(current); current = current.cause) {
18
+ seenErrors.add(current)
19
+ for (const frame of String(current.stack ?? '').split('\n')) {
20
+ const match = frame.match(location)
21
+ if (!match || seenLocations.has(`${match[2]}:${match[3]}`)) continue
22
+ seenLocations.add(`${match[2]}:${match[3]}`)
23
+ flowLocations.push({ file: flow, line: Number(match[2]), column: Number(match[3]) })
24
+ }
25
+ }
26
+ error.diagnostics = { flowLocations, ...Object.fromEntries(Object.entries(details).filter(([, value]) => value !== undefined)) }
27
+ } catch {
28
+ // A frozen error or an unavailable stack must not replace the original failure.
29
+ }
30
+ return error
31
+ }
32
+
33
+ export async function retainedFile(path) {
34
+ const info = await stat(path).catch(() => undefined)
35
+ return info?.isFile() && info.size > 0 ? path : undefined
36
+ }
37
+
38
+ export function formatFailure(error) {
39
+ const message = error instanceof Error ? error.message : String(error)
40
+ const code = error instanceof TraceframeError && error.code ? `[${error.code}] ` : ''
41
+ const lines = [`traceframe: ${code}${message.trimEnd()}`]
42
+ const diagnostics = error?.diagnostics
43
+ for (const { file, line, column } of diagnostics?.flowLocations ?? []) {
44
+ lines.push(` Flow: ${file}:${line}:${column}`)
45
+ }
46
+ if (Number.isFinite(diagnostics?.recordingTime)) lines.push(` Recording time: ${diagnostics.recordingTime.toFixed(2)} s`)
47
+ if (diagnostics?.capturePath) lines.push(` Capture: ${diagnostics.capturePath}`)
48
+ if (diagnostics?.screenshotPath) lines.push(` Screenshot: ${diagnostics.screenshotPath}`)
49
+ return `${lines.join('\n')}\n`
50
+ }
package/src/flow.mjs CHANGED
@@ -4,7 +4,6 @@ import { pathToFileURL } from 'node:url'
4
4
  import { TraceframeError } from './errors.mjs'
5
5
 
6
6
  export const LIMITS = {
7
- maxDuration: 120,
8
7
  maxMoves: 100,
9
8
  maxClicks: 100,
10
9
  maxActionDuration: 30,
@@ -56,14 +55,14 @@ export function createFlowApi({ page, trace, clock, viewport, signal }) {
56
55
 
57
56
  const moveTo = async (locator, duration, name) => {
58
57
  const seconds = durationSeconds(duration ?? MOVE_SECONDS, `${name} duration`)
59
- await locator.waitFor()
60
- const box = await locator.boundingBox()
61
- if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
62
- // A person does not chase a target that shifts under a pointer already inside it.
63
- if (pointInsideBox(pointer, box)) return pointer
64
- const point = await aimPoint(locator, box)
58
+ const target = await findTarget(locator)
59
+ // A person does not chase a target that shifts under a pointer already on it.
60
+ if (pointInsideBox(pointer, target.box)) return pointer
61
+ let point = await aimPoint(target)
65
62
  if (!insideViewport(point, viewport)) {
66
- throw new TraceframeError(`${locator} is outside the viewport. Scroll explicitly before moving to it.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
63
+ const aim = await reveal(target.element, locator)
64
+ if (!aim.visible) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
65
+ point = aim.point
67
66
  }
68
67
  await movePointer(point, seconds)
69
68
  const after = await locator.boundingBox()
@@ -109,6 +108,10 @@ export function createFlowApi({ page, trace, clock, viewport, signal }) {
109
108
  }
110
109
  await movePointer(point, durationSeconds(options.duration ?? MOVE_SECONDS, 'move duration'))
111
110
  },
111
+ async scroll(locator) {
112
+ const { shown } = await reveal((await findTarget(locator)).element, locator)
113
+ if (!shown) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
114
+ },
112
115
  async wait(seconds) {
113
116
  await sleep(durationSeconds(seconds, 'wait', { allowZero: true }) * 1000)
114
117
  },
@@ -142,21 +145,85 @@ function assertBelowLimit(events, limit, type) {
142
145
  }
143
146
  }
144
147
 
148
+ async function findTarget(locator) {
149
+ await locator.waitFor()
150
+ const element = await locator.elementHandle()
151
+ const box = await element.boundingBox()
152
+ if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
153
+ return { element, box }
154
+ }
155
+
145
156
  function pointInsideBox(point, box) {
146
157
  return point.x >= box.x && point.x <= box.x + box.width && point.y >= box.y && point.y <= box.y + box.height
147
158
  }
148
159
 
149
160
  // Aim at the middle of the first line box, as Playwright does, so a link that wraps onto two lines is not missed.
150
- async function aimPoint(locator, box) {
151
- const offset = await locator.evaluate((element) => {
152
- const bounds = element.getBoundingClientRect()
153
- const first = element.getClientRects()[0] ?? bounds
161
+ async function aimPoint({ element, box }) {
162
+ const offset = await element.evaluate((target) => {
163
+ const bounds = target.getBoundingClientRect()
164
+ const first = target.getClientRects()[0] ?? bounds
154
165
  // As fractions of the element's size, because a scaled iframe makes the page box a different size than the element's own.
155
166
  return { x: (first.left + first.width / 2 - bounds.left) / bounds.width, y: (first.top + first.height / 2 - bounds.top) / bounds.height }
156
167
  })
157
168
  return { x: box.x + offset.x * box.width, y: box.y + offset.y * box.height }
158
169
  }
159
170
 
171
+ async function reveal(element, locator) {
172
+ // Content that loads above the target during the scroll can push it out again.
173
+ for (let attempt = 0; attempt < 2; attempt++) {
174
+ const target = await scrollIntoView(element, locator)
175
+ const point = await aimPoint(target)
176
+ const visibility = await target.element.evaluate((node, offset) => new Promise((resolve) => {
177
+ new IntersectionObserver(([entry], observer) => {
178
+ observer.disconnect()
179
+ const bounds = node.getBoundingClientRect()
180
+ const x = bounds.left + offset.x * bounds.width
181
+ const y = bounds.top + offset.y * bounds.height
182
+ const rect = entry.intersectionRect
183
+ const shown = entry.intersectionRatio > 0
184
+ resolve({ shown, visible: shown && x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom })
185
+ }).observe(node)
186
+ }), relativePoint(point, target.box))
187
+ if (visibility.visible || attempt === 1) return { point, ...visibility }
188
+ element = target.element
189
+ }
190
+ }
191
+
192
+ // Awaiting the scroll waits for a scrolling panel. Chromium resolves a page scroll before it stops, and a sticky target
193
+ // or iframe can stay put while its page scrolls, so also wait for every enclosing page to stop.
194
+ async function scrollIntoView(element, locator) {
195
+ await element.evaluate((target) => target.scrollIntoView({ behavior: 'smooth', block: 'center' }))
196
+ try {
197
+ await element.waitForElementState('stable')
198
+ let frame = await element.ownerFrame()
199
+ while (frame) {
200
+ await frame.evaluate(() => new Promise((resolve) => {
201
+ let last
202
+ const check = () => {
203
+ const now = `${scrollX},${scrollY}`
204
+ if (now === last) resolve()
205
+ else requestAnimationFrame(check)
206
+ last = now
207
+ }
208
+ requestAnimationFrame(check)
209
+ }))
210
+ if (!frame.parentFrame()) break
211
+ await (await frame.frameElement()).waitForElementState('stable')
212
+ frame = frame.parentFrame()
213
+ }
214
+ // The page may replace the target while it scrolls, or remove or hide it, as a menu that closes on scroll does.
215
+ const target = await findTarget(locator).catch((error) => {
216
+ throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} disappeared when it was scrolled into view.`, { code: 'FLOW_TARGET_MISSING', cause: error }) : error
217
+ })
218
+ await target.element.waitForElementState('stable')
219
+ const box = await target.element.boundingBox()
220
+ if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
221
+ return { element: target.element, box }
222
+ } catch (error) {
223
+ throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} kept moving after it was scrolled into view.`, { code: 'FLOW_TARGET_MOVED', cause: error }) : error
224
+ }
225
+ }
226
+
160
227
  async function assertReceivesPointer(element, locator, point) {
161
228
  const box = await element.boundingBox()
162
229
  if (!box || !pointInsideBox(point, box)) {
@@ -172,7 +239,7 @@ async function assertReceivesPointer(element, locator, point) {
172
239
  return false
173
240
  }, relativePoint(point, box))
174
241
  if (!receivesPointer || !(await framesReceivePointer(element, point))) {
175
- throw new TraceframeError(`${locator} is covered at the click point.`, { code: 'FLOW_TARGET_OBSCURED' })
242
+ throw new TraceframeError(`${locator} is covered at the click point. If a dialog or toast covers it, wait until it closes; if a panel clips it or a sticky header or bar covers it, call scroll() on the target first.`, { code: 'FLOW_TARGET_OBSCURED' })
176
243
  }
177
244
  }
178
245
 
package/src/record.mjs CHANGED
@@ -4,6 +4,7 @@ import { dirname, join } from 'node:path'
4
4
 
5
5
  import { createConfig } from './config.mjs'
6
6
  import { captureFlow } from './capture.mjs'
7
+ import { addDiagnostics, retainedFile } from './diagnostics.mjs'
7
8
  import { probeFfmpeg } from './ffmpeg.mjs'
8
9
  import { loadFlow } from './flow.mjs'
9
10
  import { renderVideo } from './render.mjs'
@@ -13,13 +14,18 @@ export async function record(options = {}) {
13
14
  signal?.throwIfAborted()
14
15
  const config = createConfig(options)
15
16
  const ffmpeg = await probeFfmpeg()
16
- const flow = await loadFlow(config.flow, config.viewport)
17
+ const flow = await loadFlow(config.flow, config.viewport).catch(async (error) => {
18
+ throw await addDiagnostics(error, { flow: config.flow })
19
+ })
17
20
  await mkdir(dirname(config.output), { recursive: true })
18
21
  const directory = config.runDirectory ?? await mkdtemp(join(tmpdir(), 'traceframe-'))
19
22
  await mkdir(directory, { recursive: true })
20
23
 
24
+ let capture
25
+ let failed = false
21
26
  try {
22
- const { rawPath, trace } = await captureFlow({ config, flow, directory, signal })
27
+ capture = await captureFlow({ config, flow, directory, signal })
28
+ const { rawPath, trace } = capture
23
29
  const duration = await renderVideo({ rawPath, trace, directory, config, ffmpeg, signal })
24
30
  return {
25
31
  output: config.output,
@@ -32,8 +38,17 @@ export async function record(options = {}) {
32
38
  runDirectory: config.runDirectory,
33
39
  }
34
40
  } catch (error) {
41
+ failed = true
42
+ if (signal?.aborted) throw signal.reason
43
+ if (capture && config.runDirectory) {
44
+ await addDiagnostics(error, { flow: config.flow, capturePath: await retainedFile(capture.rawPath) })
45
+ }
35
46
  throw signal?.aborted ? signal.reason : error
36
47
  } finally {
37
- if (!config.runDirectory) await rm(directory, { recursive: true, force: true })
48
+ if (!config.runDirectory) {
49
+ const cleanup = rm(directory, { recursive: true, force: true })
50
+ if (failed) await cleanup.catch(() => {})
51
+ else await cleanup
52
+ }
38
53
  }
39
54
  }