@milkstraw/traceframe 0.0.0-stage → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Traceframe contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,178 @@
1
- # Temporary Holding Version
1
+ # Traceframe
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Traceframe records a scripted Chromium flow and renders an MP4 with a smooth cursor and click pulse. Everything runs locally: pages, flows, videos, and authentication state stay on your machine.
4
+
5
+ https://github.com/user-attachments/assets/38e02cd5-a9ba-4f3c-938c-159955353141
6
+
7
+ ## Prerequisites
8
+
9
+ Traceframe requires Node.js 22+ and system FFmpeg 5+ with `libx264` and `ffprobe` to render the MP4 (for example, `brew install ffmpeg` or `sudo apt install ffmpeg`). Playwright uses its own Chromium and FFmpeg builds to record the page. Install them with the exact Playwright version Traceframe uses; other cached versions are not used:
10
+
11
+ ```sh
12
+ npx playwright@1.64.1 install --only-shell chromium ffmpeg
13
+ ```
14
+
15
+ To use system Chrome 150 or later, pass `--browser` or set `CHROME_PATH`, and install Playwright's recording FFmpeg with `npx playwright@1.64.1 install ffmpeg`. Older Chrome versions can fail to scroll targets inside panels.
16
+
17
+ In a sandbox, such as a coding agent's, allow Playwright's browser download and let Chromium start.
18
+
19
+ To record pages behind a login, pass a Playwright storage-state file with `--storage-state`. Keep it in `.auth/` or another ignored location.
20
+
21
+ ## Record a video
22
+
23
+ Run Traceframe with `npx` without adding it to your project's dependencies. Pass your application URL and your own [flow file](#write-a-flow):
24
+
25
+ ```sh
26
+ npx @milkstraw/traceframe record \
27
+ --url http://localhost:3000 \
28
+ --flow ./flow.mjs \
29
+ --out ./recording.mp4
30
+ ```
31
+
32
+ For regular use, you can install the command globally:
33
+
34
+ ```sh
35
+ npm install -g @milkstraw/traceframe
36
+ traceframe record --url http://localhost:3000 --flow ./flow.mjs --out ./recording.mp4
37
+ ```
38
+
39
+ To check installation before touching your application, save this as `smoke.mjs`:
40
+
41
+ ```js
42
+ export default async function ({ wait }) {
43
+ await wait(0.2)
44
+ }
45
+ ```
46
+
47
+ ```sh
48
+ npx @milkstraw/traceframe record \
49
+ --url 'data:text/html,<h1>Traceframe smoke</h1>' \
50
+ --flow ./smoke.mjs --out ./smoke.mp4 \
51
+ --preset draft
52
+ ```
53
+
54
+ The [recording guide](docs/recording-flows.md) includes a runnable example and recipes for form input, asynchronous results, browser settings, and editing or sharing a recording.
55
+
56
+ Both presets use a 1280×800 CSS viewport. `draft` produces 1280×800 at 30 FPS; `standard` (the default) produces 2560×1600 at 60 FPS. The output dimensions are the viewport times `--scale`, so `--preset draft --scale 2` gives 2560×1600 at 30 FPS. The command prints a JSON summary when it finishes.
57
+
58
+ The cursor and click pulses are always drawn at the full frame rate. Chromium may capture page changes, such as scrolling and animations, at a lower rate at 2560×1600, depending on the machine. The video repeats the last page frame until the next one arrives.
59
+
60
+ Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and FFmpeg filter file, including a partial capture if the flow fails during recording. Keep that directory out of Git, because the raw capture shows everything the page showed.
61
+
62
+ ## Write a flow
63
+
64
+ The flow module exports a default function. Timing values are seconds:
65
+
66
+ ```js
67
+ export async function ready({ page }) {
68
+ await page.getByRole('heading', { name: 'Example workspace' }).waitFor()
69
+ }
70
+
71
+ export default async function ({ page, click, wait }) {
72
+ await click(page.getByRole('button', { name: 'Open settings' }), { duration: 0.8 })
73
+ await wait(1)
74
+ }
75
+ ```
76
+
77
+ The optional `ready({ page, signal })` hook runs after initial navigation and before recording starts. Export `initialPointer = { x, y }` to choose where the cursor starts; by default it starts near the lower left.
78
+
79
+ ### Helpers
80
+
81
+ The default function receives a Playwright `page`, an `AbortSignal` named `signal`, and five helpers:
82
+
83
+ | Helper | Use |
84
+ | --- | --- |
85
+ | `move({ x, y }, { duration })` | Move the pointer to viewport coordinates. |
86
+ | `hover(locator, { duration })` | Move the pointer to a locator. |
87
+ | `click(locator, { duration, dwell })` | Move, pause, and click a locator. |
88
+ | `scroll(locator)` | Center a target without moving the pointer. |
89
+ | `wait(seconds)` | Pause for reading time. |
90
+
91
+ A move takes 0.5 seconds by default. A click pauses on the target for 0.25 seconds (`dwell`) before pressing, and waits for a disabled target to become enabled. `click` and `hover` leave the pointer in place if it is already inside the target's box.
92
+
93
+ Error messages name the locator; give it a clearer name with Playwright's `locator.describe('Save settings')`.
94
+
95
+ ### Scrolling
96
+
97
+ `click` and `hover` scroll automatically only when their aim point starts outside the viewport. They scroll smoothly through the page, scrollable containers, and iframes, and wait for scrolling to stop before moving the pointer.
98
+
99
+ `scroll(target)` smoothly centers the target vertically, even if it is already visible, and waits for scrolling to stop without moving the pointer. Use it to show a result or before clicking a target clipped by a scrollable panel or covered by a sticky header. Wait for a dialog or toast to close instead.
100
+
101
+ Centering can leave a target partly visible or covered, for example by a sticky bar taller than half the height of the viewport or its scrolling panel. It cannot reveal content hidden by `overflow: clip`. `click` fails with `FLOW_TARGET_OBSCURED` if the target is still covered when it presses.
102
+
103
+ ### Timing and limits
104
+
105
+ `--ready-timeout` (default: 30 seconds) sets Playwright's default timeout during initial navigation and `ready`, for operations that accept a timeout. It does not set a deadline for the whole hook.
106
+
107
+ The default function may run for 120 seconds, including waits for your backend. For longer flows, pass `--flow-timeout 300` (`flowTimeout: 300` in the API). This does not change Playwright's action timeouts. Helper calls stop promptly when the flow timeout is reached.
108
+
109
+ A flow may contain at most 100 moves and 100 clicks, and each `duration`, `dwell`, and `wait` is at most 30 seconds.
110
+
111
+ ### What appears in the video
112
+
113
+ Playwright typing and scrolling appear in the captured page. Direct `page.mouse`, `locator.click()`, and dragging calls bypass Traceframe's cursor trace and click pulses, so use its pointer helpers for visible interactions. Only the starting page is recorded; new tabs and popups do not appear in the video.
114
+
115
+ ### Have an agent create the flow
116
+
117
+ You can ask a coding agent to inspect your running UI and write the flow from a journey:
118
+
119
+ ```text
120
+ Create a Traceframe flow for this user journey: open Settings, change the theme, and save it.
121
+ Follow the Traceframe recording guide: https://github.com/milkstrawai/traceframe/blob/main/docs/recording-flows.md
122
+ Inspect the running application, use stable role or test-id locators, add a ready hook,
123
+ and use Traceframe's click, hover, move, scroll, and wait helpers for visible interactions.
124
+ Write the result to flow.mjs and explain how to run it.
125
+ ```
126
+
127
+ ## Programmatic usage
128
+
129
+ To call Traceframe from JavaScript, install it in your project:
130
+
131
+ ```sh
132
+ npm install @milkstraw/traceframe
133
+ ```
134
+
135
+ ```js
136
+ import { record } from '@milkstraw/traceframe'
137
+
138
+ await record({
139
+ url: 'http://localhost:3000',
140
+ flow: './flow.mjs',
141
+ output: './recording.mp4',
142
+ preset: 'standard',
143
+ })
144
+ ```
145
+
146
+ Traceframe's own errors are `TraceframeError` instances (exported by the package) with a `code`, such as `READY_FAILED` or `FLOW_TARGET_OBSCURED`.
147
+
148
+ Pass `signal` to stop a recording early. Traceframe closes the browser, stops FFmpeg, and removes its temporary files before rejecting with the signal's reason. When you pass a signal, Traceframe leaves Ctrl-C and other termination signals to you. The CLI does this for you: the first Ctrl-C stops and cleans up, and a second one exits at once.
149
+
150
+ ## Sample output
151
+
152
+ The committed sample is generated from the [repository's fixture flow](examples/fixture-flow.mjs) with the `standard` preset (2560×1600 at 60 FPS) and is about 12 seconds long:
153
+
154
+ [Download the sample video](https://github.com/milkstrawai/traceframe/blob/main/examples/sample-output.mp4)
155
+
156
+ The cursor and click-pulse artwork and the fixture page's icons were made for Traceframe and are distributed under this repository's MIT license.
157
+
158
+ The fixture page uses the [Inter](https://github.com/rsms/inter) font, included under the SIL Open Font License in `examples/fonts/OFL.txt`.
159
+
160
+ ## Development
161
+
162
+ ```sh
163
+ npm ci
164
+ npm test
165
+ npm run test:smoke
166
+ TRACEFRAME_PRESET=standard npm run test:smoke
167
+ npm run test:package
168
+ ```
169
+
170
+ To release, set the version in `package.json` and `package-lock.json`, push all release changes to `main`, and publish a GitHub Release with the tag `v<version>` on the final tested commit containing those changes. The Release workflow reruns CI and stages the package on npm. A maintainer downloads it with `npm stage download <stage-id>`, checks that its files match `npm pack` at the tag, and approves it on npmjs.com with 2FA.
171
+
172
+ After editing an SVG in `src/assets`, run `npm run build:assets` to rebuild its 1× and 2× PNGs.
173
+
174
+ To re-record the committed sample:
175
+
176
+ ```sh
177
+ npx traceframe record --url "file://$PWD/examples/fixture.html" --flow ./examples/fixture-flow.mjs --out ./examples/sample-output.mp4
178
+ ```
@@ -0,0 +1,123 @@
1
+ # Write a recording flow
2
+
3
+ Use this guide yourself or give it to a coding agent with the journey you want to show. Traceframe provides pointer and scrolling helpers; the supplied Playwright page handles typing, file selection, and application checks.
4
+
5
+ ## Run the example
6
+
7
+ The package includes [a complete flow](../examples/recipes-flow.mjs) and [its local page](../examples/recipes.html). It types English and Arabic, waits for two simulated replies, selects a fake file, and clicks a final action in a panel below the fold, which `click` scrolls into view first. After a global installation and the [prerequisites](../README.md#prerequisites), run:
8
+
9
+ ```sh
10
+ TRACEFRAME_EXAMPLES="$(npm root -g)/@milkstraw/traceframe/examples"
11
+ traceframe record \
12
+ --url "file://$TRACEFRAME_EXAMPLES/recipes.html" \
13
+ --flow "$TRACEFRAME_EXAMPLES/recipes-flow.mjs" \
14
+ --preset draft --out ./recipes.mp4
15
+ ```
16
+
17
+ In a repository clone, run `npx traceframe record --url "file://$PWD/examples/recipes.html" --flow ./examples/recipes-flow.mjs --preset draft --out ./recipes.mp4` instead. These shell commands use macOS/Linux path syntax. The flow itself has no application dependencies, external services, or credentials.
18
+
19
+ ## Prepare the journey
20
+
21
+ Inspect the running application at the recording viewport. Pick a short journey with a visible result and known demo data. Use role, label, or test-id locators; `locator.describe()` can make errors clearer.
22
+
23
+ Put setup and page readiness in `ready({ page, signal })`. This runs after initial navigation and before recording. Use `--storage-state` for an existing login, and keep that file in an ignored directory. For headers, locale, or themes, see [optional startup settings](#optional-startup-settings).
24
+
25
+ ## Choose the right actions
26
+
27
+ Use Traceframe's `click`, `hover`, and `move` for visible pointer actions. Direct `locator.click()` and `page.mouse` calls move the browser pointer without updating Traceframe's overlay or click pulses. Use Playwright for the other actions below; their page changes appear in the video.
28
+
29
+ For a target clipped by a scrollable panel or covered by a sticky header, try `await scroll(target)` before the click. For a tooltip, a toast, or a closing dialog, wait until it is gone, or give the click a longer `dwell`. A covered click fails with an error that it "is covered at the click point" (`FLOW_TARGET_OBSCURED`); `hover` does not check for covers. See the [scrolling rules and limits](../README.md#scrolling).
30
+
31
+ Both automatic scrolling and `scroll(target)` can move the outer page, which closes some lists. For those lists, use their own scrolling container through Playwright, for example `await list.evaluate((element) => element.scrollTo({ top: element.scrollHeight, behavior: 'smooth' }))` to show the final options, then click the option. Wait for any opening animation before scrolling.
32
+
33
+ To show a result without a pointer action, use `await scroll(result)`. Centering a result taller than the view hides its start, so scroll to its heading or another element near its start instead.
34
+
35
+ Focus a field visibly, then type with a delay in milliseconds:
36
+
37
+ ```js
38
+ await click(message)
39
+ await message.pressSequentially('A short question', { delay: 45 })
40
+ ```
41
+
42
+ For an upload, hover the visible control, set the actual file input, and wait for the application's selected-file feedback. The visible label or drop zone and the input are often different elements:
43
+
44
+ ```js
45
+ const dropZone = page.getByText('Choose a file', { exact: true })
46
+ const fileInput = page.getByLabel('Choose a file')
47
+ await hover(dropZone)
48
+ await fileInput.setInputFiles('./sample.txt')
49
+ await page.getByText('sample.txt').waitFor()
50
+ await wait(0.6)
51
+ ```
52
+
53
+ A relative path is resolved from the directory where you run Traceframe. File selection does not show an operating-system file picker. The complete example supplies an in-memory fake file so it can run on its own.
54
+
55
+ ## Wait for the current result
56
+
57
+ Before a follow-up, wait for the application to show that the current step is complete. For a conversation that appends completed replies, the example waits for the next reply by index. For streamed answers, wait for that request's completion indicator instead; the first visible text or a previous reply is not completion.
58
+
59
+ Set a Playwright timeout explicitly when an individual operation may exceed its usual 30 seconds, such as `await newReply.waitFor({ timeout: 60_000 })`. Use Traceframe's `wait(seconds)` for reading time. Each helper wait is limited to 30 seconds, and the whole flow has a 120-second timeout that `--flow-timeout` (`flowTimeout` in the API) raises.
60
+
61
+ ## Optional startup settings
62
+
63
+ `page.emulateMedia({ colorScheme: 'dark' })` in `ready` updates CSS themes that follow the browser's color scheme. If the app reads the setting only at startup, set it before opening the app, as below.
64
+
65
+ If the application needs request headers, a browser locale, or a color scheme from startup, record `--url about:blank` and open the application yourself in `ready`. Keep only the lines you need:
66
+
67
+ ```js
68
+ export async function ready({ page }) {
69
+ await page.emulateMedia({ colorScheme: 'dark' })
70
+ await page.setExtraHTTPHeaders({ 'x-demo-access': 'demo' })
71
+ const cdp = await page.context().newCDPSession(page)
72
+ await cdp.send('Emulation.setLocaleOverride', { locale: 'ar-EG' })
73
+ await cdp.send('Emulation.setUserAgentOverride', { userAgent: '', acceptLanguage: 'ar-EG' })
74
+ await page.goto('http://localhost:3000')
75
+ await page.getByRole('heading', { name: 'Example workspace' }).waitFor()
76
+ await page.evaluate(() => document.fonts.ready)
77
+ }
78
+ ```
79
+
80
+ The two `Emulation` calls set the locale that `navigator.language`, `Intl`, and the `Accept-Language` header use; the empty `userAgent` keeps Chromium's own user agent and client hints. Traceframe waits for web fonts only on the `--url` page, so the last line waits for the application's fonts.
81
+
82
+ ## Record, inspect, and refine
83
+
84
+ Use `--preset draft` while writing the flow and the default preset for the final video.
85
+
86
+ While writing the flow, pass `--run-dir` with an ignored folder, such as `.traceframe/runs/demo`. A failed take then keeps its partial `capture.webm`, which has no cursor overlay and stops where the flow failed.
87
+
88
+ A failed take may already have submitted real jobs, so reset demo data before recording again. Check the important transitions and the final result. You can also check the file for decode errors:
89
+
90
+ ```sh
91
+ ffmpeg -v error -xerror -i ./recording.mp4 -f null -
92
+ ```
93
+
94
+ ## Edit the finished video
95
+
96
+ Edit the finished MP4 with FFmpeg rather than changing the flow. Editing finished frames keeps the page and cursor together. Keep the original, and say when a clip's pacing was edited.
97
+
98
+ To speed up an idle stretch, such as a progress bar, you need its start and end times. Read them in a video player, or log them from the flow; logged times run up to about 0.1 seconds earlier than in the video:
99
+
100
+ ```js
101
+ export default async function ({ page, click, wait }) {
102
+ const started = Date.now()
103
+ const mark = (label) => console.error(`${label}: ${((Date.now() - started) / 1000).toFixed(2)} s`)
104
+ // ...
105
+ mark('progress started')
106
+ }
107
+ ```
108
+
109
+ Log to `console.error`, because Traceframe prints its JSON summary on standard output. This command plays seconds 2 to 9 at 4× speed. Set `fps` to the video's frame rate: 60 for the default preset, 30 for draft.
110
+
111
+ ```sh
112
+ ffmpeg -i recording.mp4 -filter_complex "[0:v]trim=0:2,setpts=PTS-STARTPTS[a];[0:v]trim=2:9,setpts=(PTS-STARTPTS)/4[b];[0:v]trim=9,setpts=PTS-STARTPTS[c];[a][b][c]concat=n=3,fps=60[v]" -map "[v]" -c:v libx264 -crf 18 -movflags +faststart faster.mp4
113
+ ```
114
+
115
+ For an animated preview where an MP4 does not play inline, make a WebP. Not every FFmpeg build includes the WebP encoder.
116
+
117
+ ```sh
118
+ ffmpeg -i recording.mp4 -vf "fps=12,scale=1280:-1" -c:v libwebp_anim -quality 75 -loop 0 preview.webp
119
+ ```
120
+
121
+ ## Optional sharing
122
+
123
+ With push access to a repository, current [GitHub CLI](https://docs.github.com/en/github-cli/github-cli/attaching-files-with-github-cli) can attach the MP4 to a comment so it plays there; check `gh pr comment --help` for your version. An MP4 hosted elsewhere shows as a link, not a player, so use the WebP preview there. Upload only when asked and after checking that the recording suits the destination; recording itself uploads nothing.
@@ -0,0 +1,27 @@
1
+ export async function ready({ page }) {
2
+ await page.getByRole('heading', { name: 'Conversation demo' }).waitFor()
3
+ }
4
+
5
+ export default async function ({ page, click, hover, wait }) {
6
+ const message = page.getByRole('textbox', { name: 'Message' })
7
+ const replies = page.getByRole('list', { name: 'Replies' }).getByRole('listitem')
8
+ for (const [index, text] of ['Give me a short summary.', 'هل يمكنك التوضيح؟'].entries()) {
9
+ await click(message)
10
+ await message.pressSequentially(text, { delay: 45 })
11
+ await click(page.getByRole('button', { name: 'Send', exact: true }))
12
+ await replies.nth(index).waitFor({ timeout: 60_000 })
13
+ await wait(0.6)
14
+ }
15
+
16
+ const upload = page.getByText('Choose a sample file', { exact: true })
17
+ await hover(upload)
18
+ await page.getByLabel('Choose a sample file').setInputFiles({
19
+ name: 'sample.txt', mimeType: 'text/plain', buffer: Buffer.from('Example file for a local demo.\n'),
20
+ })
21
+ await page.getByText('Selected: sample.txt', { exact: true }).waitFor()
22
+ await wait(0.6)
23
+
24
+ await click(page.getByRole('button', { name: 'Finish demo', exact: true }))
25
+ await page.getByRole('button', { name: 'Demo complete', exact: true }).waitFor()
26
+ await wait(0.8)
27
+ }
@@ -0,0 +1,74 @@
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>Traceframe recording recipes</title>
7
+ <style>
8
+ :root { color-scheme: light; font: 18px/1.5 system-ui, sans-serif; color: #172033; background: #f4f6fa; }
9
+ * { box-sizing: border-box; }
10
+ main { max-width: 900px; margin: 40px auto; padding: 32px; border: 1px solid #dce1eb; border-radius: 16px; background: #fff; }
11
+ h1 { margin: 0; font-size: 28px; }
12
+ h2 { margin: 20px 0 8px; font-size: 18px; }
13
+ p { margin: 8px 0 20px; }
14
+ form > label { display: block; margin-bottom: 8px; }
15
+ .compose { display: flex; gap: 12px; }
16
+ input, button { font: inherit; padding: 10px 16px; border: 1px solid #aeb7c8; border-radius: 8px; }
17
+ input { min-width: 0; flex: 1; }
18
+ button { color: #fff; background: #3454b5; cursor: pointer; }
19
+ button:hover { background: #284393; }
20
+ button:disabled { opacity: 0.6; }
21
+ #status { min-height: 27px; margin: 8px 0; }
22
+ #replies { min-height: 80px; padding-left: 28px; }
23
+ #replies li { margin: 8px 0; }
24
+ .upload { display: inline-block; padding: 12px 16px; border: 1px dashed #aeb7c8; border-radius: 8px; cursor: pointer; }
25
+ .upload:hover { background: #f4f6fa; }
26
+ .scroll-area { height: 160px; overflow: auto; border: 1px solid #dce1eb; border-radius: 8px; }
27
+ .scroll-content { height: 360px; display: flex; align-items: end; padding: 20px; }
28
+ .below-fold { margin-top: 480px; }
29
+ </style>
30
+ </head>
31
+ <body>
32
+ <main>
33
+ <h1>Conversation demo</h1>
34
+ <p>Replies are simulated locally. No messages or files leave this page.</p>
35
+ <form id="conversation">
36
+ <label for="message">Message</label>
37
+ <div class="compose"><input id="message" dir="auto" required autocomplete="off"><button id="send">Send</button></div>
38
+ </form>
39
+ <p id="status" role="status">Ready for your question.</p>
40
+ <ol id="replies" aria-label="Replies"></ol>
41
+ <h2>Attach a file</h2>
42
+ <label class="upload" for="file">Choose a sample file</label><input id="file" type="file" hidden>
43
+ <p id="selected-file">No file selected.</p>
44
+ <h2 class="below-fold">Scroll to the last action</h2>
45
+ <section class="scroll-area" aria-label="More actions" tabindex="0">
46
+ <div class="scroll-content"><button id="finish" type="button">Finish demo</button></div>
47
+ </section>
48
+ </main>
49
+ <script>
50
+ const message = document.querySelector('#message')
51
+ const send = document.querySelector('#send')
52
+ const status = document.querySelector('#status')
53
+ const replies = document.querySelector('#replies')
54
+ document.querySelector('#conversation').addEventListener('submit', async (event) => {
55
+ event.preventDefault()
56
+ const text = message.value
57
+ message.value = ''
58
+ send.disabled = true
59
+ status.textContent = 'Preparing a reply…'
60
+ await new Promise((resolve) => setTimeout(resolve, 1000))
61
+ const reply = document.createElement('li')
62
+ reply.dir = 'auto'
63
+ reply.textContent = `Reply ${replies.children.length + 1}: ${text}`
64
+ replies.append(reply)
65
+ status.textContent = 'Reply complete.'
66
+ send.disabled = false
67
+ })
68
+ document.querySelector('#file').addEventListener('change', (event) => {
69
+ document.querySelector('#selected-file').textContent = `Selected: ${event.target.files[0]?.name ?? 'none'}`
70
+ })
71
+ document.querySelector('#finish').addEventListener('click', (event) => { event.currentTarget.textContent = 'Demo complete' })
72
+ </script>
73
+ </body>
74
+ </html>
package/package.json CHANGED
@@ -1,6 +1,48 @@
1
1
  {
2
2
  "name": "@milkstraw/traceframe",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
3
+ "version": "0.2.0",
4
+ "description": "Record scripted browser flows and render smooth cursor overlays into video.",
5
+ "keywords": [
6
+ "playwright",
7
+ "screen-recording",
8
+ "video",
9
+ "demo",
10
+ "cursor",
11
+ "browser-automation"
12
+ ],
13
+ "repository": {
14
+ "type": "git",
15
+ "url": "git+https://github.com/milkstrawai/traceframe.git"
16
+ },
17
+ "homepage": "https://github.com/milkstrawai/traceframe#readme",
18
+ "bugs": "https://github.com/milkstrawai/traceframe/issues",
19
+ "type": "module",
20
+ "bin": {
21
+ "traceframe": "src/cli.mjs"
22
+ },
23
+ "exports": {
24
+ ".": "./src/index.mjs"
25
+ },
26
+ "files": [
27
+ "src",
28
+ "docs/recording-flows.md",
29
+ "examples/recipes.html",
30
+ "examples/recipes-flow.mjs"
31
+ ],
32
+ "engines": {
33
+ "node": ">=22"
34
+ },
35
+ "scripts": {
36
+ "test": "node --test",
37
+ "test:smoke": "node scripts/smoke.mjs",
38
+ "test:package": "node scripts/package-smoke.mjs",
39
+ "build:assets": "node scripts/build-assets.mjs"
40
+ },
41
+ "dependencies": {
42
+ "playwright": "1.64.1"
43
+ },
44
+ "license": "MIT",
45
+ "publishConfig": {
46
+ "access": "public"
47
+ }
48
+ }
Binary file
Binary file
@@ -0,0 +1,4 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" width="30" height="30" viewBox="0 0 30 30">
2
+ <circle cx="15" cy="15" r="8" fill="none" stroke="#fff" stroke-width="5" opacity=".82"/>
3
+ <circle cx="15" cy="15" r="8" fill="none" stroke="#14191e" stroke-width="3" opacity=".78"/>
4
+ </svg>
Binary file
Binary file
@@ -0,0 +1,3 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 32 32">
2
+ <path d="M6.6 1.6Q6.6 1.05 7.05 1.42L21.2 12.55Q22.15 13.3 21.7 14.05Q21.45 14.4 20.95 14.4H15.35L18.25 19.95Q18.65 20.72 17.9 21.1L16.05 22.05Q15.3 22.45 14.9 21.7L11.85 16L8.35 19.45Q7.85 19.95 7.35 19.65Q6.9 19.4 6.9 18.8Z" fill="#000000" stroke="#FFFFFF" stroke-width="1.5" stroke-linejoin="round" stroke-linecap="round" vector-effect="non-scaling-stroke" transform="translate(6.6 1.6) scale(0.78 0.84) translate(-6.6 -1.6)"/>
3
+ </svg>
@@ -0,0 +1,111 @@
1
+ import { addAbortListener } from 'node:events'
2
+ import { join } from 'node:path'
3
+ import { setTimeout as delay } from 'node:timers/promises'
4
+ import { chromium } from 'playwright'
5
+
6
+ import { TraceframeError } from './errors.mjs'
7
+ import { createFlowApi } from './flow.mjs'
8
+
9
+ const PLAYWRIGHT_DEFAULT_TIMEOUT = 30_000
10
+ const FIRST_FRAME_TIMEOUT = 10_000
11
+
12
+ export async function captureFlow({ config, flow, directory, signal: callerSignal }) {
13
+ const signal = callerSignal ?? new AbortController().signal
14
+ const rawPath = join(directory, 'capture.webm')
15
+ const trace = { initialPointer: flow.initialPointer, moves: [], clicks: [] }
16
+
17
+ signal.throwIfAborted()
18
+ const browser = await launchBrowser(config, { handleSignals: !callerSignal })
19
+ let page
20
+ try {
21
+ page = await browser.newPage({ viewport: config.viewport, deviceScaleFactor: config.scale, storageState: config.storageState })
22
+ await raceAbort(waitForReady(page, flow, config, signal), signal)
23
+ await page.mouse.move(trace.initialPointer.x, trace.initialPointer.y)
24
+
25
+ const firstFrame = Promise.withResolvers()
26
+ await startScreencast(page, rawPath, config, ({ timestamp }) => firstFrame.resolve(timestamp))
27
+ const clock = await withDeadline(
28
+ () => firstFrame.promise,
29
+ FIRST_FRAME_TIMEOUT,
30
+ new TraceframeError(`No screencast frame arrived within ${FIRST_FRAME_TIMEOUT / 1000} seconds.`, { code: 'FIRST_FRAME_TIMEOUT' }),
31
+ signal,
32
+ )
33
+ await withDeadline(
34
+ (flowSignal) => flow.run({ page, signal: flowSignal, ...createFlowApi({ page, trace, clock, viewport: config.viewport, signal: flowSignal }) }),
35
+ config.flowTimeout * 1000,
36
+ new TraceframeError(`Flow exceeded the ${config.flowTimeout}-second flow timeout. Raise it with the flowTimeout option, or --flow-timeout.`, { code: 'FLOW_TOO_LONG' }),
37
+ signal,
38
+ )
39
+ trace.end = (Date.now() - clock) / 1000
40
+ await delay(config.postRoll * 1000, undefined, { signal })
41
+ await page.screencast.stop()
42
+ return { rawPath, trace }
43
+ } catch (error) {
44
+ // Closing the browser alone discards the screencast. Stop it so a run directory keeps what was recorded.
45
+ await page?.screencast.stop().catch(() => {})
46
+ throw error
47
+ } finally {
48
+ await browser.close()
49
+ }
50
+ }
51
+
52
+ async function launchBrowser(config, { handleSignals }) {
53
+ try {
54
+ return await chromium.launch({
55
+ executablePath: config.browser,
56
+ args: config.scale === 2 ? ['--force-device-scale-factor=2'] : [],
57
+ handleSIGINT: handleSignals,
58
+ handleSIGTERM: handleSignals,
59
+ handleSIGHUP: handleSignals,
60
+ })
61
+ } catch (error) {
62
+ if (!/Executable doesn't exist/i.test(error.message)) throw error
63
+ const message = config.browser
64
+ ? `Configured browser executable is unavailable: ${config.browser}`
65
+ : 'Playwright Chromium is not installed. Run `npx playwright@1.64.1 install --only-shell chromium`, or pass --browser to a Chrome executable and run `npx playwright@1.64.1 install ffmpeg`.'
66
+ throw new TraceframeError(message, { code: 'PLAYWRIGHT_RUNTIME_MISSING', cause: error })
67
+ }
68
+ }
69
+
70
+ async function waitForReady(page, flow, config, signal) {
71
+ try {
72
+ page.setDefaultTimeout(config.readyTimeout * 1000)
73
+ await page.goto(config.url)
74
+ await page.evaluate(() => document.fonts.ready)
75
+ await flow.ready?.({ page, signal })
76
+ page.setDefaultTimeout(PLAYWRIGHT_DEFAULT_TIMEOUT)
77
+ } catch (error) {
78
+ throw new TraceframeError(`Ready check failed: ${error instanceof Error ? error.message : error}`, { code: 'READY_FAILED', cause: error })
79
+ }
80
+ }
81
+
82
+ async function startScreencast(page, path, config, onFrame) {
83
+ try {
84
+ await page.screencast.start({ path, size: { width: config.video.width, height: config.video.height }, fps: config.video.fps, quality: 95, onFrame })
85
+ } catch (error) {
86
+ if (!/Executable doesn't exist/i.test(error.message)) throw error
87
+ throw new TraceframeError("Playwright's FFmpeg is not installed. It is needed to record, even with --browser. Run `npx playwright@1.64.1 install ffmpeg`.", { code: 'PLAYWRIGHT_RUNTIME_MISSING', cause: error })
88
+ }
89
+ }
90
+
91
+ // Settles with the task, or rejects at the deadline or on abort, even if the task ignores its signal.
92
+ async function withDeadline(task, milliseconds, error, signal) {
93
+ const controller = new AbortController()
94
+ const timer = setTimeout(() => controller.abort(error), milliseconds)
95
+ const deadline = AbortSignal.any([signal, controller.signal])
96
+ try {
97
+ return await raceAbort(task(deadline), deadline)
98
+ } finally {
99
+ clearTimeout(timer)
100
+ }
101
+ }
102
+
103
+ async function raceAbort(promise, signal) {
104
+ const aborted = Promise.withResolvers()
105
+ const listener = addAbortListener(signal, () => aborted.reject(signal.reason))
106
+ try {
107
+ return await Promise.race([promise, aborted.promise])
108
+ } finally {
109
+ listener[Symbol.dispose]()
110
+ }
111
+ }
package/src/cli.mjs ADDED
@@ -0,0 +1,111 @@
1
+ #!/usr/bin/env node
2
+
3
+ import { createRequire } from 'node:module'
4
+ import { parseArgs } from 'node:util'
5
+
6
+ import { LIMITS } from './flow.mjs'
7
+ import { record } from './record.mjs'
8
+
9
+ const { version: VERSION } = createRequire(import.meta.url)('../package.json')
10
+ const SIGNAL_EXIT_CODES = { SIGINT: 130, SIGTERM: 143, SIGHUP: 129 }
11
+ const OPTION_KEYS = {
12
+ url: 'url',
13
+ flow: 'flow',
14
+ out: 'output',
15
+ preset: 'preset',
16
+ browser: 'browser',
17
+ 'storage-state': 'storageState',
18
+ 'viewport-width': 'viewportWidth',
19
+ 'viewport-height': 'viewportHeight',
20
+ scale: 'scale',
21
+ fps: 'fps',
22
+ 'post-roll': 'postRoll',
23
+ 'ready-timeout': 'readyTimeout',
24
+ 'flow-timeout': 'flowTimeout',
25
+ 'run-dir': 'runDirectory',
26
+ }
27
+ const NUMBER_KEYS = new Set(['viewportWidth', 'viewportHeight', 'scale', 'fps', 'postRoll', 'readyTimeout', 'flowTimeout'])
28
+ const HELP = `traceframe ${VERSION}
29
+
30
+ Record a scripted Chromium flow and render a smooth cursor and click pulse.
31
+
32
+ Usage:
33
+ traceframe record --url <url> --flow <flow.mjs> [--out <video.mp4>] [options]
34
+
35
+ Options:
36
+ --preset <standard|draft> standard: 1280x800 CSS viewport, 2560x1600 at 60 FPS (default)
37
+ draft: 1280x800 CSS viewport, 1280x800 at 30 FPS
38
+ --browser <path> Chromium executable (or set CHROME_PATH)
39
+ --storage-state <path> Playwright storage state JSON
40
+ --viewport-width <pixels> CSS viewport width
41
+ --viewport-height <pixels> CSS viewport height
42
+ --scale <1|2> Device scale factor; the video is the viewport times this
43
+ --fps <30|60> Encoded frame rate
44
+ --post-roll <seconds> Tail after the flow (default: 0.3)
45
+ --ready-timeout <seconds> Playwright default timeout during navigation and ready() (default: 30)
46
+ --flow-timeout <seconds> Time limit for the flow, including waits (default: 120)
47
+ --run-dir <path> Keep the raw capture and FFmpeg filter in this directory
48
+ -h, --help Show this help
49
+ -v, --version Show the version
50
+
51
+ The flow module must export a default function. It may also export
52
+ ready({ page, signal }) to establish state before capture and initialPointer
53
+ { x, y } to set where the cursor starts.
54
+ The flow helpers are move(), hover(), click(), scroll(), and wait(). A flow
55
+ may contain at most ${LIMITS.maxMoves} move events and ${LIMITS.maxClicks} click events.
56
+ Each duration, dwell, and wait is at most ${LIMITS.maxActionDuration} seconds.
57
+ `
58
+
59
+ const interruption = new AbortController()
60
+ let interruptedBy
61
+ for (const [signal, exitCode] of Object.entries(SIGNAL_EXIT_CODES)) {
62
+ process.on(signal, () => {
63
+ if (interruption.signal.aborted) process.exit(exitCode)
64
+ interruptedBy = signal
65
+ process.stderr.write(`traceframe: Stopping and cleaning up.${signal === 'SIGINT' ? ' Press Ctrl-C again to exit now.' : ''}\n`)
66
+ interruption.abort()
67
+ })
68
+ }
69
+
70
+ try {
71
+ await main()
72
+ } catch (error) {
73
+ if (interruptedBy) {
74
+ process.stderr.write('traceframe: Recording was interrupted.\n')
75
+ process.exitCode = SIGNAL_EXIT_CODES[interruptedBy]
76
+ } else {
77
+ process.stderr.write(`traceframe: ${error instanceof Error ? error.message : String(error)}\n`)
78
+ process.exitCode = 1
79
+ }
80
+ }
81
+
82
+ async function main() {
83
+ const parsed = parseCommand(process.argv.slice(2))
84
+ if (parsed.help) return process.stdout.write(HELP)
85
+ if (parsed.version) return process.stdout.write(`${VERSION}\n`)
86
+ const summary = await record({ ...parsed.options, signal: interruption.signal })
87
+ process.stdout.write(`${JSON.stringify(summary)}\n`)
88
+ }
89
+
90
+ function parseCommand(args) {
91
+ const { values, positionals } = parseArgs({
92
+ args,
93
+ allowPositionals: true,
94
+ options: {
95
+ help: { type: 'boolean', short: 'h' },
96
+ version: { type: 'boolean', short: 'v' },
97
+ ...Object.fromEntries(Object.keys(OPTION_KEYS).map((name) => [name, { type: 'string' }])),
98
+ },
99
+ })
100
+ if (values.help || args.length === 0) return { help: true }
101
+ if (values.version) return { version: true }
102
+ const [command, ...rest] = positionals
103
+ if (command !== 'record') throw new Error('Expected the record command. Use "traceframe record --help".')
104
+ if (rest.length) throw new Error(`Unexpected argument: ${rest[0]}`)
105
+
106
+ const options = {}
107
+ for (const [name, key] of Object.entries(OPTION_KEYS)) {
108
+ if (values[name] !== undefined) options[key] = NUMBER_KEYS.has(key) ? Number(values[name]) : values[name]
109
+ }
110
+ return { options }
111
+ }
package/src/config.mjs ADDED
@@ -0,0 +1,60 @@
1
+ import { resolve } from 'node:path'
2
+
3
+ import { TraceframeError } from './errors.mjs'
4
+
5
+ const PRESETS = {
6
+ standard: { viewport: { width: 1280, height: 800 }, scale: 2, fps: 60 },
7
+ draft: { viewport: { width: 1280, height: 800 }, scale: 1, fps: 30 },
8
+ }
9
+
10
+ export function createConfig(options = {}) {
11
+ const presetName = options.preset ?? 'standard'
12
+ check(Object.hasOwn(PRESETS, presetName), `Unknown preset: ${presetName}`)
13
+ const preset = PRESETS[presetName]
14
+
15
+ const viewport = {
16
+ width: options.viewportWidth ?? preset.viewport.width,
17
+ height: options.viewportHeight ?? preset.viewport.height,
18
+ }
19
+ check(Number.isInteger(viewport.width) && viewport.width > 0, 'Viewport width must be a positive integer.')
20
+ check(Number.isInteger(viewport.height) && viewport.height > 0, 'Viewport height must be a positive integer.')
21
+ const scale = options.scale ?? preset.scale
22
+ check(scale === 1 || scale === 2, 'Scale must be 1 or 2.')
23
+ const video = {
24
+ width: viewport.width * scale,
25
+ height: viewport.height * scale,
26
+ fps: options.fps ?? preset.fps,
27
+ }
28
+ check(video.fps === 30 || video.fps === 60, 'FPS must be 30 or 60.')
29
+ check(video.width % 2 === 0 && video.height % 2 === 0, 'At scale 1, the viewport width and height must be even.')
30
+
31
+ check(options.flow, 'A flow file is required (the flow option, or --flow).')
32
+ check(options.url, 'A URL is required (the url option, or --url).')
33
+ const output = resolve(options.output ?? 'traceframe.mp4')
34
+ check(output.toLowerCase().endsWith('.mp4'), 'Output path must use the .mp4 extension.')
35
+ const postRoll = options.postRoll ?? 0.3
36
+ check(Number.isFinite(postRoll) && postRoll >= 0, 'Post-roll must be 0 or more seconds.')
37
+ const readyTimeout = options.readyTimeout ?? 30
38
+ check(Number.isFinite(readyTimeout) && readyTimeout > 0, 'Ready timeout must be more than 0 seconds.')
39
+ const flowTimeout = options.flowTimeout ?? 120
40
+ check(Number.isFinite(flowTimeout) && flowTimeout > 0, 'Flow timeout must be more than 0 seconds.')
41
+
42
+ return {
43
+ url: options.url,
44
+ flow: resolve(options.flow),
45
+ output,
46
+ storageState: options.storageState,
47
+ browser: options.browser ?? process.env.CHROME_PATH,
48
+ viewport,
49
+ video,
50
+ scale,
51
+ postRoll,
52
+ readyTimeout,
53
+ flowTimeout,
54
+ runDirectory: options.runDirectory ? resolve(options.runDirectory) : undefined,
55
+ }
56
+ }
57
+
58
+ function check(valid, message) {
59
+ if (!valid) throw new TraceframeError(message, { code: 'INVALID_CONFIG' })
60
+ }
package/src/errors.mjs ADDED
@@ -0,0 +1,7 @@
1
+ export class TraceframeError extends Error {
2
+ constructor(message, { code, ...options } = {}) {
3
+ super(message, options)
4
+ this.name = 'TraceframeError'
5
+ this.code = code
6
+ }
7
+ }
package/src/ffmpeg.mjs ADDED
@@ -0,0 +1,54 @@
1
+ import { execFile } from 'node:child_process'
2
+ import { promisify } from 'node:util'
3
+
4
+ import { TraceframeError } from './errors.mjs'
5
+
6
+ const execFileAsync = promisify(execFile)
7
+
8
+ export async function probeFfmpeg() {
9
+ const version = await runCommand('ffmpeg', ['-version'])
10
+ const major = parseFfmpegMajor(version.stdout)
11
+ if (!(major >= 5)) {
12
+ throw new TraceframeError(`FFmpeg 5 or newer is required. Found: ${version.stdout.split('\n')[0].trim()}`, { code: 'FFMPEG_UNSUPPORTED' })
13
+ }
14
+ const codecs = await runCommand('ffmpeg', ['-hide_banner', '-encoders'])
15
+ if (!codecs.stdout.includes('libx264')) {
16
+ throw new TraceframeError('System FFmpeg does not provide the libx264 encoder.', { code: 'FFMPEG_UNSUPPORTED' })
17
+ }
18
+ await runCommand('ffprobe', ['-version'])
19
+ return { filterComplexFileOption: filterComplexFileOption(major) }
20
+ }
21
+
22
+ export async function probeDuration(path, signal) {
23
+ const { stdout } = await runCommand('ffprobe', ['-v', 'error', '-show_entries', 'format=duration', '-of', 'csv=p=0', path], { signal })
24
+ return Number(stdout)
25
+ }
26
+
27
+ export async function runCommand(command, args, options = {}) {
28
+ const run = execFileAsync(command, args, options)
29
+ const closed = new Promise((resolve) => run.child.once('close', resolve))
30
+ try {
31
+ return await run
32
+ } catch (error) {
33
+ // An abort rejects at once, but the process may take a moment to exit. Wait so cleanup does not race it.
34
+ if (options.signal?.aborted) await closed
35
+ if (error.code === 'ENOENT') {
36
+ throw new TraceframeError(`${command} was not found. Install FFmpeg 5 or newer with libx264, and make sure ffmpeg and ffprobe are on your PATH.`, { code: 'FFMPEG_MISSING' })
37
+ }
38
+ const rawDetail = error.stderr?.trim() || error.stdout?.trim() || error.message
39
+ const detail = rawDetail.length > 2_000 ? `${rawDetail.slice(0, 1_900)}\n… output truncated` : rawDetail
40
+ throw new TraceframeError(`${command} failed: ${detail}`, { code: 'COMMAND_FAILED', cause: error })
41
+ }
42
+ }
43
+
44
+ export function parseFfmpegMajor(output) {
45
+ const release = output.match(/ffmpeg version [nN]?(\d+)\./)
46
+ if (release) return Number(release[1])
47
+ // Snapshot builds print no release number. libavutil 57 shipped with FFmpeg 5, 58 with 6, 59 with 7, and so on.
48
+ const libavutil = output.match(/libavutil\s+(\d+)\./)
49
+ return libavutil ? Number(libavutil[1]) - 52 : undefined
50
+ }
51
+
52
+ export function filterComplexFileOption(major) {
53
+ return major >= 7 ? '-/filter_complex' : '-filter_complex_script'
54
+ }
package/src/flow.mjs ADDED
@@ -0,0 +1,264 @@
1
+ import { setTimeout as delay } from 'node:timers/promises'
2
+ import { pathToFileURL } from 'node:url'
3
+
4
+ import { TraceframeError } from './errors.mjs'
5
+
6
+ export const LIMITS = {
7
+ maxMoves: 100,
8
+ maxClicks: 100,
9
+ maxActionDuration: 30,
10
+ }
11
+ const MOVE_SECONDS = 0.5
12
+ const DWELL_SECONDS = 0.25
13
+ // People hold a mouse button for about 100 ms, long enough for press styles such as :active to show.
14
+ const PRESS_HOLD_MS = 100
15
+ const MOVE_SAMPLE_MS = 1000 / 30
16
+
17
+ export async function loadFlow(path, viewport) {
18
+ let module
19
+ try {
20
+ module = await import(pathToFileURL(path).href)
21
+ } catch (error) {
22
+ throw new TraceframeError(`Could not load flow: ${path}: ${error instanceof Error ? error.message : error}`, { code: 'INVALID_FLOW', cause: error })
23
+ }
24
+ if (typeof module.default !== 'function') throw new TraceframeError('Flow must export a default function.', { code: 'INVALID_FLOW' })
25
+ const { x, y } = module.initialPointer ?? { x: Math.round(viewport.width / 4), y: Math.max(0, viewport.height - 80) }
26
+ if (!insideViewport({ x, y }, viewport)) {
27
+ throw new TraceframeError('Flow initialPointer must be a point inside the viewport.', { code: 'INVALID_FLOW' })
28
+ }
29
+ return { run: module.default, ready: module.ready, initialPointer: { x, y } }
30
+ }
31
+
32
+ // Events are timed in seconds from `clock`, the epoch milliseconds of the first video frame.
33
+ export function createFlowApi({ page, trace, clock, viewport, signal }) {
34
+ let pointer = trace.initialPointer
35
+ const sleep = (milliseconds) => delay(milliseconds, undefined, { signal })
36
+
37
+ const movePointer = async (point, duration) => {
38
+ assertBelowLimit(trace.moves, LIMITS.maxMoves, 'move')
39
+ const from = pointer
40
+ const to = { x: point.x, y: point.y }
41
+ trace.moves.push({ at: secondsSince(clock), duration, from, to })
42
+
43
+ // Each sample follows the clock, so a late sample skips ahead instead of slowing the move down.
44
+ const total = duration * 1000
45
+ const startedAt = performance.now()
46
+ let elapsed = 0
47
+ while (elapsed < total) {
48
+ await sleep(Math.min(MOVE_SAMPLE_MS, total - elapsed))
49
+ elapsed = Math.min(total, performance.now() - startedAt)
50
+ const eased = easeOutCubic(elapsed / total)
51
+ await page.mouse.move(from.x + (to.x - from.x) * eased, from.y + (to.y - from.y) * eased)
52
+ }
53
+ pointer = to
54
+ }
55
+
56
+ const moveTo = async (locator, duration, name) => {
57
+ const seconds = durationSeconds(duration ?? MOVE_SECONDS, `${name} duration`)
58
+ const target = await findTarget(locator)
59
+ // A person does not chase a target that shifts under a pointer already on it.
60
+ if (pointInsideBox(pointer, target.box)) return pointer
61
+ let point = await aimPoint(target)
62
+ if (!insideViewport(point, viewport)) {
63
+ const aim = await reveal(target.element, locator)
64
+ if (!aim.visible) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
65
+ point = aim.point
66
+ }
67
+ await movePointer(point, seconds)
68
+ const after = await locator.boundingBox()
69
+ if (!after || !pointInsideBox(point, after)) {
70
+ throw new TraceframeError(`${locator} moved while the pointer was arriving.`, { code: 'FLOW_TARGET_MOVED' })
71
+ }
72
+ return point
73
+ }
74
+
75
+ return {
76
+ async hover(locator, options = {}) {
77
+ await moveTo(locator, options.duration, 'hover')
78
+ },
79
+ async click(locator, options = {}) {
80
+ assertBelowLimit(trace.clicks, LIMITS.maxClicks, 'click')
81
+ const dwell = durationSeconds(options.dwell ?? DWELL_SECONDS, 'click dwell', { allowZero: true })
82
+ const point = await moveTo(locator, options.duration, 'click')
83
+ await sleep(dwell * 1000)
84
+ // Hold on to this element: a control may rename or hide itself once it is pressed.
85
+ const element = await locator.elementHandle()
86
+ await element.waitForElementState('enabled').catch((error) => {
87
+ throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} stayed disabled.`, { code: 'FLOW_TARGET_DISABLED', cause: error }) : error
88
+ })
89
+ await assertReceivesPointer(element, locator, point)
90
+ const at = secondsSince(clock)
91
+ await page.mouse.down()
92
+ try {
93
+ await sleep(PRESS_HOLD_MS)
94
+ // A press style such as :active { scale: 0.95 } can shrink the target out from under a pointer near its edge.
95
+ // A target that disappears when pressed, such as an option that closes its list, has already taken the press.
96
+ const pressed = await element.boundingBox()
97
+ if (pressed && !pointInsideBox(point, pressed)) {
98
+ throw new TraceframeError(`${locator} moved away from the pointer when pressed. Click nearer its center.`, { code: 'FLOW_TARGET_MOVED' })
99
+ }
100
+ } finally {
101
+ await page.mouse.up()
102
+ }
103
+ trace.clicks.push({ at, point })
104
+ },
105
+ async move(point, options = {}) {
106
+ if (!insideViewport(point, viewport)) {
107
+ throw new TraceframeError('move requires an { x, y } point inside the viewport.', { code: 'INVALID_FLOW' })
108
+ }
109
+ await movePointer(point, durationSeconds(options.duration ?? MOVE_SECONDS, 'move duration'))
110
+ },
111
+ async scroll(locator) {
112
+ const { shown } = await reveal((await findTarget(locator)).element, locator)
113
+ if (!shown) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
114
+ },
115
+ async wait(seconds) {
116
+ await sleep(durationSeconds(seconds, 'wait', { allowZero: true }) * 1000)
117
+ },
118
+ }
119
+ }
120
+
121
+ function insideViewport(point, viewport) {
122
+ return point?.x >= 0 && point?.y >= 0 && point.x <= viewport.width && point.y <= viewport.height
123
+ }
124
+
125
+ function secondsSince(clock) {
126
+ return (Date.now() - clock) / 1000
127
+ }
128
+
129
+ // render.mjs draws the cursor along the same curve.
130
+ function easeOutCubic(progress) {
131
+ return 1 - (1 - progress) ** 3
132
+ }
133
+
134
+ function durationSeconds(value, name, { allowZero = false } = {}) {
135
+ if (!Number.isFinite(value) || (allowZero ? value < 0 : value <= 0) || value > LIMITS.maxActionDuration) {
136
+ const range = allowZero ? 'between 0 and' : 'more than 0 and at most'
137
+ throw new TraceframeError(`The ${name} must be ${range} ${LIMITS.maxActionDuration} seconds.`, { code: 'INVALID_FLOW' })
138
+ }
139
+ return value
140
+ }
141
+
142
+ function assertBelowLimit(events, limit, type) {
143
+ if (events.length >= limit) {
144
+ throw new TraceframeError(`Flow has more than ${limit} ${type} events.`, { code: 'FLOW_LIMIT' })
145
+ }
146
+ }
147
+
148
+ async function findTarget(locator) {
149
+ await locator.waitFor()
150
+ const element = await locator.elementHandle()
151
+ const box = await element.boundingBox()
152
+ if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
153
+ return { element, box }
154
+ }
155
+
156
+ function pointInsideBox(point, box) {
157
+ return point.x >= box.x && point.x <= box.x + box.width && point.y >= box.y && point.y <= box.y + box.height
158
+ }
159
+
160
+ // Aim at the middle of the first line box, as Playwright does, so a link that wraps onto two lines is not missed.
161
+ async function aimPoint({ element, box }) {
162
+ const offset = await element.evaluate((target) => {
163
+ const bounds = target.getBoundingClientRect()
164
+ const first = target.getClientRects()[0] ?? bounds
165
+ // As fractions of the element's size, because a scaled iframe makes the page box a different size than the element's own.
166
+ return { x: (first.left + first.width / 2 - bounds.left) / bounds.width, y: (first.top + first.height / 2 - bounds.top) / bounds.height }
167
+ })
168
+ return { x: box.x + offset.x * box.width, y: box.y + offset.y * box.height }
169
+ }
170
+
171
+ async function reveal(element, locator) {
172
+ // Content that loads above the target during the scroll can push it out again.
173
+ for (let attempt = 0; attempt < 2; attempt++) {
174
+ const target = await scrollIntoView(element, locator)
175
+ const point = await aimPoint(target)
176
+ const visibility = await target.element.evaluate((node, offset) => new Promise((resolve) => {
177
+ new IntersectionObserver(([entry], observer) => {
178
+ observer.disconnect()
179
+ const bounds = node.getBoundingClientRect()
180
+ const x = bounds.left + offset.x * bounds.width
181
+ const y = bounds.top + offset.y * bounds.height
182
+ const rect = entry.intersectionRect
183
+ const shown = entry.intersectionRatio > 0
184
+ resolve({ shown, visible: shown && x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom })
185
+ }).observe(node)
186
+ }), relativePoint(point, target.box))
187
+ if (visibility.visible || attempt === 1) return { point, ...visibility }
188
+ element = target.element
189
+ }
190
+ }
191
+
192
+ // Awaiting the scroll waits for a scrolling panel. Chromium resolves a page scroll before it stops, and a sticky target
193
+ // or iframe can stay put while its page scrolls, so also wait for every enclosing page to stop.
194
+ async function scrollIntoView(element, locator) {
195
+ await element.evaluate((target) => target.scrollIntoView({ behavior: 'smooth', block: 'center' }))
196
+ try {
197
+ await element.waitForElementState('stable')
198
+ let frame = await element.ownerFrame()
199
+ while (frame) {
200
+ await frame.evaluate(() => new Promise((resolve) => {
201
+ let last
202
+ const check = () => {
203
+ const now = `${scrollX},${scrollY}`
204
+ if (now === last) resolve()
205
+ else requestAnimationFrame(check)
206
+ last = now
207
+ }
208
+ requestAnimationFrame(check)
209
+ }))
210
+ if (!frame.parentFrame()) break
211
+ await (await frame.frameElement()).waitForElementState('stable')
212
+ frame = frame.parentFrame()
213
+ }
214
+ // The page may replace the target while it scrolls, or remove or hide it, as a menu that closes on scroll does.
215
+ const target = await findTarget(locator).catch((error) => {
216
+ throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} disappeared when it was scrolled into view.`, { code: 'FLOW_TARGET_MISSING', cause: error }) : error
217
+ })
218
+ await target.element.waitForElementState('stable')
219
+ const box = await target.element.boundingBox()
220
+ if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
221
+ return { element: target.element, box }
222
+ } catch (error) {
223
+ throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} kept moving after it was scrolled into view.`, { code: 'FLOW_TARGET_MOVED', cause: error }) : error
224
+ }
225
+ }
226
+
227
+ async function assertReceivesPointer(element, locator, point) {
228
+ const box = await element.boundingBox()
229
+ if (!box || !pointInsideBox(point, box)) {
230
+ throw new TraceframeError(`${locator} moved before the click.`, { code: 'FLOW_TARGET_MOVED' })
231
+ }
232
+ const receivesPointer = await element.evaluate((target, { x, y }) => {
233
+ const rect = target.getBoundingClientRect()
234
+ let current = target.getRootNode().elementFromPoint(rect.left + x * rect.width, rect.top + y * rect.height)
235
+ while (current) {
236
+ if (current === target || target.contains(current)) return true
237
+ current = current.assignedSlot ?? current.parentElement ?? current.getRootNode().host ?? null
238
+ }
239
+ return false
240
+ }, relativePoint(point, box))
241
+ if (!receivesPointer || !(await framesReceivePointer(element, point))) {
242
+ throw new TraceframeError(`${locator} is covered at the click point. If a dialog or toast covers it, wait until it closes; if a panel clips it or a sticky header or bar covers it, call scroll() on the target first.`, { code: 'FLOW_TARGET_OBSCURED' })
243
+ }
244
+ }
245
+
246
+ async function framesReceivePointer(element, point) {
247
+ let frame = await element.ownerFrame()
248
+ while (frame?.parentFrame()) {
249
+ const frameElement = await frame.frameElement()
250
+ const box = await frameElement.boundingBox()
251
+ if (!box) return false
252
+ const hit = await frameElement.evaluate((target, { x, y }) => {
253
+ const rect = target.getBoundingClientRect()
254
+ return target.getRootNode().elementFromPoint(rect.left + x * rect.width, rect.top + y * rect.height) === target
255
+ }, relativePoint(point, box))
256
+ if (!hit) return false
257
+ frame = frame.parentFrame()
258
+ }
259
+ return true
260
+ }
261
+
262
+ function relativePoint(point, box) {
263
+ return { x: (point.x - box.x) / box.width, y: (point.y - box.y) / box.height }
264
+ }
package/src/index.mjs ADDED
@@ -0,0 +1,2 @@
1
+ export { record } from './record.mjs'
2
+ export { TraceframeError } from './errors.mjs'
package/src/record.mjs ADDED
@@ -0,0 +1,39 @@
1
+ import { mkdir, mkdtemp, rm } from 'node:fs/promises'
2
+ import { tmpdir } from 'node:os'
3
+ import { dirname, join } from 'node:path'
4
+
5
+ import { createConfig } from './config.mjs'
6
+ import { captureFlow } from './capture.mjs'
7
+ import { probeFfmpeg } from './ffmpeg.mjs'
8
+ import { loadFlow } from './flow.mjs'
9
+ import { renderVideo } from './render.mjs'
10
+
11
+ export async function record(options = {}) {
12
+ const { signal } = options
13
+ signal?.throwIfAborted()
14
+ const config = createConfig(options)
15
+ const ffmpeg = await probeFfmpeg()
16
+ const flow = await loadFlow(config.flow, config.viewport)
17
+ await mkdir(dirname(config.output), { recursive: true })
18
+ const directory = config.runDirectory ?? await mkdtemp(join(tmpdir(), 'traceframe-'))
19
+ await mkdir(directory, { recursive: true })
20
+
21
+ try {
22
+ const { rawPath, trace } = await captureFlow({ config, flow, directory, signal })
23
+ const duration = await renderVideo({ rawPath, trace, directory, config, ffmpeg, signal })
24
+ return {
25
+ output: config.output,
26
+ width: config.video.width,
27
+ height: config.video.height,
28
+ fps: config.video.fps,
29
+ duration,
30
+ moveCount: trace.moves.length,
31
+ clickCount: trace.clicks.length,
32
+ runDirectory: config.runDirectory,
33
+ }
34
+ } catch (error) {
35
+ throw signal?.aborted ? signal.reason : error
36
+ } finally {
37
+ if (!config.runDirectory) await rm(directory, { recursive: true, force: true })
38
+ }
39
+ }
package/src/render.mjs ADDED
@@ -0,0 +1,93 @@
1
+ import { rename, rm, writeFile } from 'node:fs/promises'
2
+ import { dirname, join } from 'node:path'
3
+ import { randomUUID } from 'node:crypto'
4
+
5
+ import { probeDuration, runCommand } from './ffmpeg.mjs'
6
+ import { TraceframeError } from './errors.mjs'
7
+
8
+ const ASSETS = join(import.meta.dirname, 'assets')
9
+ // The arrow tip in cursor.svg.
10
+ const CURSOR_HOTSPOT = { x: 6.6, y: 1.6 }
11
+ const CLICK = { duration: 0.24, scaleStart: 0.7, scaleEnd: 1.25 }
12
+ // eof_action=pass lets the main video drive each overlay instead of waiting on the looping image inputs; without it, rendering is many times slower.
13
+ // format=yuv444 places overlays on whole pixels. The default yuv420 snaps them to even pixels, and RGB costs more for the same result.
14
+ const OVERLAY = 'eof_action=pass:format=yuv444'
15
+
16
+ export async function renderVideo({ rawPath, trace, directory, config, ffmpeg, signal }) {
17
+ const { fps } = config.video
18
+ const outputDuration = Math.max(trace.end + config.postRoll, ...trace.clicks.map((click) => click.at + CLICK.duration))
19
+ const filterPath = join(directory, 'render.filter')
20
+ await writeFile(filterPath, buildFilterScript({ trace, scale: config.scale }))
21
+ const outputTemp = join(dirname(config.output), `.traceframe-${randomUUID()}.tmp.mp4`)
22
+ const args = [
23
+ '-y', '-hide_banner', '-loglevel', 'error',
24
+ '-i', rawPath,
25
+ '-loop', '1', '-framerate', String(fps), '-i', join(ASSETS, `cursor-${config.scale}x.png`),
26
+ ]
27
+ if (trace.clicks.length) args.push('-loop', '1', '-framerate', String(fps), '-t', String(CLICK.duration + 1 / fps), '-i', join(ASSETS, `click-pulse-${config.scale}x.png`))
28
+ args.push(
29
+ ffmpeg.filterComplexFileOption, filterPath,
30
+ '-map', '[final]',
31
+ '-t', String(number(outputDuration)),
32
+ '-r', String(fps),
33
+ '-c:v', 'libx264', '-preset', 'fast', '-crf', '18',
34
+ '-colorspace', 'bt709', '-color_primaries', 'bt709', '-color_trc', 'bt709', '-color_range', 'tv',
35
+ '-g', String(fps * 2),
36
+ '-movflags', '+faststart', outputTemp,
37
+ )
38
+ try {
39
+ await runCommand('ffmpeg', args, { signal })
40
+ const duration = await probeDuration(outputTemp, signal)
41
+ if (!(Math.abs(duration - outputDuration) <= 1.5 / fps)) {
42
+ throw new TraceframeError(`The rendered video is ${duration.toFixed(2)} seconds long, but the recording is ${outputDuration.toFixed(2)} seconds.`, { code: 'OUTPUT_MISMATCH' })
43
+ }
44
+ signal?.throwIfAborted()
45
+ await rename(outputTemp, config.output)
46
+ return duration
47
+ } catch (error) {
48
+ await rm(outputTemp, { force: true })
49
+ throw error
50
+ }
51
+ }
52
+
53
+ // Event times are seconds from the first frame of the raw video.
54
+ export function buildFilterScript({ trace, scale }) {
55
+ const { clicks, moves } = trace
56
+ // Playwright ends the raw video by the time it received the last frame, so a page that stays still can end it a little before the post-roll. The page did not change, so repeat its last frame.
57
+ const filters = [`[0:v]setpts=PTS-STARTPTS,tpad=stop_mode=clone:stop_duration=1[video0]`]
58
+ if (clicks.length) filters.push(`[2:v]format=rgba,split=${clicks.length}${clicks.map((_, index) => `[pulse${index}]`).join('')}`)
59
+ clicks.forEach((click, index) => filters.push(...clickFilter(click, index, scale)))
60
+
61
+ const cursorLabels = moves.map((_, index) => `[cursor${index}]`).join('')
62
+ filters.push(`[1:v]format=rgba,split=${moves.length + 1}${cursorLabels}[cursor${moves.length}]`)
63
+ const coordinate = (value, hotspot) => number(value * scale - hotspot * scale)
64
+ const initialEnable = moves.length ? `:enable='lt(t,${number(moves[0].at)})'` : ''
65
+ filters.push(`[video${clicks.length}][cursor0]overlay=x='${coordinate(trace.initialPointer.x, CURSOR_HOTSPOT.x)}':y='${coordinate(trace.initialPointer.y, CURSOR_HOTSPOT.y)}':${OVERLAY}${initialEnable}[cursorVideo0]`)
66
+ moves.forEach((move, index) => {
67
+ const start = number(move.at)
68
+ const end = index + 1 < moves.length ? number(moves[index + 1].at) : undefined
69
+ // Each window excludes its end, so the frame where one move hands over to the next does not draw the cursor twice.
70
+ const enable = end === undefined ? `gte(t,${start})` : `gte(t,${start})*lt(t,${end})`
71
+ // The same curve as easeOutCubic in flow.mjs, so the drawn cursor follows the real pointer.
72
+ const eased = `(1-pow(1-clip((t-${start})/${number(move.duration)},0,1),3))`
73
+ const x = `${coordinate(move.from.x, CURSOR_HOTSPOT.x)}+${number((move.to.x - move.from.x) * scale)}*${eased}`
74
+ const y = `${coordinate(move.from.y, CURSOR_HOTSPOT.y)}+${number((move.to.y - move.from.y) * scale)}*${eased}`
75
+ filters.push(`[cursorVideo${index}][cursor${index + 1}]overlay=x='${x}':y='${y}':${OVERLAY}:enable='${enable}'[cursorVideo${index + 1}]`)
76
+ })
77
+ filters.push(`[cursorVideo${moves.length}]colorspace=ispace=bt470bg:iprimaries=bt709:itrc=bt709:all=bt709:format=yuv420p[final]`)
78
+ return filters.join(';')
79
+ }
80
+
81
+ function clickFilter(click, index, scale) {
82
+ const start = number(click.at)
83
+ const end = number(start + CLICK.duration)
84
+ const growth = `(${CLICK.scaleStart}+${CLICK.scaleEnd - CLICK.scaleStart}*clip((t-${start})/${CLICK.duration},0,1))`
85
+ return [
86
+ `[pulse${index}]setpts=PTS-STARTPTS+${start}/TB,scale=w='iw*${growth}':h='ih*${growth}':eval=frame,fade=t=out:st=${start}:d=${CLICK.duration}:alpha=1[click${index}]`,
87
+ `[video${index}][click${index}]overlay=x='${number(click.point.x * scale)}-w/2':y='${number(click.point.y * scale)}-h/2':${OVERLAY}:enable='between(t,${start},${end})'[video${index + 1}]`,
88
+ ]
89
+ }
90
+
91
+ function number(value) {
92
+ return Number(value.toFixed(6))
93
+ }