@milkstraw/traceframe 0.1.0 → 0.2.0

package/README.md

@@ -4,61 +4,63 @@ Traceframe records a scripted Chromium flow and renders an MP4 with a smooth cur
 
 https://github.com/user-attachments/assets/38e02cd5-a9ba-4f3c-938c-159955353141
 
-## Install
+## Prerequisites
 
-```sh
-npm install @milkstraw/traceframe
-```
-
-Traceframe requires Node.js 22+, FFmpeg 5+ with `libx264` and `ffprobe` (for example, `brew install ffmpeg` or `sudo apt install ffmpeg`), and Playwright's Chromium:
+Traceframe requires Node.js 22+ and system FFmpeg 5+ with `libx264` and `ffprobe` to render the MP4 (for example, `brew install ffmpeg` or `sudo apt install ffmpeg`). Playwright uses its own Chromium and FFmpeg builds to record the page. Install them with the exact Playwright version Traceframe uses; other cached versions are not used:
 
 ```sh
-npx playwright@1.64.1 install --only-shell chromium
+npx playwright@1.64.1 install --only-shell chromium ffmpeg
 ```
 
-To use a system Chrome instead, pass `--browser` or set `CHROME_PATH`. Playwright still needs its own FFmpeg build to record the page, so install it with `npx playwright@1.64.1 install ffmpeg`.
+To use system Chrome 150 or later, pass `--browser` or set `CHROME_PATH`, and install Playwright's recording FFmpeg with `npx playwright@1.64.1 install ffmpeg`. Older Chrome versions can fail to scroll targets inside panels.
+
+In a sandbox, such as a coding agent's, allow Playwright's browser download and let Chromium start.
 
 To record pages behind a login, pass a Playwright storage-state file with `--storage-state`. Keep it in `.auth/` or another ignored location.
 
 ## Record a video
 
-The included fixture is available in a repository clone. It is not included in the installed package.
-
-```sh
-git clone https://github.com/milkstrawai/traceframe.git
-cd traceframe
-npm ci
-npx traceframe record \
-  --url "file://$PWD/examples/fixture.html" \
-  --flow ./examples/fixture-flow.mjs \
-  --out ./sample.mp4 \
-  --preset draft
-```
-
-For an installed package, pass your application URL and your own flow file:
+Run Traceframe with `npx` without adding it to your project's dependencies. Pass your application URL and your own [flow file](#write-a-flow):
 
 ```sh
-npx traceframe record \
+npx @milkstraw/traceframe record \
   --url http://localhost:3000 \
   --flow ./flow.mjs \
   --out ./recording.mp4
 ```
 
-The `draft` preset uses a 1280×800 CSS viewport and produces 1280×800 at 30 FPS. The `standard` preset uses the same CSS viewport and encodes it at 2560×1600 at 60 FPS. The cursor and click pulses are always drawn at the full frame rate, but Chromium may capture page changes, such as scrolling and animations, at a lower rate at 2560×1600, depending on the machine. When that happens, the video repeats the last page frame until the next one arrives. The video is always the viewport size times `--scale`, so `--preset draft --scale 2` gives 2560×1600 at 30 FPS. The command prints a JSON summary when it finishes.
+For regular use, you can install the command globally:
 
-Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and the FFmpeg filter file.
+```sh
+npm install -g @milkstraw/traceframe
+traceframe record --url http://localhost:3000 --flow ./flow.mjs --out ./recording.mp4
+```
 
-## Have an agent create the flow
+To check installation before touching your application, save this as `smoke.mjs`:
 
-For a real application, ask your coding agent to inspect the running UI and create the flow from the user journey. This keeps selectors tied to the product and avoids manually guessing coordinates. For example:
+```js
+export default async function ({ wait }) {
+  await wait(0.2)
+}
+```
 
-```text
-Create a Traceframe flow for this user journey: open Settings, change the theme, and save it.
-Inspect the running application, use stable role or test-id locators, add a ready hook,
-and use Traceframe's click, hover, move, and wait helpers for visible interactions.
-Write the result to flow.mjs and explain how to run it.
+```sh
+npx @milkstraw/traceframe record \
+  --url 'data:text/html,<h1>Traceframe smoke</h1>' \
+  --flow ./smoke.mjs --out ./smoke.mp4 \
+  --preset draft
 ```
 
+The [recording guide](docs/recording-flows.md) includes a runnable example and recipes for form input, asynchronous results, browser settings, and editing or sharing a recording.
+
+Both presets use a 1280×800 CSS viewport. `draft` produces 1280×800 at 30 FPS; `standard` (the default) produces 2560×1600 at 60 FPS. The output dimensions are the viewport times `--scale`, so `--preset draft --scale 2` gives 2560×1600 at 30 FPS. The command prints a JSON summary when it finishes.
+
+The cursor and click pulses are always drawn at the full frame rate. Chromium may capture page changes, such as scrolling and animations, at a lower rate at 2560×1600, depending on the machine. The video repeats the last page frame until the next one arrives.
+
+Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and FFmpeg filter file, including a partial capture if the flow fails during recording. Keep that directory out of Git, because the raw capture shows everything the page showed.
+
+## Write a flow
+
 The flow module exports a default function. Timing values are seconds:
 
 ```js
@@ -72,14 +74,64 @@ export default async function ({ page, click, wait }) {
 }
 ```
 
-`ready({ page, signal })` runs before recording starts. `--ready-timeout` (30 seconds by default) is Playwright's timeout for the page load and for each Playwright call in `ready`; it does not limit other work in the hook. A flow can also export `initialPointer = { x, y }` to choose where the cursor starts. By default it starts near the lower left.
+The optional `ready({ page, signal })` hook runs after initial navigation and before recording starts. Export `initialPointer = { x, y }` to choose where the cursor starts; by default it starts near the lower left.
 
-The default function receives `page`, `signal`, and four helpers: `move({ x, y }, { duration })`, `hover(locator, { duration })`, `click(locator, { duration, dwell })`, and `wait(seconds)`. A move takes 0.5 seconds unless you pass `duration`, and a click rests on the target for 0.25 seconds (`dwell`) before it presses. If the target is disabled, the click waits for it to become enabled. A flow may contain at most 100 moves, 100 clicks, and 120 seconds of work, and each `duration`, `dwell`, and `wait` is at most 30 seconds. Error messages name the locator; give it a clearer name with Playwright's `locator.describe('Save settings')`.
+### Helpers
 
-`click` and `hover` use real Playwright mouse events and add their movement to the overlay timeline. If the pointer is already inside the target, it stays where it is, so a hover effect that moves or resizes the target does not pull the cursor along. A click holds the mouse button for about 100 ms, as a person would, so press styles such as `:active` show in the video. Headless Chromium draws pages without a GPU, so on heavy pages, such as ones with blurred backgrounds, hover effects can appear a little after the cursor passes. A longer `duration`, or a `move` around hover-reactive elements, keeps this from showing. Typing, scrolling, dragging, multiple tabs, and direct `page.mouse` or `locator.click()` calls are not currently represented in that timeline. A flow receives an `AbortSignal`; helper calls stop promptly when the duration limit is reached.
+The default function receives a Playwright `page`, an `AbortSignal` named `signal`, and five helpers:
+
+| Helper | Use |
+| --- | --- |
+| `move({ x, y }, { duration })` | Move the pointer to viewport coordinates. |
+| `hover(locator, { duration })` | Move the pointer to a locator. |
+| `click(locator, { duration, dwell })` | Move, pause, and click a locator. |
+| `scroll(locator)` | Center a target without moving the pointer. |
+| `wait(seconds)` | Pause for reading time. |
+
+A move takes 0.5 seconds by default. A click pauses on the target for 0.25 seconds (`dwell`) before pressing, and waits for a disabled target to become enabled. `click` and `hover` leave the pointer in place if it is already inside the target's box.
+
+Error messages name the locator; give it a clearer name with Playwright's `locator.describe('Save settings')`.
+
+### Scrolling
+
+`click` and `hover` scroll automatically only when their aim point starts outside the viewport. They scroll smoothly through the page, scrollable containers, and iframes, and wait for scrolling to stop before moving the pointer.
+
+`scroll(target)` smoothly centers the target vertically, even if it is already visible, and waits for scrolling to stop without moving the pointer. Use it to show a result or before clicking a target clipped by a scrollable panel or covered by a sticky header. Wait for a dialog or toast to close instead.
+
+Centering can leave a target partly visible or covered, for example by a sticky bar taller than half the height of the viewport or its scrolling panel. It cannot reveal content hidden by `overflow: clip`. `click` fails with `FLOW_TARGET_OBSCURED` if the target is still covered when it presses.
+
+### Timing and limits
+
+`--ready-timeout` (default: 30 seconds) sets Playwright's default timeout during initial navigation and `ready`, for operations that accept a timeout. It does not set a deadline for the whole hook.
+
+The default function may run for 120 seconds, including waits for your backend. For longer flows, pass `--flow-timeout 300` (`flowTimeout: 300` in the API). This does not change Playwright's action timeouts. Helper calls stop promptly when the flow timeout is reached.
+
+A flow may contain at most 100 moves and 100 clicks, and each `duration`, `dwell`, and `wait` is at most 30 seconds.
+
+### What appears in the video
+
+Playwright typing and scrolling appear in the captured page. Direct `page.mouse`, `locator.click()`, and dragging calls bypass Traceframe's cursor trace and click pulses, so use its pointer helpers for visible interactions. Only the starting page is recorded; new tabs and popups do not appear in the video.
+
+### Have an agent create the flow
+
+You can ask a coding agent to inspect your running UI and write the flow from a journey:
+
+```text
+Create a Traceframe flow for this user journey: open Settings, change the theme, and save it.
+Follow the Traceframe recording guide: https://github.com/milkstrawai/traceframe/blob/main/docs/recording-flows.md
+Inspect the running application, use stable role or test-id locators, add a ready hook,
+and use Traceframe's click, hover, move, scroll, and wait helpers for visible interactions.
+Write the result to flow.mjs and explain how to run it.
+```
 
 ## Programmatic usage
 
+To call Traceframe from JavaScript, install it in your project:
+
+```sh
+npm install @milkstraw/traceframe
+```
+
 ```js
 import { record } from '@milkstraw/traceframe'
 
@@ -97,13 +149,13 @@ Pass `signal` to stop a recording early. Traceframe closes the browser, stops FF
 
 ## Sample output
 
-The committed sample is generated from the included fixture flow with the `standard` preset (2560×1600 at 60 FPS) and is about 12 seconds long:
+The committed sample is generated from the [repository's fixture flow](examples/fixture-flow.mjs) with the `standard` preset (2560×1600 at 60 FPS) and is about 12 seconds long:
 
 [Download the sample video](https://github.com/milkstrawai/traceframe/blob/main/examples/sample-output.mp4)
 
-The cursor and click-pulse artwork and the example page's icons were made for Traceframe and are distributed under this repository's MIT license.
+The cursor and click-pulse artwork and the fixture page's icons were made for Traceframe and are distributed under this repository's MIT license.
 
-The example page uses the [Inter](https://github.com/rsms/inter) font, included under the SIL Open Font License in `examples/fonts/OFL.txt`.
+The fixture page uses the [Inter](https://github.com/rsms/inter) font, included under the SIL Open Font License in `examples/fonts/OFL.txt`.
 
 ## Development
 
@@ -115,6 +167,8 @@ TRACEFRAME_PRESET=standard npm run test:smoke
 npm run test:package
 ```
 
+To release, set the version in `package.json` and `package-lock.json`, push all release changes to `main`, and publish a GitHub Release with the tag `v<version>` on the final tested commit containing those changes. The Release workflow reruns CI and stages the package on npm. A maintainer downloads it with `npm stage download <stage-id>`, checks that its files match `npm pack` at the tag, and approves it on npmjs.com with 2FA.
+
 After editing an SVG in `src/assets`, run `npm run build:assets` to rebuild its 1× and 2× PNGs.
 
 To re-record the committed sample:

package/docs/recording-flows.md

@@ -0,0 +1,123 @@
+# Write a recording flow
+
+Use this guide yourself or give it to a coding agent with the journey you want to show. Traceframe provides pointer and scrolling helpers; the supplied Playwright page handles typing, file selection, and application checks.
+
+## Run the example
+
+The package includes [a complete flow](../examples/recipes-flow.mjs) and [its local page](../examples/recipes.html). It types English and Arabic, waits for two simulated replies, selects a fake file, and clicks a final action in a panel below the fold, which `click` scrolls into view first. After a global installation and the [prerequisites](../README.md#prerequisites), run:
+
+```sh
+TRACEFRAME_EXAMPLES="$(npm root -g)/@milkstraw/traceframe/examples"
+traceframe record \
+  --url "file://$TRACEFRAME_EXAMPLES/recipes.html" \
+  --flow "$TRACEFRAME_EXAMPLES/recipes-flow.mjs" \
+  --preset draft --out ./recipes.mp4
+```
+
+In a repository clone, run `npx traceframe record --url "file://$PWD/examples/recipes.html" --flow ./examples/recipes-flow.mjs --preset draft --out ./recipes.mp4` instead. These shell commands use macOS/Linux path syntax. The flow itself has no application dependencies, external services, or credentials.
+
+## Prepare the journey
+
+Inspect the running application at the recording viewport. Pick a short journey with a visible result and known demo data. Use role, label, or test-id locators; `locator.describe()` can make errors clearer.
+
+Put setup and page readiness in `ready({ page, signal })`. This runs after initial navigation and before recording. Use `--storage-state` for an existing login, and keep that file in an ignored directory. For headers, locale, or themes, see [optional startup settings](#optional-startup-settings).
+
+## Choose the right actions
+
+Use Traceframe's `click`, `hover`, and `move` for visible pointer actions. Direct `locator.click()` and `page.mouse` calls move the browser pointer without updating Traceframe's overlay or click pulses. Use Playwright for the other actions below; their page changes appear in the video.
+
+For a target clipped by a scrollable panel or covered by a sticky header, try `await scroll(target)` before the click. For a tooltip, a toast, or a closing dialog, wait until it is gone, or give the click a longer `dwell`. A covered click fails with an error that it "is covered at the click point" (`FLOW_TARGET_OBSCURED`); `hover` does not check for covers. See the [scrolling rules and limits](../README.md#scrolling).
+
+Both automatic scrolling and `scroll(target)` can move the outer page, which closes some lists. For those lists, use their own scrolling container through Playwright, for example `await list.evaluate((element) => element.scrollTo({ top: element.scrollHeight, behavior: 'smooth' }))` to show the final options, then click the option. Wait for any opening animation before scrolling.
+
+To show a result without a pointer action, use `await scroll(result)`. Centering a result taller than the view hides its start, so scroll to its heading or another element near its start instead.
+
+Focus a field visibly, then type with a delay in milliseconds:
+
+```js
+await click(message)
+await message.pressSequentially('A short question', { delay: 45 })
+```
+
+For an upload, hover the visible control, set the actual file input, and wait for the application's selected-file feedback. The visible label or drop zone and the input are often different elements:
+
+```js
+const dropZone = page.getByText('Choose a file', { exact: true })
+const fileInput = page.getByLabel('Choose a file')
+await hover(dropZone)
+await fileInput.setInputFiles('./sample.txt')
+await page.getByText('sample.txt').waitFor()
+await wait(0.6)
+```
+
+A relative path is resolved from the directory where you run Traceframe. File selection does not show an operating-system file picker. The complete example supplies an in-memory fake file so it can run on its own.
+
+## Wait for the current result
+
+Before a follow-up, wait for the application to show that the current step is complete. For a conversation that appends completed replies, the example waits for the next reply by index. For streamed answers, wait for that request's completion indicator instead; the first visible text or a previous reply is not completion.
+
+Set a Playwright timeout explicitly when an individual operation may exceed its usual 30 seconds, such as `await newReply.waitFor({ timeout: 60_000 })`. Use Traceframe's `wait(seconds)` for reading time. Each helper wait is limited to 30 seconds, and the whole flow has a 120-second timeout that `--flow-timeout` (`flowTimeout` in the API) raises.
+
+## Optional startup settings
+
+`page.emulateMedia({ colorScheme: 'dark' })` in `ready` updates CSS themes that follow the browser's color scheme. If the app reads the setting only at startup, set it before opening the app, as below.
+
+If the application needs request headers, a browser locale, or a color scheme from startup, record `--url about:blank` and open the application yourself in `ready`. Keep only the lines you need:
+
+```js
+export async function ready({ page }) {
+  await page.emulateMedia({ colorScheme: 'dark' })
+  await page.setExtraHTTPHeaders({ 'x-demo-access': 'demo' })
+  const cdp = await page.context().newCDPSession(page)
+  await cdp.send('Emulation.setLocaleOverride', { locale: 'ar-EG' })
+  await cdp.send('Emulation.setUserAgentOverride', { userAgent: '', acceptLanguage: 'ar-EG' })
+  await page.goto('http://localhost:3000')
+  await page.getByRole('heading', { name: 'Example workspace' }).waitFor()
+  await page.evaluate(() => document.fonts.ready)
+}
+```
+
+The two `Emulation` calls set the locale that `navigator.language`, `Intl`, and the `Accept-Language` header use; the empty `userAgent` keeps Chromium's own user agent and client hints. Traceframe waits for web fonts only on the `--url` page, so the last line waits for the application's fonts.
+
+## Record, inspect, and refine
+
+Use `--preset draft` while writing the flow and the default preset for the final video.
+
+While writing the flow, pass `--run-dir` with an ignored folder, such as `.traceframe/runs/demo`. A failed take then keeps its partial `capture.webm`, which has no cursor overlay and stops where the flow failed.
+
+A failed take may already have submitted real jobs, so reset demo data before recording again. Check the important transitions and the final result. You can also check the file for decode errors:
+
+```sh
+ffmpeg -v error -xerror -i ./recording.mp4 -f null -
+```
+
+## Edit the finished video
+
+Edit the finished MP4 with FFmpeg rather than changing the flow. Editing finished frames keeps the page and cursor together. Keep the original, and say when a clip's pacing was edited.
+
+To speed up an idle stretch, such as a progress bar, you need its start and end times. Read them in a video player, or log them from the flow; logged times run up to about 0.1 seconds earlier than in the video:
+
+```js
+export default async function ({ page, click, wait }) {
+  const started = Date.now()
+  const mark = (label) => console.error(`${label}: ${((Date.now() - started) / 1000).toFixed(2)} s`)
+  // ...
+  mark('progress started')
+}
+```
+
+Log to `console.error`, because Traceframe prints its JSON summary on standard output. This command plays seconds 2 to 9 at 4× speed. Set `fps` to the video's frame rate: 60 for the default preset, 30 for draft.
+
+```sh
+ffmpeg -i recording.mp4 -filter_complex "[0:v]trim=0:2,setpts=PTS-STARTPTS[a];[0:v]trim=2:9,setpts=(PTS-STARTPTS)/4[b];[0:v]trim=9,setpts=PTS-STARTPTS[c];[a][b][c]concat=n=3,fps=60[v]" -map "[v]" -c:v libx264 -crf 18 -movflags +faststart faster.mp4
+```
+
+For an animated preview where an MP4 does not play inline, make a WebP. Not every FFmpeg build includes the WebP encoder.
+
+```sh
+ffmpeg -i recording.mp4 -vf "fps=12,scale=1280:-1" -c:v libwebp_anim -quality 75 -loop 0 preview.webp
+```
+
+## Optional sharing
+
+With push access to a repository, current [GitHub CLI](https://docs.github.com/en/github-cli/github-cli/attaching-files-with-github-cli) can attach the MP4 to a comment so it plays there; check `gh pr comment --help` for your version. An MP4 hosted elsewhere shows as a link, not a player, so use the WebP preview there. Upload only when asked and after checking that the recording suits the destination; recording itself uploads nothing.

package/examples/recipes-flow.mjs

@@ -0,0 +1,27 @@
+export async function ready({ page }) {
+  await page.getByRole('heading', { name: 'Conversation demo' }).waitFor()
+}
+
+export default async function ({ page, click, hover, wait }) {
+  const message = page.getByRole('textbox', { name: 'Message' })
+  const replies = page.getByRole('list', { name: 'Replies' }).getByRole('listitem')
+  for (const [index, text] of ['Give me a short summary.', 'هل يمكنك التوضيح؟'].entries()) {
+    await click(message)
+    await message.pressSequentially(text, { delay: 45 })
+    await click(page.getByRole('button', { name: 'Send', exact: true }))
+    await replies.nth(index).waitFor({ timeout: 60_000 })
+    await wait(0.6)
+  }
+
+  const upload = page.getByText('Choose a sample file', { exact: true })
+  await hover(upload)
+  await page.getByLabel('Choose a sample file').setInputFiles({
+    name: 'sample.txt', mimeType: 'text/plain', buffer: Buffer.from('Example file for a local demo.\n'),
+  })
+  await page.getByText('Selected: sample.txt', { exact: true }).waitFor()
+  await wait(0.6)
+
+  await click(page.getByRole('button', { name: 'Finish demo', exact: true }))
+  await page.getByRole('button', { name: 'Demo complete', exact: true }).waitFor()
+  await wait(0.8)
+}

package/examples/recipes.html

@@ -0,0 +1,74 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <title>Traceframe recording recipes</title>
+    <style>
+      :root { color-scheme: light; font: 18px/1.5 system-ui, sans-serif; color: #172033; background: #f4f6fa; }
+      * { box-sizing: border-box; }
+      main { max-width: 900px; margin: 40px auto; padding: 32px; border: 1px solid #dce1eb; border-radius: 16px; background: #fff; }
+      h1 { margin: 0; font-size: 28px; }
+      h2 { margin: 20px 0 8px; font-size: 18px; }
+      p { margin: 8px 0 20px; }
+      form > label { display: block; margin-bottom: 8px; }
+      .compose { display: flex; gap: 12px; }
+      input, button { font: inherit; padding: 10px 16px; border: 1px solid #aeb7c8; border-radius: 8px; }
+      input { min-width: 0; flex: 1; }
+      button { color: #fff; background: #3454b5; cursor: pointer; }
+      button:hover { background: #284393; }
+      button:disabled { opacity: 0.6; }
+      #status { min-height: 27px; margin: 8px 0; }
+      #replies { min-height: 80px; padding-left: 28px; }
+      #replies li { margin: 8px 0; }
+      .upload { display: inline-block; padding: 12px 16px; border: 1px dashed #aeb7c8; border-radius: 8px; cursor: pointer; }
+      .upload:hover { background: #f4f6fa; }
+      .scroll-area { height: 160px; overflow: auto; border: 1px solid #dce1eb; border-radius: 8px; }
+      .scroll-content { height: 360px; display: flex; align-items: end; padding: 20px; }
+      .below-fold { margin-top: 480px; }
+    </style>
+  </head>
+  <body>
+    <main>
+      <h1>Conversation demo</h1>
+      <p>Replies are simulated locally. No messages or files leave this page.</p>
+      <form id="conversation">
+        <label for="message">Message</label>
+        <div class="compose"><input id="message" dir="auto" required autocomplete="off"><button id="send">Send</button></div>
+      </form>
+      <p id="status" role="status">Ready for your question.</p>
+      <ol id="replies" aria-label="Replies"></ol>
+      <h2>Attach a file</h2>
+      <label class="upload" for="file">Choose a sample file</label><input id="file" type="file" hidden>
+      <p id="selected-file">No file selected.</p>
+      <h2 class="below-fold">Scroll to the last action</h2>
+      <section class="scroll-area" aria-label="More actions" tabindex="0">
+        <div class="scroll-content"><button id="finish" type="button">Finish demo</button></div>
+      </section>
+    </main>
+    <script>
+      const message = document.querySelector('#message')
+      const send = document.querySelector('#send')
+      const status = document.querySelector('#status')
+      const replies = document.querySelector('#replies')
+      document.querySelector('#conversation').addEventListener('submit', async (event) => {
+        event.preventDefault()
+        const text = message.value
+        message.value = ''
+        send.disabled = true
+        status.textContent = 'Preparing a reply…'
+        await new Promise((resolve) => setTimeout(resolve, 1000))
+        const reply = document.createElement('li')
+        reply.dir = 'auto'
+        reply.textContent = `Reply ${replies.children.length + 1}: ${text}`
+        replies.append(reply)
+        status.textContent = 'Reply complete.'
+        send.disabled = false
+      })
+      document.querySelector('#file').addEventListener('change', (event) => {
+        document.querySelector('#selected-file').textContent = `Selected: ${event.target.files[0]?.name ?? 'none'}`
+      })
+      document.querySelector('#finish').addEventListener('click', (event) => { event.currentTarget.textContent = 'Demo complete' })
+    </script>
+  </body>
+</html>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@milkstraw/traceframe",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Record scripted browser flows and render smooth cursor overlays into video.",
   "keywords": [
     "playwright",
@@ -24,7 +24,10 @@
     ".": "./src/index.mjs"
   },
   "files": [
-    "src"
+    "src",
+    "docs/recording-flows.md",
+    "examples/recipes.html",
+    "examples/recipes-flow.mjs"
   ],
   "engines": {
     "node": ">=22"

package/src/capture.mjs

@@ -4,7 +4,7 @@ import { setTimeout as delay } from 'node:timers/promises'
 import { chromium } from 'playwright'
 
 import { TraceframeError } from './errors.mjs'
-import { createFlowApi, LIMITS } from './flow.mjs'
+import { createFlowApi } from './flow.mjs'
 
 const PLAYWRIGHT_DEFAULT_TIMEOUT = 30_000
 const FIRST_FRAME_TIMEOUT = 10_000
@@ -32,8 +32,8 @@ export async function captureFlow({ config, flow, directory, signal: callerSigna
     )
     await withDeadline(
       (flowSignal) => flow.run({ page, signal: flowSignal, ...createFlowApi({ page, trace, clock, viewport: config.viewport, signal: flowSignal }) }),
-      LIMITS.maxDuration * 1000,
-      new TraceframeError(`Flow exceeded the ${LIMITS.maxDuration}-second duration limit.`, { code: 'FLOW_TOO_LONG' }),
+      config.flowTimeout * 1000,
+      new TraceframeError(`Flow exceeded the ${config.flowTimeout}-second flow timeout. Raise it with the flowTimeout option, or --flow-timeout.`, { code: 'FLOW_TOO_LONG' }),
       signal,
     )
     trace.end = (Date.now() - clock) / 1000
@@ -62,7 +62,7 @@ async function launchBrowser(config, { handleSignals }) {
     if (!/Executable doesn't exist/i.test(error.message)) throw error
     const message = config.browser
       ? `Configured browser executable is unavailable: ${config.browser}`
-      : 'Playwright Chromium is not installed. Run `npx playwright@1.64.1 install --only-shell chromium` or pass --browser to a Chrome executable.'
+      : 'Playwright Chromium is not installed. Run `npx playwright@1.64.1 install --only-shell chromium`, or pass --browser to a Chrome executable and run `npx playwright@1.64.1 install ffmpeg`.'
     throw new TraceframeError(message, { code: 'PLAYWRIGHT_RUNTIME_MISSING', cause: error })
   }
 }

package/src/cli.mjs

@@ -21,9 +21,10 @@ const OPTION_KEYS = {
   fps: 'fps',
   'post-roll': 'postRoll',
   'ready-timeout': 'readyTimeout',
+  'flow-timeout': 'flowTimeout',
   'run-dir': 'runDirectory',
 }
-const NUMBER_KEYS = new Set(['viewportWidth', 'viewportHeight', 'scale', 'fps', 'postRoll', 'readyTimeout'])
+const NUMBER_KEYS = new Set(['viewportWidth', 'viewportHeight', 'scale', 'fps', 'postRoll', 'readyTimeout', 'flowTimeout'])
 const HELP = `traceframe ${VERSION}
 
 Record a scripted Chromium flow and render a smooth cursor and click pulse.
@@ -41,7 +42,8 @@ Options:
   --scale <1|2>                Device scale factor; the video is the viewport times this
   --fps <30|60>                Encoded frame rate
   --post-roll <seconds>        Tail after the flow (default: 0.3)
-  --ready-timeout <seconds>    Playwright timeout for the page load and calls in ready() (default: 30)
+  --ready-timeout <seconds>    Playwright default timeout during navigation and ready() (default: 30)
+  --flow-timeout <seconds>     Time limit for the flow, including waits (default: 120)
   --run-dir <path>             Keep the raw capture and FFmpeg filter in this directory
   -h, --help                   Show this help
   -v, --version                Show the version
@@ -49,8 +51,8 @@ Options:
 The flow module must export a default function. It may also export
 ready({ page, signal }) to establish state before capture and initialPointer
 { x, y } to set where the cursor starts.
-The flow helpers are move(), hover(), click(), and wait(). A flow may contain
-at most ${LIMITS.maxMoves} move events, ${LIMITS.maxClicks} click events, and ${LIMITS.maxDuration} seconds of work.
+The flow helpers are move(), hover(), click(), scroll(), and wait(). A flow
+may contain at most ${LIMITS.maxMoves} move events and ${LIMITS.maxClicks} click events.
 Each duration, dwell, and wait is at most ${LIMITS.maxActionDuration} seconds.
 `
 

package/src/config.mjs

@@ -36,6 +36,8 @@ export function createConfig(options = {}) {
   check(Number.isFinite(postRoll) && postRoll >= 0, 'Post-roll must be 0 or more seconds.')
   const readyTimeout = options.readyTimeout ?? 30
   check(Number.isFinite(readyTimeout) && readyTimeout > 0, 'Ready timeout must be more than 0 seconds.')
+  const flowTimeout = options.flowTimeout ?? 120
+  check(Number.isFinite(flowTimeout) && flowTimeout > 0, 'Flow timeout must be more than 0 seconds.')
 
   return {
     url: options.url,
@@ -48,6 +50,7 @@ export function createConfig(options = {}) {
     scale,
     postRoll,
     readyTimeout,
+    flowTimeout,
     runDirectory: options.runDirectory ? resolve(options.runDirectory) : undefined,
   }
 }

package/src/flow.mjs

@@ -4,7 +4,6 @@ import { pathToFileURL } from 'node:url'
 import { TraceframeError } from './errors.mjs'
 
 export const LIMITS = {
-  maxDuration: 120,
   maxMoves: 100,
   maxClicks: 100,
   maxActionDuration: 30,
@@ -56,14 +55,14 @@ export function createFlowApi({ page, trace, clock, viewport, signal }) {
 
   const moveTo = async (locator, duration, name) => {
     const seconds = durationSeconds(duration ?? MOVE_SECONDS, `${name} duration`)
-    await locator.waitFor()
-    const box = await locator.boundingBox()
-    if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
-    // A person does not chase a target that shifts under a pointer already inside it.
-    if (pointInsideBox(pointer, box)) return pointer
-    const point = await aimPoint(locator, box)
+    const target = await findTarget(locator)
+    // A person does not chase a target that shifts under a pointer already on it.
+    if (pointInsideBox(pointer, target.box)) return pointer
+    let point = await aimPoint(target)
     if (!insideViewport(point, viewport)) {
-      throw new TraceframeError(`${locator} is outside the viewport. Scroll explicitly before moving to it.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
+      const aim = await reveal(target.element, locator)
+      if (!aim.visible) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
+      point = aim.point
     }
     await movePointer(point, seconds)
     const after = await locator.boundingBox()
@@ -109,6 +108,10 @@ export function createFlowApi({ page, trace, clock, viewport, signal }) {
       }
       await movePointer(point, durationSeconds(options.duration ?? MOVE_SECONDS, 'move duration'))
     },
+    async scroll(locator) {
+      const { shown } = await reveal((await findTarget(locator)).element, locator)
+      if (!shown) throw new TraceframeError(`${locator} could not be revealed by scrolling.`, { code: 'FLOW_TARGET_OUTSIDE_VIEWPORT' })
+    },
     async wait(seconds) {
       await sleep(durationSeconds(seconds, 'wait', { allowZero: true }) * 1000)
     },
@@ -142,21 +145,85 @@ function assertBelowLimit(events, limit, type) {
   }
 }
 
+async function findTarget(locator) {
+  await locator.waitFor()
+  const element = await locator.elementHandle()
+  const box = await element.boundingBox()
+  if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
+  return { element, box }
+}
+
 function pointInsideBox(point, box) {
   return point.x >= box.x && point.x <= box.x + box.width && point.y >= box.y && point.y <= box.y + box.height
 }
 
 // Aim at the middle of the first line box, as Playwright does, so a link that wraps onto two lines is not missed.
-async function aimPoint(locator, box) {
-  const offset = await locator.evaluate((element) => {
-    const bounds = element.getBoundingClientRect()
-    const first = element.getClientRects()[0] ?? bounds
+async function aimPoint({ element, box }) {
+  const offset = await element.evaluate((target) => {
+    const bounds = target.getBoundingClientRect()
+    const first = target.getClientRects()[0] ?? bounds
     // As fractions of the element's size, because a scaled iframe makes the page box a different size than the element's own.
     return { x: (first.left + first.width / 2 - bounds.left) / bounds.width, y: (first.top + first.height / 2 - bounds.top) / bounds.height }
   })
   return { x: box.x + offset.x * box.width, y: box.y + offset.y * box.height }
 }
 
+async function reveal(element, locator) {
+  // Content that loads above the target during the scroll can push it out again.
+  for (let attempt = 0; attempt < 2; attempt++) {
+    const target = await scrollIntoView(element, locator)
+    const point = await aimPoint(target)
+    const visibility = await target.element.evaluate((node, offset) => new Promise((resolve) => {
+      new IntersectionObserver(([entry], observer) => {
+        observer.disconnect()
+        const bounds = node.getBoundingClientRect()
+        const x = bounds.left + offset.x * bounds.width
+        const y = bounds.top + offset.y * bounds.height
+        const rect = entry.intersectionRect
+        const shown = entry.intersectionRatio > 0
+        resolve({ shown, visible: shown && x >= rect.left && x <= rect.right && y >= rect.top && y <= rect.bottom })
+      }).observe(node)
+    }), relativePoint(point, target.box))
+    if (visibility.visible || attempt === 1) return { point, ...visibility }
+    element = target.element
+  }
+}
+
+// Awaiting the scroll waits for a scrolling panel. Chromium resolves a page scroll before it stops, and a sticky target
+// or iframe can stay put while its page scrolls, so also wait for every enclosing page to stop.
+async function scrollIntoView(element, locator) {
+  await element.evaluate((target) => target.scrollIntoView({ behavior: 'smooth', block: 'center' }))
+  try {
+    await element.waitForElementState('stable')
+    let frame = await element.ownerFrame()
+    while (frame) {
+      await frame.evaluate(() => new Promise((resolve) => {
+        let last
+        const check = () => {
+          const now = `${scrollX},${scrollY}`
+          if (now === last) resolve()
+          else requestAnimationFrame(check)
+          last = now
+        }
+        requestAnimationFrame(check)
+      }))
+      if (!frame.parentFrame()) break
+      await (await frame.frameElement()).waitForElementState('stable')
+      frame = frame.parentFrame()
+    }
+    // The page may replace the target while it scrolls, or remove or hide it, as a menu that closes on scroll does.
+    const target = await findTarget(locator).catch((error) => {
+      throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} disappeared when it was scrolled into view.`, { code: 'FLOW_TARGET_MISSING', cause: error }) : error
+    })
+    await target.element.waitForElementState('stable')
+    const box = await target.element.boundingBox()
+    if (!box) throw new TraceframeError(`Could not find ${locator}.`, { code: 'FLOW_TARGET_MISSING' })
+    return { element: target.element, box }
+  } catch (error) {
+    throw error.name === 'TimeoutError' ? new TraceframeError(`${locator} kept moving after it was scrolled into view.`, { code: 'FLOW_TARGET_MOVED', cause: error }) : error
+  }
+}
+
 async function assertReceivesPointer(element, locator, point) {
   const box = await element.boundingBox()
   if (!box || !pointInsideBox(point, box)) {
@@ -172,7 +239,7 @@ async function assertReceivesPointer(element, locator, point) {
     return false
   }, relativePoint(point, box))
   if (!receivesPointer || !(await framesReceivePointer(element, point))) {
-    throw new TraceframeError(`${locator} is covered at the click point.`, { code: 'FLOW_TARGET_OBSCURED' })
+    throw new TraceframeError(`${locator} is covered at the click point. If a dialog or toast covers it, wait until it closes; if a panel clips it or a sticky header or bar covers it, call scroll() on the target first.`, { code: 'FLOW_TARGET_OBSCURED' })
   }
 }