@milkstraw/traceframe 0.6.0 → 0.6.2

package/README.md

@@ -65,12 +65,14 @@ Use `--capture-diagnostics` (`captureDiagnostics: true` with `record` or `record
 
 Since v0.6.0, the retained raw file is `capture.mkv` instead of `capture.webm`; update scripts that inspect it. Final output remains MP4.
 
-Traceframe saves Chromium's JPEG frames and presentation timestamps directly into a temporary `capture.mkv`, then encodes the final MP4 after the flow ends. This avoids a live intermediate video encode and works with CPU or GPU browser rendering. Chromium frame timestamps are retained with millisecond precision; a 150 ms buffer orders delayed arrivals. The first received frame establishes the audio clock, so subsequently delivered frames older than that origin are dropped. Slow storage applies backpressure. Moving content needs more temporary disk space than an inter-frame codec: at high resolution, allow hundreds of megabytes for short takes and potentially gigabytes for longer ones. Static pages need only a few stored frames. If your temporary directory is backed by RAM, choose a disk-backed `--run-dir` for large takes and remove it afterward. The final renderer supplies the requested frame rate before drawing the cursor.
+Traceframe saves Chromium's JPEG frames and presentation timestamps directly into a temporary `capture.mkv`, then encodes the final MP4 after the flow ends. This avoids a live intermediate video encode and works with CPU or GPU browser rendering. Chromium frame timestamps are retained with microsecond precision; a 150 ms buffer orders delayed arrivals. The first received frame establishes the audio clock, so subsequently delivered frames older than that origin are dropped. Slow storage applies backpressure. Moving content needs more temporary disk space than an inter-frame codec: at high resolution, allow hundreds of megabytes for short takes and potentially gigabytes for longer ones. Static pages need only a few stored frames. If your temporary directory is backed by RAM, choose a disk-backed `--run-dir` for large takes and remove it afterward. The final renderer supplies the requested frame rate while keeping cursor animations smooth.
 
 GPU rendering and hardware video decoding can help the browser, but frame export still costs CPU. The bundled headless browser used software rendering on the tested Linux setup; selecting an executable or adding a GPU flag does not guarantee acceleration. For a caller-owned browser, configure and verify its GPU use before `recordPage`; use full Chromium for extensions. A GPU capability flag alone does not establish that the active video is hardware-decoded. When measuring performance, keep the browser version, source codec, viewport and device scale fixed, and compare recorded image updates and audio timing as well as file metadata.
 
 Start performance comparisons with the Chromium build bundled with Playwright. A custom `--browser` executable can have different screencast performance even when recording succeeds. Traceframe preserves your requested resolution and quality. If the workload cannot sustain the desired cadence, increasing the encoded FPS repeats existing frames; choose a lower quality or smaller dimensions explicitly if that tradeoff suits your recording.
 
+For video-heavy flows, see [checking video playback](docs/recording-flows.md#check-video-playback), including optional page effects and media sizing.
+
 Use `--run-dir .traceframe/runs/name` (`runDirectory` in the API) to keep the raw capture and FFmpeg filter file, including a partial capture if the flow fails during recording. An interrupted or failed capture can be incomplete; a retained file is not proof that recording finalized successfully. On a preparation or recording failure, Traceframe also tries to save a viewport screenshot as `failure.png`. Keep that directory out of Git, because captures and screenshots can contain private page data.
 
 When available, failure output includes the error code, flow-file locations, elapsed recording time, and paths to artifacts saved for that attempt. Without `--run-dir`, temporary captures are removed and no failure screenshot is saved. See [diagnosing a failed take](docs/recording-flows.md#diagnose-a-failed-take).

package/docs/recording-flows.md

@@ -24,6 +24,16 @@ Put setup and page readiness in `ready({ page, signal })`. This runs after initi
 
 If your script already owns the page or an extension context, use [`recordPage`](../README.md#record-a-page-you-already-opened) with an inline callback. Prepare the page before calling it; the caller owns navigation and browser cleanup.
 
+### Check video playback
+
+For a video-heavy journey, check playback in your first take at the final recording settings; a separate check before recording is not needed. Record that take with `--capture-diagnostics` (`captureDiagnostics: true` in the API) and [inspect the saved motion](#record-inspect-and-refine) of its video segment. Capture diagnostics count delivered frames, which can contain repeated pictures; a final MP4 reporting 60 FPS does not establish 60 distinct captured images per second, and a 30 FPS source cannot provide 60 distinct video frames per second. If the video looks smooth, later takes need no new playback check until the browser, recording settings or workload changes.
+
+On YouTube, the player's **Settings (gear) → Ambient mode → Off** can reduce expensive background drawing during CPU-only playback. Ambient mode is available only with YouTube's dark theme; check the setting under the theme you will record. Disable this decorative effect only when it is unnecessary for the demonstration, and verify the setting before the take. Use the site's setting in `ready`, or prepare the caller-owned page before `recordPage`; Traceframe does not change it automatically.
+
+For video assets you control, matching the source dimensions to the displayed physical pixels can avoid costly browser resizing during CPU-only playback. Physical dimensions include device scale: a video displayed 1512 CSS pixels wide at scale 2 occupies 3024 physical pixels. Preparing another asset can change image quality, so compare it with the original and preserve the intended framing. Do not silently replace third-party media, crop content or lower the required output resolution to claim a performance improvement.
+
+If the video looks choppy, apply any preparation above that suits the demonstration and make one more take at the same recording settings. If it is still choppy, or nothing applies, keep the requested output dimensions, playback speed and source quality, finish the requested recording and report the limitation. For optional diagnosis of playback without recording, let buffering and any extension processing warm up, then read the video's `getVideoPlaybackQuality()` counters before and after a representative segment with recording off, for example in `ready` or before `recordPage`. Page preparation can improve playback while capture remains expensive.
+
 ## Save and switch login state
 
 Add `.auth/` to your project's `.gitignore`; saved state can grant access to the account. To create it interactively, install full Chromium and open Playwright's login window:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@milkstraw/traceframe",
-  "version": "0.6.0",
+  "version": "0.6.2",
   "description": "Record scripted browser flows and render smooth cursor overlays into video.",
   "keywords": [
     "playwright",

package/src/jpeg-capture.mjs

@@ -6,6 +6,9 @@ import { finished } from 'node:stream/promises'
 // A short reorder window handles JPEGs that finish compression out of order.
 const REORDER_MS = 150
 const MAX_BUFFER_BYTES = 64 * 1024 * 1024
+// Store microseconds. Millisecond rounding can move distinct pictures into the
+// same output frame when export resamples to a fixed frame rate.
+const TIMESTAMP_SCALE_NS = 1_000
 
 export function createJpegCapture(path, { width, height, fps, onError }) {
   const output = createWriteStream(path)
@@ -25,7 +28,7 @@ export function createJpegCapture(path, { width, height, fps, onError }) {
     try {
       await new Promise((resolve, reject) => {
         output.cork()
-        output.write(cluster(Math.round(frame.timestamp - origin), frame.data.length))
+        output.write(cluster(ticks(frame.timestamp - origin), frame.data.length))
         output.write(frame.data, cause => cause ? reject(cause) : resolve())
         output.uncork()
       })
@@ -70,7 +73,7 @@ export function createJpegCapture(path, { width, height, fps, onError }) {
           await finished(output)
           if (last) {
             const duration = Buffer.from('4489880000000000000000', 'hex')
-            duration.writeDoubleBE(Math.round(last.timestamp - origin) + 1000 / fps, 3)
+            duration.writeDoubleBE(ticks(last.timestamp - origin + 1000 / fps), 3)
             const file = await open(path, 'r+')
             try {
               const { bytesWritten } = await file.write(duration, 0, duration.length, durationOffset)
@@ -94,7 +97,10 @@ export function createJpegCapture(path, { width, height, fps, onError }) {
 }
 
 // The fixed Matroska subset contains one MJPEG track and one keyframe per cluster.
-// Timestamps are milliseconds; JPEG dimensions and pixels remain unchanged.
+// Timestamps are microseconds; JPEG dimensions and pixels remain unchanged.
+function ticks(milliseconds) {
+  return Math.round(milliseconds * 1e6 / TIMESTAMP_SCALE_NS)
+}
 function uint(value) {
   const bytes = []
   do { bytes.unshift(value % 256); value = Math.floor(value / 256) } while (value)
@@ -119,7 +125,7 @@ function header(width, height, fps) {
   ])
   // Reserve a Void element, then replace it with Duration after the last write.
   const durationSlot = Buffer.from('ec89000000000000000000', 'hex')
-  const info = element('1549a966', [element('2ad7b1', 1_000_000), element('4d80', 'traceframe'), element('5741', 'traceframe'), durationSlot])
+  const info = element('1549a966', [element('2ad7b1', TIMESTAMP_SCALE_NS), element('4d80', 'traceframe'), element('5741', 'traceframe'), durationSlot])
   const bytes = Buffer.concat([prefix, info,
     element('1654ae6b', [element('ae', [element('d7', 1), element('73c5', 1), element('83', 1), element('9c', 0), element('86', 'V_MJPEG'), element('23e383', Math.round(1e9 / fps)), element('e0', [element('b0', width), element('ba', height)])])]),
   ])

package/src/render.mjs

@@ -57,20 +57,24 @@ export async function renderVideo({ rawPath, trace, directory, config, ffmpeg, s
 // Event times are seconds from the first frame of the raw video.
 export function buildFilterScript({ trace, scale, fps = 60, audio, overlayEofAction = 'pass' }) {
   const { clicks, moves } = trace
+  const animatedOverlays = clicks.length > 0 || moves.length > 0
+  const cadence = `tpad=stop_mode=clone:stop_duration=1,fps=${fps}`
   // Pass mode avoids waiting on looping image inputs; FFmpeg 7 needs repeat for padding.
   // yuv444 places overlays on whole pixels rather than snapping them to even pixels.
   const overlay = `eof_action=${overlayEofAction}:format=yuv444`
-  // Expand sparse capture timestamps before overlays so the cursor stays smooth
-  // even on a static page. Padding also covers a sub-frame take or final rounding.
+  // Animated overlays need every output tick, even on a static page.
+  // With stationary overlays, composite before duplicating identical frames.
+  // Padding also covers a sub-frame take or final rounding.
   // Chrome JPEGs are full range; normalize before compositing limited-range overlays.
   // Convert to the overlay format before duplicating frames. Converting afterward
   // can queue a full-size buffer for every repeated frame in a long static gap.
-  const filters = [`[0:v]scale=out_range=tv,format=yuv420p,format=yuv444p,setpts=PTS-STARTPTS,tpad=stop_mode=clone:stop_duration=1,fps=${fps}[video0]`]
+  const filters = [`[0:v]scale=out_range=tv,format=yuv420p,format=yuv444p,setpts=PTS-STARTPTS${animatedOverlays ? ',' + cadence : ''}[video0]`]
   if (clicks.length) filters.push(`[2:v]format=rgba,split=${clicks.length}${clicks.map((_, index) => `[pulse${index}]`).join('')}`)
   clicks.forEach((click, index) => filters.push(...clickFilter(click, index, scale, overlay)))
 
   const cursorLabels = moves.map((_, index) => `[cursor${index}]`).join('')
-  filters.push(`[1:v]format=rgba,split=${moves.length + 1}${cursorLabels}[cursor${moves.length}]`)
+  // Convert the cursor once before splitting it into movement segments.
+  filters.push(`[1:v]format=yuva444p,split=${moves.length + 1}${cursorLabels}[cursor${moves.length}]`)
   const coordinate = (value, hotspot) => number(value * scale - hotspot * scale)
   const initialEnable = moves.length ? `:enable='lt(t,${number(moves[0].at)})'` : ''
   filters.push(`[video${clicks.length}][cursor0]overlay=x='${coordinate(trace.initialPointer.x, CURSOR_HOTSPOT.x)}':y='${coordinate(trace.initialPointer.y, CURSOR_HOTSPOT.y)}':${overlay}${initialEnable}[cursorVideo0]`)
@@ -85,7 +89,7 @@ export function buildFilterScript({ trace, scale, fps = 60, audio, overlayEofAct
     const y = `${coordinate(move.from.y, CURSOR_HOTSPOT.y)}+${number((move.to.y - move.from.y) * scale)}*${eased}`
     filters.push(`[cursorVideo${index}][cursor${index + 1}]overlay=x='${x}':y='${y}':${overlay}:enable='${enable}'[cursorVideo${index + 1}]`)
   })
-  filters.push(`[cursorVideo${moves.length}]colorspace=ispace=bt470bg:iprimaries=bt709:itrc=bt709:all=bt709:format=yuv420p[final]`)
+  filters.push(`[cursorVideo${moves.length}]colorspace=ispace=bt470bg:iprimaries=bt709:itrc=bt709:all=bt709:format=yuv420p${animatedOverlays ? '' : ',' + cadence}[final]`)
   if (audio) {
     const input = clicks.length ? 3 : 2
     const trim = number(Math.max(0, -audio.offset))