browsertime 28.0.0 → 28.2.0

package/CHANGELOG.md

@@ -1,5 +1,30 @@
 # Browsertime changelog (we do [semantic versioning](https://semver.org))
 
+## 28.2.0 - 2026-07-20
+
+### Added
+* New `videoParams.noiseTolerance`: a percentage of a frame's pixels that is allowed to differ before duplicate-frame elimination still treats the frame as unchanged. Browsers re-rasterize a visually-done page long after it settles, leaving invisible sub-pixel noise on glyph edges that the default handful-of-pixels tolerance counts as real change, inflating Last Visual Change, VisualComplete95/99 and Speed Index. Off by default [#2528](https://github.com/sitespeedio/browsertime/pull/2528).
+* New `videoParams.filmstripDiff` frame diffs: writes `diff_<ms>.png` next to each filmstrip frame (the frame dimmed with every changed pixel painted red, using the same per-pixel threshold report UIs use at runtime) plus a Frame Diffs metric. Report UIs cannot read frame pixels from `file://` results, so an exact "what changed in this frame" view was impossible exactly where people debug local results. Off by default [#2531](https://github.com/sitespeedio/browsertime/pull/2531).
+* Visual instability heatmap: the same `videoParams.filmstripDiff` run now also writes one `instability.png` showing how many times each pixel changed across the video (amber for changed once, deep red for ten or more), plus a Frame Instability summary in the JSON. Answers "which parts of the page keep changing", so invisible re-rasterization noise can be told apart from a genuinely restless element [#2533](https://github.com/sitespeedio/browsertime/pull/2533).
+* `cpu.styleInvalidations` (Chrome only) now splits every reason, trigger and source at first paint: new `afterFirstPaint` counts and `styleRecalcsAfterFirstPaint` / `layoutInvalidationsAfterFirstPaint` totals separate building the page from churn on a page the user is already looking at. The fields only exist when the trace carried the FCP event, so absent means unknown rather than zero [#2530](https://github.com/sitespeedio/browsertime/pull/2530).
+
+### Fixed
+* Visual metrics no longer require OpenCV and pyssim unless the metrics that need them are requested: OpenCV is only used by `--contentful` and pyssim only by `--perceptual`, so a default install now runs on just ffmpeg, Numpy and Pillow. The Docker image keeps installing everything so both metrics keep working there [#2535](https://github.com/sitespeedio/browsertime/pull/2535).
+* `VisualComplete99` is now reset when the page repaints backward. Two copy-paste typos reset `VisualComplete95` twice and never reset `VisualComplete99`, so on a page that reaches 100%, repaints backward and then recovers, `VisualComplete99` could report a time earlier than `VisualComplete95` (99% complete before 95%, which cannot happen) [#2537](https://github.com/sitespeedio/browsertime/pull/2537).
+
+### Tech
+* `compare()`, the hot path of frame processing, now counts differing pixels with `count_nonzero` on the boolean mask instead of materializing an index array of every matching pixel on every call. Byte-identical metrics on a real recording [#2536](https://github.com/sitespeedio/browsertime/pull/2536).
+* Removed dead code from visual metrics: the unreachable ImageMagick-based `visualmetrics.py`, an advertised-but-unimplemented `--render` option, uncalled functions and leftover regexes in the portable script, plus a regex escape that printed a `SyntaxWarning` on Python 3.12+. No behaviour change [#2534](https://github.com/sitespeedio/browsertime/pull/2534).
+* Added CI test coverage for the Speed Index, visual progress and visual change math in `visualmetrics-portable.py`, running the shipped script against the repo's test frames with the computed values pinned. A dedicated GitHub Action runs the suite in under a second [#2532](https://github.com/sitespeedio/browsertime/pull/2532).
+
+## 28.1.0 - 2026-07-17
+
+### Added
+* Long Animation Frames (`pageinfo.loaf`) now ship every per-script field the LoAF API exposes (`startTime`, `duration`, `executionStart`, `pauseDuration`) plus `startTime` and `firstUIEventTimestamp` per frame — enough to attribute blocking time to individual scripts, place each frame on the page timeline and spot frames where an input was waiting [#2522](https://github.com/sitespeedio/browsertime/pull/2522) [#2523](https://github.com/sitespeedio/browsertime/pull/2523) [#2526](https://github.com/sitespeedio/browsertime/pull/2526).
+* New `pageinfo.loafSummary` key (`totalFrames`, `totalBlockingDuration`, `totalDuration`): `pageinfo.loaf` only ships the 10 frames with the most blocking, so totals computed from it understate busy pages [#2524](https://github.com/sitespeedio/browsertime/pull/2524).
+* LoAF script URLs get the same ResourceLoader `label`s as `cpu.urls` (`load.php[startup]` and friends) [#2525](https://github.com/sitespeedio/browsertime/pull/2525).
+* Multi-module `load.php` batches are labeled with their first module name — pages lazy-load several general batches, so the bare `load.php[scripts]` label collided as soon as there was more than one [#2527](https://github.com/sitespeedio/browsertime/pull/2527).
+
 ## 28.0.0 - 2026-07-17
 
 ### Breaking

package/browserscripts/pageinfo/loaf.js

@@ -17,6 +17,11 @@
     // re-package
     for (let entry of longestBlockingLoAFs) {
       const info = {};
+      // startTime places the frame on the page timeline (before or
+      // after FCP/LCP/load); firstUIEventTimestamp > 0 means an input
+      // was waiting while this frame blocked — the INP connection.
+      info.startTime = entry.startTime;
+      info.firstUIEventTimestamp = entry.firstUIEventTimestamp;
       info.blockingDuration = entry.blockingDuration;
       info.duration = entry.duration;
       info.styleAndLayoutStart = entry.styleAndLayoutStart;
@@ -29,6 +34,16 @@
       info.scripts = [];
       for (let script of entry.scripts) {
         const s = {};
+        // startTime + duration make per-script attribution possible:
+        // without them a frame's blocking can only be credited to
+        // every script that ran in it.
+        s.startTime = script.startTime;
+        s.duration = script.duration;
+        // executionStart - startTime is queue/compile delay before the
+        // script ran; pauseDuration separates "computed the whole time"
+        // from "sat in a sync pause" (alert, sync XHR).
+        s.executionStart = script.executionStart;
+        s.pauseDuration = script.pauseDuration;
         s.forcedStyleAndLayoutDuration = script.forcedStyleAndLayoutDuration;
         s.invoker = script.invoker;
         s.invokerType = script.invokerType;

package/browserscripts/pageinfo/loafSummary.js

@@ -0,0 +1,24 @@
+(function () {
+  // The loaf script ships only the 10 frames with the most blocking
+  // time, so its numbers understate busy pages. These totals are
+  // computed over every long animation frame before the cut.
+  if (
+    PerformanceObserver.supportedEntryTypes.includes('long-animation-frame')
+  ) {
+    const observer = new PerformanceObserver(list => {});
+    observer.observe({type: 'long-animation-frame', buffered: true});
+    const entries = observer.takeRecords();
+
+    let totalBlockingDuration = 0;
+    let totalDuration = 0;
+    for (let entry of entries) {
+      totalBlockingDuration += entry.blockingDuration;
+      totalDuration += entry.duration;
+    }
+    return {
+      totalFrames: entries.length,
+      totalBlockingDuration,
+      totalDuration
+    };
+  }
+})();

package/lib/chrome/mediawikiResourceLoader.js

@@ -96,12 +96,14 @@ export function labelForUrl(url) {
     return 'load.php[startup]';
   }
   const kind = only === 'styles' ? 'styles' : 'scripts';
-  if (modules.length === 1) {
+  // Every multi-module batch carries its (alphabetically) first module
+  // name. Real pages lazy-load several general batches (no `only=`)
+  // over time, so a bare load.php[scripts] label collides as soon as
+  // there is more than one — and the single-module case already names
+  // general batches like raw ones.
+  if (modules.length > 0) {
     return `load.php[${kind}:${modules[0]}]`;
   }
-  if (modules.length > 1 && only === 'scripts') {
-    return `load.php[scripts:${modules[0]}]`;
-  }
   return `load.php[${kind}]`;
 }
 

package/lib/chrome/trace/style-invalidations.js

@@ -26,6 +26,22 @@
  * they cause is already in cpu.categories.styleLayout and the
  * recalculateStyle summaries.
  *
+ * When the trace carries a firstContentfulPaint event, every reason,
+ * trigger and source also gets an afterFirstPaint count and the
+ * result gets styleRecalcsAfterFirstPaint /
+ * layoutInvalidationsAfterFirstPaint totals; with a
+ * largestContentfulPaint::Candidate event the same happens for
+ * afterLargestContentfulPaint (final candidate, mirroring how
+ * getRenderBlocking windows the recalculate-style work to both
+ * paints). Invalidations before first paint are mostly the page being
+ * built; between the paints they are what delayed the largest paint;
+ * after it they are churn on a page the user is already looking at.
+ * Absent fields mean the paint event was missing from the trace —
+ * consumers can fall back to guessing from the reason categories, and
+ * should mind that LCP can land before FCP in real traces (the counts
+ * stay truthful per boundary, an "in between" window is then simply
+ * not derivable by subtraction).
+ *
  * When the optional `bundles` map (url → location resolver from the
  * coverage path) is passed, source frames inside concatenated
  * bundles resolve to the owning module — thousands of invalidations
@@ -44,9 +60,13 @@ const MAX_REASONS = 10;
 const MAX_TRIGGERS = 15;
 const MAX_SOURCES = 10;
 
-function increment(map, key) {
+function increment(map, key, afterFirstPaint, afterLargestPaint) {
   if (!key) return;
-  map.set(key, (map.get(key) || 0) + 1);
+  const entry = map.get(key) || { count: 0, after: 0, afterLcp: 0 };
+  entry.count++;
+  if (afterFirstPaint) entry.after++;
+  if (afterLargestPaint) entry.afterLcp++;
+  map.set(key, entry);
 }
 
 // Source identity for an invalidation: the innermost stack frame
@@ -54,7 +74,13 @@ function increment(map, key) {
 // a known concatenated bundle (trace stack-frame positions are
 // 1-based, matching the resolvers). Without bundles the identity is
 // just the URL — the pre-existing behaviour.
-function sourceEntryFor(sources, data, bundles) {
+function sourceEntryFor(
+  sources,
+  data,
+  bundles,
+  afterFirstPaint,
+  afterLargestPaint
+) {
   for (const frame of data.stackTrace || []) {
     if (!frame.url) continue;
     let module_;
@@ -67,18 +93,20 @@ function sourceEntryFor(sources, data, bundles) {
     const key = module_ ? `${frame.url}|${module_.name}` : frame.url;
     let entry = sources.get(key);
     if (!entry) {
-      entry = { url: frame.url, count: 0 };
+      entry = { url: frame.url, count: 0, after: 0, afterLcp: 0 };
       if (module_) entry.module = module_.name;
       sources.set(key, entry);
     }
     entry.count++;
+    if (afterFirstPaint) entry.after++;
+    if (afterLargestPaint) entry.afterLcp++;
     return;
   }
 }
 
 function topEntries(map, limit, publish) {
   return [...map.entries()]
-    .toSorted((a, b) => b[1] - a[1])
+    .toSorted((a, b) => b[1].count - a[1].count)
     .slice(0, limit)
     .map(entry => publish(entry));
 }
@@ -90,38 +118,108 @@ export function computeStyleInvalidations(trace, bundles) {
   const sources = new Map();
   let styleRecalcs = 0;
   let layoutInvalidations = 0;
+  let styleRecalcsAfter = 0;
+  let layoutInvalidationsAfter = 0;
+  let styleRecalcsAfterLcp = 0;
+  let layoutInvalidationsAfterLcp = 0;
   let sawInvalidations = false;
 
+  // Same events getRenderBlocking uses to window the recalculate-style
+  // work — one boundary per paint, so "after first paint" and "after
+  // the largest paint" mean the same thing in both places. LCP is the
+  // final candidate, like getLargestContentfulPaintEvent picks it.
+  const fcpEvent = trace.traceEvents.find(
+    task => task.name === 'firstContentfulPaint'
+  );
+  const fcpTs = fcpEvent ? fcpEvent.ts : undefined;
+  let lcpTs;
+  for (const task of trace.traceEvents) {
+    if (
+      task.name === 'largestContentfulPaint::Candidate' &&
+      (lcpTs === undefined || task.ts > lcpTs)
+    ) {
+      lcpTs = task.ts;
+    }
+  }
+
   for (const event of trace.traceEvents) {
     const data = (event.args && event.args.data) || {};
+    const afterFirstPaint = fcpTs !== undefined && event.ts > fcpTs;
+    const afterLargestPaint = lcpTs !== undefined && event.ts > lcpTs;
     switch (event.name) {
       case 'StyleRecalcInvalidationTracking': {
         sawInvalidations = true;
         styleRecalcs++;
-        increment(recalcReasons, data.reason);
-        sourceEntryFor(sources, data, bundles);
+        if (afterFirstPaint) styleRecalcsAfter++;
+        if (afterLargestPaint) styleRecalcsAfterLcp++;
+        increment(
+          recalcReasons,
+          data.reason,
+          afterFirstPaint,
+          afterLargestPaint
+        );
+        sourceEntryFor(
+          sources,
+          data,
+          bundles,
+          afterFirstPaint,
+          afterLargestPaint
+        );
         break;
       }
       case 'LayoutInvalidationTracking': {
         sawInvalidations = true;
         layoutInvalidations++;
-        increment(layoutReasons, data.reason);
-        sourceEntryFor(sources, data, bundles);
+        if (afterFirstPaint) layoutInvalidationsAfter++;
+        if (afterLargestPaint) layoutInvalidationsAfterLcp++;
+        increment(
+          layoutReasons,
+          data.reason,
+          afterFirstPaint,
+          afterLargestPaint
+        );
+        sourceEntryFor(
+          sources,
+          data,
+          bundles,
+          afterFirstPaint,
+          afterLargestPaint
+        );
         break;
       }
       case 'ScheduleStyleInvalidationTracking': {
         sawInvalidations = true;
         if (data.changedClass) {
-          increment(triggers, `class|${data.changedClass}`);
+          increment(
+            triggers,
+            `class|${data.changedClass}`,
+            afterFirstPaint,
+            afterLargestPaint
+          );
         }
         if (data.changedAttribute) {
-          increment(triggers, `attribute|${data.changedAttribute}`);
+          increment(
+            triggers,
+            `attribute|${data.changedAttribute}`,
+            afterFirstPaint,
+            afterLargestPaint
+          );
         }
         if (data.changedId) {
-          increment(triggers, `id|${data.changedId}`);
+          increment(
+            triggers,
+            `id|${data.changedId}`,
+            afterFirstPaint,
+            afterLargestPaint
+          );
         }
         if (data.changedPseudo) {
-          increment(triggers, `pseudo|${data.changedPseudo}`);
+          increment(
+            triggers,
+            `pseudo|${data.changedPseudo}`,
+            afterFirstPaint,
+            afterLargestPaint
+          );
         }
         break;
       }
@@ -132,31 +230,49 @@ export function computeStyleInvalidations(trace, bundles) {
   }
   if (!sawInvalidations) return;
 
-  return {
+  // The after fields only exist when the trace had the matching paint
+  // event — an absent field means "unknown", never "zero".
+  const withAfter = (published, entry) => {
+    if (fcpTs !== undefined) published.afterFirstPaint = entry.after;
+    if (lcpTs !== undefined) {
+      published.afterLargestContentfulPaint = entry.afterLcp;
+    }
+    return published;
+  };
+
+  const result = {
     styleRecalcs,
     layoutInvalidations,
-    recalcReasons: topEntries(
-      recalcReasons,
-      MAX_REASONS,
-      ([reason, count]) => ({
-        reason,
-        count
-      })
+    recalcReasons: topEntries(recalcReasons, MAX_REASONS, ([reason, entry]) =>
+      withAfter({ reason, count: entry.count }, entry)
     ),
-    layoutReasons: topEntries(
-      layoutReasons,
-      MAX_REASONS,
-      ([reason, count]) => ({
-        reason,
-        count
-      })
+    layoutReasons: topEntries(layoutReasons, MAX_REASONS, ([reason, entry]) =>
+      withAfter({ reason, count: entry.count }, entry)
     ),
-    triggers: topEntries(triggers, MAX_TRIGGERS, ([key, count]) => {
+    triggers: topEntries(triggers, MAX_TRIGGERS, ([key, entry]) => {
       const [kind, ...name] = key.split('|');
-      return { kind, name: name.join('|'), count };
+      return withAfter(
+        { kind, name: name.join('|'), count: entry.count },
+        entry
+      );
     }),
     sources: [...sources.values()]
       .toSorted((a, b) => b.count - a.count)
       .slice(0, MAX_SOURCES)
+      .map(entry => {
+        const published = { url: entry.url, count: entry.count };
+        if (entry.module) published.module = entry.module;
+        return withAfter(published, entry);
+      })
   };
+  if (fcpTs !== undefined) {
+    result.styleRecalcsAfterFirstPaint = styleRecalcsAfter;
+    result.layoutInvalidationsAfterFirstPaint = layoutInvalidationsAfter;
+  }
+  if (lcpTs !== undefined) {
+    result.styleRecalcsAfterLargestContentfulPaint = styleRecalcsAfterLcp;
+    result.layoutInvalidationsAfterLargestContentfulPaint =
+      layoutInvalidationsAfterLcp;
+  }
+  return result;
 }

package/lib/chrome/webdriver/chromium.js

@@ -395,6 +395,20 @@ export class Chromium {
     // this hack fixes that
     if (result.browserScripts && result.browserScripts.pageinfo) {
       result.browserScripts.pageinfo.longTask = this.longTaskInfo;
+
+      // Resolve LoAF script URLs to ResourceLoader module labels the
+      // same way cpu.urls gets them — several frames all blaming one
+      // giant load.php URL say nothing without the module name.
+      if (Array.isArray(result.browserScripts.pageinfo.loaf)) {
+        for (const frame of result.browserScripts.pageinfo.loaf) {
+          for (const script of frame.scripts || []) {
+            const label = labelForUrl(script.sourceURL);
+            if (label) {
+              script.label = label;
+            }
+          }
+        }
+      }
     }
 
     if (this.chrome.collectNetLog && this.chrome.android) {

package/lib/core/engine/collector.js

@@ -212,8 +212,14 @@ export class Collector {
           this._logVisualMetrics(data);
         }
         for (let key of Object.keys(data.visualMetrics)) {
-          // Skip VisualProgress/ContentfulProgress etc
-          if (!key.includes('Progress')) {
+          // Skip VisualProgress/ContentfulProgress etc, FrameDiffs (an
+          // array of per-frame objects) and FrameInstability (a summary
+          // object) — nothing to summarize.
+          if (
+            !key.includes('Progress') &&
+            key !== 'FrameDiffs' &&
+            key !== 'FrameInstability'
+          ) {
             const d = { visualMetrics: {} };
             d['visualMetrics'][key] = data.visualMetrics[key];
             statistics.addDeep(d);

package/lib/support/cli.js

@@ -732,6 +732,13 @@ export function parseCommandLine() {
       describe: 'Create filmstrip screenshots.',
       group: 'video'
     })
+    .option('videoParams.filmstripDiff', {
+      type: 'boolean',
+      default: false,
+      describe:
+        'Also write a diff image per filmstrip frame (the frame dimmed with every pixel changed since the previous frame painted red) plus changed-pixel counts. Lets report UIs show exact frame diffs without pixel access, for example when a report is opened from file://.',
+      group: 'video'
+    })
     .option('videoParams.nice', {
       default: 0,
       describe:
@@ -750,6 +757,13 @@ export function parseCommandLine() {
         'Convert the original video to a viewable format (for most video players). Turn that off to make a faster run.',
       group: 'video'
     })
+    .option('videoParams.noiseTolerance', {
+      type: 'number',
+      default: 0,
+      describe:
+        'Percentage (0-100) of pixels that are allowed to differ between video frames when Visual Metrics looks for visual changes. Browsers can re-rasterize the page long after it is visually done, leaving invisible sub-pixel noise that inflates Last Visual Change, Speed Index and the visual progress curve. Try 0.05 if you see a too late Last Visual Change with no visible change in the filmstrip. 0 (default) keeps the old behavior of allowing 5 pixels.',
+      group: 'video'
+    })
     .option('videoParams.threads', {
       default: threads,
       describe:

package/lib/video/postprocessing/visualmetrics/extraMetrics.js

@@ -21,16 +21,19 @@ export function extraMetrics(metrics) {
 
       // Oh noo the painting on the screen goes backward
       // see https://github.com/sitespeedio/sitespeed.io/issues/2259#issuecomment-456878707
+      // Dropping below a threshold invalidates that milestone and every
+      // higher one, so they are re-found at the later timestamp where the
+      // page climbs back up.
       if (metrics.VisualComplete85 && Number.parseInt(percent, 10) < 85) {
         metrics.VisualComplete85 = undefined;
         metrics.VisualComplete95 = undefined;
-        metrics.VisualComplete95 = undefined;
+        metrics.VisualComplete99 = undefined;
       } else if (
         metrics.VisualComplete95 &&
         Number.parseInt(percent, 10) < 95
       ) {
         metrics.VisualComplete95 = undefined;
-        metrics.VisualComplete95 = undefined;
+        metrics.VisualComplete99 = undefined;
       } else if (
         metrics.VisualComplete99 &&
         Number.parseInt(percent, 10) < 99

package/lib/video/postprocessing/visualmetrics/visualMetrics.js

@@ -52,6 +52,8 @@ export async function run(
 
   const thumbsize = getProperty(options, 'videoParams.thumbsize', 400);
 
+  const noiseTolerance = getProperty(options, 'videoParams.noiseTolerance', 0);
+
   const scriptArguments = [
     '--video',
     videoPath,
@@ -69,6 +71,10 @@ export async function run(
     100
   ];
 
+  if (noiseTolerance > 0) {
+    scriptArguments.push('--noisetolerance', noiseTolerance);
+  }
+
   if (options.visualMetricsPerceptual) {
     scriptArguments.push('--perceptual');
   }
@@ -111,6 +117,9 @@ export async function run(
     } else if (thumbsize !== 400) {
       scriptArguments.push('--thumbsize', thumbsize);
     }
+    if (getProperty(options, 'videoParams.filmstripDiff', false)) {
+      scriptArguments.push('--framediff');
+    }
   }
 
   const visualMetricsLogFile = path.join(

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "browsertime",
   "description": "Get performance metrics from your web page using Browsertime.",
-  "version": "28.0.0",
+  "version": "28.2.0",
   "bin": "./bin/browsertime.js",
   "type": "module",
   "types": "./scripting.d.ts",

package/types/chrome/mediawikiResourceLoader.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"mediawikiResourceLoader.d.ts","sourceRoot":"","sources":["../../lib/chrome/mediawikiResourceLoader.js"],"names":[],"mappings":"AAmBA,mEAEC;AA6DD,8CAuBC;AAcD,8DAUC;AAwBD;;;;;EAkCC;AAOD,6EA4BC"}
+{"version":3,"file":"mediawikiResourceLoader.d.ts","sourceRoot":"","sources":["../../lib/chrome/mediawikiResourceLoader.js"],"names":[],"mappings":"AAmBA,mEAEC;AA6DD,8CAyBC;AAcD,8DAUC;AAwBD;;;;;EAkCC;AAOD,6EA4BC"}

package/types/chrome/trace/style-invalidations.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"style-invalidations.d.ts","sourceRoot":"","sources":["../../../lib/chrome/trace/style-invalidations.js"],"names":[],"mappings":"AAqFA;;;;;;;EA4EC"}
+{"version":3,"file":"style-invalidations.d.ts","sourceRoot":"","sources":["../../../lib/chrome/trace/style-invalidations.js"],"names":[],"mappings":"AAiHA;;;;;;;EAoKC"}

package/types/video/postprocessing/visualmetrics/extraMetrics.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"extraMetrics.d.ts","sourceRoot":"","sources":["../../../../lib/video/postprocessing/visualmetrics/extraMetrics.js"],"names":[],"mappings":"AAAA;;;EA8CC"}
+{"version":3,"file":"extraMetrics.d.ts","sourceRoot":"","sources":["../../../../lib/video/postprocessing/visualmetrics/extraMetrics.js"],"names":[],"mappings":"AAAA;;;EAiDC"}

package/types/video/postprocessing/visualmetrics/visualMetrics.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"visualMetrics.d.ts","sourceRoot":"","sources":["../../../../lib/video/postprocessing/visualmetrics/visualMetrics.js"],"names":[],"mappings":"AAoBA,kDAKC;AACD,0KAqIC"}
+{"version":3,"file":"visualMetrics.d.ts","sourceRoot":"","sources":["../../../../lib/video/postprocessing/visualmetrics/visualMetrics.js"],"names":[],"mappings":"AAoBA,kDAKC;AACD,0KA8IC"}

package/visualmetrics/visualmetrics-portable.py

@@ -66,13 +66,17 @@ def compare(img1, img2, fuzz=0.10):
         img1_data = np.array(img1)
         img2_data = np.array(img2)
 
-        inds = np.argwhere(
+        # count_nonzero instead of len(argwhere(...)): the count is all
+        # we need and argwhere materializes an index array of every
+        # matching pixel, which is most of the frame on every call in
+        # the duplicate-elimination hot path.
+        close = (
             np.isclose(img1_data[:, :, 0], img2_data[:, :, 0], atol=fuzz * 255)
             & np.isclose(img1_data[:, :, 1], img2_data[:, :, 1], atol=fuzz * 255)
             & np.isclose(img1_data[:, :, 2], img2_data[:, :, 2], atol=fuzz * 255)
         )
 
-        return (img1_data.shape[0] * img1_data.shape[1]) - len(inds)
+        return (img1_data.shape[0] * img1_data.shape[1]) - int(np.count_nonzero(close))
     except BaseException as e:
         logging.exception(e)
         return None
@@ -409,16 +413,14 @@ def video_to_frames(
                 gc.collect()
                 if extract_frames(video, directory, full_resolution, viewport):
                     client_viewport = None
-                    directories = [directory]
-                    for dir in directories:
-                        if orange_file is not None:
-                            remove_frames_before_orange(dir, orange_file)
-                            remove_orange_frames(dir, orange_file)
-                        find_render_start(dir, orange_file, cropped, is_mobile)
-                        adjust_frame_times(dir)
-                        eliminate_duplicate_frames(dir, cropped, is_mobile)
-                        crop_viewport(dir)
-                        gc.collect()
+                    if orange_file is not None:
+                        remove_frames_before_orange(directory, orange_file)
+                        remove_orange_frames(directory, orange_file)
+                    find_render_start(directory, orange_file, cropped, is_mobile)
+                    adjust_frame_times(directory)
+                    eliminate_duplicate_frames(directory, cropped, is_mobile)
+                    crop_viewport(directory)
+                    gc.collect()
                 else:
                     logging.critical("Error extracting the video frames from %s", video)
             else:
@@ -500,7 +502,7 @@ def find_recording_platform(video):
     lines = result.stderr.splitlines()
 
     is_mobile = False
-    matcher = re.compile(".*com\.android\.version.*")
+    matcher = re.compile(r".*com\.android\.version.*")
     for line in lines:
         if matcher.search(line):
             is_mobile = True
@@ -902,6 +904,33 @@ def eliminate_duplicate_frames(directory, cropped, is_mobile):
                 )
                 Path(duplicate).unlink()
 
+            # With a noise tolerance set, do a last pass over the remaining
+            # frames and drop every frame that differs from the previously
+            # kept frame by no more than the allowance. Browsers can
+            # re-rasterize the whole page long after it is visually done,
+            # leaving invisible sub-pixel noise on glyph edges spread over
+            # hundreds of pixels on a page-sized frame; the passes above
+            # allow only a handful of differing pixels, so that noise counts
+            # as a visual change and pollutes Last Visual Change, the visual
+            # progress curve, Speed Index and VisualComplete85/95/99.
+            # Comparing against the last KEPT frame (not the deleted
+            # neighbour) keeps slow real changes: once the accumulated
+            # difference crosses the allowance the frame is kept and becomes
+            # the new reference.
+            if options.noisetolerance > 0:
+                max_differences = max(
+                    5, int(width * height * options.noisetolerance / 100)
+                )
+                files = sorted(glob.glob(str(Path(directory) / "ms_*.png")))
+                if len(files) > 2:
+                    kept = files[0]
+                    for file in files[1:]:
+                        if frames_match(kept, file, 15, max_differences, crop, None):
+                            logging.debug("Removing noise frame {0}".format(file))
+                            Path(file).unlink()
+                        else:
+                            kept = file
+
     except BaseException:
         logging.exception("Error processing frames for duplicates")
 
@@ -950,18 +979,6 @@ def get_decimate_filter():
     return decimate
 
 
-def clean_directory(directory):
-    files = glob.glob(str(Path(directory) / "*.png"))
-    for file in files:
-        Path(file).unlink()
-    files = glob.glob(str(Path(directory) / "*.jpg"))
-    for file in files:
-        Path(file).unlink()
-    files = glob.glob(str(Path(directory) / "*.json"))
-    for file in files:
-        Path(file).unlink()
-
-
 def is_color_frame(file, color_file):
     """Check a section from the middle, top and bottom of the viewport to see if it matches"""
     global frame_cache
@@ -1189,6 +1206,101 @@ def save_screenshot(directory, dest, quality):
 ##########################################################################
 
 
+def create_frame_diffs(directory):
+    """Write a diff image next to every filmstrip frame and return the
+    changed-pixel counts.
+
+    For each adjacent pair of kept frames, diff_<ms>.png shows the
+    current frame dimmed toward white with every changed pixel painted
+    red — precomputed here because the report's lightbox cannot read
+    pixels when opened from file:// (tainted canvas), and this is where
+    the frames and Pillow already are. The per-pixel threshold (sum of
+    channel deltas > 30) matches the report's canvas fallback, so both
+    paths show the same picture. PNG on purpose: mostly-white plus
+    sparse red compresses better than JPEG and keeps the marks crisp.
+
+    Runs before convert_to_jpeg (which only globs ms_*.png) while the
+    frames are still lossless PNG.
+
+    The same loop also counts how many times each pixel changed across
+    the whole run and writes instability.png: the final frame ghosted
+    toward white with every changed pixel tinted from amber (changed
+    once) to deep red (changed 10 or more times). The color scale is
+    fixed, not normalized per run, so heatmaps from two runs are
+    comparable side by side. A page that paints once and stays put is
+    all amber; text that re-rasterizes over and over shows up as red
+    speckle. The summary counts are returned next to the per-frame
+    diffs, with "repeatedly" meaning 5 or more changes.
+    """
+    diffs = []
+    instability = None
+    try:
+        import numpy as np
+        from PIL import Image
+
+        files = sorted(glob.glob(str(Path(directory) / "ms_*.png")))
+        counts = None
+        final = None
+        for previous_file, current_file in zip(files, files[1:]):
+            with Image.open(previous_file) as previous_image, Image.open(
+                current_file
+            ) as current_image:
+                previous = np.array(previous_image.convert("RGB"), dtype=int)
+                current = np.array(current_image.convert("RGB"), dtype=int)
+            if previous.shape != current.shape:
+                continue
+            delta = np.abs(previous - current).sum(axis=2)
+            changed = delta > 30
+            if counts is None:
+                counts = np.zeros(changed.shape, dtype=int)
+            if counts.shape == changed.shape:
+                counts += changed
+                final = current
+            out = 255 - (255 - current) * 0.2
+            out[changed] = [224, 28, 60]
+            stem = Path(current_file).stem
+            diff_name = "diff_" + stem[3:] + ".png"
+            Image.fromarray(out.astype(np.uint8)).save(
+                str(Path(directory) / diff_name)
+            )
+            changed_pixels = int(changed.sum())
+            diffs.append(
+                {
+                    "time": int(stem[3:]),
+                    "changedPixels": changed_pixels,
+                    "changedShare": round(
+                        changed_pixels / changed.size * 100, 2
+                    ),
+                }
+            )
+        if counts is not None and final is not None:
+            cap = 10
+            heat = np.sqrt(np.minimum(counts, cap) / cap)
+            amber = np.array([255.0, 196.0, 60.0])
+            red = np.array([201.0, 16.0, 46.0])
+            color = amber + (red - amber) * heat[:, :, None]
+            alpha = np.where(counts > 0, 0.35 + 0.65 * heat, 0)[:, :, None]
+            ghost = 255 - (255 - final) * 0.15
+            heatmap = ghost * (1 - alpha) + color * alpha
+            Image.fromarray(heatmap.astype(np.uint8)).save(
+                str(Path(directory) / "instability.png")
+            )
+            changed_ever = int((counts > 0).sum())
+            changed_repeatedly = int((counts >= 5).sum())
+            instability = {
+                "changedPixels": changed_ever,
+                "changedShare": round(changed_ever / counts.size * 100, 2),
+                "repeatedlyChangedPixels": changed_repeatedly,
+                "repeatedlyChangedShare": round(
+                    changed_repeatedly / counts.size * 100, 2
+                ),
+                "maxChanges": int(counts.max()),
+            }
+    except BaseException:
+        logging.exception("Error creating frame diffs")
+    return diffs, instability
+
+
 def convert_to_jpeg(directory, quality):
     logging.debug("Converting video frames to JPEG")
     directory = str(Path(directory).resolve())
@@ -1395,12 +1507,6 @@ def calculate_key_color_frames(histograms, key_colors):
     for key in key_colors:
         key_color_frames[key] = []
 
-    current = None
-    current_key = None
-    total = 0
-    matched = 0
-    buckets = 256
-    channels = ["r", "g", "b"]
     histograms = histograms.copy()
 
     while len(histograms) > 0:
@@ -1486,17 +1592,6 @@ def calculate_frame_progress(histogram, start, final):
     return math.floor(progress * 100)
 
 
-def find_visually_complete(progress):
-    time = 0
-    for p in progress:
-        if int(p["progress"]) == 100:
-            time = p["time"]
-            break
-        elif time == 0:
-            time = p["time"]
-    return time
-
-
 def calculate_speed_index(progress):
     si = 0
     last_ms = progress[0]["time"]
@@ -1510,11 +1605,6 @@ def calculate_speed_index(progress):
 
 
 def calculate_contentful_speed_index(progress, directory):
-    # convert output comes out with lines that have this format:
-    # <pixel count>: <rgb color> #<hex color> <gray color>
-    # This is CLI dependant and very fragile
-    matcher = re.compile(r"(\d+?):")
-
     try:
         from PIL import Image
 
@@ -1715,7 +1805,14 @@ def calculate_hero_time(progress, directory, hero, viewport):
 ##########################################################################
 
 
-def check_config():
+def check_config(need_opencv=False, need_ssim=False):
+    """Check dependencies.
+
+    OpenCV is only used by --contentful/--contentful-video and pyssim
+    only by --perceptual, so their absence is fatal only when those
+    options are in play — everything else needs just ffmpeg, Numpy and
+    Pillow.
+    """
     ok = True
 
     if get_decimate_filter() is not None:
@@ -1739,45 +1836,44 @@ def check_config():
         ok = False
 
     try:
-        import cv2
+        from PIL import Image, ImageCms, ImageDraw, ImageOps  # noqa
 
-        logging.debug("OpenCV-Python found")
+        logging.debug("Pillow found")
     except BaseException:
-        print("OpenCV-Python: FAIL")
+        print("Pillow: FAIL")
         ok = False
 
     try:
-        from PIL import Image, ImageCms, ImageDraw, ImageOps  # noqa
+        import cv2
 
-        logging.debug("Pillow found")
+        logging.debug("OpenCV-Python found")
     except BaseException:
-        print("Pillow: FAIL")
-        ok = False
+        if need_opencv:
+            print("OpenCV-Python: FAIL")
+            ok = False
+        else:
+            # Not print: in --json runs stdout must stay pure JSON.
+            logging.info(
+                "OpenCV-Python is not installed (only needed for --contentful)"
+            )
 
     try:
         from ssim import compute_ssim  # noqa
 
         logging.debug("SSIM found")
     except BaseException:
-        print("SSIM: FAIL")
-        ok = False
+        if need_ssim:
+            print("SSIM: FAIL")
+            ok = False
+        else:
+            # Not print: in --json runs stdout must stay pure JSON.
+            logging.info(
+                "SSIM is not installed (only needed for --perceptual)"
+            )
 
     return ok
 
 
-def check_process(command, output):
-    ok = False
-    try:
-        out = subprocess.check_output(
-            command, stderr=subprocess.STDOUT, shell=True, encoding="UTF-8"
-        )
-        if out.find(output) > -1:
-            ok = True
-    except BaseException:
-        ok = False
-    return ok
-
-
 ##########################################################################
 #   Main Entry Point
 ##########################################################################
@@ -1821,9 +1917,6 @@ def main():
         help="Directory of video frames "
         "(as input if exists or as output if a video file is specified).",
     )
-    parser.add_argument(
-        "--render", help="Render the video frames to the given mp4 video file."
-    )
     parser.add_argument(
         "--screenshot",
         help="Save the last frame of video as an image to the path provided.",
@@ -1926,6 +2019,27 @@ def main():
         help="Ignore the center X%% of the frame when looking for "
         "the first rendered frame (useful for Opera mini).",
     )
+    parser.add_argument(
+        "--framediff",
+        action="store_true",
+        default=False,
+        help="Write a diff image next to every filmstrip frame (the "
+        "frame dimmed with every pixel changed since the previous frame "
+        "painted red) and report the changed-pixel counts as a Frame "
+        "Diffs metric. Lets report UIs show exact diffs without pixel "
+        "access.",
+    )
+    parser.add_argument(
+        "--noisetolerance",
+        type=float,
+        default=0,
+        help="Percentage (0-100) of the compared area that is allowed to "
+        "differ between frames when eliminating duplicate frames. Use it "
+        "to keep invisible sub-pixel browser rendering noise from "
+        "inflating Last Visual Change, Speed Index and the visual "
+        "progress curve. 0 (default) keeps the fixed allowance of 5 "
+        "pixels.",
+    )
     parser.add_argument(
         "-k",
         "--perceptual",
@@ -2007,7 +2121,10 @@ def main():
             # Run a quick check to make sure all requirements exist,
             # otherwise failures might be silent due to how this code is
             # structured.
-            ok = check_config()
+            ok = check_config(
+                need_opencv=options.contentful or options.contentful_video,
+                need_ssim=options.perceptual,
+            )
             if not ok:
                 raise Exception("Please install requirements before running.")
 
@@ -2057,6 +2174,15 @@ def main():
                 key_colors,
             )
 
+            if options.framediff and options.dir is not None and metrics is not None:
+                framediffs, instability = create_frame_diffs(directory)
+                if framediffs:
+                    metrics.append({"name": "Frame Diffs", "value": framediffs})
+                if instability:
+                    metrics.append(
+                        {"name": "Frame Instability", "value": instability}
+                    )
+
             if options.screenshot is not None:
                 quality = 30
                 if options.quality is not None: