browsertime 19.1.0 → 19.3.0

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Browsertime changelog (we do [semantic versioning](https://semver.org))
 
+## 19.3.0 - 2023-12-18
+
+### Added
+* Updated Chromedriver and Edgedriver to 120 [#2033](https://github.com/sitespeedio/browsertime/pull/2033).
+
+### Tech
+* Moved the visual metrics python scripts to a new home [#2032](https://github.com/sitespeedio/browsertime/pull/2032).
+
+## 19.2.0 - 2023-12-11
+### Added
+* Updated to Chrome and Edge 120 in the Docker container [#2031](https://github.com/sitespeedio/browsertime/pull/2031).
+
+### Fixed
+* Updated to Selenium 4.16.0 [#2026](https://github.com/sitespeedio/browsertime/pull/2026).
+* Updated dev dependencies and find-up [#2029](https://github.com/sitespeedio/browsertime/pull/2029).
+* Recommend NodeJS 18 or later in the package.json [#2030](https://github.com/sitespeedio/browsertime/pull/2030). Browsertime 16.0.0 removed support for NodeJS 16.
+
 ## 19.1.0 - 2023-11-27
 
 ### Added

package/README.md

@@ -1,4 +1,4 @@
-# Browsertime - Your browser, your page, your scripts!
+# Browsertime
 ![Run Docker](https://github.com/sitespeedio/browsertime/actions/workflows/docker.yml/badge.svg?branch=main)
 ![Unit tests](https://github.com/sitespeedio/browsertime/actions/workflows/unittests.yml/badge.svg?branch=main)
 ![Windows Edge](https://github.com/sitespeedio/browsertime/actions/workflows/windows.yml/badge.svg?branch=main)
@@ -12,125 +12,113 @@
 
 ![Browsertime](browsertime.png)
 
-Access the Web Performance Timeline, from your browser, in your terminal!
+# Browsertime: Measure and Optimize Web Performance
 
-## Introduction
+Browsertime is a powerful, open-source Node.js tool designed for engineers who are building their own performance measurement tools. It serves as the core engine of projects like sitespeed.io and is a trusted tool used by Mozilla to measure the performance of Firefox.
 
-**Browsertime lets you *automate running JavaScript in your browser* primary used to collect performance metrics. What exactly does that mean?**
+# Table of Contents
+1. [Introduction](#browsertime-measure-and-optimize-web-performance)
+2. [Key Features](#key-features)
+3. [Installation](#installation)
+    - [NodeJS](#nodejs)
+    - [Docker](#docker)
+4. [Usage](#usage)
+    - [Basic Usage](#basic-usage)
+    - [Advanced Options](#advanced-options)
+    - [Connectivity](#connectivity)
+    - [Navigate in a script](#navigate-in-a-script)
+    - [Test on your mobile device](#test-on-your-mobile-device)
+    - [Using WebPageReplay](#using-webpagereplay)
+    - [Speed Index and video](#speed-index-and-video)
+5. [Browser Support](#browser-support)
+6. [How does it work](#how-does-it-work)
+7. [Contributing](#contributing)
+9. [Community and Support](#community-and-support)
+9. [License](#license)
 
-We think of a Browsertime as having four key capabilities:
+## Key Features
 
- - It handles everything with the browser (Firefox/Chrome/Edge/Safari and other browser that can be driven using WebDriver).
- - It executes a batch of default and configurable JavaScript when the URL has finished loading in the browser.
- - It records a video of the Browser screen used to calculate [Visual Metrics](https://github.com/WPO-Foundation/visualmetrics).
- - It lets you run your [scripting file to create and measure your users journey](https://www.sitespeed.io/documentation/sitespeed.io/scripting/).
+- **Robust Performance Testing:** Browsertime allows you to perform comprehensive performance tests on your web pages, including page load times, resource loading, and user interactions.
 
-**What is Browsertime good for?**
+- **Deep Metrics:** Gain access to a wide range of performance metrics, such as page load times, visual metrics (when things are painted on the screen), largest contentful paint (LCP), and more, helping you pinpoint areas for improvement.
 
-It is usually used for two different things:
+- **Scripting Capabilities:** Customize your tests and user interactions with JavaScript scripting, enabling advanced scenarios and detailed analysis.
 
- - You run it as a standalone tool to collect performance timing metrics of your web site.
- - You integrate it in your tool as a JavaScript runner that collects whatever JavaScript metrics/information you want.
+- **Docker Support:** Easily integrate Browsertime into your CI/CD pipelines and containerized environments with Docker support.
+##
 
-To understand how Browsertime do these things, let's talk about how it works. Here's an example of what happens when you give Browsertime a URL to test:
+## Installation
 
-1. You give your configuration to Browsertime.
-2. Browsertime uses the [WebDriver](https://www.w3.org/TR/webdriver/) (through [Selenium](http://seleniumhq.github.io/selenium/docs/api/javascript/index.html)) to start Firefox/Chrome/Safari/Edge.
-3. Browsertime starts FFMPEG to record a video of the browser screen
-4. The browser access the URL.
-5. When the page is finished loading (you can define yourself when that happens), Browsertime collects:
-   - [Navigation Timing metrics](http://kaaes.github.io/timing/info.html)
-   - [User Timing metrics](http://www.html5rocks.com/en/tutorials/webperformance/usertiming/)
-   - [Element Timing metrics](https://wicg.github.io/element-timing/)
-   - [Paint Timings](https://w3c.github.io/paint-timing/)
-   - [Googles Web Vitals](https://web.dev/vitals/)
-   - [CPU metrics CPU Long Tasks]((https://developer.mozilla.org/en-US/docs/Web/API/Long_Tasks_API))
-6. You can also collect internal trace logs from the browser using `--firefox.geckoProfiler` for Firefox and `--chrome.timeline` for Chromium browsers.
-7. It also collects a [HAR](http://www.softwareishard.com/blog/har-12-spec/) file that shows all requests/responses on the page.
-8. FFMpeg is stopped and the video is analysed. Browsertime collect Visual Metrics like Speed Index, First Visual Change and Last Visual Change.
+If you're an engineer working on performance measurement tools or simply want to harness the power of Browsertime for your web performance needs, follow the installation and usage instructions below.
 
-The result of the run is a JSON file with all the JavaScript metrics collected, a HAR file, a video recording of the screen and a screenshot.
-
-## A simple example
+### NodeJS
+```shell
+npm install -g browsertime
+browsertime https://example.com
+```
 
-Use our Docker image (with Chrome, Firefox, Edge, XVFB and the dependencies needed to record a video):
-<pre>
-$ docker run --rm -v "$(pwd)":/browsertime sitespeedio/browsertime https://www.sitespeed.io/
-</pre>
+### Docker
+```shell
+docker run --rm -v "$(pwd)":/browsertime sitespeedio/browsertime https://www.sitespeed.io/
+```
 
-Or using node:
-<pre>
-$ npm install browsertime -g
-$ browsertime https://www.sitespeed.io/
-</pre>
+## Usage
+Browsertime is designed to be straightforward to use, regardless of your experience level. Here's a quick guide on how to get started.
 
-Load https://www.sitespeed.io/ in Chrome three times. Results are stored in a JSON file (browsertime.json) with the timing data, and a HAR file (browsertime.har) in browsertime-results/www.sitespeed.io/$date/
+### Basic Usage
 
-## I want more examples
-Checkout the [examples](docs/examples/README.md).
+```shell
+browsertime https://www.example.com --browser chrome
+```
 
-## Browsers
-Browsertime supports Firefox, Chrome, and Edge (Chromium version) on desktop and Safari on Mac OS. On Android we support Chrome and Firefox (from 8.0) and Safari on iOS. You can also use the Safari simulator on Mac OS.
+This command will test https://www.example.com using Chrome.
 
-## How does it work
-Browsertime uses Selenium NodeJS to drive the browser. It starts the browser, load a URL, executes configurable Javascripts to collect metrics, collect a HAR file.
+### Advanced Options
 
-To get the HAR from Firefox we use the [HAR Export Trigger](https://github.com/firebug/har-export-trigger) and Chrome we use [Chrome-HAR](https://github.com/sitespeedio/chrome-har) to parse the timeline log and generate the HAR file.
+Browsertime offers various advanced options for more detailed analysis, such as specifying the number of runs, choosing a browser, or setting custom metrics. Here are some examples:
 
-# Speed Index and video
-It's easiest to run [our ready made Docker container](https://hub.docker.com/r/sitespeedio/browsertime/) to be able to record a video and calculate SpeedIndex because then you get all dependencies needed for free to run [VisualMetrics](https://github.com/WPO-Foundation/visualmetrics).
+#### Run the test multiple times
 
-The default video will include a timer and showing when the metrics happens, but you can turn that off using <code>--video.addTimer false</code>.
+```shell
+browsertime -n 5 https://www.example.com
+```
 
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/video-example.gif">
+#### Specify a different browser
+```shell
+browsertime --browser firefox https://www.example.com
+```
 
-## Test using Docker
-You can build and test changes using Docker locally.
+#### Custom metrics
+```shell
+browsertime --script "return {'myMetric': window.myApp.customMetric}" https://www.example.com
+```
 
-<pre>
-$ docker build -t sitespeedio/browsertime .
-$ docker run --rm -v "$(pwd)":/browsertime sitespeedio/browsertime -n 1 https://www.sitespeed.io/
-</pre>
+For a full list of options, run <code>$ browsertime --help</code> and you can see the configuration options.
 
-## Connectivity
+#### Connectivity
 
 You can throttle the connection to make the connectivity slower to make it easier to catch regressions. The best way to do that is to setup a network bridge in Docker or use our connectivity engine Throttle. Read more about how to do that in the [documentation](https://www.sitespeed.io/documentation/sitespeed.io/connectivity/).
 
-
-## Navigate in a script
+#### Navigate in a script
 If you need a more complicated test scenario, you can define your own (Selenium)test script that will do the testing. Use your own test script when you want to test your page as a logged in user, the login page or if you want to add things to your cart.
 
 We have a full section in the documentation about [scripting](https://www.sitespeed.io/documentation/sitespeed.io/scripting/).
 
-## Test on your mobile device
+#### Test on your mobile device
 Browsertime supports Chrome and Firefox on Android: Collecting SpeedIndex, HAR and video! 
 
 You need to [install adb](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#desktop) and [prepare your phone](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#on-your-phone) before you start.
 
-If you want to set connectivity you need to use something like [gnirehtet](https://github.com/Genymobile/gnirehtet) or [TSProxy](https://github.com/WPO-Foundation/tsproxy). Read more information [here](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#connectivity).
+If you want to set connectivity you need to use something like [gnirehtet](https://github.com/Genymobile/gnirehtet). Read more information [here](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#connectivity).
 
 <pre>
 $ browsertime --chrome.android.package com.android.chrome https://www.sitespeed.io --video --visualMetrics
 </pre>
 
-If you are on Linux (we have tested Ubuntu 18) you can use our Docker container to drive your Android phone. A couple of things to remember:
- * You need to run in privileged mode *--privileged* if you share the full usb bus
- * You need to share the USB ports *-v /dev/bus/usb:/dev/bus/usb* or share a specific port with *--device=/dev/bus/usb/001/017* (use *lsusb* to find the right mapping)
- * Add *-e START_ADB_SERVER=true* to start the adb server
-
-If you use Docker you will automatically get support for video and SpeedIndex. You can get that without Docker but then need to [install VisualMetrics dependencies](https://github.com/sitespeedio/docker-visualmetrics-deps/blob/main/Dockerfile) yourself.
-
-<pre>
-$ docker run --privileged -v /dev/bus/usb:/dev/bus/usb -e START_ADB_SERVER=true --rm -v "$(pwd)":/browsertime-results sitespeedio/browsertime -n 1 --android --visualMetrics --video https://en.m.wikipedia.org/wiki/Barack_Obama
-</pre>
-
-## Configuration
-Run <code>$ bin/browsertime.js --help</code> and you can see the configuration options.
-
-## Using WebPageReplay
-Our Docker container now included [WebPageReplay](https://github.com/catapult-project/catapult/blob/main/web_page_replay_go/README.md).
+#### Using WebPageReplay
+Our Docker container now includes [WebPageReplay](https://github.com/catapult-project/catapult/blob/main/web_page_replay_go/README.md).
 
-WebPageReplay will let you replay your page locally (getting rid of server latency etc) and makes it easier to find front end regressions.
+WebPageReplay will let you replay your page locally (getting rid of server latency) and makes it easier to find front end regressions.
 
 It works like this:
 1. The start script starts WebPageReplay in record mode
@@ -150,23 +138,59 @@ docker run --cap-add=NET_ADMIN --rm -v "$(pwd)":/browsertime -e REPLAY=true -e L
 
 Use Firefox:
 
-```
-docker run --cap-add=NET_ADMIN --rm -v "$(pwd)":/browsertime -e REPLAY=true -e LATENCY=100 sitespeedio/browsertime:12.0.0 -b firefox -n 11 --firefox.acceptInsecureCerts https://en.wikipedia.org/wiki/Barack_Obama
+```shell
+docker run --cap-add=NET_ADMIN --rm -v "$(pwd)":/browsertime -e REPLAY=true -e LATENCY=100 sitespeedio/browsertime:19.2.0 -b firefox -n 11 --firefox.acceptInsecureCerts https://en.wikipedia.org/wiki/Barack_Obama
 ```
 
 And Chrome on your Android phone. This will only work on Linux because you need to be able to mount the usb port in Docker:
 
-```
+```shell
 docker run --privileged -v /dev/bus/usb:/dev/bus/usb -e START_ADB_SERVER=true --cap-add=NET_ADMIN --rm -v “$(pwd)“:/browsertime -e REPLAY=true -e LATENCY=100 sitespeedio/browsertime https://en.m.wikipedia.org/wiki/Barack_Obama --android --chrome.args ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I= -n 11 --chrome.args user-data-dir=/data/tmp/chrome
 ```
 
-## Send metrics to Graphite
-The easiest way to send metrics is to install [jq](https://stedolan.github.io/jq/) and use it to pick the values you wanna track.
+#### Speed Index and video
+It's easiest to run [our ready made Docker container](https://hub.docker.com/r/sitespeedio/browsertime/) to be able to record a video and calculate SpeedIndex because then you get all dependencies needed for free to run [VisualMetrics](https://github.com/WPO-Foundation/visualmetrics).
 
-Here's an example on how you can pickup the median SpeedIndex from Browsertime and send it to your Graphite instance.
-<pre>
-echo "browsertime.your.key.SpeedIndex.median" $(cat /tmp/browsertime/browsertime.json | jq .[0].statistics.visualMetrics.SpeedIndex.median) "`date +%s`" | nc -q0 my.graphite.com 2003
-</pre>
+The default video will include a timer and showing when the metrics happens, but you can turn that off using <code>--video.addTimer false</code>.
+
+<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/video-example.gif">
+
+
+## Browser Support
+Browsertime supports Firefox, Chrome, and Edge (Chromium version) on desktop and Safari on Mac OS. On Android we support Chrome and Firefox. Safari on iOS has limited support: there's no HAR file and no visual metrics. You can also use the Safari simulator on Mac OS.
+
+## How does it work
+Browsertime uses Selenium NodeJS to drive the browser. It starts the browser, load a URL, executes configurable Javascripts to collect metrics, collect a HAR file.
+
+To get the HAR from Firefox we use the [HAR Export Trigger](https://github.com/firebug/har-export-trigger) and Chrome we use [Chrome-HAR](https://github.com/sitespeedio/chrome-har) to parse the timeline log and generate the HAR file.
+
+## Contributing
+
+We welcome contributions from the community! Whether you're fixing a bug, adding a feature, or improving documentation, your help is valuable. Here’s how you can contribute:
+
+1. **Fork and Clone**: Fork the repository and clone it locally.
+2. **Create a Branch**: Create a new branch for your feature or bug fix.
+3. **Develop**: Make your changes. Ensure you adhere to the coding standards and write tests if applicable.
+4. **Test**: Run tests to ensure everything works as expected.
+5. **Submit a Pull Request**: Push your changes to your fork and submit a pull request to the main repository.
+
+Before contributing, please read our [CONTRIBUTING.md](CONTRIBUTING.md) for more detailed information on how to contribute.
+
+### Reporting Issues
+Found a bug or have a feature request? Please use the [GitHub Issues](https://github.com/sitespeedio/browsertime/issues) to report them. Be sure to check existing issues to avoid duplicates.
+
+## Community and Support
+
+Join our community! Whether you need help, want to share your experience, or discuss potential improvements, there are several ways to get involved:
+
+- **Slack**: Connect with fellow users and the development team on [Slack](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw).
+- **GitHub Issues**: For technical questions, feature requests, and bug reports, use our [GitHub issues](https://github.com/sitespeedio/browsertime/issues).
+
+We're excited to have you in our community and look forward to your contributions and interactions!
+
+
+## License
+[Apache License version 2.0](LICENSE).
 
 [travis-image]: https://img.shields.io/travis/sitespeedio/browsertime.svg?style=flat-square
 [travis-url]: https://travis-ci.org/sitespeedio/browsertime

package/lib/core/engine/command/chromeTrace.js

@@ -47,6 +47,8 @@ export class ChromeTrace {
 
         result.renderBlocking = render.renderBlocking;
 
+        // The trace do not know about the HAR file so we cannot backfill
+        /*
         if (!this.options.skipHar) {
           for (let harRequest of this.hars[this.index - 1].log.entries) {
             if (render.renderBlockingInfo[harRequest.request.url]) {
@@ -54,7 +56,8 @@ export class ChromeTrace {
                 render.renderBlockingInfo[harRequest.request.url];
             }
           }
-        }
+          
+        }*/
       }
     } else {
       throw new Error('Trace only works in Chrome');

package/lib/video/postprocessing/visualmetrics/visualMetrics.js

@@ -14,7 +14,7 @@ const SCRIPT_PATH = join(
   '..',
   '..',
   '..',
-  'browsertime',
+  'visualmetrics',
   'visualmetrics.py'
 );
 
@@ -24,7 +24,7 @@ const PORTABLE_SCRIPT_PATH = join(
   '..',
   '..',
   '..',
-  'browsertime',
+  'visualmetrics',
   'visualmetrics-portable.py'
 );
 

package/package.json

@@ -1,14 +1,14 @@
 {
   "name": "browsertime",
   "description": "Get performance metrics from your web page using Browsertime.",
-  "version": "19.1.0",
+  "version": "19.3.0",
   "bin": "./bin/browsertime.js",
   "type": "module",
   "dependencies": {
     "@cypress/xvfb": "1.2.4",
     "@devicefarmer/adbkit": "3.2.5",
-    "@sitespeed.io/chromedriver": "119.0.6045-105",
-    "@sitespeed.io/edgedriver": "119.0.2151-42",
+    "@sitespeed.io/chromedriver": "120.0.6099-71",
+    "@sitespeed.io/edgedriver": "120.0.2210-77",
     "@sitespeed.io/geckodriver": "0.33.0-c",
     "@sitespeed.io/throttle": "5.0.0",
     "@sitespeed.io/tracium": "0.3.3",
@@ -19,7 +19,7 @@
     "dayjs": "1.11.10",
     "execa": "8.0.1",
     "fast-stats": "0.0.6",
-    "find-up": "6.3.0",
+    "find-up": "7.0.0",
     "get-port": "7.0.0",
     "hasbin": "1.2.3",
     "intel": "1.2.0",
@@ -29,24 +29,24 @@
     "lodash.merge": "4.6.2",
     "lodash.pick": "4.4.0",
     "lodash.set": "4.3.2",
-    "selenium-webdriver": "4.15.0",
+    "selenium-webdriver": "4.16.0",
     "yargs": "17.7.2"
   },
   "optionalDependencies": {
     "jimp": "0.22.10"
   },
   "devDependencies": {
-    "ava": "5.3.1",
-    "eslint": "8.53.0",
-    "eslint-config-prettier": "9.0.0",
+    "ava": "6.0.1",
+    "eslint": "8.55.0",
+    "eslint-config-prettier": "9.1.0",
     "eslint-plugin-prettier": "5.0.1",
     "eslint-plugin-unicorn": "49.0.0",
-    "prettier": "3.0.3",
+    "prettier": "3.1.1",
     "serve-handler": "6.1.5",
     "serve": "14.2.1"
   },
   "engines": {
-    "node": ">=14.19.1"
+    "node": ">=18.0.0"
   },
   "files": [
     "CHANGELOG.md",

package/browsertime/__init__.py

@@ -1 +0,0 @@
-#

package/browsertime/test_visualmetrics.py

@@ -1,34 +0,0 @@
-import unittest
-import os
-
-from browsertime.visualmetrics import (
-    calculate_contentful_speed_index,
-    calculate_perceptual_speed_index,
-)
-
-HERE = os.path.dirname(__file__)
-
-
-class TestVisualMetrics(unittest.TestCase):
-    def setUp(self):
-        self.directory = "test_data"
-        images = os.listdir(os.path.join(HERE, self.directory))
-
-        def _p(image):
-            p = {}
-            p["time"] = int(image.split(".")[0].split("ms_")[-1])
-            return p
-
-        progress = [_p(image) for image in images if image.startswith("ms_")]
-        self.sorted_progress = sorted(progress,
-                                      key = lambda image: image['time'])
-
-    def test_calculate_contentful_speed_index(self):
-        res = calculate_contentful_speed_index(self.sorted_progress,
-                                               self.directory)
-        self.assertEqual(res[0], 1188)
-
-    def test_calculate_perceptual_speed_index(self):
-        res = calculate_perceptual_speed_index(self.sorted_progress,
-                                               self.directory)
-        self.assertEqual(res[0], 946)