sitespeed.io 30.9.0 → 30.11.0

package/.github/CONTRIBUTING.md

@@ -21,4 +21,4 @@ If you have an idea or something that you need sitespeed.io to handle, add an is
 
 Thanks for your time & support!
 
-Peter, Tobias & Jonathan
+Peter

package/.github/ISSUE_TEMPLATE/FEATURE_IMPROVEMENT.yml

@@ -10,6 +10,6 @@ body:
     id: your-idea
     attributes:
       label: Feature/improvement
-      description: You can also disuss new features/improvements in the [sitespeed.io Slack channel](https://sitespeedio.herokuapp.com/).
+      description: You can also disuss new features/improvements in the [sitespeed.io Slack channel](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw).
     validations:
       required: true

package/.github/ISSUE_TEMPLATE/QUESTION.yml

@@ -10,6 +10,6 @@ body:
     id: your-question
     attributes:
       label: Your question
-      description: Please double check that this question hasn't already answered in the [documentation](https://www.sitespeed.io/documentation/sitespeed.io/) (use the `Search`) or [old GitHub issues](https://github.com/sitespeedio/sitespeed.io/issues?q=is%3Aissue+is%3Aclosed). You can also ask questions in the [sitespeed.io Slack channel](https://sitespeedio.herokuapp.com/). And if your question is more like a bug, please [use the bug report form](https://github.com/sitespeedio/sitespeed.io/issues/new?assignees=&labels=bug&template=BUG_REPORT.yml)
+      description: Please double check that this question hasn't already answered in the [documentation](https://www.sitespeed.io/documentation/sitespeed.io/) (use the `Search`) or [old GitHub issues](https://github.com/sitespeedio/sitespeed.io/issues?q=is%3Aissue+is%3Aclosed). You can also ask questions in the [sitespeed.io Slack channel](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw). And if your question is more like a bug, please [use the bug report form](https://github.com/sitespeedio/sitespeed.io/issues/new?assignees=&labels=bug&template=BUG_REPORT.yml)
     validations:
       required: true

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # CHANGELOG - sitespeed.io  (we use [semantic versioning](https://semver.org))
 
+## 30.11.0 - 2023-12-21
+### Fixed
+* Moved functionality from co2 to sitespeed.io see [#4034](https://github.com/sitespeedio/sitespeed.io/pull/4034).
+
+### Added 
+* Upgraded the Docker container to use Firefox 121 [#4033](https://github.com/sitespeedio/sitespeed.io/pull/4033).
+* Updated to [Browsertime 19.3.0](https://github.com/sitespeedio/browsertime/blob/main/CHANGELOG.md#1931---2023-12-18) [#4031](https://github.com/sitespeedio/sitespeed.io/pull/4031).
+
+## 30.10.0 - 2023-12-15
+### Added
+* Add isSignificant metric to send to Graphite when using the compare plugin. Will send 0 if there's no siginificant change, else send the Cliffs delta to know how large the change is [#4030](https://github.com/sitespeedio/sitespeed.io/pull/4030).
+
 ## 30.9.0 - 2023-12-12
 ### Added
 * Chrome and Edge 120 in the Docker container [#4027](https://github.com/sitespeedio/sitespeed.io/pull/4027).

package/Dockerfile

@@ -1,4 +1,4 @@
-FROM sitespeedio/webbrowsers:chrome-120.0-firefox-120.0-edge-120.0
+FROM sitespeedio/webbrowsers:chrome-120.0-firefox-121.0-edge-120.0
 
 ARG TARGETPLATFORM=linux/amd64
 

package/HELP.md

@@ -2,7 +2,7 @@
 We want to make sitespeed.io one of the best web performance tool in the world and we hope you can help us!
 
 ## Developers
-We love to have more people involved in improving sitespeed.io. We are constantly working on adding more documentation and trying to write more information in the issues so its easier to help out. If there's an [issue](https://github.com/sitespeedio/sitespeed.io/issues) that you want to take on, ping the the issue and we can help you get started. You can also [join our Slack channel](https://sitespeedio.herokuapp.com/) if you need help!
+We love to have more people involved in improving sitespeed.io. We are constantly working on adding more documentation and trying to write more information in the issues so its easier to help out. If there's an [issue](https://github.com/sitespeedio/sitespeed.io/issues) that you want to take on, ping the the issue and we can help you get started. You can also [join our Slack channel](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw) if you need help!
 
 ## Designers
 As a designer there's a lot you can do: You can help us improve the HTML result pages. Maybe we should restructure the metrics  ? Or could the header/footer look better? You could also have look at [https://www.sitespeed.io](https://www.sitespeed.io/) where we have all the documentation. You can pretty much help us with everything, no one in the core team got design skills :)

package/LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2012,2013,2014,2015,2016,2017,2018,2019 Peter Hedenskog & Tobias Lidskog
+Copyright (c) 2012-2023 Peter Hedenskog
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -12,161 +12,222 @@
 [![Changelog #212][changelog-image]][changelog-url]
 
 
-[Website](https://www.sitespeed.io/) | [Documentation](https://www.sitespeed.io/documentation/sitespeed.io/) | [Changelog](https://github.com/sitespeedio/sitespeed.io/blob/main/CHANGELOG.md) | [Twitter](https://twitter.com/SiteSpeedio)
+[Website](https://www.sitespeed.io/) | [Documentation](https://www.sitespeed.io/documentation/sitespeed.io/) | [Changelog](https://github.com/sitespeedio/sitespeed.io/blob/main/CHANGELOG.md) | [Mastodon](https://fosstodon.org/@sitespeedio)
 
-## Welcome to the wonderful world of web performance!
 
-### What is sitespeed.io
+# Table of Contents
+- [Welcome to the Wonderful World of Web Performance](#welcome-to-the-wonderful-world-of-web-performance)
+  - [What is sitespeed.io?](#what-is-sitespeedio)
+  - [Why Choose sitespeed.io?](#why-choose-sitespeedio)
+  - [Dive Into Our Documentation](#dive-into-our-documentation)
+- [Installation](#installation)
+  - [Docker](#docker)
+  - [NodeJS](#nodejs)
+- [Usage](#usage)
+  - [Basic Usage](#basic-usage)
+  - [Advanced Configuration](#advanced-configuration)
+  - [Mobile Performance Testing](#mobile-performance-testing)
+- [Examples](#examples)
+- [Contributing](#contributing)
+- [Reporting Issues](#reporting-issues)
+- [Community and Support](#community-and-support)
+- [License](#license)
 
-**Sitespeed.io is a *complete web performance tool* that helps you measure the performance of your website. What exactly does that mean?**
 
-Before we start telling you all about sitespeed.io you should just try it out:
 
+# Welcome to the wonderful world of web performance!
 
-Using NodeJS/npm (you need Chrome, Firefox, Edge or Safari installed or Chrome/Firefox on Android)
+Welcome to `sitespeed.io`, the comprehensive web performance tool designed for everyone passionate about web speed. Whether you're a developer, a site owner, or just someone curious about website performance, `sitespeed.io` offers a powerful yet user-friendly way to analyze and optimize your website.
 
- ```bash
- npm i -g sitespeed.io && sitespeed.io https://www.sitespeed.io/
- ```
+## What is sitespeed.io?
 
-Or using Docker:
+`sitespeed.io` is more than just a tool; it's a complete solution for measuring, monitoring, and improving your website's performance. Built with simplicity and efficiency in mind, it enables you to:
 
- ```bash
- docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://www.sitespeed.io/
- ```
+- **Test Websites Using Real Browsers**: Simulate real user interactions and conditions to get accurate performance data.
+- **Speed Optimization Feedback**: Get detailed insights into your website's construction and discover opportunities for enhancing speed.
+- **Track Performance Over Time**: Monitor changes and trends in your website's performance to stay ahead of potential issues.
 
+Use cases on when to use `sitespeed.io`.
+- **Web performance audit**: Run performance tests from your terminal.
+- **Continuous Integration**: Detect web performance regressions early in the development cycle.
+- **Production Monitoring**: Monitor performance in production and get alerted on regressions.
 
+## Why Choose sitespeed.io?
 
-Ok, now you have tried it, let us tell you more about sitespeed.io. We think of a complete web performance tool as having three key capabilities:
+- **Open Source and Community-Driven**: Built and maintained by a community, ensuring continuous improvement and innovation.
+- **Versatile and Extensible**: Whether you're running a simple blog or a complex e-commerce site, `sitespeed.io` adapts to your needs.
+- **Seamless Integration**: Easily incorporate `sitespeed.io` into your development workflow, continuous integration systems, and monitoring setups.
 
- - It test web sites using real browsers, simulating real users connectivity and collect important user centric metrics like Speed Index and First Visual Render.
- - It analyse how your page is built and give feedback how you can make it faster for the end user.
- - It collect and keep data how your pages is built so you easily can track changes.
+## Dive Into Our Documentation
 
-**What is sitespeed.io good for?**
+We've put countless hours into our [documentation](https://www.sitespeed.io/documentation/sitespeed.io/) to help you get the most out of `sitespeed.io`. From installation guides to advanced usage scenarios, our documentation is a treasure trove of information and tips.
 
-It is usually used in two different areas:
 
- - Running in your continuous integration to find web performance regressions early: on commits or when you move code to your test environment
- - Monitoring your performance in production, alerting on regressions.
+# Installation
 
-To understand how sitespeed.io does these things, let's talk about how it works.
+Getting started with `sitespeed.io` is straightforward. You can install it using Docker or NodeJS, depending on your preference and setup. Follow these simple steps to begin optimizing your website's performance.
 
-First a few key concepts:
+## Docker
 
- - Sitespeed.io is built upon a couple of other Open Source tools in the sitespeed.io suite.
- - [Browsertime](https://github.com/sitespeedio/browsertime) is the tool that drives the browser and collect metrics.
- - [The Coach](https://github.com/sitespeedio/coach) knows how to build fast websites and analyse your page and give you feedback what you should change.
- - Visual Metrics is metrics collected from a video recording of the browser screen.
- - Everything in sitespeed.io is a [plugin](https://www.sitespeed.io/documentation/sitespeed.io/plugins/) and they communicate by passing messages on a queue.
+Using Docker is the easiest way to get started with `sitespeed.io`, especially if you don't want to handle dependencies manually. Run the following command to use `sitespeed.io` in a Docker container:
 
-When you as user choose to test a URL, this is what happens on a high level:
+ ```bash
+ docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://www.sitespeed.io/
+ ```
 
- 1. sitespeed.io starts and initialise all configured plugins.
- 2. The URL is passed around the plugins through the queue.
-    1. Browsertime gets the URL and opens the browser.
-    2. It starts to record a video of the browser screen.
-    3. The browser access the URL.
-    4. When the page is finished, Browsertime takes a screenshot of the page.
-    5. Then run some JavaScripts to analyse the page (using Coach and Browsertime scripts).
-    6. Stop the video and close the browser.
-    7. Analyse the video to get Visual Metrics like First Visual Change and Speed Index.
-    8. Browsertime passes all metrics and data on the queue so other plugins can use it.
- 3. The HTML/Graphite/InfluxDB plugin collects the metrics in queue.
- 4. When all URLs are tested, sitespeed sends a message telling plugins to summarise the metrics and then render it.
- 5. Plugins pickup the render message and the HTML plugin writes the HTML to disk.
+This command pulls the latest sitespeed.io Docker image and runs a test on the sitespeed.io website. The **-v "$(pwd)":/sitespeed.io** part mounts the current directory into the container, allowing you to easily access test results.
 
- ## Try it out
+## NodeJS
 
- Using Docker (use latest Docker):
+If you prefer installing `sitespeed.io` as an npm package, ensure you have NodeJS installed on your system. Then, install `sitespeed.io` globally using npm:
 
  ```bash
- docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://www.sitespeed.io/
+ npm i -g sitespeed.io
  ```
 
- Or install using npm:
+ After installation, you can start using sitespeed.io by running:
+  
+```bash
+sitespeed.io https://www.example.com
+```
+
+Replace https://www.example.com with the URL you wish to test. Note that using NodeJS might require additional dependencies like FFmpeg and Python. Detailed installation instructions for these dependencies can be found [here](https://www.sitespeed.io/documentation/sitespeed.io/installation/).
+
+Choose the method that best suits your environment and get ready to dive into web performance optimization with sitespeed.io!
+
+# Usage
+
+`sitespeed.io` is tailored to be user-friendly, making web performance testing accessible regardless of your technical expertise. Here's a straightforward guide to help you begin your web performance optimization journey.
+
+## Basic Usage
+
+To start testing your website, simply run `sitespeed.io` with the URL of the site you want to analyze. For example:
 
  ```bash
- npm i -g sitespeed.io
+sitespeed.io https://www.example.com --browser chrome -n 5
  ```
 
- Or clone the repo and test the latest changes:
+This command tests https://www.example.com using Chrome and performs 5 iterations of the test. This approach helps in obtaining a more accurate median performance measurement by testing the site multiple times.
 
- ```bash
- git clone https://github.com/sitespeedio/sitespeed.io.git
- cd sitespeed.io
- npm install
- bin/sitespeed.js --help
- bin/sitespeed.js https://www.sitespeed.io/
+## Advanced Configuration
+
+sitespeed.io offers a wide range of configuration options to tailor the tests to your specific needs. You can specify different browsers, adjust connectivity settings, and much more. For a comprehensive list of all available options, visit our [configuration documentation](https://www.sitespeed.io/documentation/sitespeed.io/configuration/).
+
+Additionally, for a quick overview of all command-line options, you can run:
+
+```bash
+sitespeed.io --help
 ```
 
-## More details
+This command displays all the available flags and settings you can use with sitespeed.io, helping you fine-tune your performance testing to fit your unique requirements.
 
-Using sitespeed.io you can:
-* Test your web site against Web Performance best practices using the [Coach](https://github.com/sitespeedio/coach).
-* Collect Navigation Timing API, User Timing API and Visual Metrics from Firefox/Chrome using [Browsertime](https://github.com/sitespeedio/browsertime).
-* Run your custom-made JavaScript and collect whichever metric(s) you need.
-* Test one or multiple pages, across one or many runs to get more-accurate metrics.
-* Create HTML-result pages or store the metrics in Graphite.
-* Write your own plugins that can do whatever tests you want/need.
+Whether you're running a quick check or a detailed analysis, sitespeed.io provides the flexibility and power you need to deeply understand and improve your website's performance.
 
-See all the latest changes in the [Changelog](https://github.com/sitespeedio/sitespeed.io/blob/main/CHANGELOG.md).
+## Mobile Performance Testing
 
-Checkout our example [dashboard.sitespeed.io](https://dashboard.sitespeed.io/)
+In today's mobile-first world, ensuring your website performs optimally on smartphones and tablets is crucial. With `sitespeed.io`, you can simulate and analyze the performance of your website on mobile devices, helping you understand and improve the user experience for mobile audiences.
 
-A summary report in HTML:
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/start-readme.jpg">
+### Why Test on Mobile?
 
-Individual page report:
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/start-url-readme.jpg">
+- **User Experience**: A significant portion of web traffic comes from mobile devices. Testing on mobile ensures your site is optimized for these users.
+- **Search Engine Ranking**: Search engines like Google prioritize mobile-friendly websites in their search results.
+- **Performance Insights**: Mobile devices have different performance characteristics than desktops, such as CPU limitations and network variability.
 
-Collected metrics from a URL in Graphite/Grafana:
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/grafana-readme.jpg">
+### How sitespeed.io Helps
 
-And look at trends in Grafana:
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/grafana-trends-readme.jpg">
+- **Real Browser Testing**: Simulate mobile browsers to get accurate performance data as experienced by real users.
+- **Device-Specific Metrics**: Gain insights into how your site performs on different mobile devices and networks.
+- **Responsive Design Analysis**: Test how well your site adapts to various screen sizes and orientations.
 
-Video - easiest using Docker. This gif is optimized, the quality is much better IRL:
+### Getting Started
 
-<img src="https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/barack.gif">
+To start testing your website’s mobile performance, you need to setup your mobile phone for testing. We got [documentation for setting up your Android phone](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#test-on-android) and [iOS](https://www.sitespeed.io/documentation/sitespeed.io/mobile-phones/#test-on-ios). 
 
-## Test using WebPageReplay
-We have a special Docker container that comes with [WebPageReplay](https://github.com/catapult-project/catapult/blob/main/web_page_replay_go/README.md) installed. This is a really early alpha release but we think you should try it out.
+When your setup is ready, you can run tests on your Android phone.
 
-WebPageReplay will let you replay your page locally (getting rid of server latency etc) and makes it easier to have stable metrics and find front end regressions.
+```bash
+sitespeed.io https://www.example.com --android
+```
 
-It works like this:
-1. WebPageReplay is started in record mode
-2. Browsertime access the URLs you choose one time (so it is recorded)
-3. WebPageReplay is closed down
-4. WebPageReplay in replay mode is started
-5. Sitespeed.io (using Browsertime) test the URL so many times you choose
-6. WebPageReplay in replay mode is closed down
+## Examples
 
-You can change latency by setting a Docker environment variable. Use REPLAY to turn on the replay functionality.
+`sitespeed.io` provides insightful HTML reports that help you visualize and understand your website's performance. Here are some examples to illustrate what you can achieve:
 
-Default browser is Chrome:
+### Summary Report
 
-```
-docker run --cap-add=NET_ADMIN --rm -v "$(pwd)":/sitespeed.io -e REPLAY=true -e LATENCY=100 sitespeedio/sitespeed.io -n 5 -b chrome https://en.wikipedia.org/wiki/Barack_Obama
-```
+Here's an example of a summary report in HTML, offering a comprehensive overview of your site's performance metrics:
 
-Use Firefox:
+![Summary Report](https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/start-readme.jpg)
 
-```
-docker run --cap-add=NET_ADMIN --rm -v "$(pwd)":/sitespeed.io -e REPLAY=true -e LATENCY=100 sitespeedio/sitespeed.io -n 11 -b firefox https://en.wikipedia.org/wiki/Barack_Obama
-```
+This report includes key performance indicators like load times, page size, and request counts, giving you a quick snapshot of your site’s overall health.
+
+### Individual Page Report
+
+For more detailed analysis, here's an individual page report:
+
+![Individual Page Report](https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/start-url-readme.jpg)
+
+This report dives deeper into a single page's performance, providing metrics on aspects like scripting, rendering, and network activity, crucial for pinpointing specific areas of improvement.
+
+### Performance Monitoring Dashboard
+
+To monitor your website’s performance over time, check out our live setup at [dashboard.sitespeed.io](https://dashboard.sitespeed.io/), which integrates `sitespeed.io` with Graphite and Grafana.
+
+#### Metrics in Graphite/Grafana
+
+Collected metrics from a URL visualized in Graphite/Grafana:
+
+![Graphite/Grafana Metrics](https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/grafana-readme.jpg)
+
+This setup allows for continuous tracking of performance, helping you identify trends and potential issues.
+
+#### Trends in Grafana
+
+Trends over time in Grafana provide a long-term view of your site's performance:
+
+![Grafana Trends](https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/grafana-trends-readme.jpg)
+
+With these insights, you can make informed decisions about optimizations and track the impact of changes you make.
+
+### Video Performance Analysis
+
+For visual feedback, `sitespeed.io` can generate videos, making it easier to see how your site loads in real-time. Here's an sample video:
+
+![Video Analysis](https://raw.githubusercontent.com/sitespeedio/sitespeed.io/main/docs/img/barack.gif)
+
+Video analysis is most easily done using Docker and offers a unique perspective on user experience, highlighting areas that need attention.
+
+
+# Contributing
+
+We welcome contributions from the community! Whether you're fixing a bug, adding a feature, or improving documentation, your help is valuable. Here’s how you can contribute:
+
+1. **Create an Issue**: Create an issue and discuss with us how to implement the issue.
+2. **Fork and Clone**: Fork the repository and clone it locally.
+3. **Create a Branch**: Create a new branch for your feature or bug fix.
+4. **Develop**: Make your changes. Ensure you adhere to the coding standards and write tests if applicable.
+5. **Test**: Run tests to ensure everything works as expected.
+6. **Submit a Pull Request**: Push your changes to your fork and submit a pull request to the main repository.
+
+Before contributing, please read our [CONTRIBUTING.md](.gitub/CONTRIBUTING.md) for more detailed information on how to contribute.
+
+# Reporting Issues
+Found a bug or have a feature request? Please use the [GitHub Issues](https://github.com/sitespeedio/sitespeed.io/issues) to report them. Be sure to check existing issues to avoid duplicates.
+
+# Community and Support
 
-<hr>
+Join our community! Whether you need help, want to share your experience, or discuss potential improvements, there are several ways to get involved:
 
-# Sponsors
+- **Slack**: Connect with fellow users and the development team on [Slack](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw).
+- **GitHub Issues**: For technical questions, feature requests, and bug reports, use our [GitHub issues](https://github.com/sitespeedio/sitespeed.io/issues).
+- **RSS/Changelog**: Latest releases and information can always be found in our [RSS feed](https://github.com/sitespeedio/sitespeed.io/releases.atom) and in our [changelog](https://github.com/sitespeedio/sitespeed.io/blob/main/CHANGELOG.md).
+- **Mastodon**: Follow us on Mastodon [https://fosstodon.org/@sitespeedio](https://fosstodon.org/@sitespeedio).
 
-<a href="https://www.macstadium.com"><img src="https://uploads-ssl.webflow.com/5ac3c046c82724970fc60918/5c019d917bba312af7553b49_MacStadium-developerlogo.png"></a>
+We're excited to have you in our community and look forward to your contributions and interactions!
 
-We have a Mac Mini sponsored by [MacStadium](https://www.macstadium.com) and you too can sponsor us to help us keep sitespeed.io running and making sure we can have the best test setup as possible. Read our [sponsor page](https://github.com/sponsors/soulgalore) for more info.
+# License
+[The MIT License (MIT)](LICENSE).
 
-[travis-image]: https://img.shields.io/travis/sitespeedio/sitespeed.io.svg?style=flat-square
-[travis-url]: https://travis-ci.org/sitespeedio/sitespeed.io
 [stars-url]: https://github.com/sitespeedio/sitespeed.io/stargazers
 [stars-image]: https://img.shields.io/github/stars/sitespeedio/sitespeed.io.svg?style=flat-square
 [downloads-image]: https://img.shields.io/npm/dt/sitespeed.io.svg?style=flat-square

package/ROADMAP.md

@@ -2,6 +2,6 @@
 
 This roadmap is the plan for the core team, priorities can and and will change over time. This will give you a view of our current vision and plan.
 
-### The rest of 2020
+### 2024
 
-We gonna focus refactor the core engine [Browsertime](https://github.com/sitespeedio/browsertime) and fine tune the [Coach](https://github.com/sitespeedio/coach).
+Lets get the online version if sitespeed.io up and running.

package/SUPPORT.md

@@ -2,7 +2,7 @@
 ------------------
 First, check the official [sitespeed.io documentation](https://www.sitespeed.io/documentation/).
 
-If you require further help or support then check [new and old issues on GitHub](https://github.com/sitespeedio/sitespeed.io/issues) or join the [sitespeed.io Slack](https://sitespeedio.herokuapp.com).
+If you require further help or support then check [new and old issues on GitHub](https://github.com/sitespeedio/sitespeed.io/issues) or join the [sitespeed.io Slack](https://join.slack.com/t/sitespeedio/shared_invite/zt-296jzr7qs-d6DId2KpEnMPJSQ8_R~WFw).
 
  **Please note:**
 - The sitespeed.io project uses GitHub for tracking bugs and feature requests.

package/lib/cli/cli.js

@@ -1558,18 +1558,18 @@ export async function parseCommandLine() {
     })
     .option('slack.type', {
       describe:
-        'Send summary for a run, metrics from all URLs, only on errors or all to Slack.',
+        'Send summary for a tested URL, metrics from all URLs (summary), only on errors from your tests or all to Slack.',
       default: slackConfig.type,
       choices: ['summary', 'url', 'error', 'all'],
       group: 'Slack'
     })
     .option('slack.limitWarning', {
-      describe: 'The limit to get a warning in Slack using the limitMetric',
+      describe: 'The limit to get a warning in Slack using the limitMetric.',
       default: slackConfig.limitWarning,
       group: 'Slack'
     })
     .option('slack.limitError', {
-      describe: 'The limit to get a error in Slack using the limitMetric',
+      describe: 'The limit to get a error in Slack using the limitMetric.',
       default: slackConfig.limitError,
       group: 'Slack'
     })

package/lib/plugins/compare/helper.js

@@ -356,6 +356,10 @@ export function getMetrics(data) {
   };
 }
 
+export function getIsSignificant(u, cliffs) {
+  return u < 0.05 ? cliffs : 0;
+}
+
 export function cliffsDelta(x, y) {
   const n_x = x.length;
   const n_y = y.length;

package/lib/plugins/compare/index.js

@@ -11,7 +11,8 @@ import {
   getStatistics,
   runStatisticalTests,
   getMetrics,
-  cliffsDelta
+  cliffsDelta,
+  getIsSignificant
 } from './helper.js';
 import { getBaseline, saveBaseline } from './baseline.js';
 
@@ -39,7 +40,16 @@ const DEFAULT_METRICS_PAGESUMMARY = [
   'metrics.renderBlocking.*.statisticalTestU',
   'metrics.elementTimings.*.statisticalTestU',
   'metrics.userTimings.*.statisticalTestU',
-  'metrics.extras.*.statisticalTestU'
+  'metrics.extras.*.statisticalTestU',
+  'metrics.timings.*.isSignificant',
+  'metrics.cpu.*.isSignificant',
+  'metrics.cdp.*.isSignificant',
+  'metrics.visualMetrics.*.isSignificant',
+  'metrics.googleWebVitals.*.isSignificant',
+  'metrics.renderBlocking.*.isSignificant',
+  'metrics.elementTimings.*.isSignificant',
+  'metrics.userTimings.*.isSignificant',
+  'metrics.extras.*.isSignificant'
 ];
 
 export default class ComparePlugin extends SitespeedioPlugin {
@@ -174,6 +184,7 @@ export default class ComparePlugin extends SitespeedioPlugin {
                 baselineMetrics[group][metricName].getValues()
               );
 
+              const cliffs = cliffsDelta(currentStats.data, baselineStats.data);
               finalResult[group][metricName] = {
                 current: {
                   stdev: currentStats.stddev(),
@@ -188,7 +199,8 @@ export default class ComparePlugin extends SitespeedioPlugin {
                   values: baselineStats.data
                 },
                 statisticalTestU: result['p-value'],
-                cliffsDelta: cliffsDelta(currentStats.data, baselineStats.data)
+                cliffsDelta: cliffs,
+                isSignificant: getIsSignificant(result['p-value'], cliffs)
               };
             }
           }

package/lib/plugins/compare/pug/index.pug

@@ -74,8 +74,8 @@ table
             td
               a(href=createGraphLink(groupName, metricName)) 
                 b #{groupName + '.' + metricName}
-            if values.statisticalTestU === "N/A"
-              td N/A
+            if values.statisticalTestU === "N/A" || values.statisticalTestU === "Datasets are identical"
+              td #{values.statisticalTestU}
             else
               td #{h.decimals(values.statisticalTestU)}
             td #{h.decimals(values.baseline.mean)}
@@ -84,7 +84,7 @@ table
             td #{h.decimals(values.current.median)}
             td #{h.decimals(values.baseline.stdev)}
             td #{h.decimals(values.current.stdev)}
-            if values.statisticalTestU === "N/A"
+            if values.statisticalTestU === "N/A" || values.statisticalTestU === "Datasets are identical"
               td No Test Conducted
             else
               td #{values.statisticalTestU < 0.05 ? 'Yes - ' + cliffDeltaHelper(values.cliffsDelta) : 'No'}
@@ -96,7 +96,7 @@ each metricGroup, groupName in compare.metrics
   each values, metricName in metricGroup
     - var fullMetricName = groupName + '.' + metricName
     - var metricId = fullMetricName.replace(/\./g, '_')
-    h3 #{fullMetricName} #{values.statisticalTestU === "N/A"? '' : values.statisticalTestU < 0.05 ? '(significant change)' : ''}
+    h3 #{fullMetricName} #{values.statisticalTestU === "N/A" || values.statisticalTestU === "Datasets are identical" ? '' : values.statisticalTestU < 0.05 ? '(significant change)' : ''}
     .ct-chart(id=`chart-${metricId}`)
       .ct-legend
         span.ct-legend-item

package/lib/plugins/compare/statistical.py

@@ -2,16 +2,17 @@ import sys
 import json
 from scipy.stats import wilcoxon, mannwhitneyu
 
-
 def has_variability(sample):
     """Check if the sample has more than one unique value."""
     return len(set(sample)) > 1
 
-
 def perform_test(test_type, baseline, current, **kwargs):
     """Perform the statistical test based on the test type."""
     if not has_variability(baseline) or not has_variability(current):
-        return None, "No variability"
+        if baseline == current:
+            return None, "Datasets are identical"
+        else:
+            return None, "No variability"
 
     if test_type == 'wilcoxon':
         return wilcoxon(current, baseline, **kwargs)
@@ -20,7 +21,6 @@ def perform_test(test_type, baseline, current, **kwargs):
     else:
         raise ValueError("Invalid test type. Choose 'wilcoxon' or 'mannwhitneyu'.")
 
-
 input_data = json.loads(sys.stdin.read())
 options = input_data['options']
 test_type = options.pop('test_type')
@@ -31,8 +31,8 @@ for group_name, metrics in input_data['metrics'].items():
     group_results = {}
     for metric_name, metric_data in metrics.items():
         stat, p = perform_test(test_type, metric_data['baseline'], metric_data['current'], **options)
-        if p == "No variability":
-            group_results[metric_name] = {'statistic': "N/A", 'p-value': "N/A"}
+        if p == "No variability" or p == "Datasets are identical":
+            group_results[metric_name] = {'statistic': "N/A", 'p-value': p}
         else:
             group_results[metric_name] = {'statistic': stat, 'p-value': p}
     final_results[group_name] = group_results

package/lib/plugins/sustainable/helper.js

@@ -0,0 +1,104 @@
+// Moved from the co2.js project
+// see https://github.com/thegreenwebfoundation/co2.js/issues/182
+
+export function perDomain(pageXray, greenDomains, CO2) {
+  const co2PerDomain = [];
+  for (let domain of Object.keys(pageXray.domains)) {
+    let co2;
+    co2 =
+      greenDomains && greenDomains.includes(domain)
+        ? CO2.perByte(pageXray.domains[domain].transferSize, true)
+        : CO2.perByte(pageXray.domains[domain].transferSize);
+    co2PerDomain.push({
+      domain,
+      co2,
+      transferSize: pageXray.domains[domain].transferSize
+    });
+  }
+  co2PerDomain.sort((a, b) => b.co2 - a.co2);
+
+  return co2PerDomain;
+}
+
+export function perPage(pageXray, green, CO2) {
+  // Accept an xray object, and if we receive a boolean as the second
+  // argument, we assume every request we make is sent to a server
+  // running on renwewable power.
+
+  // if we receive an array of domains, return a number accounting the
+  // reduced CO2 from green hosted domains
+
+  const domainCO2 = perDomain(pageXray, green, CO2);
+  let totalCO2 = 0;
+  for (let domain of domainCO2) {
+    totalCO2 += domain.co2;
+  }
+  return totalCO2;
+}
+
+export function perContentType(pageXray, greenDomains, CO2) {
+  const co2PerContentType = {};
+  for (let asset of pageXray.assets) {
+    const domain = new URL(asset.url).domain;
+    const transferSize = asset.transferSize;
+    const co2ForTransfer = CO2.perByte(
+      transferSize,
+      greenDomains && greenDomains.includes(domain)
+    );
+    const contentType = asset.type;
+    if (!co2PerContentType[contentType]) {
+      co2PerContentType[contentType] = { co2: 0, transferSize: 0 };
+    }
+    co2PerContentType[contentType].co2 += co2ForTransfer;
+    co2PerContentType[contentType].transferSize += transferSize;
+  }
+  // restructure and sort
+  const all = [];
+  for (let type of Object.keys(co2PerContentType)) {
+    all.push({
+      type,
+      co2: co2PerContentType[type].co2,
+      transferSize: co2PerContentType[type].transferSize
+    });
+  }
+  all.sort((a, b) => b.co2 - a.co2);
+  return all;
+}
+
+export function getDirtiestResources(pageXray, greenDomains, CO2) {
+  const allAssets = [];
+  for (let asset of pageXray.assets) {
+    const domain = new URL(asset.url).domain;
+    const transferSize = asset.transferSize;
+    const co2ForTransfer = CO2.perByte(
+      transferSize,
+      greenDomains && greenDomains.includes(domain)
+    );
+    allAssets.push({ url: asset.url, co2: co2ForTransfer, transferSize });
+  }
+  allAssets.sort((a, b) => b.co2 - a.co2);
+
+  return allAssets.slice(0, allAssets.length > 10 ? 10 : allAssets.length);
+}
+
+export function perParty(pageXray, greenDomains, CO2) {
+  let firstParty = 0;
+  let thirdParty = 0;
+  // calculate co2 per first/third party
+  const firstPartyRegEx = pageXray.firstPartyRegEx;
+  for (let d of Object.keys(pageXray.domains)) {
+    // eslint-disable-next-line unicorn/prefer-regexp-test
+    if (d.match(firstPartyRegEx)) {
+      thirdParty += CO2.perByte(
+        pageXray.domains[d].transferSize,
+        greenDomains && greenDomains.includes(d)
+      );
+    } else {
+      firstParty += CO2.perByte(
+        pageXray.domains[d].transferSize,
+        greenDomains && greenDomains.includes(d)
+      );
+    }
+  }
+  return { firstParty, thirdParty };
+}

package/lib/plugins/sustainable/index.js

@@ -8,6 +8,13 @@ import intel from 'intel';
 import { co2, hosting } from '@tgwf/co2';
 import { SitespeedioPlugin } from '@sitespeed.io/plugin';
 import { Aggregator } from './aggregator.js';
+import {
+  getDirtiestResources,
+  perParty,
+  perContentType,
+  perPage,
+  perDomain
+} from './helper.js';
 
 const fsp = fs.promises;
 const __dirname = fileURLToPath(new URL('.', import.meta.url));
@@ -129,7 +136,7 @@ export default class SustainablePlugin extends SitespeedioPlugin {
         }
 
         const CO2 = new co2(this.sustainableOptions.model);
-        const co2PerDomain = CO2.perDomain(message.data, hostingGreenCheck);
+        const co2PerDomain = perDomain(message.data, hostingGreenCheck, CO2);
         const baseDomain = message.data.baseDomain;
 
         const hostingInfo = {
@@ -142,21 +149,23 @@ export default class SustainablePlugin extends SitespeedioPlugin {
           hostingInfo.green = true;
         }
 
-        const co2PerParty = CO2.perParty(message.data, hostingGreenCheck);
+        const co2PerParty = perParty(message.data, hostingGreenCheck, CO2);
         // Fetch the resources with the largest CO2 impact. ie,
         // the resources to optimise, host somewhere green, or contact
         // a supplier about
-        const dirtiestResources = CO2.dirtiestResources(
+        const dirtiestResources = getDirtiestResources(
           message.data,
-          hostingGreenCheck
+          hostingGreenCheck,
+          CO2
         );
 
-        const co2PerContentType = CO2.perContentType(
+        const co2PerContentType = perContentType(
           message.data,
-          hostingGreenCheck
+          hostingGreenCheck,
+          CO2
         );
 
-        const co2PerPageView = CO2.perPage(message.data, hostingGreenCheck);
+        const co2PerPageView = perPage(message.data, hostingGreenCheck, CO2);
 
         const totalCO2 = this.sustainableOptions.pageViews
           ? this.sustainableOptions.pageViews * co2PerPageView

package/npm-shrinkwrap.json

@@ -1,21 +1,21 @@
 {
   "name": "sitespeed.io",
-  "version": "30.9.0",
+  "version": "30.11.0",
   "lockfileVersion": 2,
   "requires": true,
   "packages": {
     "": {
       "name": "sitespeed.io",
-      "version": "30.9.0",
+      "version": "30.11.0",
       "license": "MIT",
       "dependencies": {
         "@google-cloud/storage": "6.9.5",
         "@influxdata/influxdb-client": "1.33.2",
         "@sitespeed.io/plugin": "0.0.6",
-        "@tgwf/co2": "0.13.6",
+        "@tgwf/co2": "0.13.10",
         "aws-sdk": "2.1327.0",
         "axe-core": "4.8.2",
-        "browsertime": "19.2.0",
+        "browsertime": "19.3.1",
         "cli-color": "2.0.3",
         "coach-core": "8.0.2",
         "concurrent-queue": "7.0.2",
@@ -943,9 +943,9 @@
       }
     },
     "node_modules/@sitespeed.io/chromedriver": {
-      "version": "119.0.6045-105",
-      "resolved": "https://registry.npmjs.org/@sitespeed.io/chromedriver/-/chromedriver-119.0.6045-105.tgz",
-      "integrity": "sha512-DfQQaqTB28e05kG3CWjC9OWKeNTWiqgu5cl6CvYQsd2MTDDDRUQ0a+VZ8KTSrRx6xZCsTBgzZK2kNBNiMvNH8w==",
+      "version": "120.0.6099-71",
+      "resolved": "https://registry.npmjs.org/@sitespeed.io/chromedriver/-/chromedriver-120.0.6099-71.tgz",
+      "integrity": "sha512-45hQNdnZRisMobAuq+tPdwUV0Dr69LBNalm7FY33fTFf9koqOHWbJf+s2GKeEoSfvxGo++Y0UijlsuBJzQpH8w==",
       "hasInstallScript": true,
       "dependencies": {
         "node-downloader-helper": "2.1.9",
@@ -964,9 +964,9 @@
       }
     },
     "node_modules/@sitespeed.io/edgedriver": {
-      "version": "119.0.2151-42",
-      "resolved": "https://registry.npmjs.org/@sitespeed.io/edgedriver/-/edgedriver-119.0.2151-42.tgz",
-      "integrity": "sha512-+jGP9BmWgh/yoNcJKyiYP0anF0m2H6+cjk1MaHvzgkIdrFMVfJQIN9+tmwCBiN4Ave52IHjDdHhEjK7B+SWvrA==",
+      "version": "120.0.2210-77",
+      "resolved": "https://registry.npmjs.org/@sitespeed.io/edgedriver/-/edgedriver-120.0.2210-77.tgz",
+      "integrity": "sha512-vltszaEdgi667M8taVXv9DCTNP08NQtEnyof5/ypcMeFnBCJNGX5lsXzBjuRyVMaaZiU4LXXWQpj90VD12VEfg==",
       "hasInstallScript": true,
       "dependencies": {
         "node-downloader-helper": "2.1.7",
@@ -1026,9 +1026,9 @@
       }
     },
     "node_modules/@tgwf/co2": {
-      "version": "0.13.6",
-      "resolved": "https://registry.npmjs.org/@tgwf/co2/-/co2-0.13.6.tgz",
-      "integrity": "sha512-uV8y2voiAb9iQwzK+5dTFLNBu5P5/oCyMAvqD3/+X74sMCc/N2+4MhIY9JmpO1gBQykE647dd4l/40GiE7cKSw==",
+      "version": "0.13.10",
+      "resolved": "https://registry.npmjs.org/@tgwf/co2/-/co2-0.13.10.tgz",
+      "integrity": "sha512-iQZMYdLP+l1Kyi03EEE4jAZgzlAA8HRAbfgT3GgN6sUgbSoNS/UHijICUqgkum9aHIkDSOrtwkgbB/x3Gc/cQA==",
       "engines": {
         "node": ">=14.0.0"
       }
@@ -1759,14 +1759,14 @@
       }
     },
     "node_modules/browsertime": {
-      "version": "19.2.0",
-      "resolved": "https://registry.npmjs.org/browsertime/-/browsertime-19.2.0.tgz",
-      "integrity": "sha512-4BM5AeiO0ZNkQie3mBIW3nrcgvnTBxR3gZmtJ6q96k3gltzqPoH8Hhz9onn7wbb8U+63qIK5IBmEguG2tpIobQ==",
+      "version": "19.3.1",
+      "resolved": "https://registry.npmjs.org/browsertime/-/browsertime-19.3.1.tgz",
+      "integrity": "sha512-MjS3YvbfFMMzZcKNZjdzRMxqIC/fUiNgJ8GiLgOhit1OmqaPPFSHjWv5vZxyjIl3zzzMVgQDv+u3MGDrLGSkPw==",
       "dependencies": {
         "@cypress/xvfb": "1.2.4",
         "@devicefarmer/adbkit": "3.2.5",
-        "@sitespeed.io/chromedriver": "119.0.6045-105",
-        "@sitespeed.io/edgedriver": "119.0.2151-42",
+        "@sitespeed.io/chromedriver": "120.0.6099-71",
+        "@sitespeed.io/edgedriver": "120.0.2210-77",
         "@sitespeed.io/geckodriver": "0.33.0-c",
         "@sitespeed.io/throttle": "5.0.0",
         "@sitespeed.io/tracium": "0.3.3",
@@ -9985,9 +9985,9 @@
       }
     },
     "@sitespeed.io/chromedriver": {
-      "version": "119.0.6045-105",
-      "resolved": "https://registry.npmjs.org/@sitespeed.io/chromedriver/-/chromedriver-119.0.6045-105.tgz",
-      "integrity": "sha512-DfQQaqTB28e05kG3CWjC9OWKeNTWiqgu5cl6CvYQsd2MTDDDRUQ0a+VZ8KTSrRx6xZCsTBgzZK2kNBNiMvNH8w==",
+      "version": "120.0.6099-71",
+      "resolved": "https://registry.npmjs.org/@sitespeed.io/chromedriver/-/chromedriver-120.0.6099-71.tgz",
+      "integrity": "sha512-45hQNdnZRisMobAuq+tPdwUV0Dr69LBNalm7FY33fTFf9koqOHWbJf+s2GKeEoSfvxGo++Y0UijlsuBJzQpH8w==",
       "requires": {
         "node-downloader-helper": "2.1.9",
         "node-stream-zip": "1.15.0"
@@ -10001,9 +10001,9 @@
       }
     },
     "@sitespeed.io/edgedriver": {
-      "version": "119.0.2151-42",
-      "resolved": "https://registry.npmjs.org/@sitespeed.io/edgedriver/-/edgedriver-119.0.2151-42.tgz",
-      "integrity": "sha512-+jGP9BmWgh/yoNcJKyiYP0anF0m2H6+cjk1MaHvzgkIdrFMVfJQIN9+tmwCBiN4Ave52IHjDdHhEjK7B+SWvrA==",
+      "version": "120.0.2210-77",
+      "resolved": "https://registry.npmjs.org/@sitespeed.io/edgedriver/-/edgedriver-120.0.2210-77.tgz",
+      "integrity": "sha512-vltszaEdgi667M8taVXv9DCTNP08NQtEnyof5/ypcMeFnBCJNGX5lsXzBjuRyVMaaZiU4LXXWQpj90VD12VEfg==",
       "requires": {
         "node-downloader-helper": "2.1.7",
         "node-stream-zip": "1.15.0"
@@ -10048,9 +10048,9 @@
       }
     },
     "@tgwf/co2": {
-      "version": "0.13.6",
-      "resolved": "https://registry.npmjs.org/@tgwf/co2/-/co2-0.13.6.tgz",
-      "integrity": "sha512-uV8y2voiAb9iQwzK+5dTFLNBu5P5/oCyMAvqD3/+X74sMCc/N2+4MhIY9JmpO1gBQykE647dd4l/40GiE7cKSw=="
+      "version": "0.13.10",
+      "resolved": "https://registry.npmjs.org/@tgwf/co2/-/co2-0.13.10.tgz",
+      "integrity": "sha512-iQZMYdLP+l1Kyi03EEE4jAZgzlAA8HRAbfgT3GgN6sUgbSoNS/UHijICUqgkum9aHIkDSOrtwkgbB/x3Gc/cQA=="
     },
     "@tokenizer/token": {
       "version": "0.3.0",
@@ -10601,14 +10601,14 @@
       }
     },
     "browsertime": {
-      "version": "19.2.0",
-      "resolved": "https://registry.npmjs.org/browsertime/-/browsertime-19.2.0.tgz",
-      "integrity": "sha512-4BM5AeiO0ZNkQie3mBIW3nrcgvnTBxR3gZmtJ6q96k3gltzqPoH8Hhz9onn7wbb8U+63qIK5IBmEguG2tpIobQ==",
+      "version": "19.3.1",
+      "resolved": "https://registry.npmjs.org/browsertime/-/browsertime-19.3.1.tgz",
+      "integrity": "sha512-MjS3YvbfFMMzZcKNZjdzRMxqIC/fUiNgJ8GiLgOhit1OmqaPPFSHjWv5vZxyjIl3zzzMVgQDv+u3MGDrLGSkPw==",
       "requires": {
         "@cypress/xvfb": "1.2.4",
         "@devicefarmer/adbkit": "3.2.5",
-        "@sitespeed.io/chromedriver": "119.0.6045-105",
-        "@sitespeed.io/edgedriver": "119.0.2151-42",
+        "@sitespeed.io/chromedriver": "120.0.6099-71",
+        "@sitespeed.io/edgedriver": "120.0.2210-77",
         "@sitespeed.io/geckodriver": "0.33.0-c",
         "@sitespeed.io/throttle": "5.0.0",
         "@sitespeed.io/tracium": "0.3.3",

package/package.json

@@ -5,8 +5,8 @@
     "sitespeed.io": "./bin/sitespeed.js",
     "sitespeed.io-wpr": "./bin/browsertimeWebPageReplay.js"
   },
-  "version": "30.9.0",
-  "description": "Analyze the web performance of your site",
+  "version": "30.11.0",
+  "description": "sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.",
   "keywords": [
     "performance",
     "web",
@@ -80,10 +80,10 @@
   "dependencies": {
     "@google-cloud/storage": "6.9.5",
     "@influxdata/influxdb-client": "1.33.2",
-    "@tgwf/co2": "0.13.6",
+    "@tgwf/co2": "0.13.10",
     "aws-sdk": "2.1327.0",
     "axe-core": "4.8.2",
-    "browsertime": "19.2.0",
+    "browsertime": "19.3.1",
     "coach-core": "8.0.2",
     "cli-color": "2.0.3",
     "concurrent-queue": "7.0.2",