pagean 9.0.0 → 10.0.0

package/LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2019 - 2023 Aaron Goldenthal
+Copyright (c) 2019 - 2024 Aaron Goldenthal
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

package/README.md

@@ -1,10 +1,15 @@
 # Pagean
 
-Pagean is a web page analysis tool designed to automate tests requiring web pages to be loaded in a browser window (e.g. 404 error loading an external resource, page renders with horizontal scrollbars). The specific tests are outlined below, but are all general tests that do not include any page-specific logic.
+Pagean is a web page analysis tool designed to automate tests requiring web
+pages to be loaded in a browser window (for example 404 error loading an
+external resource, page renders with horizontal scrollbars). The specific tests
+are outlined below, but are all general tests that do not include any
+page-specific logic.
 
 ## Installation
 
-Install Pagean globally (as shown below), or locally, via [npm](https://www.npmjs.com/).
+Install Pagean globally (as shown below), or locally, via
+[npm](https://www.npmjs.com/).
 
 ```
 npm install -g pagean
@@ -27,19 +32,31 @@ Options:
   -h, --help           display help for command
 ```
 
-Pagean requires a configuration file named, which can be specified via the CLI as detailed above, or use the default file `.pageanrc.json` in the project root. This file provides the URLs to be tested and options to configure the tests and reports. Details on the available tests and the configuration file format are provided below.
+Pagean requires a configuration file named, which can be specified via the CLI
+as detailed previously, or use the default file `.pageanrc.json` in the project
+root. This file provides the URLs to be tested and options to configure the
+tests and reports. Details on the available tests and the configuration file
+format are provided below.
 
-## Test Cases
+## Test cases
 
-The tests use [Puppeteer](https://github.com/GoogleChrome/puppeteer) to launch a headless Chrome browser. The URLs defined in the configuration file are each loaded once, and after page load the applicable tests are executed. Test results are `passed` or `failed`, but can be configured to report `warning` instead of failure. Only a `failed` test will cause the test process to fail and exit with an error code (a `warning` will not).
+The tests use [Puppeteer](https://github.com/GoogleChrome/puppeteer) to launch
+a headless Chrome browser. The URLs defined in the configuration file are each
+loaded once, and after page load the applicable tests are executed. Test
+results are `passed` or `failed`, but can be configured to report `warning`
+instead of failure. Only a `failed` test causes the test process to fail and
+exit with an error code (a `warning` doesn't).
 
-### Horizontal Scrollbar Test
+### Horizontal scrollbar test
 
-The horizontal scrollbar test fails if the rendered page has a horizontal scrollbar. If a specific browser viewport size is desired for this test, that can be configured in the `puppeteerLaunchOptions`.
+The horizontal scrollbar test fails if the rendered page has a horizontal
+scrollbar. If a specific browser viewport size is desired for this test, that
+can be configured in the `puppeteerLaunchOptions`.
 
-### Console Output Test
+### Console output test
 
-The console output test fails if any output is written to the browser console. An array is included in the report with all entries, as shown below:
+The console output test fails if any output is written to the browser console.
+An array is included in the report with all entries, as shown below:
 
 ```json
 [
@@ -53,13 +70,20 @@ The console output test fails if any output is written to the browser console. A
 ]
 ```
 
-### Console Error Test
+### Console error test
 
-The console error test fails if any error is written to the browser console, but is otherwise the same as the console output test. This separation allows for testing for console errors, but allowing any other console output.
+The console error test fails if any error is written to the browser console,
+but is otherwise the same as the console output test. This separation allows
+for testing for console errors, but allowing any other console output.
 
-### Rendered HTML Test
+### Rendered HTML test
 
-The rendered HTML test is intended for cases where content is dynamically created prior to page load (i.e. the `load` event firing). The rendered HTML is returned and checked with [HTML Hint](https://www.npmjs.com/package/htmlhint) and the test fails if any issues are found. An array is included in the report with all HTML Hint issues, as shown below:
+The rendered HTML test is intended for cases where content is dynamically
+created prior to page load (that is, the `load` event firing). The rendered
+HTML is returned and checked with
+[HTML Hint](https://www.npmjs.com/package/htmlhint) and the test fails if any
+issues are found. An array is included in the report with all HTML Hint issues,
+as shown below:
 
 ```json
 [
@@ -79,23 +103,43 @@ The rendered HTML test is intended for cases where content is dynamically create
 ]
 ```
 
-An htmlhintrc file can be specified in the configuration file, otherwise the default "./.htmlhintrc" file will be used (if it exists). See the Configuration section below.
+An htmlhintrc file can be specified in the configuration file, otherwise the
+default "./.htmlhintrc" file is used (if it exists). See the Configuration
+section below.
 
-Note: This test may not find some errors in the original HTML that are removed/resolved as the page is parsed (e.g. closing tags with no opening tags).
+Note: this test may not find some errors in the original HTML that are
+removed/resolved as the page is parsed (for example closing tags with no
+opening tags).
 
-### Page Load Time Test
+### Page load time test
 
-The page load time test fails if the page load time (from start through the `load` event) exceeds the defined threshold in the configuration file (or the default of 2 seconds). The actual load time is included in the report. Tests will time out at twice the page load time threshold.
+The page load time test fails if the page load time (from start through the
+`load` event) exceeds the defined threshold in the configuration file (or the
+default of 2 seconds). The actual load time is included in the report. Tests
+time out at twice the page load time threshold.
 
-### External Script Test
+### External script test
 
-The external script test is intended to identify any externally loaded javascript files (e.g. loaded from a CDN) and aggregate those files so they can undergo further analysis (e.g. dependency vulnerability scanning). The test is included here since these tests load fully rendered pages, therefore allowing the aggregation of this data for pages generated using any language or framework. By default the test returns a warning if the page includes any javascript files loaded from a different domain than the page (although this could be overridden to fail instead via setting `failWarn: false`, see the Configuration section below). These files are then downloaded and saved in the "pagean-external-files" directory in the project root. Subdirectories are created for each domain, then following the URL path. For example, the following script...
+The external script test is intended to identify any externally loaded
+JavaScript files (for example loaded from a CDN) and aggregate those files so
+they can undergo further analysis (for example dependency vulnerability
+scanning). The test is included here since these tests load fully rendered
+pages, therefore allowing the aggregation of this data for pages generated
+using any language or framework. By default the test returns a warning if the
+page includes any JavaScript files loaded from a different domain than the page
+(although this could be overridden to fail instead via setting
+`failWarn: false`, see the Configuration section below). These files are then
+downloaded and saved in the "pagean-external-files" directory in the project
+root. Subdirectories are created for each domain, then following the URL path.
+For example, the following script…
 
 ```html
 <script src="https://bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js"></script>
 ```
 
-...will be saved as `./bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js`. The `data` array in the test report includes the original file URL and the local saved filename or applicable error, as shown below.
+…is saved as `./bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js`. The
+`data` array in the test report includes the original file URL and the local
+saved filename or applicable error, as shown below.
 
 ```json
 [
@@ -110,21 +154,51 @@ The external script test is intended to identify any externally loaded javascrip
 ]
 ```
 
-Each external script is saved only once, but will be reported on any page where it is referenced.
-
-### Broken Link Test
-
-The broken link test checks for broken links on the page. It checks any `<a>` tag on the page with `href` pointing to another location on the current page or another page (i.e. only `http(s)` or `file` protocols).
-
-- For links within the page, this test checks for existence of the element on the page, passing if the element exists and failing otherwise (and passing for cases that are always valid, e.g. `#` or `#top` for the current page). It does not check the visibility of the element. Failing tests return a response of "#element Not Found" (where `#element` identifies the specific element).
-- For links to other pages, the test tries to most efficiently confirm whether the target link is valid. It first makes a `HEAD` request for that URL and checks the response. If an erroneous response is returned (>= 400 with no execution error) and not code 429 (Too Many Requests), the request is retried with a `GET` request. The test passes for HTTP responses < 400 and fails otherwise (if HTTP response is >= 400 or another error occurs).
-  - This can result in false failure indications, specifically for `file:` links (`404` or `ECONNREFUSED`) or where the browser passes a domain identity with the request (page loads when tested, but `401` response for links to that page). For these cases, or other false failures, the test configuration allows a boolean `checkWithBrowser` option that will instead check links by loading the target in the browser (via `puppeteer`). Note this can increase test execution time, in some cases substantially, due to the time to open a new browser tab and plus load the page and all assets.
-  - Note that `file:` links can only be tested with the `checkWithBrowser` option.
-  - If the link to another page includes a hash it is removed prior to checking. The test in this case is confirming a valid link, not that the element exists, which is only done for the current page.
-  - The test configuration allows an `ignoredLinks` array listing link URLs to ignore for this test. Note this only applies to links to other pages, not links within the page, which are always checked.
-- To optimize performance, link test results are cached and those links are not re-tested for the entire test run (across all tested URLs). The test configuration allows a boolean `ignoreDuplicates` option that can be set to `false` to bypass this behavior and re-test all links. The results for any failed links are included in the reports in any case.
-
-For any failing test, the `data` array in the test report includes the original URL and the response code or error as shown below.
+Each external script is saved only once, but is reported on any page where it's
+referenced.
+
+### Broken link test
+
+The broken link test checks for broken links on the page. It checks any `<a>`
+tag on the page with `href` pointing to another location on the current page or
+another page (that is, only `http(s)` or `file` protocols).
+
+- For links within the page, this test checks for existence of the element on
+  the page, passing if the element exists and failing otherwise (and passing
+  for cases that are always valid, for example `#` or `#top` for the current
+  page). It doesn't check the visibility of the element. Failing tests return a
+  response of "#element Not Found" (where `#element` identifies the specific
+  element).
+- For links to other pages, the test tries to most efficiently confirm whether
+  the target link is valid. It first makes a `HEAD` request for that URL and
+  checks the response. If an erroneous response is returned (>= 400 with no
+  execution error) and not code 429 (Too Many Requests), the request is retried
+  with a `GET` request. The test passes for HTTP responses < 400 and fails
+  otherwise (if HTTP response is >= 400 or another error occurs).
+  - This can result in false failure indications, specifically for `file:`
+    links (`404` or `ECONNREFUSED`) or where the browser passes a domain
+    identity with the request (page loads when tested, but `401` response for
+    links to that page). For these cases, or other false failures, the test
+    configuration allows a Boolean `checkWithBrowser` option that instead
+    checks links by loading the target in the browser (via `puppeteer`). Note
+    this can increase test execution time, in some cases substantially, due to
+    the time to open a new browser tab and plus load the page and all assets.
+  - Note that `file:` links can only be tested with the `checkWithBrowser`
+    option.
+  - If the link to another page includes a hash it's removed prior to checking.
+    The test in this case is confirming a valid link, not that the element
+    exists, which is only done for the current page.
+  - The test configuration allows an `ignoredLinks` array listing link URLs to
+    ignore for this test. Note this only applies to links to other pages, not
+    links within the page, which are always checked.
+- To optimize performance, link test results are cached and those links aren't
+  re-tested for the entire test run (across all tested URLs). The test
+  configuration allows a Boolean `ignoreDuplicates` option that can be set to
+  `false` to bypass this behavior and re-test all links. The results for any
+  failed links are included in the reports in any case.
+
+For any failing test, the `data` array in the test report includes the original
+URL and the response code or error as shown below.
 
 ```json
 [
@@ -143,38 +217,71 @@ For any failing test, the `data` array in the test report includes the original
 ]
 ```
 
-Note: This test will check all links on the page, and does not respect mechanisms inteded to limit web crawlers such as `robots.txt` or `noindex` tags.
+Note: this test checks all links on the page, and doesn't respect mechanisms
+intended to limit web crawlers such as `robots.txt` or `noindex` tags.
 
 ## Reports
 
-Based on the `reporters` configuration, Pagean results may be displayed in the console and saved in two reports in the project root directory (any or all of the three):
+Based on the `reporters` configuration, Pagean results may be displayed in the
+console and saved in two reports in the project root directory (any or all of
+the three):
 
-- A JSON report named [`pagean-results.json`](https://gitlab-ci-utils.gitlab.io/pagean/pagean-results.json)
-- An HTML report named [`pagean-results.html`](https://gitlab-ci-utils.gitlab.io/pagean/pagean-results.html)
+- A JSON report named
+  [`pagean-results.json`](https://gitlab-ci-utils.gitlab.io/pagean/pagean-results.json)
+- An HTML report named
+  [`pagean-results.html`](https://gitlab-ci-utils.gitlab.io/pagean/pagean-results.html)
 
 Both reports contain:
 
 - The time of test execution
 - A summary of the total tests and results (passed, warning, failed)
-- The detailed test results, including the URL tested, list of tests performed on that URL with results, and, if applicable, any relevant data associated with the test failure (e.g. the console errors if the console error test fails).
+- The detailed test results, including the URL tested, list of tests performed
+  on that URL with results, and, if applicable, any relevant data associated
+  with the test failure (for example the console errors if the console error
+  test fails).
 
-Complete reports for the example case in this project (the tests as specified in the project [`.pageanrc.json`](https://gitlab.com/gitlab-ci-utils/pagean/-/blob/master/.pageanrc.json) file) can be found at the links above.
+Complete reports for the example case in this project (the tests as specified
+in the project
+[`.pageanrc.json`](https://gitlab.com/gitlab-ci-utils/pagean/-/blob/master/.pageanrc.json)
+file) can be found at the preceding links.
 
 ## Configuration
 
-Pagean looks for a configuration file as specified via the CLI, or defaults to a file named `.pageanrc.json` in the project root. If the configuration file is not found, is not valid JSON, or does not contain any URLs to check the job will fail.
-
-Below is an example `.pageanrc.json` file, which is broken into seven major properties:
-
-- `htmlhintrc`: An optional path to an htmlhintrc file to be used in the rendered HTML test
-- `project`: An optional name of the project, which is included in HTML and JSON reports.
-- `puppeteerLaunchOptions`: An optional set of options to pass to Puppeteer on launch. There are no default options. The complete list of available options can be found at https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions.
-- `reporters`: An optional array of reporters indicating the test reports that should be provided. There are three possible options - `cli`, `html`, and `json`. The `cli` option reports all test details to the console, but the final results summary is always output (even with `cli` disabled). If `reporters` is specified, at least one reporter must be included. The default value, as specified below, is all three reporters enabled.
-- `settings`: These settings enable/disable or configure tests, and are applied to all tests overriding the default values.
-  - The shorthand notation allows easy enabling/disabling of tests. In this format the test name is given with a boolean value to enable or disable the test. In this case any other test-specific settings use the default values.
-  - The longhand version includes an object for each test. Every test includes two possible properties (some tests include additional settings):
-    - `enabled`: A boolean value to enable/disable the test, and some tests include additional settings (default `true` for all tests).
-    - `failWarn`: A boolean value causing a failed test to report a warning instead of failure. A warning result will not cause the test process to fail (exit with an error code). The default value for all tests is `false` except the `externalScriptTest`, as shown below.
+Pagean looks for a configuration file as specified via the CLI, or defaults to
+a file named `.pageanrc.json` in the project root. If the configuration file is
+not found, is not valid JSON, or doesn't contain any URLs to check the job
+fails.
+
+Below is an example `.pageanrc.json` file, which is broken into seven major
+properties:
+
+- `htmlhintrc`: An optional path to an htmlhintrc file to be used in the
+  rendered HTML test
+- `project`: An optional name of the project, which is included in HTML and
+  JSON reports.
+- `puppeteerLaunchOptions`: An optional set of options to pass to Puppeteer on
+  launch. There are no default options. The complete list of available options
+  can be found at
+  https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#puppeteerlaunchoptions.
+- `reporters`: An optional array of reporters indicating the test reports that
+  should be provided. There are three possible options - `cli`, `html`, and
+  `json`. The `cli` option reports all test details to the console, but the
+  final results summary is always output (even with `cli` disabled). If
+  `reporters` is specified, at least one reporter must be included. The default
+  value, as specified below, is all three reporters enabled.
+- `settings`: These settings enable/disable or configure tests, and are applied
+  to all tests overriding the default values.
+  - The shorthand notation allows easy enabling/disabling of tests. In this
+    format the test name is given with a Boolean value to enable or disable the
+    test. In this case any other test-specific settings use the default values.
+  - The longhand version includes an object for each test. Every test includes
+    two possible properties (some tests include additional settings):
+    - `enabled`: A Boolean value to enable/disable the test, and some tests
+      include additional settings (default `true` for all tests).
+    - `failWarn`: A Boolean value causing a failed test to report a warning
+      instead of failure. A warning result doesn't cause the test process to
+      fail (exit with an error code). The default value for all tests is
+      `false` except the `externalScriptTest`, as shown below.
 
 The shorthand:
 
@@ -197,12 +304,28 @@ is equivalent to the longhand:
 
 All available settings with the default values are shown below.
 
-- `sitemap`: Specify a sitemap with URLs to test. If a sitemap is specified, the URLs from the sitemap are added to the `urls` array. If a URL is in the `urls` array with `settings`, those settings are retained. Note that `<sitemapindex>` is currently not supported. The `sitemap` object can have the following properties:
-  - `url`: The URL of the sitemap (required if `sitemap` is included). This can be either an actual URL or a local file.
-  - `find`: A string to search for in sitemap URLs (e.g. `https://somehere.test`) (required if `replace` is specified).
-  - `replace`: The string to replace the `find` string with (e.g. `http://localhost:3000`) (required if `find` is specified).
-  - `exclude`: An array of strings with regular expressions to exclude URLs from the sitemap (e.g. `['\.pdf$']` to exclude any PDF files). Since these are string representations of regular expressions, the backslash must be escaped (e.g. `\\.`). Exclude is performed before find/replace, so uses the original URLs from the sitemap.
-- `urls`: An array of URLs to be tested, which must contain at least one value. Each array entry can either be a URL string, or an object that contains a `url` string and an optional `settings` object. This object can contain any of the `settings` values identified above and will override that setting for testing that URL. The `url` string can be either an actual URL or a local file, as shown in the example below.
+- `sitemap`: Specify a sitemap with URLs to test. If a sitemap is specified,
+  the URLs from the sitemap are added to the `urls` array. If a URL is in the
+  `urls` array with `settings`, those settings are retained. Note that
+  `<sitemapindex>` is currently not supported. The `sitemap` object can have
+  the following properties:
+  - `url`: The URL of the sitemap (required if `sitemap` is included). This can
+    be either an actual URL or a local file.
+  - `find`: A string to search for in sitemap URLs (for example
+    `https://somehere.test`) (required if `replace` is specified).
+  - `replace`: The string to replace the `find` string with (for example
+    `http://localhost:3000`) (required if `find` is specified).
+  - `exclude`: An array of strings with regular expressions to exclude URLs
+    from the sitemap (for example `['\.pdf$']` to exclude any PDF files). Since
+    these are string representations of regular expressions, the backslash must
+    be escaped (for example `\\.`). Exclude is performed before find/replace,
+    so uses the original URLs from the sitemap.
+- `urls`: An array of URLs to be tested, which must contain at least one value.
+  Each array entry can either be a URL string, or an object that contains a
+  `url` string and an optional `settings` object. This object can contain any
+  of the `settings` values identified previously and overrides that setting for
+  testing that URL. The `url` string can be either an actual URL or a local
+  file, as shown in the example below.
 
 ```json
 {
@@ -256,19 +379,31 @@ All available settings with the default values are shown below.
 }
 ```
 
-## Docker Images
+## Container images
 
-Provided with the Pagean project are Docker images configured to run the tests. All available image tags can be found in the `gitlab-ci-utils/pagean` repository at https://gitlab.com/gitlab-ci-utils/pagean/container_registry. Details on each release can be found on the [Releases](https://gitlab.com/gitlab-ci-utils/pagean/releases) page.
+Provided with the Pagean project are container images configured to run the
+tests. All available image tags can be found in the `gitlab-ci-utils/pagean`
+repository at https://gitlab.com/gitlab-ci-utils/pagean/container_registry.
+Details on each release can be found on the
+[Releases](https://gitlab.com/gitlab-ci-utils/pagean/releases) page.
 
-**Note:** Any images in the `gitlab-ci-utils/pagean/tmp` repository are temporary images used during the build process and may be deleted at any point.
+**Note:** any images in the `gitlab-ci-utils/pagean/tmp` repository are
+temporary images used during the build process and may be deleted at any point.
 
-### Puppeteer Cache Location
+### Puppeteer cache location
 
-In [Puppeteer v19](https://github.com/puppeteer/puppeteer/releases/tag/v19.0.0) the default cache location for installing the Chrome binary was changed from within the project's `node_modules` folder to `~/.cache/puppeteer`. To simplify execution in a container, the `PUPPETEER_CACHE_DIR` environment variable is set to install the Chrome binaries in `/home/pptruser/.cache/puppeteer` during container build, so setting to another value before execution can cause errors where Puppeteer cannot find the Chrome binary.
+In [Puppeteer v19](https://github.com/puppeteer/puppeteer/releases/tag/v19.0.0)
+the default cache location for installing the Chrome binary was changed from
+within the project's `node_modules` folder to `~/.cache/puppeteer`. To simplify
+execution in a container, the `PUPPETEER_CACHE_DIR` environment variable is set
+to install the Chrome binaries in `/home/pptruser/.cache/puppeteer` during
+container build, so setting to another value before execution can cause errors
+where Puppeteer can't find the Chrome binary.
 
-## GitLab CI Configuration
+## GitLab CI configuration
 
-The following is an example job from a .gitlab-ci.yml file to use this image to run Pagean against another project in GitLab CI:
+The following is an example job from a .gitlab-ci.yml file to use this image to
+run Pagean against another project in GitLab CI:
 
 ```yaml
 pagean:
@@ -284,9 +419,17 @@ pagean:
       - pagean-external-scripts/
 ```
 
-### Testing With Static HTTP Server
+### Testing with static HTTP server
 
-The Docker image shown above includes [`serve`](https://www.npmjs.com/package/serve) and [`wait-on`](https://www.npmjs.com/package/wait-on) installed globally to run a local HTTP server for testing static content. The example job below illustrates how to use this for Pagean tests. The script starts the server in this project's `./tests/fixtures/site` directory and uses `wait-on` to hold the script until the server is running and returns a valid response. The referenced `pageanrc` file is the same as the project default `pageanrc`, but references all test URLs from the local server.
+The container image shown previously includes
+[`serve`](https://www.npmjs.com/package/serve) and
+[`wait-on`](https://www.npmjs.com/package/wait-on) installed globally to run a
+local HTTP server for testing static content. The example job below illustrates
+how to use this for Pagean tests. The script starts the server in this
+project's `./tests/fixtures/site` directory and uses `wait-on` to hold the
+script until the server is running and returns a valid response. The referenced
+`pageanrc` file is the same as the project default `pageanrc`, but references
+all test URLs from the local server.
 
 ```yaml
 pagean:
@@ -306,9 +449,10 @@ pagean:
       - pagean-external-scripts/
 ```
 
-## Linting Pageanrc Files
+## Linting pageanrc files
 
-A command line tool is also available to lint pageanrc files, which is executed as follows:
+A command line tool is also available to lint pageanrc files, which is executed
+as follows:
 
 ```
 Installed globally:
@@ -325,21 +469,28 @@ Options:
   -h, --help     display help for command
 ```
 
-The `--json` option outputs the JSON results to stdout in all cases for consistency (`[]` if no errors found, so that it always outputs valid JSON). Otherwise errors are output to stderr, for example:
+The `--json` option outputs the JSON results to stdout in all cases for
+consistency (`[]` if no errors found, so that it always outputs valid
+JSON). Otherwise errors are output to stderr, for example:
 
 ```sh
 .\tests\test-configs\cli-tests\some-test.pageanrc.json
   <pageanrc>.puppeteerLaunchOptions                  must NOT have fewer than 1 properties
   <pageanrc>.reporters[0]                            must be equal to one of the allowed values (cli, html, json)
-  <pageanrc>.settings.consoleOutputTest              must be either boolean or object with the appropriate properties
+  <pageanrc>.settings.consoleOutputTest              must be either Boolean or object with the appropriate properties
   <pageanrc>.settings.pageLoadTimeTest.foo           must NOT contain additional properties: "foo"
-  <pageanrc>.settings.pageLoadTimeTest               must be either boolean or object with the appropriate properties
+  <pageanrc>.settings.pageLoadTimeTest               must be either Boolean or object with the appropriate properties
   <pageanrc>.sitemap                                 must use 'find' and 'replace' together
-  <pageanrc>.urls[2].settings.consoleOutputTest      must be either boolean or object with the appropriate properties
+  <pageanrc>.urls[2].settings.consoleOutputTest      must be either Boolean or object with the appropriate properties
   <pageanrc>.urls[3]                                 must be either URL string or object with the appropriate properties
   <pageanrc>.urls[5]                                 must have required property 'url'
 ```
 
-In some cases, a single error might result in multiple messages based on the options in the schema definition, especially for cases that can be either a single value or an object with specific properties (e.g. the errors for `<pageanrc>.settings.pageLoadTimeTest` in the example above).
+In some cases, a single error might result in multiple messages based on the
+options in the schema definition, especially for cases that can be either a
+single value or an object with specific properties (for example the errors for
+`<pageanrc>.settings.pageLoadTimeTest` in the preceding example).
 
-Note that because of the large number of options, which are dependent on an external project, the linting of `puppeteerLaunchOptions` only checks that at least one property is provided, it does not check the detailed settings.
+Note that because of the large number of options, which are dependent on an
+external project, the linting of `puppeteerLaunchOptions` only checks that at
+least one property is provided, it doesn't check the detailed settings.

package/docs/upgrade-guide.md

@@ -1,10 +1,15 @@
-# Version Upgrade Guide
+# Version upgrade guide
 
 ## Upgrading from v4.x to v5.0
 
-1. Update to a Node.js current or LTS release (`^12.20.0 || ^14.15.0 || >=16.0.0`). Pagean may still work in other releases, but is not specifically tested or supported.
+1. Update to a Node.js current or LTS release
+   (`^12.20.0 || ^14.15.0 || >=16.0.0`). Pagean may still work in other
+   releases, but is not specifically tested or supported.
 
-2. The Broken Link Test default setting was originally to fail with warning, but this was updated to fail (without warning) to be consistent with the other tests. To return to the previous behavior, update your configuration settings with:
+2. The Broken Link Test default setting was originally to fail with warning,
+   but this was updated to fail (without warning) to be consistent with the
+   other tests. To return to the previous behavior, update your configuration
+   settings with:
 
 ```json
 {
@@ -18,9 +23,13 @@
 
 ## Upgrading from v3.x (`page-load-tests`) to v4.0 (`pagean`)
 
-1. Change the configurations file name from `.pltconfig.json` to `.pageanrc.json`
+1. Change the configurations filename from `.pltconfig.json` to
+   `.pageanrc.json`
 
-2. Pagean v4.0.0 updated the configuration file format to allow test-specific settings. With this change, `pageLoadTimeThreshold` must now appears as a setting of the `pageLoadTimeTest` test, rather than at the same level as the other tests. See the example from/to below:
+2. Pagean v4.0.0 updated the configuration file format to allow test-specific
+   settings. With this change, `pageLoadTimeThreshold` must now appears as a
+   setting of the `pageLoadTimeTest` test, rather than at the same level as the
+   other tests. See the example from/to below:
 
 **From:**
 

package/index.js

@@ -38,6 +38,8 @@ const executeAllTests = async (config) => {
                 type: message.type()
             })
         );
+        // User provided test input required.
+        // nosemgrep: puppeteer-goto-injection
         await page.goto(testUrl.url, { waitUntil: 'load' });
 
         const testContext = {

package/lib/config.js

@@ -6,8 +6,8 @@
  * @module config
  */
 
-const fs = require('fs');
-const path = require('path');
+const fs = require('node:fs');
+const path = require('node:path');
 const Ajv = require('ajv/dist/2019');
 const draft7MetaSchema = require('ajv/dist/refs/json-schema-draft-07.json');
 const ajvErrors = require('ajv-errors');
@@ -108,20 +108,29 @@ const processUrls = async (config) => {
 };
 
 const getHtmlHintConfig = (htmlHintConfigFilename) => {
+    // Allow users to specify a custom htmlhintrc file.
+    // nosemgrep: eslint.detect-non-literal-fs-filename
     if (!fs.existsSync(htmlHintConfigFilename)) {
         return;
     }
-    /* eslint-disable-next-line consistent-return -- return undefined
-       if no settings */
+    /* eslint-disable consistent-return -- return undefined if no settings */
+    // Allow users to specify a custom htmlhintrc file.
+    // nosemgrep: eslint.detect-non-literal-fs-filename
     return JSON.parse(fs.readFileSync(htmlHintConfigFilename, 'utf8'));
+    /* eslint-enable consistent-return -- return undefined if no settings */
 };
 
 const validateConfigSchema = (config) => {
     const schema = JSON.parse(
+        // All values hardcoded.
+        // nosemgrep: eslint.detect-non-literal-fs-filename
         fs.readFileSync(
             path.join(__dirname, '../', 'schemas', 'pageanrc.schema.json')
         )
     );
+    // Allow allErrors to lint the entire config file, although users could
+    // ReDoS themselves.
+    // nosemgrep: ajv-allerrors-true
     const ajv = new Ajv({ allErrors: true });
     ajv.addMetaSchema(draft7MetaSchema);
     ajvErrors(ajv);
@@ -131,6 +140,8 @@ const validateConfigSchema = (config) => {
 };
 
 const getConfigFromFile = (configFileName) =>
+    // Allow users to specify config filename.
+    // nosemgrep: eslint.detect-non-literal-fs-filename
     JSON.parse(fs.readFileSync(configFileName, 'utf8'));
 
 /**

package/lib/external-file-utils.js

@@ -5,8 +5,8 @@
  *
  * @module external-file-utils
  */
-const fs = require('fs');
-const path = require('path');
+const fs = require('node:fs');
+const path = require('node:path');
 
 const axios = require('axios');
 
@@ -48,10 +48,16 @@ const saveExternalScript = async (script) => {
             scriptUrl.hostname,
             scriptUrl.pathname
         );
+        // Path generated above from URL, not from user input.
+        // nosemgrep: eslint.detect-non-literal-fs-filename
         if (!fs.existsSync(pathName)) {
             // Axios will throw for any error response
             const response = await axios.get(script);
+            // Path generated above from URL, not from user input.
+            // nosemgrep: eslint.detect-non-literal-fs-filename
             fs.mkdirSync(path.dirname(pathName), { recursive: true });
+            // Path generated above from URL, not from user input.
+            // nosemgrep: eslint.detect-non-literal-fs-filename
             fs.writeFileSync(pathName, response.data);
         }
         result.localFile = pathName;

package/lib/link-utils.js

@@ -6,7 +6,7 @@
  * @module link-utils
  */
 
-const https = require('https');
+const https = require('node:https');
 const axios = require('axios');
 const normalizeUrl = require('normalize-url');
 const cssesc = require('cssesc');

package/lib/report-template.handlebars

@@ -4,25 +4,40 @@
 <head>
 	<title>Pagean Results</title>
 	<style>
+		:root {
+			/* Default colors */
+			--color-text-default: #333333;
+			--color-background-data-hover: rgba(255 255 255 / 10%);
+			--color-background-summary: #f4f4f4;
+			--color-border-summary: #333333;
+
+			/* Test result colors */
+			--color-background-passed: #dff2bf;
+			--color-text-passed: #44760f;
+			--color-background-failed: #ffbaba;
+			--color-text-failed: #ad000c;
+			--color-background-warning: #fdec96;
+			--color-text-warning: #7e6902;
+		}
+
 		html {
 			margin: 0;
 			padding: 0;
 		}
 
 		body {
-			color: #333333;
+			color: var(--color-text-default);
 			font-family: system-ui, sans-serif;
 			font-size: 0.85rem;
 			margin: auto;
-			max-width: 1000px;
+			max-inline-size: 1000px;
 			padding: 1rem;
 		}
 
 		ul {
-			margin: 0;
 			margin-block: 0;
 			margin-inline: 0;
-			padding: 0;
+			padding-block: 0;
 			padding-inline: 0;
 		}
 
@@ -31,7 +46,6 @@
 		}
 
 		h1 {
-			margin: 1rem 0 0.75rem;
 			margin-block: 0;
 			margin-inline: 0;
 		}
@@ -48,54 +62,55 @@
 		.project,
 		.started {
 			margin: 0;
-			padding-bottom: 0.5rem;
+			padding-block-end: 0.5rem;
 		}
 
 		.test-summary {
 			display: flex;
 			flex-direction: row;
-			margin-bottom: 2rem;
-			max-width: 50%;
+			margin-block-end: 2rem;
+			max-inline-size: 50%;
 		}
 
 		.summary {
-			background-color: #f4f4f4;
-			border: 1px solid #333333;
+			background-color: var(--color-background-summary);
+			border: 1px solid var(--color-border-summary);
 			display: inline-block;
 			flex: 1 1 10%;
-			margin: 0 0.25rem;
-			padding: 0.25rem 0.5rem;
+			margin-block: 0;
+			margin-inline: 0.25rem;
+			padding-block: 0.25rem;
+			padding-inline: 0.5rem;
 		}
 
 		.summary:first-of-type {
-			margin-left: 0;
+			margin-inline-start: 0;
 		}
 
 		.summary:last-of-type {
-			margin-right: 0;
+			margin-inline-end: 0;
 		}
 
 		.passed {
-			background-color: #dff2bf;
-			border: 1px solid #44760f;
-			color: #44760f;
+			background-color: var(--color-background-passed);
+			border: 1px solid var(--color-text-passed);
+			color: var(--color-text-passed);
 		}
 
 		.failed {
-			background-color: #ffbaba;
-			border: 1px solid #ad000c;
-			color: #ad000c;
+			background-color: var(--color-background-failed);
+			border: 1px solid var(--color-text-failed);
+			color: var(--color-text-failed);
 		}
 
 		.warning {
-			background-color: #fdec96;
-			border: 1px solid #7e6902;
-			color: #7e6902;
+			background-color: var(--color-background-warning);
+			border: 1px solid var(--color-text-warning);
+			color: var(--color-text-warning);
 		}
 
 		.test-results h2 {
-			margin-block-end: 0;
-			margin-block-start: 1rem;
+			margin-block: 1rem 0;
 		}
 
 		details summary .name::after,
@@ -104,7 +119,7 @@
 			font-family: monospace;
 			font-weight: bold;
 			opacity: 0.8;
-    		padding-left: 0.5rem;
+    		padding-inline-start: 0.5rem;
 		}
 
 		details[open] summary .name::after {
@@ -121,8 +136,10 @@
 		}
 
 		li.test {
-			margin: 0.25rem 0;
-			padding: 0.5rem 1rem;
+			margin-block: 0.25rem;
+			margin-inline: 0;
+			padding-block: 0.5rem;
+			padding-inline: 1rem;
 		}
 
 		.test .header {
@@ -136,20 +153,20 @@
 
 		.test .header .result {
 			flex: 1 1 25%;
-			text-align: right;
+			text-align: end;
 		}
 
 		.test .data,
 		.test .time,
 		.test .error {
-			padding-left: 10px;
+			padding-inline-start: 0.75rem;
 		}
 
 		.test .data h3 {
 			font-size: inherit;
 			font-weight: normal;
-			margin: 0.25rem 0 0;
 			margin-block: 0;
+			margin-inline: 0;
 		}
 
 		.test .data li {
@@ -159,18 +176,20 @@
 		}
 
 		.test .data li:hover {
-			background-color: rgba(255 255 255 / 10%);
+			background-color: var(--color-background-data-hover);
 			border: 1px dotted;
-			border-right: 4px solid;
+			border-inline-end: 4px solid;
 			padding: calc(0.25rem - 1px);
 		}
 
 		.test .data .pre {
-			margin: 0 0 0.25rem;
+			margin-block: 0 0.25rem;
+			margin-inline: 0;
 		}
 
 		.test .time {
-			margin: 0.25rem 0 0;
+			margin-block: 0.25rem 0;
+			margin-inline: 0;
 		}
 	</style>
 </head>

package/lib/reporter.js

@@ -5,8 +5,8 @@
  *
  * @module reporter
  */
-const fs = require('fs');
-const path = require('path');
+const fs = require('node:fs');
+const path = require('node:path');
 const handlebars = require('handlebars');
 
 const htmlReportTemplateName = 'report-template.handlebars';
@@ -22,6 +22,8 @@ const saveHtmlReport = (results) => {
     const templateFile = path.resolve(
         path.join(__dirname, htmlReportTemplateName)
     );
+    // Path hardcoded above, not from user input.
+    // nosemgrep: eslint.detect-non-literal-fs-filename
     const htmlReportTemplate = fs.readFileSync(templateFile, 'utf8');
     const template = handlebars.compile(htmlReportTemplate);
     const htmlReport = template(results);

package/lib/sitemap.js

@@ -1,10 +1,9 @@
 'use strict';
 
+const fs = require('node:fs');
 const axios = require('axios');
 const { parseStringPromise } = require('xml2js');
 
-const fs = require('fs');
-
 /**
  * Gets a sitemap, via file path or URL, and returns the string contents.
  *
@@ -21,6 +20,8 @@ const getSitemap = async (url) => {
             const response = await axios.get(url);
             return response.data;
         }
+        // Allow users to specify a local sitemap filename.
+        // nosemgrep: eslint.detect-non-literal-fs-filename
         return fs.readFileSync(url, 'utf8');
     } catch {
         throw new Error(`Error retrieving sitemap "${url}"`);
@@ -54,6 +55,8 @@ const parseSitemap = async (sitemapXml) => {
  * @private
  */
 const removeExcludedUrls = (urls, exclude = []) => {
+    // Allow URLs to be excluded by regular expression.
+    // nosemgrep: eslint.detect-non-literal-regexp
     const excludeRegex = exclude.map((url) => new RegExp(url));
     return urls.filter((url) => {
         for (const excludeUrlRegex of excludeRegex) {

package/package.json

@@ -1,24 +1,24 @@
 {
   "name": "pagean",
-  "version": "9.0.0",
+  "version": "10.0.0",
   "description": "Pagean is a web page analysis tool designed to automate tests requiring web pages to be loaded in a browser window (e.g. horizontal scrollbar, console errors)",
   "bin": {
     "pagean": "./bin/pagean.js",
     "pageanrc-lint": "./bin/pageanrc-lint.js"
   },
   "scripts": {
-    "hooks-pre-commit": "npm run lint && npm run prettier-check",
-    "hooks-pre-push": "npm audit --audit-level=high && npm test",
-    "lint": "npm run lint-css && npm run lint-html && npm run lint-js && npm run lint-md",
-    "lint-css": "stylelint ./lib/report-template.handlebars",
-    "lint-html": "htmlhint ./lib/report-template.handlebars",
-    "lint-js": "eslint .",
-    "lint-md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\" \"#Archive\"",
-    "prettier-check": "prettier --check .",
-    "prettier-fix": "prettier --write .",
+    "hooks:pre-commit": "npm run lint && npm run prettier:check",
+    "hooks:pre-push": "npm audit --audit-level=high && npm test",
+    "lint": "npm run lint:css && npm run lint:html && npm run lint:js && npm run lint:md",
+    "lint:css": "stylelint ./lib/report-template.handlebars",
+    "lint:html": "htmlhint ./lib/report-template.handlebars",
+    "lint:js": "eslint .",
+    "lint:md": "markdownlint-cli2 \"**/*.md\" \"#node_modules\" \"#Archive\"",
+    "prettier:check": "prettier --check .",
+    "prettier:fix": "prettier --write .",
     "start": "node ./bin/pagean.js",
-    "start-lint": "node ./bin/pageanrc-lint.js",
-    "start-lint-all": "npm run start-lint && npm run start-lint static-server.pageanrc.json && npm run start-lint ./lib/default-config.json",
+    "start:lint": "node ./bin/pageanrc-lint.js",
+    "start:lint-all": "npm run start:lint && npm run start:lint static-server.pageanrc.json && npm run start:lint ./lib/default-config.json",
     "test": "jest --ci --config jest.config.json"
   },
   "repository": {
@@ -37,7 +37,7 @@
   "author": "Aaron Goldenthal <npm@aarongoldenthal.com>",
   "license": "MIT",
   "engines": {
-    "node": "^16.13.0 || ^18.12.0 || >=20.0.0"
+    "node": "^18.12.0 || >=20.0.0"
   },
   "files": [
     "index.js",
@@ -51,30 +51,30 @@
   },
   "homepage": "https://gitlab.com/gitlab-ci-utils/pagean",
   "devDependencies": {
-    "@aarongoldenthal/eslint-config-standard": "^22.1.0",
-    "@aarongoldenthal/stylelint-config-standard": "^14.0.0",
-    "bin-tester": "^4.0.1",
-    "eslint": "^8.44.0",
-    "jest": "^29.6.0",
+    "@aarongoldenthal/eslint-config-standard": "^25.0.0",
+    "@aarongoldenthal/stylelint-config-standard": "^17.0.0",
+    "bin-tester": "^5.0.0",
+    "eslint": "^8.56.0",
+    "jest": "^29.7.0",
     "jest-junit": "^16.0.0",
-    "markdownlint-cli2": "^0.8.1",
-    "prettier": "^2.8.8",
+    "markdownlint-cli2": "^0.12.1",
+    "prettier": "^3.2.4",
     "strip-ansi": "^6.0.1",
-    "stylelint": "^15.10.0"
+    "stylelint": "^16.2.0"
   },
   "dependencies": {
     "ajv": "^8.12.0",
     "ajv-errors": "^3.0.0",
-    "axios": "^1.4.0",
+    "axios": "^1.6.7",
     "ci-logger": "^6.0.0",
-    "commander": "^11.0.0",
+    "commander": "^11.1.0",
     "cssesc": "^3.0.0",
-    "handlebars": "^4.7.7",
+    "handlebars": "^4.7.8",
     "htmlhint": "^1.1.4",
     "kleur": "^4.1.5",
     "normalize-url": "^6.1.0",
     "protocolify": "^3.0.0",
-    "puppeteer": "^20.7.4",
-    "xml2js": "^0.6.0"
+    "puppeteer": "^21.9.0",
+    "xml2js": "^0.6.2"
   }
 }