@code-pushup/lighthouse-plugin 0.42.0

package/CONTRIBUTING.md

@@ -0,0 +1,117 @@
+# Contributing
+
+## Setup
+
+Make sure to install dependencies:
+
+```sh
+npm install
+```
+
+### Chrome path
+
+In this plugin we provide Lighthouse functionality exposed over the `lighthousePlugin`.
+To test lighthouse properly we work with a predefined testing setup.
+
+On some OS there could be a problem finding the path to Chrome.
+
+We try to detect it automatically in the [`chrome-path.setup.ts` script](../../testing/test-setup/src/lib/chrome-path.setup.ts).
+There we use `getChromePath` and have `chromium` installed as NPM package, so detecting the path should not cause any problem.
+
+However, if no chrome path is detected automatically the error looks like this:
+
+```bash
+Runtime error encountered: No Chrome installations found.
+```
+
+To prevent this from happening you have to provide the path manually in your `.env`:
+
+```bash
+CUSTOM_CHROME_PATH=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
+```
+
+In the CI you can set a static path if needed over the env variable like this:
+
+```yml
+# ...
+- name: Set custom Chrome path for Windows only
+  if: matrix.os == 'windows-latest'
+  run: |
+    echo "CUSTOM_CHROME_PATH=C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
+  shell: pwsh
+
+# Optional debug log
+- name: Log all environment variables
+  run: printenv
+# ...
+```
+
+We consider this path in our `beforeAll` hook in a [`chrome-path.setup.ts` script](../../testing/test-setup/src/lib/chrome-path.setup.ts).
+
+### Testing chrome flags
+
+1. run `npx chrome-debug --<chromium-flag>` to pass terminal arguments to Chrome. E.g. `npx chrome-debug --headless=shell`.
+   `npx chrome-debug --headless=shell --@TODO-PUT-OTHER-EXAMPLE-FOR-FLAG`
+
+For a full list of available flags check out [this document](https://peter.sh/experiments/chromium-command-line-switches/).
+
+> [!NOTE]
+> To pass chrome flags to lighthouse you have to provide them under `--chrome-flags="<chrome-flags-as-array>"`.
+> E.g. `lighthouse https://example.com --chrome-flage="--headless=shell"`
+
+2. Check if the flag got accepted. This is quite unintuitive as we would expect the passed flag to be visible under `chrome://flags/` but as you can see in the screenshot it is not visible.
+   <img width="1202" alt="chrome-flags" src="./docs/images/chrome-flags.png">
+   Instead open `chrome://version/` and look under the "Command Line" section.  
+   <img width="1202" alt="chrome-chrome-version" src="./docs/images/chrome-version.png">
+
+### Chrome User Data
+
+To bootstrap Chrome with a predefined for setting we have to provide a couple of config files that we located under `<project-root>/mock/chromium-user-data`.
+When executing Lighthouse we provide the path to this folder over the `Flag` object.
+
+To generate initialise or edit the file structure under `chromium-user-data` do the following steps:
+
+1. Spin up Chrome by running `npx chrome-debug --user-data-dir=./packages/plugin-lighthouse/mock/chromium-user-data`
+   <img width="1202" alt="chrome-blank-screen" src="./docs/images/chrome-blank-screen.png">
+
+2. If you do this the first time you should already see content under `<project-root>/mock/chromium-user-data`
+3. Edit the configuration over the Chrome UI. E.g. adding a profile
+4. Close chromium and open it again, and you should see chromium bootstraps as the configured user
+   <img width="1202" alt="chrome-blank-screen-pre-configured" src="./docs/images/chrome-blank-screen-pre-configure.png">
+
+To reset the above just delete the folder and apply the settings again.
+
+_A helpful chromium setup is preconfigured with the following settings:_
+
+- A user profile is set up. This enables certain debugging related options as well as help to visually distinguish between test setups as the header bar is colored.
+  <img width="1202" alt="chrome-settings-manage-profile" src="./docs/images/chrome-settings-manage-profile.png">
+
+#### Resources
+
+- [chromium flags guide](https://www.chromium.org/developers/how-tos/run-chromium-with-flags/)
+
+## Troubleshooting
+
+1. Verify Chrome Installation  
+   Ensure Chrome is correctly installed and accessible to the Lighthouse process.
+   Run `npx chrome-debug` to test it. Read further under [chrome-path](#chrome-path)
+
+2. Increase Timeout  
+   Lighthouse has a longer runtime which can time out in different environments.
+   **Try increasing the test timeout** in `lighthouse-plugin.integration.test.ts` for `runner creation and execution` test suite.
+
+3. Turn on debug mode  
+   Show debug logs of Lighthouse. Set the following environment variable: `DEBUG='*'`
+
+4. Understand error messages (⏳ could also be because of timeout problems :D )
+
+- Lighthouse error - `Runtime error encountered: No Chrome installations found.`
+  Read further under [chrome-path](#chrome-path)
+- Could not find `report.json` (⏳)
+  ![lighthouse-error-2.png](./docs/images/lighthouse-error-2.png)
+- Lighthouse Error - `Could Not Connect to Chrome` (⏳)
+  ![lighthouse-error-1.png](./docs/images/lighthouse-error-1.png)
+  Your Chrome path is set incorrectly. Read further under [chrome-path](#chrome-path)
+- Lighthouse Error - `start lh:<any>:<performancemark>" performance mark has not been set` (⏳)  
+  ![lighthouse-error-3.png](./docs/images/lighthouse-error-3.png)
+  If this error pops up you are able to launch Chrome but had problems to communicate over the ports.

package/README.md

@@ -0,0 +1,203 @@
+# @code-pushup/lighthouse-plugin
+
+[![npm](https://img.shields.io/npm/v/%40code-pushup%2Flighthouse-plugin.svg)](https://www.npmjs.com/package/@code-pushup/lighthouse-plugin)
+[![downloads](https://img.shields.io/npm/dm/%40code-pushup%2Flighthouse-plugin)](https://npmtrends.com/@code-pushup/lighthouse-plugin)
+[![dependencies](https://img.shields.io/librariesio/release/npm/%40code-pushup/lighthouse-plugin)](https://www.npmjs.com/package/@code-pushup/lighthouse-plugin?activeTab=dependencies)
+
+🕵️ **Code PushUp plugin for measuring web performance and quality with Lighthouse.** 🔥
+
+---
+
+The plugin parses your Lighthouse configuration and lints all audits of the official [Lighthouse](https://github.com/GoogleChrome/lighthouse/blob/main/readme.md#lighthouse-------) CLI.
+
+Detected Lighthouse audits are mapped to Code PushUp audits. Audit reports are calculated based on the [original implementation](https://googlechrome.github.io/lighthouse/scorecalc/).
+Additionally, Lighthouse categories are mapped to Code PushUp groups which can make it easier to assemble the categories.
+
+For more infos visit the [official docs](https://developer.chrome.com/docs/lighthouse/overview).
+
+## Getting started
+
+1. If you haven't already, install [@code-pushup/cli](../cli/README.md) and create a configuration file.
+
+2. Install as a dev dependency with your package manager:
+
+   ```sh
+   npm install --save-dev @code-pushup/lighthouse-plugin
+   ```
+
+   ```sh
+   yarn add --dev @code-pushup/lighthouse-plugin
+   ```
+
+   ```sh
+   pnpm add --save-dev @code-pushup/lighthouse-plugin
+   ```
+
+3. Add this plugin to the `plugins` array in your Code PushUp CLI config file (e.g. `code-pushup.config.ts`).
+
+   Pass in the URL you want to measure, along with optional [flags](#flags) and [config](#config) data.
+
+   ```ts
+   import lighthousePlugin from '@code-pushup/lighthouse-plugin';
+
+   export default {
+     // ...
+     plugins: [
+       // ...
+       await lighthousePlugin('https://example.com'),
+     ],
+   };
+   ```
+
+4. Run the CLI with `npx code-pushup collect` and view or upload the report (refer to [CLI docs](../cli/README.md)).
+
+### Optionally set up categories
+
+Reference audits (or groups) which you wish to include in custom categories (use `npx code-pushup print-config --onlyPlugins=lighthouse` to list audits and groups).
+
+Assign weights based on what influence each Lighthouse audit has on the overall category score (assign weight 0 to only include as extra info, without influencing category score).
+The plugin exports the helper `lighthouseAuditRef` and `lighthouseGroupRef` to reference Lighthouse category references for audits and groups.
+
+#### Reference audits directly with `lighthouseGroupRef`
+
+```ts
+import { lighthouseGroupRef } from './utils';
+
+export default {
+  // ...
+  categories: [
+    {
+      slug: 'performance',
+      title: 'Performance',
+      refs: [lighthouseGroupRef('performance')],
+    },
+    {
+      slug: 'a11y',
+      title: 'Accessibility',
+      refs: [lighthouseGroupRef('accessibility')],
+    },
+    {
+      slug: 'best-practices',
+      title: 'Best Practices',
+      refs: [lighthouseGroupRef('best-practices')],
+    },
+    {
+      slug: 'seo',
+      title: 'SEO',
+      refs: [lighthouseGroupRef('seo')],
+    },
+    {
+      slug: 'pwa',
+      title: 'PWA',
+      isBinary: true,
+      refs: [lighthouseGroupRef('pwa')],
+    },
+  ],
+};
+```
+
+#### Reference groups with `lighthouseAuditRef`
+
+The Lighthouse categories are reflected as groups.
+Referencing individual audits offers more granularity. However, keep maintenance costs of a higher number of audits in mind as well.
+
+```ts
+import { lighthouseAuditRef } from './utils';
+
+export default {
+  // ...
+  categories: [
+    {
+      slug: 'pwa',
+      title: 'PWA',
+      isBinary: true,
+      refs: [lighthouseAuditRef('installable-manifest', 2), lighthouseAuditRef('splash-screen', 1), lighthouseAuditRef('themed-omnibox', 1), lighthouseAuditRef('content-width', 1), lighthouseAuditRef('themed-omnibox', 2), lighthouseAuditRef('viewport', 2), lighthouseAuditRef('maskable-icon', 1), lighthouseAuditRef('pwa-cross-browser', 0), lighthouseAuditRef('pwa-page-transitions', 0), lighthouseAuditRef('pwa-each-page-has-url', 0)],
+    },
+  ],
+};
+```
+
+## Flags
+
+The plugin accepts a second optional argument, `flags`.
+
+`flags` is the Lighthouse [CLI flags](https://github.com/GoogleChrome/lighthouse/blob/7d80178c37a1b600ea8f092fc0b098029799a659/cli/cli-flags.js#L80) as a JS object.
+
+Within the flags object a couple of other external configuration files can be referenced. E.g. `configPath` , `preset` or `budgetPath` reference external `json` or JavaScript files.
+
+For a complete list the [official documentation of CLI flags](https://github.com/GoogleChrome/lighthouse/blob/main/readme.md#cli-options)
+
+> [!TIP]  
+> If you are not used to work with the Lighthouse CLI you would pass flags like this:
+> `lighthouse https://example.com --output=json --chromeFlags='--headless=shell'`
+>
+> Now with the plugin it would look like this:
+>
+> ```ts
+> // code-pushup.config.ts
+> ...
+> lighthousePlugin('https://example.com', { output: 'json', chromeFlags: ['--headless=shell']});
+> ```
+
+> [!note]
+> The following flags are **not supported** in the current implementation:
+>
+> - `list-all-audits` - Prints a list of all available audits and exits. Alternative: `npx code-pushup print-config --onlyPlugins lighthouse`
+> - `list-locales` - Prints a list of all supported locales and exits.
+> - `list-trace-categories` - Prints a list of all required trace categories and exits.
+> - `view` - Open HTML report in your browser
+
+## Chrome Flags for Tooling
+
+We recommend using Chrome flags for more stable runs in a tooling environment.
+The [`chrome-launcher`](https://www.npmjs.com/package/chrome-launcher) package provides a set of flags dedicated to tooling that they also documented very well.
+
+> ```ts
+> // code-pushup.config.ts
+> import { DEFAULT_FLAGS } from 'chrome-launcher/dist/flags.js';
+> ...
+> lighthousePlugin('https://example.com', { output: 'json', chromeFlags: DEFAULT_FLAGS });
+> ```
+
+## Config
+
+The plugin accepts a third optional argument, `config`.
+
+`config` is the Lighthouse [configuration](https://github.com/GoogleChrome/lighthouse/blob/7d80178c37a1b600ea8f092fc0b098029799a659/types/config.d.ts#L21) as a JS object.
+
+For a complete guide on Lighthouse configuration read the [official documentation on configuring](https://github.com/GoogleChrome/lighthouse/blob/main/docs/configuration.md)
+
+> [!TIP]  
+> If you are not used to work with the Lighthouse CLI you would pass a config like this:
+> `lighthouse --config-path=path/to/custom-config.js https://example.com`
+>
+> And in a separate file you would place the following object:
+>
+> ```typescript
+> // custom-config.js file
+> export default {
+>   extends: 'lighthouse:default',
+>   settings: {
+>     onlyAudits: ['first-meaningful-paint', 'speed-index', 'interactive'],
+>   },
+> };
+> ```
+>
+> Now with the plugin it would look like this:
+>
+> ```ts
+> // code-pushup.config.ts
+> ...
+> lighthousePlugin('https://example.com', undefined, {
+>   extends: 'lighthouse:default',
+>   settings: {
+>     onlyAudits: [
+>       'first-meaningful-paint',
+>       'speed-index',
+>       'interactive',
+>      ],
+>  }
+> })
+> ```
+
+If you want to contribute, please refer to [CONTRIBUTING.md](./CONTRIBUTING.md).