lighthouse 13.5.0-dev.20261008 → 13.5.0-dev.20261012

package/.agents/skills/update-ard-port/SKILL.md

@@ -0,0 +1,170 @@
+---
+name: update-ard-port
+description: Sync Lighthouse's Agentic Resource Discovery (ARD) port with upstream ards-project/ard-spec. Invoke when `node core/scripts/update-ard-spec.js --check` reports the ARD spec is out of sync, when the weekly cron CI job fails on it, or when asked to update the ARD conformance script, schema, or discovery rules.
+---
+
+# Update the ARD port
+
+Lighthouse vendors a JavaScript port of the upstream ARD conformance tester. When
+upstream changes, the port must be brought back to **1:1 parity** while preserving the
+Lighthouse-specific adaptations listed in `third-party/ard/README.md`.
+
+## What lives where
+
+| Upstream (ards-project/ard-spec) | Lighthouse | Notes |
+|---|---|---|
+| `conformance/bin/conformance-test` → `validate_manifest` and helpers | `third-party/ard/ard.js` | `ConformanceTester` (`validate_manifest`, `run_json_schema_validation`, `add_error/warning/info`), `parse_media_type`, `classify_media_type`, `format_schema_error`. Keep upstream `snake_case` names and **word-for-word message text**. |
+| `spec/schemas/ard-entry.schema.json` | `third-party/ard/spec/schemas/ard-entry.schema.json` | Must stay byte-identical to upstream. |
+| (generated) | `third-party/ard/schema-validator.js` | Precompiled Ajv standalone validator, built by `build/build-ard-schema.js` (`yarn build-ard-schema`). Never hand-edit. |
+| `spec/ard.md` §5.1 Discovery Mechanisms (+ `resolve_publisher` in the script) | `core/gather/gatherers/agentic/ard.js` | Candidate order: robots.txt `agentmap` → `<link rel="ard">` → `Link` header → `/.well-known/ard.json` → legacy `ai-catalog` equivalents. Source names in `Artifacts.ArdDiscoverySource` (`types/artifacts.d.ts`). |
+| severity / scoring (Lighthouse-only) | `core/audits/agentic/ard-schema.js` | errors → score 0; warnings/legacy/broken-link → 0.9; clean → 1. `infos` are not surfaced. |
+| pinned SHA + reviewed hashes + deviation list | `third-party/ard/README.md` | `**Pinned Commit SHA**` and the `**Reviewed Upstream Files**` SHA-256 lines are parsed/written by the update script. Never edit the hashes by hand — they are the proof that the port was reviewed against that exact upstream content. |
+
+Tests:
+- `third-party/ard/ard-test.js` — port unit tests
+- `core/test/gather/gatherers/agentic/ard-test.js` — gatherer
+- `core/test/audits/agentic/ard-schema-test.js` — audit
+- `cli/test/smokehouse/test-definitions/ard.js`, `ard-invalid.js` + fixtures in `cli/test/fixtures/agentic/`
+- `core/test/results/artifacts/artifacts.json` — sample `AgentResourceDiscovery` artifact
+
+## Procedure
+
+### 1. See what changed
+
+```
+node core/scripts/update-ard-spec.js --check
+```
+
+"In sync" means upstream HEAD matches the vendored schema byte-for-byte **and** the
+SHA-256 of `conformance/bin/conformance-test` and `spec/ard.md` match the hashes recorded
+in the README. The output lists which of the three differ, plus old/new SHA and a compare
+link. Pull the upstream files at **both** SHAs for side-by-side review (the compare API
+patch is often truncated):
+
+```
+https://raw.githubusercontent.com/ards-project/ard-spec/<sha>/conformance/bin/conformance-test
+https://raw.githubusercontent.com/ards-project/ard-spec/<sha>/spec/schemas/ard-entry.schema.json
+https://raw.githubusercontent.com/ards-project/ard-spec/<sha>/spec/ard.md
+```
+
+Save them in the conversation scratch dir and diff old vs new. Also check
+`conformance/` for new test fixtures/expected outputs and `docs/adr/` for ADRs explaining
+renamed or removed fields (e.g. `collections`, media type renames).
+
+### 2. Sync the schema
+
+```
+yarn update:ard-spec
+```
+
+This writes `third-party/ard/spec/schemas/ard-entry.schema.json` verbatim from upstream
+and prints the `conformance-test` diff. If a hand-ported file changed it **exits 1 and
+does not bump the pinned SHA** — that is expected; the SHA is bumped in step 8 after
+porting. Do not pass `--ack-port` yet.
+
+### 3. Port conformance-test changes into `third-party/ard/ard.js`
+
+For every hunk in the upstream `conformance-test` diff decide whether it touches:
+- **Validation rules** (new/removed checks, changed thresholds such as the
+  `representativeQueries` 2–5 range, new deprecated media types, URN regex) → port 1:1.
+- **Severity** (error ↔ warning ↔ info) → mirror it; this changes audit scoring.
+- **Message text** → copy exactly; the audit shows these strings to users and tests
+  assert on them.
+- **CLI/ANSI output, argparse, `main()`, `resolve_publisher`** → not ported (see README
+  deviations); skip unless the discovery logic itself changed (then see step 4).
+- **`format_schema_error`** → only adapt if upstream changes how schema errors are
+  phrased. Ajv's messages differ from Python `jsonschema`; keep the
+  `'x' is a required property` mapping.
+
+Rules to preserve:
+- Treat non-object roots/entries as empty instead of throwing.
+- Push `{element, message}` objects; never return formatted strings.
+- Log through `lighthouse-logger`, not `console`.
+- Only `ConformanceTester` and `classify_media_type` are exported; add exports only if
+  tests need them.
+
+If `classify_media_type` / `parse_media_type` changed, run a parity check against the
+real Python script on a list of tricky inputs (params, quoted strings, casing,
+deprecated/unknown `application/*` types) and fix any mismatch.
+
+### 4. If `spec/ard.md` changed
+
+Read §5.1 (and any section the script's `resolve_publisher` references). If the
+mechanisms, well-known path, `rel` names, header names, or precedence changed:
+- Update the candidate list/order in `core/gather/gatherers/agentic/ard.js`
+  (`resolveManifest`) and the `getArdLinksInDOM` rel names.
+- Update `Artifacts.ArdDiscoverySource` and `discoverySignals` in `types/artifacts.d.ts`.
+- Update `LEGACY_SOURCES` / UIStrings in `core/audits/agentic/ard-schema.js` if what counts
+  as legacy changed.
+
+### 5. Rebuild the validator (only if the schema changed)
+
+```
+yarn build-ard-schema
+```
+
+Review the `schema-validator.js` diff: it should only reflect schema changes. The build
+throws if any `require(` survives or a `$id` is missing — fix `build/build-ard-schema.js`
+rather than the output.
+
+### 6. Update tests and fixtures
+
+- `third-party/ard/ard-test.js`: add a case for every new/changed rule and message.
+- Gatherer/audit tests if discovery or scoring changed.
+- Smoke: `cli/test/fixtures/agentic/ard.json` / `ai-catalog-invalid.json` and the
+  expectations in `cli/test/smokehouse/test-definitions/ard-invalid.js`. Smokehouse
+  asserts the **exact count and order** of `details.items`.
+- `core/test/results/artifacts/artifacts.json` if the artifact shape changed.
+
+### 7. Update docs
+
+- README "Modifications for Lighthouse" if a new deviation was introduced, and the
+  "Updating Conformance Script" steps if the process changed.
+- `@fileoverview` comments in `ard.js`, the gatherer and the audit that mention the spec
+  version.
+- Audit `UIStrings` if user-facing wording or the spec link changed.
+
+### 8. Verify, then acknowledge
+
+```
+yarn mocha third-party/ard/ard-test.js core/test/gather/gatherers/agentic/ard-test.js core/test/audits/agentic/ard-schema-test.js
+yarn smoke ard ardInvalid
+yarn type-check
+yarn lint --fix
+yarn update:sample-json
+```
+
+`yarn update:sample-json` also runs `i18n:collect-strings`; audit tests resolve
+UIStrings through `shared/localization/locales/en-US.json`, so new strings fail tests
+until this runs. Do not run `yarn unit`.
+
+Only once everything above is green:
+
+```
+yarn update:ard-spec --ack-port                 # records reviewed hashes + bumps pinned SHA
+node core/scripts/update-ard-spec.js --check   # must now report in sync
+```
+
+`--ack-port` is the single statement that "the port matches this upstream content". Never
+run it to silence the check without having ported the diff.
+
+### 9. Report
+
+Summarize for the user: upstream SHA range, each rule/message/severity change and its
+effect on audit scoring, any discovery changes, and any deviation added to the README.
+Do not commit or open a PR unless asked.
+
+## Gotchas
+
+- The check compares *content*, not commit IDs. A green check after a deps upgrade means
+  nothing changed upstream, not that someone looked. `upgrade-deps.sh` only runs
+  `--check`; it cannot acknowledge changes.
+- `run_command` rejects `grep`; use `grep_search`, `awk`, or `sed`.
+- Upstream v0.91 has no `specVersion` check, logs unknown root members only, warns on
+  `collections` (removed by ADR-0003), does not check HTTPS, and does not validate
+  `capabilities`. Do not add Lighthouse-only checks to the port — put them in the audit
+  if ever wanted.
+- The `[label]` prefix upstream prepends to entry messages is carried in `element`, not
+  in `message`.
+- The gatherer, not the port, decides which manifest URL is validated; the port only
+  receives `raw_content` and `source_label`.

package/cli/cli-flags.js

@@ -20,7 +20,6 @@ import {isObjectOfUnknownValues} from '../shared/type-verifiers.js';
  */
 function getYargsParser(manualArgv) {
   const y = manualArgv ?
-    // @ts-expect-error - undocumented, but yargs() supports parsing a single `string`.
     yargs(manualArgv) :
     yargs(yargsHelpers.hideBin(process.argv));
 

package/cli/sentry-prompt.js

@@ -60,16 +60,15 @@ function prompt() {
 function askPermission() {
   return Promise.resolve().then(_ => {
     const configstore = new Configstore('lighthouse');
-    let isErrorReportingEnabled = configstore.get('isErrorReportingEnabled');
+    const isErrorReportingEnabled = configstore.get('isErrorReportingEnabled');
     if (typeof isErrorReportingEnabled === 'boolean') {
       return Promise.resolve(isErrorReportingEnabled);
     }
 
     return prompt()
       .then(response => {
-        isErrorReportingEnabled = response;
-        configstore.set('isErrorReportingEnabled', isErrorReportingEnabled);
-        return isErrorReportingEnabled;
+        configstore.set('isErrorReportingEnabled', response);
+        return response;
       });
   // Error accessing configstore; default to false.
   }).catch(_ => false);

package/cli/test/smokehouse/config/exclusions.js

@@ -23,7 +23,7 @@ const exclusions = {
     'redirects-http',
     // Disabled because these tests use settings that cannot be fully configured in
     // DevTools (e.g. throttling method "provided").
-    'metrics-tricky-tti', 'metrics-tricky-tti-late-fcp', 'screenshot',
+    'metrics-delayed-lcp', 'metrics-tricky-tti', 'metrics-tricky-tti-late-fcp', 'screenshot',
     // Disabled because of differences that need further investigation
     'byte-efficiency', 'byte-gzip', 'perf-preload',
     // Disabled because a renderer crash also breaks devtools frontend

package/core/audits/agentic/ard-schema.d.ts

@@ -1,5 +1,15 @@
 export default ArdSchema;
 declare class ArdSchema extends Audit {
+    /**
+     * @param {LH.Artifacts['AgentResourceDiscovery']['failedSources'][number]} failedSource
+     * @param {LH.IcuMessage} severity
+     * @return {{element: string, issue: LH.IcuMessage, severity: LH.IcuMessage}}
+     */
+    static makeFailedSourceItem({ url, status }: LH.Artifacts["AgentResourceDiscovery"]["failedSources"][number], severity: LH.IcuMessage): {
+        element: string;
+        issue: LH.IcuMessage;
+        severity: LH.IcuMessage;
+    };
     /**
      * @param {LH.Artifacts} artifacts
      * @return {LH.Audit.Product}
@@ -11,6 +21,9 @@ export namespace UIStrings {
     let failureTitle: string;
     let description: string;
     let explanation: string;
+    let legacyDiscovery: string;
+    let sourceUnavailable: string;
+    let sourceFetchFailed: string;
     let columnIssue: string;
     let columnSeverity: string;
     let itemSeverityError: string;

package/core/audits/agentic/ard-schema.js

@@ -5,29 +5,48 @@
  */
 
 /**
- * @fileoverview Audit that validates ai-catalog.json manifests against the Agentic Resource Discovery (ARD) specification.
+ * @fileoverview Audit that validates a site's Agentic Resource Discovery (ARD) manifest.
  *
- * This implementation is a direct JavaScript port of `validate_manifest` from the official ARD Conformance Test suite:
+ * Validation itself is done by `third-party/ard/ard.js`, a port of `validate_manifest` from the
+ * official ARD conformance test (see that directory's README for the pinned upstream commit).
+ * This audit maps its errors and warnings onto a score, and adds discovery-related items the
+ * conformance test reports in its publisher-resolution mode.
  * @see https://github.com/ards-project/ard-spec/blob/main/conformance/bin/conformance-test
  * @see https://agenticresourcediscovery.org/spec/
- * @version ARD Spec 1.0 / ADR-0003
  */
 
 import {Audit} from '../audit.js';
 import * as i18n from '../../lib/i18n/i18n.js';
 import {ConformanceTester} from '../../../third-party/ard/ard.js';
 
+const HTTP_CLIENT_ERROR_CODE_LOW = 400;
+const HTTP_SERVER_ERROR_CODE_LOW = 500;
+
 const UIStrings = {
-  /** Title of a Lighthouse audit that evaluates whether ai-catalog.json conforms to the ARD specification. Shown when valid. */
-  title: '`ai-catalog.json` schema is valid',
-  /** Title of a Lighthouse audit that evaluates whether ai-catalog.json conforms to the ARD specification. Shown when invalid. */
-  failureTitle: '`ai-catalog.json` schema is invalid',
-  /** Description of a Lighthouse audit that tells the user why ai-catalog.json must match the ARD specification. */
-  description: 'Valid `ai-catalog.json` manifests are required for autonomous ' +
-    'AI agents and registries to discover and verify your resources. ' +
+  /** Title of a Lighthouse audit that evaluates whether the site's Agentic Resource Discovery (ARD) manifest conforms to the ARD specification. Shown when valid. "ARD" should not be translated. */
+  title: 'ARD manifest is valid',
+  /** Title of a Lighthouse audit that evaluates whether the site's Agentic Resource Discovery (ARD) manifest conforms to the ARD specification. Shown when invalid. "ARD" should not be translated. */
+  failureTitle: 'ARD manifest is invalid',
+  /** Description of a Lighthouse audit that tells the user why their Agentic Resource Discovery (ARD) manifest must match the ARD specification. "ARD", "/.well-known/ard.json" and "rel="ard"" should not be translated. This is displayed after a user expands the section to see more. No character length limits. The last sentence starting with 'Learn' becomes link text to additional documentation. */
+  description: 'A valid Agentic Resource Discovery (ARD) manifest, served at ' +
+    '`/.well-known/ard.json` or linked with `rel="ard"`, lets AI agents and registries ' +
+    'discover and verify your resources. ' +
     '[Learn more about the ARD specification](https://agenticresourcediscovery.org/spec/).',
-  /** Explanatory message stating that ai-catalog.json could not be loaded for schema validation. */
-  explanation: 'Catalog file could not be loaded for schema validation.',
+  /** Explanatory message stating that the Agentic Resource Discovery (ARD) manifest could not be loaded for schema validation. "ARD" should not be translated. */
+  explanation: 'ARD manifest could not be loaded for schema validation.',
+  /** Table item describing that the Agentic Resource Discovery (ARD) manifest was only found at a legacy location that ARD consumers are not required to check. "ARD", "/.well-known/ai-catalog.json", "rel="ai-catalog"", "/.well-known/ard.json" and "rel="ard"" should not be translated. */
+  legacyDiscovery: 'Manifest was only found through a legacy location ' +
+    '(`/.well-known/ai-catalog.json` or `rel="ai-catalog"`). ARD consumers are not ' +
+    'required to check these, so ARD consumers may not discover your resources. Serve the ' +
+    'manifest at `/.well-known/ard.json` or link it with `rel="ard"`.',
+  /**
+   * @description Table item describing that a location the site advertises for its Agentic Resource Discovery (ARD) manifest (for example, a link on the page) returned an unsuccessful HTTP status code. "ARD" should not be translated.
+   * @example {404} statusCode
+   */
+  sourceUnavailable: 'Advertised ARD manifest location could not be loaded ' +
+    '(HTTP status {statusCode}).',
+  /** Table item describing that a location the site advertises for its Agentic Resource Discovery (ARD) manifest (for example, a link on the page) could not be fetched at all. "ARD" should not be translated. */
+  sourceFetchFailed: 'Advertised ARD manifest location could not be loaded.',
   /** Header of the table column which displays the issue. */
   columnIssue: 'Issue',
   /** Header of the table column which displays the severity. */
@@ -36,6 +55,9 @@ const UIStrings = {
   itemSeverityError: 'Error',
 };
 
+/** @type {Set<LH.Artifacts.ArdDiscoverySource>} */
+const LEGACY_SOURCES = new Set(['legacyHtmlLink', 'legacyHttpHeaderLink', 'legacyWellKnown']);
+
 const str_ = i18n.createIcuMessageFn(import.meta.url, UIStrings);
 
 class ArdSchema extends Audit {
@@ -53,6 +75,18 @@ class ArdSchema extends Audit {
     };
   }
 
+  /**
+   * @param {LH.Artifacts['AgentResourceDiscovery']['failedSources'][number]} failedSource
+   * @param {LH.IcuMessage} severity
+   * @return {{element: string, issue: LH.IcuMessage, severity: LH.IcuMessage}}
+   */
+  static makeFailedSourceItem({url, status}, severity) {
+    const issue = status === null ?
+      str_(UIStrings.sourceFetchFailed) :
+      str_(UIStrings.sourceUnavailable, {statusCode: status});
+    return {element: url, issue, severity};
+  }
+
   /**
    * @param {LH.Artifacts} artifacts
    * @return {LH.Audit.Product}
@@ -64,30 +98,26 @@ class ArdSchema extends Audit {
     const hasExplicitSignal = Boolean(
       signals.robotsTxtAgentmap ||
       signals.htmlLink ||
-      signals.httpHeaderLink
+      signals.httpHeaderLink ||
+      signals.legacyHtmlLink ||
+      signals.legacyHttpHeaderLink
+    );
+    const isClientError = Boolean(
+      ard.status &&
+      ard.status >= HTTP_CLIENT_ERROR_CODE_LOW &&
+      ard.status < HTTP_SERVER_ERROR_CODE_LOW
     );
-    const hasCatalog = hasExplicitSignal || ard.status === 200;
 
-    if (!hasCatalog) {
+    if (!hasExplicitSignal && isClientError) {
       return {
         score: 1,
         notApplicable: true,
       };
     }
 
-    if (ard.status !== 200 || !ard.content) {
-      return {
-        score: 0,
-        explanation: str_(UIStrings.explanation),
-      };
-    }
-
     const itemSeverityError = str_(UIStrings.itemSeverityError);
     const itemSeverityLow = str_(i18n.UIStrings.itemSeverityLow);
 
-    /** @type {Array<{element: string, issue: string | LH.IcuMessage, severity: LH.IcuMessage}>} */
-    const issues = [];
-
     /** @type {LH.Audit.Details.Table['headings']} */
     const headings = [
       {key: 'element', valueType: 'text', label: str_(i18n.UIStrings.columnElement)},
@@ -95,8 +125,21 @@ class ArdSchema extends Audit {
       {key: 'severity', valueType: 'text', label: str_(UIStrings.columnSeverity)},
     ];
 
+    if (!ard.status || ard.status >= HTTP_CLIENT_ERROR_CODE_LOW || !ard.content) {
+      const failedItems = ard.failedSources.map(source =>
+        ArdSchema.makeFailedSourceItem(source, itemSeverityError));
+      return {
+        score: 0,
+        explanation: str_(UIStrings.explanation),
+        details: failedItems.length ? Audit.makeTableDetails(headings, failedItems) : undefined,
+      };
+    }
+
+    /** @type {Array<{element: string, issue: string | LH.IcuMessage, severity: LH.IcuMessage}>} */
+    const issues = [];
+
     const tester = new ConformanceTester();
-    tester.validate_manifest(ard.content, 'ai-catalog.json');
+    tester.validate_manifest(ard.content, ard.catalogUrl);
 
     for (const err of tester.errors) {
       issues.push({
@@ -114,16 +157,21 @@ class ArdSchema extends Audit {
       });
     }
 
-    const hasErrors = issues.some(i => i.severity === itemSeverityError);
-    const hasLowSeverity = issues.some(i => i.severity === itemSeverityLow);
+    for (const failedSource of ard.failedSources) {
+      issues.push(ArdSchema.makeFailedSourceItem(failedSource, itemSeverityLow));
+    }
 
-    let score = 1;
-    if (hasErrors) {
-      score = 0;
-    } else if (hasLowSeverity) {
-      score = 0.9;
+    if (LEGACY_SOURCES.has(ard.discoverySource)) {
+      issues.push({
+        element: ard.catalogUrl,
+        issue: str_(UIStrings.legacyDiscovery),
+        severity: itemSeverityLow,
+      });
     }
 
+    const hasErrors = tester.errors.length > 0;
+    const score = hasErrors ? 0 : (issues.length ? 0.9 : 1);
+
     return {
       score,
       details: issues.length ? Audit.makeTableDetails(headings, issues) : undefined,

package/core/audits/agentic/llms-txt.js

@@ -82,7 +82,7 @@ class LlmsTxt extends Audit {
       };
     }
 
-    if (status >= HTTP_SERVER_ERROR_CODE_LOW) {
+    if (status >= HTTP_SERVER_ERROR_CODE_LOW || status === 429) {
       return {
         score: 0,
         displayValue: str_(UIStrings.displayValueHttpBadCode, {statusCode: status}),
@@ -95,7 +95,10 @@ class LlmsTxt extends Audit {
     }
 
     if (content === null) {
-      throw new Error(`Status ${status} was valid, but content was null`);
+      return {
+        score: 0,
+        explanation: str_(UIStrings.explanation),
+      };
     }
 
     const hasH1 = /^\s*#\s+.+/m.test(content);

package/core/audits/long-tasks.js

@@ -192,9 +192,10 @@ class LongTasks extends Audit {
       taskTimingsByEvent = new Map();
 
       const simulatorOptions = {devtoolsLog, settings: context.settings};
+      const fromTrace = process.env.INTERNAL_LANTERN_USE_TRACE !== undefined;
       const pageGraph =
         // eslint-disable-next-line max-len
-        await PageDependencyGraph.request({settings, trace, devtoolsLog, URL, SourceMaps, HostDPR, fromTrace: false}, context);
+        await PageDependencyGraph.request({settings, trace, devtoolsLog, URL, SourceMaps, HostDPR, fromTrace}, context);
       const simulator = await LoadSimulator.request(simulatorOptions, context);
       const simulation = simulator.simulate(pageGraph, {label: 'long-tasks-diagnostic'});
       for (const [node, timing] of simulation.nodeTimings.entries()) {

package/core/audits/screenshot-thumbnails.d.ts

@@ -2,19 +2,6 @@ export default ScreenshotThumbnails;
 export type SpeedlineFrame = LH.Artifacts.Speedline["frames"][0];
 /** @typedef {LH.Artifacts.Speedline['frames'][0]} SpeedlineFrame */
 declare class ScreenshotThumbnails extends Audit {
-    /**
-     * Scales down an image to THUMBNAIL_WIDTH using nearest neighbor for speed, maintains aspect
-     * ratio of the original thumbnail.
-     *
-     * @param {ReturnType<SpeedlineFrame['getParsedImage']>} imageData
-     * @param {number} scaledWidth
-     * @return {{width: number, height: number, data: Uint8Array}}
-     */
-    static scaleImageToThumbnail(imageData: ReturnType<SpeedlineFrame["getParsedImage"]>, scaledWidth: number): {
-        width: number;
-        height: number;
-        data: Uint8Array;
-    };
     /**
      * @param {LH.Artifacts} artifacts
      * @param {LH.Audit.Context} context

package/core/audits/screenshot-thumbnails.js

@@ -4,8 +4,6 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import jpeg from 'jpeg-js';
-
 import {Audit} from './audit.js';
 import {LighthouseError} from '../lib/lh-error.js';
 import {Speedline} from '../computed/speedline.js';
@@ -28,42 +26,6 @@ class ScreenshotThumbnails extends Audit {
     };
   }
 
-  /**
-   * Scales down an image to THUMBNAIL_WIDTH using nearest neighbor for speed, maintains aspect
-   * ratio of the original thumbnail.
-   *
-   * @param {ReturnType<SpeedlineFrame['getParsedImage']>} imageData
-   * @param {number} scaledWidth
-   * @return {{width: number, height: number, data: Uint8Array}}
-   */
-  static scaleImageToThumbnail(imageData, scaledWidth) {
-    const scaleFactor = imageData.width / scaledWidth;
-    const scaledHeight = Math.floor(imageData.height / scaleFactor);
-
-    const outPixels = new Uint8Array(scaledWidth * scaledHeight * 4);
-
-    for (let i = 0; i < scaledWidth; i++) {
-      for (let j = 0; j < scaledHeight; j++) {
-        const origX = Math.floor(i * scaleFactor);
-        const origY = Math.floor(j * scaleFactor);
-
-        const origPos = (origY * imageData.width + origX) * 4;
-        const outPos = (j * scaledWidth + i) * 4;
-
-        outPixels[outPos] = imageData.data[origPos];
-        outPixels[outPos + 1] = imageData.data[origPos + 1];
-        outPixels[outPos + 2] = imageData.data[origPos + 2];
-        outPixels[outPos + 3] = imageData.data[origPos + 3];
-      }
-    }
-
-    return {
-      width: scaledWidth,
-      height: scaledHeight,
-      data: outPixels,
-    };
-  }
-
   /**
    * @param {LH.Artifacts} artifacts
    * @param {LH.Audit.Context} context
@@ -79,7 +41,6 @@ class ScreenshotThumbnails extends Audit {
     // Make the minimum time range 3s so sites that load super quickly don't get a single screenshot
     const minimumTimelineDuration = context.options.minimumTimelineDuration || 3000;
     const numberOfThumbnails = context.options.numberOfThumbnails || NUMBER_OF_THUMBNAILS;
-    const thumbnailWidth = context.options.thumbnailWidth || null;
 
     const thumbnails = [];
     const analyzedFrames = speedline.frames.filter(frame => !frame.isProgressInterpolated());
@@ -112,12 +73,6 @@ class ScreenshotThumbnails extends Audit {
       const cachedThumbnail = cachedThumbnails.get(frameForTimestamp);
       if (cachedThumbnail) {
         base64Data = cachedThumbnail;
-      } else if (thumbnailWidth !== null) {
-        const imageData = frameForTimestamp.getParsedImage();
-        const thumbnailImageData =
-          ScreenshotThumbnails.scaleImageToThumbnail(imageData, thumbnailWidth);
-        base64Data = jpeg.encode(thumbnailImageData, 90).data.toString('base64');
-        cachedThumbnails.set(frameForTimestamp, base64Data);
       } else {
         base64Data = frameForTimestamp.getImage().toString('base64');
         cachedThumbnails.set(frameForTimestamp, base64Data);

package/core/audits/seo/robots-txt.js

@@ -27,7 +27,7 @@ const DIRECTIVE_SAFELIST = new Set([
   DIRECTIVE_ALLOW, DIRECTIVE_SITEMAP, // universally supported
   'crawl-delay', // yahoo, bing, yandex
   'clean-param', 'host', // yandex
-  'request-rate', 'visit-time', 'noindex', 'content-signal', // not officially supported, but used in the wild
+  'request-rate', 'visit-time', 'noindex', 'content-signal', 'license', // not officially supported, but used in the wild
 ]);
 const SITEMAP_VALID_PROTOCOLS = new Set(['https:', 'http:', 'ftp:']);
 
@@ -209,7 +209,7 @@ class RobotsTxt extends Audit {
     } = artifacts.RobotsTxt;
 
     // Do specific error messages first.
-    if (status && status >= HTTP_SERVER_ERROR_CODE_LOW) {
+    if (status && (status >= HTTP_SERVER_ERROR_CODE_LOW || status === 429)) {
       return {
         score: 0,
         displayValue: str_(UIStrings.displayValueHttpBadCode, {statusCode: status}),
@@ -228,18 +228,13 @@ class RobotsTxt extends Audit {
       };
     }
 
-    if (!status) {
+    if (!status || content === null) {
       return {
         score: 0,
         explanation: str_(UIStrings.explanation),
       };
     }
 
-    // If status is good, content must be not null.
-    if (content === null) {
-      throw new Error(`Status ${status} was valid, but content was null`);
-    }
-
     const validationErrors = validateRobots(content);
 
     /** @type {LH.Audit.Details.Table['headings']} */

package/core/audits/unsized-images.d.ts

@@ -1,12 +1,5 @@
 export default UnsizedImages;
 declare class UnsizedImages extends Audit {
-    /**
-     * An img size attribute prevents layout shifts if it is a non-negative integer (incl zero!).
-     * @url https://html.spec.whatwg.org/multipage/embedded-content-other.html#dimension-attributes
-     * @param {string | null} attrValue
-     * @return {boolean}
-     */
-    static doesHtmlAttrProvideExplicitSize(attrValue: string | null): boolean;
     /**
      * An img css size property prevents layout shifts if it is defined, not empty, and not equal to 'auto'.
      * @param {string | null} property

package/core/audits/unsized-images.js

@@ -10,6 +10,7 @@
 
 
 import {Audit} from './audit.js';
+import ImageElements from '../gather/gatherers/image-elements.js';
 import * as i18n from './../lib/i18n/i18n.js';
 import UrlUtils from '../lib/url-utils.js';
 
@@ -40,28 +41,6 @@ class UnsizedImages extends Audit {
     };
   }
 
-  /**
-   * An img size attribute prevents layout shifts if it is a non-negative integer (incl zero!).
-   * @url https://html.spec.whatwg.org/multipage/embedded-content-other.html#dimension-attributes
-   * @param {string | null} attrValue
-   * @return {boolean}
-   */
-  static doesHtmlAttrProvideExplicitSize(attrValue) {
-    if (!attrValue) return false;
-
-    // First, superweird edge case of using the positive-sign. The spec _sorta_ says it's valid...
-    // https://html.spec.whatwg.org/multipage/common-microsyntaxes.html#rules-for-parsing-integers
-    //   > Otherwise, if the character is … (+): Advance position to the next character.
-    //   > (The "+" is ignored, but it is not conforming.)
-    // lol.  Irrelevant though b/c Chrome (at least) doesn't ignore. It rejects this as a non-conforming value.
-    if (attrValue.startsWith('+')) return false;
-
-    // parseInt isn't exactly the same as the html's spec for parsing integers, but it's close enough
-    // https://tc39.es/ecma262/#sec-parseint-string-radix
-    const int = parseInt(attrValue, 10);
-    return int >= 0;
-  }
-
   /**
    * An img css size property prevents layout shifts if it is defined, not empty, and not equal to 'auto'.
    * @param {string | null} property
@@ -78,6 +57,12 @@ class UnsizedImages extends Audit {
    * @return {boolean}
    */
   static isSizedImage(image) {
+    const attrWidth = image.attributeWidth;
+    const attrHeight = image.attributeHeight;
+    const htmlWidthIsExplicit = ImageElements.doesHtmlAttrProvideExplicitSize(attrWidth);
+    const htmlHeightIsExplicit = ImageElements.doesHtmlAttrProvideExplicitSize(attrHeight);
+    if (htmlWidthIsExplicit && htmlHeightIsExplicit) return true;
+
     // Perhaps we hit reachedGatheringBudget before collecting this image's cssWidth/Height
     // in fetchSourceRules. In this case, we don't have enough information to determine if it's sized.
     // We don't want to show the user a false positive, so we'll call it sized to give it as pass.
@@ -85,14 +70,10 @@ class UnsizedImages extends Audit {
     // Handwavey TODO: explore ways to avoid this.
     if (image.cssEffectiveRules === undefined) return true;
 
-    const attrWidth = image.attributeWidth;
-    const attrHeight = image.attributeHeight;
     const cssWidth = image.cssEffectiveRules.width;
     const cssHeight = image.cssEffectiveRules.height;
     const cssAspectRatio = image.cssEffectiveRules.aspectRatio;
-    const htmlWidthIsExplicit = UnsizedImages.doesHtmlAttrProvideExplicitSize(attrWidth);
     const cssWidthIsExplicit = UnsizedImages.isCssPropExplicitlySet(cssWidth);
-    const htmlHeightIsExplicit = UnsizedImages.doesHtmlAttrProvideExplicitSize(attrHeight);
     const cssHeightIsExplicit = UnsizedImages.isCssPropExplicitlySet(cssHeight);
     const explicitAspectRatio = UnsizedImages.isCssPropExplicitlySet(cssAspectRatio);
     const explicitWidth = htmlWidthIsExplicit || cssWidthIsExplicit;