kjeks-scanner 0.2.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Per Søderlind
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,186 @@
+# Kjeks discovery scanner
+
+A standalone Playwright scanner that discovers cookies and similar technologies
+across a WordPress Multisite, in each consent state. It runs **separately from
+the WordPress runtime** — nothing here is loaded by the plugin.
+
+Part of the **kjeks family**. It integrates with the
+[Kjeks plugin](https://github.com/soderlind/kjeks) over REST only
+(`scan-config` + `import`); see the
+[kjeks ecosystem overview](https://github.com/soderlind/kjeks/blob/main/docs/architecture.md#9-ecosystem-the-kjeks-family).
+
+> Discovery is observational. It records what a real browser encountered; it
+> **cannot prove the absence** of tracking, and results vary by geo/IP. Imported
+> observations are always **unreviewed** until an administrator classifies them.
+
+## Why real Chromium
+
+The scanner drives a version-pinned, real headless Chromium via Playwright's
+CDP-backed API. Only CDP exposes HttpOnly cookies and full network events, and
+only a real browser executes third-party code the way a visitor's browser does.
+See [docs/adr/0005-scanner-uses-real-chromium.md](docs/adr/0005-scanner-uses-real-chromium.md).
+
+## How it works end-to-end
+
+1. **Select** — the WordPress plugin's `scan-config` auto-picks representative URLs per
+   site (home, newest post/page, embed-bearing pages), capped. The scanner fetches this
+   over REST or from a file.
+2. **Scan** — for each site (in a bounded parallel pool with a per-host ceiling) the
+   scanner opens a fresh browser context per consent state, injects the consent record,
+   visits the URLs, and records cookies, storage, third-party requests, scripts, iframes,
+   and beacons.
+3. **Attribute** — every observation is tagged with `source_urls`, the page(s) it loaded on.
+4. **Diff** — results are written as one deterministic JSON file per site and diffed
+   against the committed baseline; any change exits non-zero for review.
+5. **Target** — the next run always re-scans pages that previously produced a tracker
+   (from `source_urls`), so coverage never silently regresses.
+6. **Import** — observations are POSTed to the plugin as *unreviewed* (in the same run
+   with `--import`, or separately) for an administrator to classify — optionally via the
+   [AI Reviewer](https://github.com/soderlind/kjeks-ai-reviewer).
+
+Design rationale is recorded under [docs/adr/](docs/adr): see
+[0006 — REST-driven scanning](docs/adr/0006-rest-driven-scanning.md) and
+[0005 — real Chromium](docs/adr/0005-scanner-uses-real-chromium.md).
+
+## Install
+
+```bash
+npm ci
+npx playwright install --with-deps chromium
+```
+
+### Or run without installing (npx)
+
+Run it directly — no clone required:
+
+```bash
+npx kjeks-scanner --config-url "https://network.example.com/wp-json/kjeks/v1/scan-config" --out scan
+```
+
+Use the package name with npx: `npx kjeks-scanner`. After a global or local
+install, both the `kjeks-scanner` and `kjeks-scan` commands are available.
+The first run downloads a pinned Chromium (~100+ MB) via Playwright; later runs
+reuse it.
+
+## Run a scan
+
+```bash
+# From a config file (recommended for multisite):
+node src/cli.js --config config.json --out scan
+
+# Or a single URL:
+node src/cli.js --url https://example.com --blog-id 1 --out scan
+
+# Fetch the site list from WordPress over REST (recommended for CI):
+KJEKS_USER=admin KJEKS_APP_PASSWORD='xxxx xxxx xxxx xxxx' \
+  node src/cli.js --config-url "https://network.example.com/wp-json/kjeks/v1/scan-config" --out scan
+
+# Against Cloudflare Browser Run instead of local Chromium (opt-in):
+node src/cli.js --config config.json --endpoint "wss://…/browser-run/…"
+
+# Scan several sites in parallel (default 3; polite per-host cap 2):
+node src/cli.js --config-url "https://network.example.com/wp-json/kjeks/v1/scan-config" --concurrency 4 --out scan
+
+# Scan and import in one step:
+KJEKS_USER=admin KJEKS_APP_PASSWORD='xxxx xxxx xxxx xxxx' \
+  node src/cli.js --config-url "https://network.example.com/wp-json/kjeks/v1/scan-config" --import --out scan
+```
+
+### Flags
+
+| Flag | Default | Purpose |
+| --- | --- | --- |
+| `--concurrency <n>` | 3 | Sites scanned in parallel. |
+| `--per-host <n>` | 2 | Parallel scans allowed to share one hostname (politeness for subdirectory multisites on one server). |
+| `--full` | off | Scan the server selection as-is; skip re-scanning pages that previously produced a tracker. |
+| `--import [<url>]` | off | After scanning, POST observations to the Kjeks import endpoint. Base URL from the value, `--site`, or `--config-url`. |
+
+Config shape: `{ "sites": [ { "url", "blog_id", "policy_version", "paths",
+"scenarios" } ] }`. Generate it with `wp kjeks scan-config` or fetch it over
+REST (below); `paths`/`scenarios` shape is shown in `overlay.example.json`.
+
+### Where the config comes from
+
+Three interchangeable sources:
+
+1. **Static file** — `--config config.json`.
+2. **REST** — `--config-url .../wp-json/kjeks/v1/scan-config` fetches the live
+   site list (auth: `KJEKS_USER` + `KJEKS_APP_PASSWORD`, caller needs
+   `manage_network`). Best for CI: no committed config, new subsites appear
+   automatically. Add `--overlay overlay.json` to merge repo-side `paths` and
+   `scenarios` by `blog_id` (see `overlay.example.json`).
+3. **WP-CLI** — generate a static file locally:
+
+   ```bash
+   wp kjeks scan-config > config.json                    # auto-selects URLs per site
+   wp kjeks scan-config --cap=15 --include=1,3            # cap the auto-selection
+   wp kjeks scan-config --paths=/,/about > config.json    # explicit paths (override)
+   ```
+
+Omit `--paths` (CLI) or the `paths` query param (REST) and the plugin
+**auto-selects representative URLs per site** via `WP_Query` — the home page, the
+newest post and page, the posts archive, and pages whose content shows an embed /
+inline-script signal — capped by `--cap` (default 10). Passing explicit paths
+overrides the selection.
+
+The REST endpoint and `wp kjeks scan-config` share the same builder, so they
+produce identical output.
+
+For every site the scanner opens a **fresh browser context** per consent state:
+`before-choice`, `reject-all`, `only-preferences`, `only-analytics`,
+`only-marketing`, `accept-all`. It collects Set-Cookie headers, context cookies
+(incl. HttpOnly/Secure/SameSite), `document.cookie`, localStorage,
+sessionStorage, IndexedDB names, third-party requests, redirects, scripts,
+iframes, and beacon/pixel requests.
+
+Each derived observation records **`source_urls`** — the page(s) it actually
+loaded on — so reviewers can see *where* a tracker fires. On the next run the
+scanner always re-scans those pages (on top of the server selection), so a page
+that once produced a tracker is never dropped by sampling. Use `--full` to scan
+the server selection as-is.
+
+## Output and diff
+
+One deterministic JSON file per site at `<out>/<host>.json` (stable key order,
+volatile fields normalized). The CLI prints a per-subsite diff (new / changed /
+removed) against the previous file and exits non-zero when anything changed, so
+CI can flag it for review. Commit `scan/<host>.json` as the baseline.
+
+## Import into WordPress
+
+```bash
+KJEKS_USER=admin KJEKS_APP_PASSWORD='xxxx xxxx xxxx xxxx' \
+  node src/import.js --site https://network.example.com scan/*.json
+```
+
+Uses a WordPress application password (HTTP Basic) against
+`/wp-json/kjeks/v1/import`; the caller needs `manage_network`. Never commit the
+password — pass it via environment / CI secrets. Locally you can instead use
+`wp kjeks import <file>`.
+
+To scan and import in a single command, pass `--import` to `src/cli.js` (see
+Flags above) instead of running `src/import.js` separately.
+
+## Tests
+
+```bash
+npm run test:unit                       # fast unit tests (node:test), no browser
+BASE_URL=http://plugins.local/ npx playwright test   # end-to-end, needs a site
+```
+
+The end-to-end suite confirms optional cookies, storage, and third-party requests
+do not occur before a consent choice, and that gated scripts stay inert.
+
+## Scheduled scanning
+
+`.github/workflows/scan.yml` runs the scan weekly, uploads the full artifact,
+imports observations (if secrets are set), and commits baseline changes. Set
+`KJEKS_SITE_URL`, `KJEKS_USER`, and `KJEKS_APP_PASSWORD` as repository secrets.
+
+## Known limitations
+
+- First/third-party classification uses an eTLD+1 approximation (last two
+  labels), not the full Public Suffix List.
+- Consent states are injected via the shared record schema, not by clicking the
+  banner; a UI regression could pass the state matrix yet break the real banner.
+- A single run from one location cannot capture geo/consent variations.

package/package.json

@@ -0,0 +1,47 @@
+{
+	"name": "kjeks-scanner",
+	"version": "0.2.0",
+	"description": "Standalone Playwright discovery scanner for the Kjeks cookie-consent plugin. Runs separately from the WordPress runtime.",
+	"license": "MIT",
+	"author": "Per Søderlind",
+	"keywords": [
+		"kjeks",
+		"cookie-consent",
+		"playwright",
+		"tracker-scanner",
+		"cookie-scanner",
+		"gdpr",
+		"privacy",
+		"wordpress"
+	],
+	"homepage": "https://github.com/soderlind/kjeks-scanner",
+	"repository": {
+		"type": "git",
+		"url": "https://github.com/soderlind/kjeks-scanner.git"
+	},
+	"private": false,
+	"type": "module",
+	"files": [
+		"src",
+		"README.md",
+		"LICENSE"
+	],
+	"bin": {
+		"kjeks-scan": "src/cli.js",
+		"kjeks-scanner": "src/cli.js"
+	},
+	"engines": {
+		"node": ">=20.0.0"
+	},
+	"scripts": {
+		"scan": "node src/cli.js",
+		"test": "playwright test",
+		"test:unit": "node --test tests/*.test.js"
+	},
+	"dependencies": {
+		"playwright": "^1.48.0"
+	},
+	"devDependencies": {
+		"@playwright/test": "^1.48.0"
+	}
+}

package/src/cli.js

@@ -0,0 +1,278 @@
+#!/usr/bin/env node
+/**
+ * Kjeks discovery scanner CLI.
+ *
+ * Usage:
+ *   node src/cli.js --config config.json --out scan
+ *   node src/cli.js --url https://example.com --blog-id 1 --out scan
+ *   node src/cli.js --config-url https://net.example/wp-json/kjeks/v1/scan-config --out scan
+ *   node src/cli.js --config-url <url> --overlay overlay.json --out scan
+ *   node src/cli.js --config config.json --endpoint wss://…browser-run…  (Browser Run)
+ *
+ * --config-url fetches the site list from WordPress (auth: KJEKS_USER +
+ * KJEKS_APP_PASSWORD env). --overlay merges repo-side paths/scenarios by blog_id.
+ *
+ * Options:
+ *   --concurrency <n>  Sites scanned in parallel (default 3).
+ *   --per-host <n>     Parallel scans sharing a hostname (default 2).
+ *   --full             Scan the server selection as-is; skip re-scanning pages
+ *                      that previously produced a tracker.
+ *   --import [<url>]   After scanning, POST observations to the Kjeks import
+ *                      endpoint (base from the value, --site, or --config-url).
+ *
+ * Writes one deterministic JSON file per site to <out>/<host>[_<path>].json and prints a
+ * per-subsite diff against any previous file of the same name. Exits non-zero when a
+ * subsite changed, a site errored, or an import failed.
+ */
+
+import { readFile, writeFile, mkdir } from 'node:fs/promises';
+import { existsSync } from 'node:fs';
+import { join } from 'node:path';
+import { runScan } from './scan.js';
+import { diffScans } from './diff.js';
+import { priorTrackerPaths, mergePaths } from './targeting.js';
+import { importSites } from './import.js';
+
+async function main() {
+	const args = parseArgs( process.argv.slice( 2 ) );
+	const outDir = args.out || 'scan';
+
+	await mkdir( outDir, { recursive: true } );
+
+	// Targeted re-scan: always revisit pages that previously produced a tracker,
+	// on top of the server-selected paths. --full scans the selection as-is.
+	const config = await applyTargeting( await loadConfig( args ), outDir, Boolean( args.full ) );
+
+	const result = await runScan( config, {
+		endpoint: args.endpoint,
+		concurrency: args.concurrency ? Number( args.concurrency ) : undefined,
+		perHost: args[ 'per-host' ] ? Number( args[ 'per-host' ] ) : undefined,
+	} );
+	let anyChanged = false;
+
+	for ( const site of result.sites ) {
+		const slug = siteFileSlug( site );
+		const file = join( outDir, `${ slug }.json` );
+		const previous = existsSync( file ) ? JSON.parse( await readFile( file, 'utf8' ) ) : null;
+
+		const [ siteDiff ] = diffScans( previous, { sites: [ site ] } );
+
+		const single = { generated_at: result.generated_at, sites: [ site ] };
+		await writeFile( file, stableStringify( single ) + '\n', 'utf8' );
+
+		printDiff( slug, siteDiff );
+
+		if ( siteDiff && ( siteDiff.added.length || siteDiff.changed.length || siteDiff.removed.length ) ) {
+			anyChanged = true;
+		}
+	}
+
+	// Surface isolated per-site scan failures (the run continued past them).
+	const scanErrors = result.errors || [];
+	for ( const failure of scanErrors ) {
+		process.stderr.write( `scan error (blog ${ failure.site && failure.site.blog_id }): ${ failure.message }\n` );
+	}
+
+	// Optional: import the reviewed-less observations in the same run.
+	let importFailures = 0;
+	if ( args.import ) {
+		const base = ( typeof args.import === 'string' ? args.import : null )
+			|| args.site
+			|| originOf( args[ 'config-url' ] );
+		if ( ! base ) {
+			throw new Error( '--import needs a base URL: use --import <url>, --site <url>, or --config-url.' );
+		}
+		const outcome = await importSites( base, result.sites, {
+			log: ( message, level ) => ( level === 'error' ? process.stderr : process.stdout ).write( message + '\n' ),
+		} );
+		importFailures = outcome.failures;
+	}
+
+	// Non-zero exit if any subsite changed, any site errored, or an import failed.
+	process.exitCode = ( anyChanged || scanErrors.length > 0 || importFailures > 0 ) ? 1 : 0;
+}
+
+function originOf( url ) {
+	if ( ! url || typeof url !== 'string' ) {
+		return null;
+	}
+	try {
+		return new URL( url ).origin;
+	} catch ( e ) {
+		return null;
+	}
+}
+
+// Distinct output name per site. Subdirectory multisites share one host, so the
+// path is folded in; subdomain/domain-mapped sites keep their host-only name.
+function siteFileSlug( site ) {
+	let path = '';
+	try {
+		path = new URL( site.url ).pathname;
+	} catch ( e ) {
+		path = '';
+	}
+	const trimmed = path.replace( /^\/+|\/+$/g, '' );
+	const suffix = trimmed ? '_' + trimmed.replace( /[^a-z0-9._-]+/gi, '-' ) : '';
+	return `${ site.host }${ suffix }`;
+}
+
+function printDiff( host, diff ) {
+	if ( ! diff ) {
+		return;
+	}
+	const total = diff.added.length + diff.changed.length + diff.removed.length;
+	if ( total === 0 ) {
+		process.stdout.write( `${ host }: no changes\n` );
+		return;
+	}
+	process.stdout.write(
+		`${ host }: ${ diff.added.length } new, ${ diff.changed.length } changed, ${ diff.removed.length } removed\n`
+	);
+	for ( const o of diff.added ) {
+		process.stdout.write( `  + ${ o.storage_type } ${ o.name }${ o.domain ? ' @ ' + o.domain : '' }\n` );
+	}
+	for ( const o of diff.removed ) {
+		process.stdout.write( `  - ${ o.storage_type } ${ o.name }${ o.domain ? ' @ ' + o.domain : '' }\n` );
+	}
+}
+
+/**
+ * Adds paths that previously produced a tracker (read from each site's prior
+ * committed scan file) to the config paths, so known-tracker pages are always
+ * re-checked. Skipped when `full` is set.
+ */
+async function applyTargeting( config, outDir, full ) {
+	if ( full ) {
+		return config;
+	}
+
+	const sites = [];
+	for ( const site of config.sites || [] ) {
+		const file = join( outDir, `${ siteFileSlug( site ) }.json` );
+		let prior = null;
+		if ( existsSync( file ) ) {
+			try {
+				prior = JSON.parse( await readFile( file, 'utf8' ) );
+			} catch ( e ) {
+				prior = null;
+			}
+		}
+		sites.push( { ...site, paths: mergePaths( site.paths, priorTrackerPaths( prior ) ) } );
+	}
+
+	return { ...config, sites };
+}
+
+async function loadConfig( args ) {
+	if ( args.config ) {
+		return JSON.parse( await readFile( args.config, 'utf8' ) );
+	}
+	if ( args[ 'config-url' ] ) {
+		return applyOverlay( await fetchConfig( args[ 'config-url' ] ), args.overlay );
+	}
+	if ( args.url ) {
+		return {
+			sites: [
+				{
+					url: args.url,
+					blog_id: args[ 'blog-id' ] ? Number( args[ 'blog-id' ] ) : 1,
+					paths: args.path ? [].concat( args.path ) : [ '/' ],
+				},
+			],
+		};
+	}
+	throw new Error( 'Provide --config <file>, --config-url <url>, or --url <url>.' );
+}
+
+/**
+ * Fetches the scanner config from the WordPress REST endpoint.
+ *
+ * Authenticates with an application password via HTTP Basic auth, supplied
+ * through KJEKS_USER / KJEKS_APP_PASSWORD in the environment.
+ */
+async function fetchConfig( url ) {
+	const user = process.env.KJEKS_USER;
+	const password = process.env.KJEKS_APP_PASSWORD;
+	if ( ! user || ! password ) {
+		throw new Error( '--config-url requires KJEKS_USER and KJEKS_APP_PASSWORD in the environment.' );
+	}
+	const auth = 'Basic ' + Buffer.from( `${ user }:${ password }` ).toString( 'base64' );
+	const response = await fetch( url, { headers: { authorization: auth } } );
+	if ( ! response.ok ) {
+		throw new Error( `config-url returned ${ response.status }` );
+	}
+	return response.json();
+}
+
+/**
+ * Merges a repo-side overlay (paths / scenarios keyed by blog_id) into the
+ * site list fetched from REST. Operator intent stays in the repo; the live
+ * site list stays authoritative.
+ */
+async function applyOverlay( config, overlayPath ) {
+	if ( ! overlayPath ) {
+		return config;
+	}
+	const overlay = JSON.parse( await readFile( overlayPath, 'utf8' ) );
+	const byBlog = new Map( ( overlay.sites || [] ).map( ( s ) => [ s.blog_id, s ] ) );
+
+	config.sites = ( config.sites || [] ).map( ( site ) => {
+		const extra = byBlog.get( site.blog_id );
+		if ( ! extra ) {
+			return site;
+		}
+		return {
+			...site,
+			paths: extra.paths && extra.paths.length ? extra.paths : site.paths,
+			scenarios: extra.scenarios || site.scenarios,
+		};
+	} );
+
+	return config;
+}
+
+function parseArgs( argv ) {
+	const args = {};
+	for ( let i = 0; i < argv.length; i++ ) {
+		const token = argv[ i ];
+		if ( token.startsWith( '--' ) ) {
+			const key = token.slice( 2 );
+			const next = argv[ i + 1 ];
+			if ( ! next || next.startsWith( '--' ) ) {
+				args[ key ] = true;
+			} else {
+				args[ key ] = args[ key ] ? [].concat( args[ key ], next ) : next;
+				i++;
+			}
+		}
+	}
+	return args;
+}
+
+/**
+ * Stable JSON: object keys sorted recursively so output is byte-stable.
+ */
+function stableStringify( value ) {
+	return JSON.stringify( sortKeys( value ), null, '\t' );
+}
+
+function sortKeys( value ) {
+	if ( Array.isArray( value ) ) {
+		return value.map( sortKeys );
+	}
+	if ( value && typeof value === 'object' ) {
+		return Object.keys( value )
+			.sort()
+			.reduce( ( acc, key ) => {
+				acc[ key ] = sortKeys( value[ key ] );
+				return acc;
+			}, {} );
+	}
+	return value;
+}
+
+main().catch( ( error ) => {
+	process.stderr.write( `kjeks-scan: ${ error.message }\n` );
+	process.exitCode = 2;
+} );

package/src/collect.js

@@ -0,0 +1,235 @@
+/**
+ * Per-state forensic collection.
+ *
+ * Attaches network listeners, drives a page, and reads all client-side storage
+ * surfaces. HttpOnly cookies are only available via context.cookies() (CDP),
+ * never document.cookie — which is exactly why a real, CDP-capable browser is
+ * required (see docs/adr/0005).
+ */
+
+/**
+ * Approximate registrable domain (eTLD+1). Good enough for first/third-party
+ * classification; documented as a known limitation.
+ *
+ * @param {string} host
+ */
+export function registrableDomain( host ) {
+	const labels = String( host || '' ).split( '.' ).filter( Boolean );
+	if ( labels.length <= 2 ) {
+		return labels.join( '.' );
+	}
+	return labels.slice( -2 ).join( '.' );
+}
+
+function isBeacon( request ) {
+	const type = request.resourceType();
+	if ( type === 'beacon' || type === 'ping' ) {
+		return true;
+	}
+	const url = request.url();
+	return /\.(gif|png)(\?|$)/i.test( url ) && /(pixel|track|beacon|collect|pageview)/i.test( url );
+}
+
+/**
+ * Collects observations for a single already-configured page.
+ *
+ * @param {import('playwright').Page} page
+ * @param {import('playwright').BrowserContext} context
+ * @param {object} options
+ * @param {string} options.firstPartyDomain  Registrable domain of the site.
+ * @param {string[]} options.paths            Paths to visit (relative or absolute).
+ * @param {string} options.baseUrl            Site base URL.
+ */
+export async function collect( page, context, { firstPartyDomain, paths, baseUrl } ) {
+	const requests = [];
+	const redirects = [];
+	const setCookies = [];
+	let currentPath = paths && paths.length ? String( paths[ 0 ] ) : '/';
+
+	page.on( 'request', ( request ) => {
+		let host = '';
+		try {
+			host = new URL( request.url() ).hostname;
+		} catch ( e ) {
+			host = '';
+		}
+		const party = registrableDomain( host ) === firstPartyDomain ? 'first' : 'third';
+		requests.push( {
+			url: request.url(),
+			host,
+			method: request.method(),
+			resourceType: request.resourceType(),
+			party,
+			beacon: isBeacon( request ),
+			path: currentPath,
+		} );
+	} );
+
+	page.on( 'response', ( response ) => {
+		const headers = response.headers();
+		if ( headers[ 'set-cookie' ] ) {
+			setCookies.push( { url: response.url(), setCookie: headers[ 'set-cookie' ] } );
+		}
+		if ( response.status() >= 300 && response.status() < 400 && headers.location ) {
+			redirects.push( { from: response.url(), to: headers.location } );
+		}
+	} );
+
+	// First path where each cumulative cookie / storage key appears — the basis
+	// for per-URL attribution (which page a tracker actually loads on).
+	const cookieFirstSeen = new Map();
+	const localFirstSeen = new Map();
+	const sessionFirstSeen = new Map();
+	const idbFirstSeen = new Map();
+
+	let lastCookies = [];
+	let lastStorage = { documentCookie: [], localStorage: [], sessionStorage: [], indexedDB: [], scripts: [], iframes: [] };
+
+	// Resolve paths relative to the site base so subdirectory multisites work:
+	// new URL( '/', 'https://host/sub/' ) would resolve to the domain root and
+	// scan the wrong site. Treat a leading-slash path as relative to the base.
+	const base = baseUrl.endsWith( '/' ) ? baseUrl : baseUrl + '/';
+	for ( const path of paths ) {
+		currentPath = String( path );
+		const target = path.startsWith( 'http' )
+			? path
+			: new URL( String( path ).replace( /^\/+/, '' ), base ).toString();
+		await page.goto( target, { waitUntil: 'networkidle', timeout: 30000 } ).catch( () => {} );
+
+		// Snapshot after each path; the delta attributes new items to this path.
+		lastCookies = await context.cookies();
+		lastStorage = await page.evaluate( readStorage );
+
+		for ( const c of lastCookies ) {
+			const key = `${ c.name }|${ c.domain }`;
+			if ( ! cookieFirstSeen.has( key ) ) {
+				cookieFirstSeen.set( key, currentPath );
+			}
+		}
+		for ( const k of lastStorage.localStorage ) {
+			if ( ! localFirstSeen.has( k ) ) {
+				localFirstSeen.set( k, currentPath );
+			}
+		}
+		for ( const k of lastStorage.sessionStorage ) {
+			if ( ! sessionFirstSeen.has( k ) ) {
+				sessionFirstSeen.set( k, currentPath );
+			}
+		}
+		for ( const k of lastStorage.indexedDB ) {
+			if ( ! idbFirstSeen.has( k ) ) {
+				idbFirstSeen.set( k, currentPath );
+			}
+		}
+	}
+
+	const cookies = lastCookies;
+	const storage = lastStorage;
+	const sources = buildSources( { requests, cookieFirstSeen, localFirstSeen, sessionFirstSeen, idbFirstSeen } );
+
+	return {
+		cookies: cookies.map( ( c ) => ( {
+			name: c.name,
+			domain: c.domain,
+			path: c.path,
+			secure: c.secure,
+			http_only: c.httpOnly,
+			same_site: c.sameSite,
+			session: c.expires === -1,
+			party: registrableDomain( ( c.domain || '' ).replace( /^\./, '' ) ) === firstPartyDomain ? 'first' : 'third',
+		} ) ),
+		documentCookie: storage.documentCookie,
+		localStorage: storage.localStorage,
+		sessionStorage: storage.sessionStorage,
+		indexedDB: storage.indexedDB,
+		scripts: storage.scripts,
+		iframes: storage.iframes,
+		requests,
+		redirects,
+		setCookies,
+		sources,
+	};
+}
+
+/**
+ * Maps observation keys to the sorted list of paths that produced them. Keys
+ * match scan.js deriveObservations() so attribution merges cleanly.
+ */
+function buildSources( { requests, cookieFirstSeen, localFirstSeen, sessionFirstSeen, idbFirstSeen } ) {
+	const sources = {};
+	const add = ( key, path ) => {
+		if ( ! path ) {
+			return;
+		}
+		( sources[ key ] ||= new Set() ).add( path );
+	};
+
+	for ( const r of requests ) {
+		if ( r.party === 'third' ) {
+			add( `script|${ r.host }`, r.path );
+		}
+		if ( r.beacon ) {
+			add( `pixel|${ stripQuery( r.url ) }`, r.path );
+		}
+	}
+	for ( const [ key, path ] of cookieFirstSeen ) {
+		add( `cookie|${ key }`, path );
+	}
+	for ( const [ key, path ] of localFirstSeen ) {
+		add( `localstorage|${ key }`, path );
+	}
+	for ( const [ key, path ] of sessionFirstSeen ) {
+		add( `sessionstorage|${ key }`, path );
+	}
+	for ( const [ key, path ] of idbFirstSeen ) {
+		add( `indexeddb|${ key }`, path );
+	}
+
+	const out = {};
+	for ( const key of Object.keys( sources ) ) {
+		out[ key ] = Array.from( sources[ key ] ).sort();
+	}
+	return out;
+}
+
+function stripQuery( url ) {
+	try {
+		const u = new URL( url );
+		return u.origin + u.pathname;
+	} catch ( e ) {
+		return url;
+	}
+}
+
+/* Runs in the page context. */
+function readStorage() {
+	const names = ( store ) => {
+		try {
+			return Object.keys( store );
+		} catch ( e ) {
+			return [];
+		}
+	};
+	return Promise.resolve()
+		.then( async () => {
+			let idb = [];
+			try {
+				if ( indexedDB.databases ) {
+					idb = ( await indexedDB.databases() ).map( ( d ) => d.name ).filter( Boolean );
+				}
+			} catch ( e ) {
+				idb = [];
+			}
+			return {
+				documentCookie: ( document.cookie || '' )
+					.split( ';' )
+					.map( ( c ) => c.split( '=' )[ 0 ].trim() )
+					.filter( Boolean ),
+				localStorage: names( window.localStorage ),
+				sessionStorage: names( window.sessionStorage ),
+				indexedDB: idb,
+				scripts: Array.from( document.querySelectorAll( 'script[src]' ) ).map( ( s ) => s.src ),
+				iframes: Array.from( document.querySelectorAll( 'iframe[src]' ) ).map( ( f ) => f.src ),
+			};
+		} );
+}

package/src/consent.js

@@ -0,0 +1,58 @@
+/**
+ * Consent-state matrix and the shared consent-record schema.
+ *
+ * Mirrors the PHP `Soderlind\Kjeks\Consent\ConsentSchema` wire format so the
+ * scanner can drive each consent state deterministically by injecting the
+ * record directly, rather than clicking the banner UI.
+ */
+
+export const COOKIE_NAME = 'kjeks_consent';
+export const STORAGE_KEY = 'kjeks_consent';
+
+// Must match Categories::optional() on the PHP side.
+export const OPTIONAL_CATEGORIES = [ 'preferences', 'analytics', 'marketing' ];
+
+/**
+ * Builds the ordered list of consent states to scan for one site.
+ *
+ * @returns {{ id: string, choices: Record<string, boolean> | null }[]}
+ */
+export function consentStates() {
+	const states = [
+		// Before any choice: no record injected at all.
+		{ id: 'before-choice', choices: null },
+		{ id: 'reject-all', choices: denied() },
+	];
+
+	// Each optional category granted on its own.
+	for ( const category of OPTIONAL_CATEGORIES ) {
+		states.push( { id: `only-${ category }`, choices: { ...denied(), [ category ]: true } } );
+	}
+
+	states.push( { id: 'accept-all', choices: accepted() } );
+	return states;
+}
+
+export function denied() {
+	return Object.fromEntries( OPTIONAL_CATEGORIES.map( ( c ) => [ c, false ] ) );
+}
+
+export function accepted() {
+	return Object.fromEntries( OPTIONAL_CATEGORIES.map( ( c ) => [ c, true ] ) );
+}
+
+/**
+ * Encodes a consent record in the shared wire format.
+ *
+ * @param {Record<string, boolean>} choices  Optional-category choices.
+ * @param {number} version  Policy version.
+ * @param {number} blogId   Blog id.
+ * @param {number} time     Unix seconds.
+ */
+export function encodeRecord( choices, version, blogId, time ) {
+	const c = {};
+	for ( const category of OPTIONAL_CATEGORIES ) {
+		c[ category ] = choices[ category ] ? 1 : 0;
+	}
+	return { v: version, t: time, b: blogId, c };
+}

package/src/diff.js

@@ -0,0 +1,82 @@
+/**
+ * Deterministic per-subsite diff against a previous scan.
+ */
+
+/**
+ * Diffs two scan results, per site, by observation identity.
+ *
+ * @param {object|null} previous  Previous scan result (or null for first run).
+ * @param {object} current        Current scan result.
+ * @returns {{ host: string, added: object[], changed: object[], removed: object[] }[]}
+ */
+export function diffScans( previous, current ) {
+	const previousByHost = indexByHost( previous );
+	const out = [];
+
+	for ( const site of current.sites ) {
+		const before = previousByHost.get( site.host );
+		out.push( diffSite( site.host, before ? before.observations : [], site.observations ) );
+	}
+
+	return out;
+}
+
+function diffSite( host, beforeObs, afterObs ) {
+	const before = indexObservations( beforeObs );
+	const after = indexObservations( afterObs );
+
+	const added = [];
+	const changed = [];
+	const removed = [];
+
+	for ( const [ key, observation ] of after ) {
+		if ( ! before.has( key ) ) {
+			added.push( observation );
+		} else if ( ! sameAttributes( before.get( key ), observation ) ) {
+			changed.push( { from: before.get( key ), to: observation } );
+		}
+	}
+
+	for ( const [ key, observation ] of before ) {
+		if ( ! after.has( key ) ) {
+			removed.push( observation );
+		}
+	}
+
+	return {
+		host,
+		added: added.sort( byName ),
+		changed: changed.sort( ( a, b ) => byName( a.to, b.to ) ),
+		removed: removed.sort( byName ),
+	};
+}
+
+function observationKey( o ) {
+	return `${ o.storage_type }|${ o.name }|${ o.domain || '' }`;
+}
+
+function indexObservations( list ) {
+	return new Map( ( list || [] ).map( ( o ) => [ observationKey( o ), o ] ) );
+}
+
+function indexByHost( scan ) {
+	return new Map( ( scan && scan.sites ? scan.sites : [] ).map( ( s ) => [ s.host, s ] ) );
+}
+
+/**
+ * Compares attributes that matter, ignoring volatile fields like triggered_by
+ * ordering (already sorted) and observation timestamps.
+ */
+function sameAttributes( a, b ) {
+	const fields = [ 'party', 'secure', 'http_only', 'same_site', 'retention', 'path' ];
+	for ( const field of fields ) {
+		if ( ( a[ field ] ?? null ) !== ( b[ field ] ?? null ) ) {
+			return false;
+		}
+	}
+	return JSON.stringify( a.triggered_by || [] ) === JSON.stringify( b.triggered_by || [] );
+}
+
+function byName( a, b ) {
+	return observationKey( a ).localeCompare( observationKey( b ) );
+}

package/src/import.js

@@ -0,0 +1,91 @@
+#!/usr/bin/env node
+/**
+ * Posts scan results to the Kjeks REST import endpoint.
+ *
+ * Reads one or more scan JSON files and imports each site's observations as
+ * UNREVIEWED. Authentication uses a WordPress application password via HTTP
+ * Basic auth — supply it through the environment, never on the command line or
+ * in the repo.
+ *
+ * Usage:
+ *   KJEKS_USER=admin KJEKS_APP_PASSWORD='xxxx xxxx xxxx' \
+ *     node src/import.js --site https://network.example.com scan/*.json
+ */
+
+import { readFile } from 'node:fs/promises';
+import { fileURLToPath } from 'node:url';
+
+export function basicAuthFromEnv() {
+	const user = process.env.KJEKS_USER;
+	const password = process.env.KJEKS_APP_PASSWORD;
+	if ( ! user || ! password ) {
+		throw new Error( 'Set KJEKS_USER and KJEKS_APP_PASSWORD in the environment.' );
+	}
+	return 'Basic ' + Buffer.from( `${ user }:${ password }` ).toString( 'base64' );
+}
+
+/**
+ * POSTs each site's observations to the Kjeks import endpoint as UNREVIEWED.
+ *
+ * @param {string} base   Network base URL.
+ * @param {object[]} sites  Sites with { blog_id, observations }.
+ * @param {object} [opts]
+ * @param {string} [opts.auth]  Authorization header value (default: env basic auth).
+ * @param {(message: string, level?: string) => void} [opts.log]
+ * @returns {Promise<{ imported: number, failures: number }>}
+ */
+export async function importSites( base, sites, opts = {} ) {
+	const auth = opts.auth || basicAuthFromEnv();
+	const log = opts.log || ( () => {} );
+	const endpoint = new URL( '/wp-json/kjeks/v1/import', base ).toString();
+
+	let imported = 0;
+	let failures = 0;
+	for ( const site of sites || [] ) {
+		const response = await fetch( endpoint, {
+			method: 'POST',
+			headers: { 'content-type': 'application/json', authorization: auth },
+			body: JSON.stringify( { blog_id: site.blog_id, observations: site.observations } ),
+		} );
+		const body = await response.json().catch( () => ( {} ) );
+		if ( ! response.ok ) {
+			log( `Import failed for blog ${ site.blog_id }: ${ response.status } ${ JSON.stringify( body ) }`, 'error' );
+			failures++;
+			continue;
+		}
+		imported += Number( body.imported ) || 0;
+		log( `blog ${ site.blog_id }: imported ${ body.imported } unreviewed observation(s)` );
+	}
+
+	return { imported, failures };
+}
+
+async function main() {
+	const args = process.argv.slice( 2 );
+	const siteIndex = args.indexOf( '--site' );
+	if ( siteIndex === -1 ) {
+		throw new Error( 'Provide --site <network-base-url>.' );
+	}
+	const base = args[ siteIndex + 1 ];
+	const files = args.filter( ( a, i ) => i !== siteIndex && i !== siteIndex + 1 && ! a.startsWith( '--' ) );
+
+	let failures = 0;
+	for ( const file of files ) {
+		const scan = JSON.parse( await readFile( file, 'utf8' ) );
+		const result = await importSites( base, scan.sites, {
+			log: ( message, level ) => ( level === 'error' ? process.stderr : process.stdout ).write( message + '\n' ),
+		} );
+		failures += result.failures;
+	}
+	if ( failures ) {
+		process.exitCode = 1;
+	}
+}
+
+// Only run as a CLI when executed directly (not when imported by scan CLI).
+if ( process.argv[ 1 ] && fileURLToPath( import.meta.url ) === process.argv[ 1 ] ) {
+	main().catch( ( error ) => {
+		process.stderr.write( `kjeks-import: ${ error.message }\n` );
+		process.exitCode = 2;
+	} );
+}

package/src/scan.js

@@ -0,0 +1,304 @@
+/**
+ * Scan orchestration.
+ *
+ * For each site and each consent state, opens a FRESH browser context, injects
+ * the consent record for that state, visits the configured paths, and collects
+ * observations. Produces a deterministic result per site.
+ */
+
+import { chromium } from 'playwright';
+import { COOKIE_NAME, STORAGE_KEY, consentStates, encodeRecord } from './consent.js';
+import { collect, registrableDomain } from './collect.js';
+
+const IGNORED_COOKIES = new Set( [ COOKIE_NAME ] );
+
+/**
+ * @param {object} config  Parsed scanner config.
+ * @param {object} [options]
+ * @param {string} [options.endpoint]     CDP endpoint (e.g. Cloudflare Browser Run). Omit for local Chromium.
+ * @param {number} [options.concurrency]  Max sites scanned in parallel (default 3).
+ * @param {number} [options.perHost]      Max parallel scans sharing a hostname (default 2).
+ */
+export async function runScan( config, options = {} ) {
+	const browser = options.endpoint
+		? await chromium.connectOverCDP( options.endpoint )
+		: await chromium.launch( { headless: true } );
+
+	const concurrency = Math.max( 1, Number( options.concurrency ) || 3 );
+	const perHost = Math.max( 1, Number( options.perHost ) || 2 );
+
+	try {
+		const results = await mapWithLimit(
+			config.sites || [],
+			{ concurrency, perHost, hostOf: siteHost },
+			( site ) => scanSite( browser, site, config )
+		);
+
+		const sites = [];
+		const errors = [];
+		results.forEach( ( result, i ) => {
+			if ( result && result.__error ) {
+				errors.push( { site: config.sites[ i ], message: String( result.__error && result.__error.message || result.__error ) } );
+			} else if ( result ) {
+				sites.push( result );
+			}
+		} );
+
+		return {
+			// generated_at is excluded from diffs; kept for provenance only.
+			generated_at: Math.floor( Date.now() / 1000 ),
+			sites: sites.sort( ( a, b ) => a.host.localeCompare( b.host ) ),
+			errors,
+		};
+	} finally {
+		await browser.close();
+	}
+}
+
+function siteHost( site ) {
+	try {
+		return new URL( site.url ).hostname;
+	} catch ( e ) {
+		return '';
+	}
+}
+
+/**
+ * Runs `worker` over `items` with a global concurrency cap and a per-host cap.
+ * A failing item is isolated: its slot holds `{ __error }` and the run continues.
+ * Results preserve input order. Exported for testing without a browser.
+ *
+ * @param {any[]} items
+ * @param {{ concurrency: number, perHost: number, hostOf: (item:any)=>string }} limits
+ * @param {(item:any, index:number)=>Promise<any>} worker
+ * @returns {Promise<any[]>}
+ */
+export function mapWithLimit( items, { concurrency, perHost, hostOf }, worker ) {
+	const results = new Array( items.length );
+	const queue = items.map( ( item, index ) => ( { item, index } ) );
+	const hostActive = new Map();
+	let active = 0;
+
+	return new Promise( ( resolve ) => {
+		const pump = () => {
+			if ( queue.length === 0 && active === 0 ) {
+				resolve( results );
+				return;
+			}
+
+			for ( let qi = 0; qi < queue.length && active < concurrency; ) {
+				const { item, index } = queue[ qi ];
+				const host = hostOf( item );
+				if ( ( hostActive.get( host ) || 0 ) >= perHost ) {
+					qi++;
+					continue;
+				}
+
+				queue.splice( qi, 1 );
+				active++;
+				hostActive.set( host, ( hostActive.get( host ) || 0 ) + 1 );
+
+				Promise.resolve()
+					.then( () => worker( item, index ) )
+					.then( ( value ) => {
+						results[ index ] = value;
+					} )
+					.catch( ( error ) => {
+						results[ index ] = { __error: error };
+					} )
+					.finally( () => {
+						active--;
+						hostActive.set( host, hostActive.get( host ) - 1 );
+						pump();
+					} );
+				// queue shifted; do not advance qi.
+			}
+		};
+
+		pump();
+	} );
+}
+
+async function scanSite( browser, site, config ) {
+	const baseUrl = site.url;
+	const host = new URL( baseUrl ).hostname;
+	const firstPartyDomain = registrableDomain( host );
+	const paths = site.paths && site.paths.length ? site.paths : [ '/' ];
+	const blogId = site.blog_id || 1;
+	const policyVersion = site.policy_version || 1;
+
+	const states = [];
+	const sourcesByState = [];
+	for ( const state of consentStates() ) {
+		const context = await browser.newContext();
+
+		if ( state.choices !== null ) {
+			const record = encodeRecord( state.choices, policyVersion, blogId, 0 );
+			const json = JSON.stringify( record );
+			await context.addCookies( [
+				{
+					name: COOKIE_NAME,
+					value: encodeURIComponent( json ),
+					domain: host,
+					path: '/',
+					sameSite: 'Lax',
+				},
+			] );
+			await context.addInitScript(
+				( { key, value } ) => {
+					try {
+						window.localStorage.setItem( key, value );
+					} catch ( e ) {}
+				},
+				{ key: STORAGE_KEY, value: json }
+			);
+		}
+
+		const page = await context.newPage();
+		const collected = await collect( page, context, { firstPartyDomain, paths, baseUrl } );
+
+		if ( site.scenarios ) {
+			await runScenarios( page, site.scenarios );
+		}
+
+		states.push( { state: state.id, ...normalizeState( collected, firstPartyDomain ) } );
+		sourcesByState.push( { state: state.id, sources: collected.sources || {} } );
+		await context.close();
+	}
+
+	return {
+		host,
+		url: baseUrl,
+		blog_id: blogId,
+		states: states.sort( ( a, b ) => a.state.localeCompare( b.state ) ),
+		observations: deriveObservations( states, sourcesByState ),
+	};
+}
+
+async function runScenarios( page, scenarios ) {
+	for ( const scenario of scenarios ) {
+		for ( const step of scenario.steps || [] ) {
+			if ( step.action === 'click' && step.selector ) {
+				await page.click( step.selector, { timeout: 5000 } ).catch( () => {} );
+			}
+			if ( step.action === 'wait' && step.ms ) {
+				await page.waitForTimeout( step.ms );
+			}
+		}
+	}
+}
+
+/**
+ * Deterministically normalizes one state's collected data.
+ */
+function normalizeState( collected, firstPartyDomain ) {
+	const cookies = collected.cookies
+		.filter( ( c ) => ! IGNORED_COOKIES.has( c.name ) )
+		.sort( byKey( ( c ) => `${ c.name }|${ c.domain }|${ c.path }` ) );
+
+	const thirdPartyHosts = uniqueSorted(
+		collected.requests.filter( ( r ) => r.party === 'third' ).map( ( r ) => r.host )
+	);
+
+	const beacons = uniqueSorted(
+		collected.requests.filter( ( r ) => r.beacon ).map( ( r ) => stripQuery( r.url ) )
+	);
+
+	return {
+		cookies,
+		localStorage: uniqueSorted( collected.localStorage ).filter( ( k ) => k !== STORAGE_KEY ),
+		sessionStorage: uniqueSorted( collected.sessionStorage ),
+		indexedDB: uniqueSorted( collected.indexedDB ),
+		thirdPartyHosts,
+		beacons,
+		scripts: uniqueSorted( collected.scripts.map( stripQuery ) ),
+		iframes: uniqueSorted( collected.iframes.map( stripQuery ) ),
+		redirects: collected.redirects.map( ( r ) => ( { from: stripQuery( r.from ), to: stripQuery( r.to ) } ) ),
+	};
+}
+
+/**
+ * Builds a flat, import-ready observation list from all states.
+ *
+ * @param {object[]} states
+ * @param {Array<{state: string, sources: object}>} [sourcesByState]  Per-state observation-key -> source paths.
+ */
+export function deriveObservations( states, sourcesByState = [] ) {
+	const sourcesById = new Map( sourcesByState.map( ( s ) => [ s.state, s.sources || {} ] ) );
+	const map = new Map();
+
+	const add = ( key, observation, stateId ) => {
+		if ( ! map.has( key ) ) {
+			map.set( key, { ...observation, triggered_by: [], source_urls: [] } );
+		}
+		const existing = map.get( key );
+		if ( ! existing.triggered_by.includes( stateId ) ) {
+			existing.triggered_by.push( stateId );
+		}
+		const srcs = ( sourcesById.get( stateId ) || {} )[ key ];
+		if ( srcs ) {
+			for ( const url of srcs ) {
+				if ( ! existing.source_urls.includes( url ) ) {
+					existing.source_urls.push( url );
+				}
+			}
+		}
+	};
+
+	for ( const state of states ) {
+		for ( const c of state.cookies ) {
+			add( `cookie|${ c.name }|${ c.domain }`, {
+				name: c.name,
+				storage_type: 'cookie',
+				domain: c.domain,
+				path: c.path,
+				party: c.party,
+				secure: c.secure,
+				http_only: c.http_only,
+				same_site: c.same_site,
+				retention: c.session ? 'session' : 'persistent',
+			}, state.state );
+		}
+		for ( const key of state.localStorage ) {
+			add( `localstorage|${ key }`, { name: key, storage_type: 'localstorage', party: 'first' }, state.state );
+		}
+		for ( const key of state.sessionStorage ) {
+			add( `sessionstorage|${ key }`, { name: key, storage_type: 'sessionstorage', party: 'first' }, state.state );
+		}
+		for ( const db of state.indexedDB ) {
+			add( `indexeddb|${ db }`, { name: db, storage_type: 'indexeddb', party: 'first' }, state.state );
+		}
+		for ( const host of state.thirdPartyHosts ) {
+			add( `script|${ host }`, { name: host, storage_type: 'script', domain: host, party: 'third' }, state.state );
+		}
+		for ( const url of state.beacons ) {
+			add( `pixel|${ url }`, { name: url, storage_type: 'pixel', party: 'third' }, state.state );
+		}
+	}
+
+	for ( const observation of map.values() ) {
+		observation.triggered_by.sort();
+		observation.source_urls.sort();
+	}
+
+	return Array.from( map.values() ).sort(
+		byKey( ( o ) => `${ o.storage_type }|${ o.name }|${ o.domain || '' }` )
+	);
+}
+
+function byKey( keyFn ) {
+	return ( a, b ) => keyFn( a ).localeCompare( keyFn( b ) );
+}
+
+function uniqueSorted( list ) {
+	return Array.from( new Set( list.filter( Boolean ) ) ).sort();
+}
+
+function stripQuery( url ) {
+	try {
+		const u = new URL( url );
+		return u.origin + u.pathname;
+	} catch ( e ) {
+		return url;
+	}
+}

package/src/targeting.js

@@ -0,0 +1,44 @@
+/**
+ * Incremental targeting: always re-scan URLs that previously loaded a tracker.
+ *
+ * Pure helpers (no filesystem) so they unit-test without a browser. The CLI
+ * reads the prior committed per-site scan file and feeds its observations here.
+ */
+
+/**
+ * Union of `source_urls` across every observation in a prior scan file.
+ *
+ * @param {object|null} priorScan  Parsed previous per-site scan ({ sites: [...] }).
+ * @returns {string[]} Sorted, unique paths that previously produced a tracker.
+ */
+export function priorTrackerPaths( priorScan ) {
+	if ( ! priorScan || ! Array.isArray( priorScan.sites ) ) {
+		return [];
+	}
+	const set = new Set();
+	for ( const site of priorScan.sites ) {
+		for ( const observation of site.observations || [] ) {
+			for ( const url of observation.source_urls || [] ) {
+				set.add( url );
+			}
+		}
+	}
+	return Array.from( set ).sort();
+}
+
+/**
+ * Config paths plus prior tracker paths, de-duped and order-stable.
+ *
+ * @param {string[]} configPaths
+ * @param {string[]} priorPaths
+ * @returns {string[]}
+ */
+export function mergePaths( configPaths, priorPaths ) {
+	const out = [];
+	for ( const path of [ ...( configPaths || [] ), ...( priorPaths || [] ) ] ) {
+		if ( ! out.includes( path ) ) {
+			out.push( path );
+		}
+	}
+	return out.length ? out : [ '/' ];
+}