@analitikisler/ai-tag-debugger 0.6.2 → 0.7.0

package/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.7.0 (2026-10-12)
+
+Fixes from the first audit run across 18 real shops.
+
+- **GA4 hits with a `gaf` parameter were missed and sent to the real GA4 property** (since 0.6.2). Some sites send `gaf=2` on every normal hit, so `view_item`, `add_to_cart` and others were reported as missing, and the test visits reached the site's analytics. Only copies to `www.google.com` and `doubleclick.net` are now left out, whatever their parameters, and every GA4 request is answered locally again.
+- More Google tag pings are answered locally instead of reaching the real accounts: Google Ads remarketing (`www.google.com/rmkt/collect`), Merchant Center (`merchant-center-analytics.goog/mc/collect`), the Google tag's `/g/s/collect`, and consent mode pings on any host, including Google tag gateway paths on the site's own domain (`/<id>/gs/ccm/collect`).
+- One browsing step can no longer stall a run: a model call that doesn't answer within 3 minutes, or a browser action that doesn't finish within 2, ends the journey as incomplete with a note.
+- **Saved sites and `replay`.** `run --save-site <dir>` also saves every response the site sent (as a Playwright HAR file per journey and viewport, without tracking requests, cookies sent or set, authorization headers, form bodies, or requests with user data in their address) and the steps taken. `replay <dir>...` audits those pages again offline, with no network and no AI model, and lists findings that are new or gone. It exits with code 1 when any changed; `--update` makes the new findings the expected ones. Steps the AI took by ref are saved as CSS paths, or as visible text inside iframes. Images, fonts and video are left out of the recording; when the browser closed early, there is no recording to clean instead of an error.
+
+## 0.6.3 (2026-10-12)
+
+- The audited site's own email address in a `mailto:` link or a link field (`link_url`, `link_text`, `gtm.elementUrl`, `gtm.elementText`), such as the contact link in a GTM `gtm.linkClick` push, is no longer reported as unhashed user data or redacted. An own-domain address anywhere else, and every named user data field (`email`, `user_data`, `em`, …), is still redacted and reported.
+
 ## 0.6.2 (2026-10-12)
 
 Fixes from Bahadır's audit of analitikisler.com.

package/README.md

@@ -169,6 +169,19 @@ Each browsing step is one model call, so usage grows with **journeys × viewport
 - **Replay without AI.** After one AI run, copy the steps into `journeys[].steps` and use `--scripted` (with `--no-ai` for zero model calls) on every deploy.
 - **Use a chat app.** Over MCP, the chat app's own model does the browsing, so no API key is used at all.
 
+### Check a new version against saved sites
+
+`--save-site <dir>` also saves every response the site sent and the steps taken. `replay` then audits the same pages again offline, with no network and no AI model, so it costs nothing. It lists findings that are new or gone compared with the saved run:
+
+```bash
+npx @analitikisler/ai-tag-debugger run --site https://www.yourshop.com --journey purchase --save-site saved/yourshop
+npx @analitikisler/ai-tag-debugger replay saved/*
+✗ https://www.yourshop.com (saved/yourshop): 0 new, 1 gone
+    - warning duplicate purchase desktop ga4 page_view
+```
+
+The first run's findings become the expected ones. When a change is right, for example a false alarm that is fixed, `replay --update` makes the new findings the expected ones. Use it to check a new version of the tool against sites you know, or a site change against its last good run. A saved site is usually a few MB. It leaves out tracking requests, images and video, cookies, authorization headers, form data, and requests with an email or other user data in their address. It still holds the site's pages and scripts, so keep sites you don't own out of public repositories.
+
 ## Configure your site
 
 `ai-tag-debugger.config.json`:
@@ -229,6 +242,8 @@ ai-tag-debugger run [--site <url>] [--config <file>] [--journey <id>] [--goal "<
                 [--max-steps <n>] [--out <dir>] [--viewport desktop|mobile] [--providers <list>]
                 [--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--base-url <url>]
                 [--lang en|tr] [--allow-host <host>] [--keep-bodies] [--scripted] [--no-ai] [--fail-on broken|risk|warning|never] [--headed]
+                [--save-site <dir>]
+ai-tag-debugger replay <dir>... [--update] [--out <dir>]
 ai-tag-debugger mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
                 [--allow-private-network]
 ```
@@ -316,7 +331,7 @@ A hosted server, so you don't need a tunnel, is part of the [Analitik İşler](h
 - The browser stays on the audited site and its subdomains, plus any `allowed_hosts`. It opens only `http` and `https` pages.
 - The model is told that page content is data, never instructions. A prompt rule can't guarantee a model is never misled, so the navigation limits above contain what a misled model can do.
 - The browser runs without your API keys and other secrets in its environment.
-- **User data sent for ad matching is redacted** before it is stored in `capture.json` or shown in the report: emails, phone numbers, names, addresses and user ids in GA4 and Google Ads enhanced conversions (`em`, `ph`, `uid`, …), Meta advanced matching (`ud[*]`, `udff[*]`), TikTok `context.user`, and the `user_data` the site pushes to the dataLayer for enhanced conversions. Any value that looks like an email address is redacted whatever its field is called. Each value becomes a marker such as `[redacted: 64 chars, looks hashed]`, and a value that wasn't hashed is reported as a personal data risk. `--keep-bodies` (`"keep_bodies": true`) stores requests as sent, for debugging.
+- **User data sent for ad matching is redacted** before it is stored in `capture.json` or shown in the report: emails, phone numbers, names, addresses and user ids in GA4 and Google Ads enhanced conversions (`em`, `ph`, `uid`, …), Meta advanced matching (`ud[*]`, `udff[*]`), TikTok `context.user`, and the `user_data` the site pushes to the dataLayer for enhanced conversions. Any value that looks like an email address is redacted whatever its field is called, except the audited site's own address in a `mailto:` link or a link field (`link_url`, `link_text`, `gtm.elementUrl`, …), such as a contact link in a GTM click event, which is not visitor data. Each value becomes a marker such as `[redacted: 64 chars, looks hashed]`, and a value that wasn't hashed is reported as a personal data risk. `--keep-bodies` (`"keep_bodies": true`) stores requests as sent, for debugging.
 - Stored request bodies are capped at 32 KB per request and 2 MB per journey. Past that, the report says the body wasn't stored.
 - Only run it against sites you own or are authorized to test.
 - Hosting the HTTP server for other people? Also block private and cloud metadata address ranges at the network level (an egress firewall or an isolated network), as defense in depth.

package/README.tr.md

@@ -171,6 +171,19 @@ Her gezinme adımı bir model çağrısıdır. Yani kullanım **senaryo × ekran
 - **Yapay zekâ olmadan tekrar oynatın.** Bir yapay zekâ çalıştırmasından sonra adımları `journeys[].steps` içine kopyalayın ve her deploy'da `--scripted` (hiç model çağrısı olmaması için `--no-ai` ile birlikte) kullanın.
 - **Bir yapay zekâ asistanı kullanın.** MCP üzerinden gezinmeyi asistanın kendi modeli yapar, hiç API anahtarı kullanılmaz.
 
+### Yeni bir sürümü kaydedilmiş sitelerle karşılaştırın
+
+`--save-site <klasör>`, sitenin gönderdiği her yanıtı ve atılan adımları da kaydeder. `replay` aynı sayfaları ağ ve yapay zekâ modeli olmadan, çevrimdışı olarak yeniden denetler; bu yüzden hiçbir maliyeti yoktur. Kaydedilen çalıştırmaya göre yeni çıkan ya da kaybolan bulguları listeler:
+
+```bash
+npx @analitikisler/ai-tag-debugger run --site https://www.magazaniz.com --journey purchase --save-site saved/magazaniz
+npx @analitikisler/ai-tag-debugger replay saved/*
+✗ https://www.magazaniz.com (saved/magazaniz): 0 new, 1 gone
+    - warning duplicate purchase desktop ga4 page_view
+```
+
+İlk çalıştırmanın bulguları beklenen bulgular olur. Bir değişiklik doğruysa (örneğin düzeltilmiş bir yanlış alarm), `replay --update` yeni bulguları beklenen bulgular yapar. Aracın yeni bir sürümünü bildiğiniz sitelerle ya da bir site değişikliğini son sağlıklı çalıştırmayla karşılaştırmak için kullanın. Kaydedilmiş bir site genellikle birkaç MB'tır. İzleme isteklerini, görselleri ve videoları, çerezleri, yetkilendirme başlıklarını, form verilerini ve adresinde e-posta ya da başka kullanıcı verisi olan istekleri içermez. Yine de sitenin sayfalarını ve betiklerini içerdiği için size ait olmayan siteleri herkese açık depolara koymayın.
+
 ## Sitenizi ayarlayın
 
 `ai-tag-debugger.config.json`:
@@ -231,6 +244,8 @@ ai-tag-debugger run [--site <url>] [--config <file>] [--journey <id>] [--goal "<
                 [--max-steps <n>] [--out <dir>] [--viewport desktop|mobile] [--providers <list>]
                 [--provider anthropic|openai|gemini|openai-compatible] [--model <id>] [--base-url <url>]
                 [--lang en|tr] [--allow-host <host>] [--keep-bodies] [--scripted] [--no-ai] [--fail-on broken|risk|warning|never] [--headed]
+                [--save-site <dir>]
+ai-tag-debugger replay <dir>... [--update] [--out <dir>]
 ai-tag-debugger mcp [--http] [--port <n>] [--host <addr>] [--token <secret>] [--public-url <https url>] [--reports <dir>]
                 [--allow-private-network]
 ```
@@ -318,7 +333,7 @@ Tünele gerek kalmayan, hazır bir adresle barındırılan sunucu, [Analitik İ
 - Tarayıcı test edilen sitede ve alt alan adlarında, ayrıca `allowed_hosts` içindeki adreslerde kalır. Sadece `http` ve `https` sayfalarını açar.
 - Modele, sayfa içeriğinin talimat değil veri olduğu söylenir. Bir prompt kuralı, modelin hiçbir zaman yanıltılmayacağını garanti edemez; bu yüzden yukarıdaki gezinme sınırları, yanıltılmış bir modelin yapabileceklerini kısıtlar.
 - Tarayıcı, ortamında API anahtarlarınız ve diğer gizli bilgiler olmadan çalışır.
-- **Reklam eşleştirmesi için gönderilen kullanıcı verileri**, `capture.json` dosyasına kaydedilmeden ya da raporda gösterilmeden önce **gizlenir**: GA4 ve Google Ads enhanced conversions (`em`, `ph`, `uid`, …), Meta advanced matching (`ud[*]`, `udff[*]`), TikTok `context.user` içindeki e-postalar, telefon numaraları, adlar, adresler ve kullanıcı kimlikleri ile sitenin enhanced conversions için dataLayer'a gönderdiği `user_data`. E-posta adresine benzeyen her değer, alanın adı ne olursa olsun gizlenir. Her değer `[redacted: 64 chars, looks hashed]` gibi bir işarete dönüşür; hash'lenmemiş bir değer kişisel veri riski olarak raporlanır. `--keep-bodies` (`"keep_bodies": true`) hata ayıklama için istekleri gönderildiği haliyle saklar.
+- **Reklam eşleştirmesi için gönderilen kullanıcı verileri**, `capture.json` dosyasına kaydedilmeden ya da raporda gösterilmeden önce **gizlenir**: GA4 ve Google Ads enhanced conversions (`em`, `ph`, `uid`, …), Meta advanced matching (`ud[*]`, `udff[*]`), TikTok `context.user` içindeki e-postalar, telefon numaraları, adlar, adresler ve kullanıcı kimlikleri ile sitenin enhanced conversions için dataLayer'a gönderdiği `user_data`. E-posta adresine benzeyen her değer, alanın adı ne olursa olsun gizlenir. Denetlenen sitenin kendi adresi bir `mailto:` bağlantısında ya da bir bağlantı alanında (`link_url`, `link_text`, `gtm.elementUrl`, …) geçiyorsa, örneğin bir GTM tıklama olayındaki iletişim bağlantısında, ziyaretçi verisi olmadığı için bu kuralın dışında kalır. Her değer `[redacted: 64 chars, looks hashed]` gibi bir işarete dönüşür; hash'lenmemiş bir değer kişisel veri riski olarak raporlanır. `--keep-bodies` (`"keep_bodies": true`) hata ayıklama için istekleri gönderildiği haliyle saklar.
 - Saklanan istek gövdeleri istek başına 32 KB ve senaryo başına 2 MB ile sınırlıdır. Bu sınırın üstünde rapor, gövdenin saklanmadığını belirtir.
 - Sadece size ait ya da test etme yetkiniz olan sitelerde çalıştırın.
 - HTTP sunucusunu başkaları için mi barındırıyorsunuz? Ek bir güvenlik katmanı olarak özel ağ ve bulut metadata adres aralıklarını ağ seviyesinde de engelleyin (bir egress firewall ya da izole bir ağ ile).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@analitikisler/ai-tag-debugger",
-  "version": "0.6.2",
+  "version": "0.7.0",
   "description": "Open-source AI agent that QA-tests your website's analytics, ad and consent tracking (GA4, GTM, Google Ads, Meta, TikTok). Bring your own model (Claude, OpenAI, Gemini) or use it from your chat app over MCP.",
   "type": "module",
   "bin": {

package/src/agent.js

@@ -1,3 +1,4 @@
+import path from "node:path";
 import { AuditSession } from "./browser.js";
 import { messages } from "./i18n.js";
 import { createProvider, PROVIDERS } from "./providers.js";
@@ -87,6 +88,19 @@ export function validateAction(input) {
   return null;
 }
 
+// One hung page or model request must not stall a whole run.
+const MODEL_TIMEOUT_MS = 180_000;
+const ACTION_TIMEOUT_MS = 120_000;
+
+class StepTimeout extends Error {}
+
+/** The promise's value, or a StepTimeout with this message once ms have passed. */
+function withTimeout(promise, ms, message) {
+  let timer;
+  const timeout = new Promise((_, reject) => (timer = setTimeout(() => reject(new StepTimeout(message)), ms)));
+  return Promise.race([promise, timeout]).finally(() => clearTimeout(timer));
+}
+
 /**
  * Lets the AI model drive the browser through one journey.
  * @param {AuditSession} session
@@ -106,7 +120,7 @@ export async function runAgentJourney(session, journey, opts = {}) {
   });
 
   for (let turn = 0; turn < maxSteps; turn++) {
-    const { toolCalls, stop } = await chat.next();
+    const { toolCalls, stop } = await withTimeout(chat.next(), MODEL_TIMEOUT_MS, "The model did not answer within 3 minutes.");
     if (stop === "refusal") return { completed: false, note: "The model declined to continue this journey." };
     if (toolCalls.length === 0) {
       if (stop === "max_tokens") return { completed: false, note: "The model response was cut off." };
@@ -132,7 +146,7 @@ export async function runAgentJourney(session, journey, opts = {}) {
         continue;
       }
       log(`  ${call.input.reason}`);
-      const error = await session.act(call.input);
+      const error = await withTimeout(session.act(call.input), ACTION_TIMEOUT_MS, "The browser action did not finish within 2 minutes.");
       results.push({ id: call.id, content: pageView(await session.snapshot(), error) });
     }
     if (finished) return finished;
@@ -153,20 +167,29 @@ export async function runScriptedJourney(session, journey, opts = {}) {
   return { completed: true, note: "All scripted steps completed." };
 }
 
+/** The file a saved site keeps one journey's recorded responses in. */
+export const harName = (journeyId, viewport) => `${String(journeyId).replace(/[^\w.-]+/g, "_")}-${viewport}.har`;
+
 /**
  * Runs one journey on one viewport and returns everything the checks need.
+ * opts.saveSite records the site's responses into that folder; opts.replaySite answers from one.
  */
 export async function runJourney(config, journey, viewport, opts = {}) {
-  const session = new AuditSession({ viewport, site: config.site, allowedHosts: config.allowed_hosts, providers: config.providers, keepBodies: config.keep_bodies === true, blockHits: config.block_hits !== false, headless: !opts.headed });
+  const har = (dir) => (dir ? path.join(dir, harName(journey.id, viewport)) : undefined);
+  const session = new AuditSession({
+    viewport, site: config.site, allowedHosts: config.allowed_hosts, providers: config.providers, keepBodies: config.keep_bodies === true,
+    blockHits: config.block_hits !== false, headless: !opts.headed, recordHar: har(opts.saveSite), replayHar: har(opts.replaySite),
+  });
   try {
     await session.start();
     const startUrl = new URL(journey.start ?? "/", config.site).href;
     const startError = await session.act({ action: "goto", url: startUrl, reason: `Open ${startUrl}` });
     if (startError) return { ...collect(session, journey, viewport), completed: false, note: `Could not open the start page: ${startError}` };
 
-    const outcome = opts.scripted
-      ? await runScriptedJourney(session, journey, opts)
-      : await runAgentJourney(session, journey, opts);
+    const outcome = await (opts.scripted ? runScriptedJourney(session, journey, opts) : runAgentJourney(session, journey, opts)).catch((err) => {
+      if (err instanceof StepTimeout) return { completed: false, note: err.message };
+      throw err;
+    });
     return { ...collect(session, journey, viewport), ...outcome, cookies: await session.cookies() };
   } finally {
     await session.close();

package/src/audit.js

@@ -9,6 +9,7 @@ import { LANGUAGES, messages } from "./i18n.js";
 import { createProvider } from "./providers.js";
 import { filterFindings, planRun } from "./journeys.js";
 import { resolveProviders } from "./parsers.js";
+import { writeSavedSite } from "./replay.js";
 
 export async function readJson(file, baseDir = ".") {
   const full = path.resolve(baseDir, file);
@@ -48,10 +49,12 @@ export async function writeReports({ outDir, config, plan, runs, findings, expla
 /**
  * @param {{ config: any, plan?: any, gtmContainer?: any, journeys?: string[], goals?: string[], events?: string[], viewports?: string[],
  *   scripted?: boolean, ai?: boolean, provider?: string, model?: string, baseUrl?: string, maxSteps?: number, lang?: string, headed?: boolean,
- *   outDir?: string, branding?: boolean, providers?: string[], log?: (m: string) => void }} opts
+ *   outDir?: string, branding?: boolean, providers?: string[], saveSite?: string, replaySite?: string, log?: (m: string) => void }} opts
  *   journeys: config journey ids or ready-made journeys (purchase, add_to_cart, sign_up, lead, search).
  *   goals: extra journeys described in plain language. events: only check these events.
  *   providers: tracking providers to check (default: config "providers", else GA4, Google Ads, Meta, TikTok).
+ *   saveSite: record the site's responses and the steps taken into this folder, for replaySite (see replay.js).
+ *   replaySite: answer every request from a folder saved with saveSite, without the network.
  */
 export async function runAudit(opts) {
   const { gtmContainer } = opts;
@@ -76,17 +79,22 @@ export async function runAudit(opts) {
   const provider = useAi ? await createProvider({ name: opts.provider ?? config.provider, model: opts.model ?? config.model, baseUrl: opts.baseUrl ?? config.base_url }) : null;
   if (provider) log(`Using ${provider.name} (${provider.model})`);
 
+  if (opts.saveSite) await mkdir(opts.saveSite, { recursive: true });
   const runs = [];
   for (const journey of journeys) {
     for (const viewport of viewports) {
       log(`\n▶ ${journey.id} (${viewport})`);
-      const run = await runJourney(config, journey, viewport, { scripted: opts.scripted, provider, maxSteps: opts.maxSteps, headed: opts.headed, lang, log });
+      const run = await runJourney(config, journey, viewport, { scripted: opts.scripted, provider, maxSteps: opts.maxSteps, headed: opts.headed, lang, log, saveSite: opts.saveSite, replaySite: opts.replaySite });
       log(`  ${run.completed ? "✓" : "✗"} ${run.note} · ${run.hits.length} tracking hits captured`);
       runs.push(run);
     }
   }
 
   const findings = filterFindings(runs.flatMap((run) => checkRun(plan, run, { lang })), events);
+  if (opts.saveSite) {
+    await writeSavedSite(opts.saveSite, { config, plan, runs, findings });
+    log(`\nSaved the site for offline replay in ${opts.saveSite}`);
+  }
   let explained = null;
   if (provider) {
     log("\nAsking the model to explain the findings…");

package/src/browser.js

@@ -1,3 +1,4 @@
+import { readFile, rm, writeFile } from "node:fs/promises";
 import { chromium, devices } from "playwright";
 import { parseTrackingRequest, isCollectionEndpoint, providerOf, vendorOf, resolveProviders } from "./parsers.js";
 import { redactBody, redactParams, redactPush, redactUrl } from "./redact.js";
@@ -22,6 +23,14 @@ export const VIEWPORTS = {
 // log survives page navigations.
 /** A page address without its hash, for matching page_location to navigations. */
 const pageKey = (url) => String(url ?? "").split("#")[0];
+/** The host of a URL, or null. */
+const hostOf = (url) => {
+  try {
+    return new URL(url).hostname || null;
+  } catch {
+    return null;
+  }
+};
 
 const INIT_SCRIPT = `(() => {
   const record = (entry) => {
@@ -188,6 +197,62 @@ function trackingBody(request) {
   return body ? (body.length > MAX_BODY ? body.slice(0, MAX_BODY) : body) : null;
 }
 
+/**
+ * How a scripted run can find the element an action used again: the selector it was given,
+ * else a CSS path to the element, else its visible text. Refs only mean something in one run.
+ */
+async function replayTarget(a, locator) {
+  if (a.selector) return { selector: a.selector };
+  const found = await locator
+    .evaluate((el) => {
+      const text = (el.innerText || el.value || el.getAttribute("aria-label") || "").replace(/\s+/g, " ").trim().slice(0, 80);
+      if (window !== window.top) return { text };
+      const parts = [];
+      for (let n = el; n && n !== document.documentElement; n = n.parentElement) {
+        if (n.id && document.querySelectorAll(`#${CSS.escape(n.id)}`).length === 1) {
+          parts.unshift(`#${CSS.escape(n.id)}`);
+          break;
+        }
+        let i = 1;
+        for (let s = n.previousElementSibling; s; s = s.previousElementSibling) if (s.tagName === n.tagName) i++;
+        parts.unshift(`${n.tagName.toLowerCase()}:nth-of-type(${i})`);
+      }
+      return { selector: parts.join(" > "), text };
+    }, null, { timeout: 5000 })
+    .catch(() => null);
+  if (found?.selector) return { selector: found.selector };
+  return found?.text ? { text: found.text } : a.text ? { text: a.text } : {};
+}
+
+// Images, fonts and video don't change what tracking sends, and are most of a recording's size.
+const MEDIA = /^(image|font|video|audio)\//;
+
+const SECRET_REQUEST_HEADER = /^(cookie|authorization|proxy-authorization)$/i;
+
+/**
+ * Drops images, fonts, video and tracking requests from a recording (they are answered locally on replay, and their
+ * URLs can hold the user data redacted everywhere else), requests with user data in their address, form bodies,
+ * cookies both ways and authorization headers.
+ */
+
+async function cleanRecording(file) {
+  const har = JSON.parse(await readFile(file, "utf8"));
+  har.log.entries = har.log.entries
+    .filter((e) => !isCollectionEndpoint(e.request.url) && !MEDIA.test(e.response?.content?.mimeType ?? ""))
+    // A request whose address holds user data (an email in a form GET, for example) is left out whole.
+    .filter((e) => redactUrl(e.request.url).url === e.request.url)
+    .map((e) => {
+      // Form bodies are dropped; replay then matches POSTs by address and method.
+      const { postData: _postData, ...request } = e.request;
+      return {
+        ...e,
+        request: { ...request, cookies: [], headers: request.headers.filter((h) => !SECRET_REQUEST_HEADER.test(h.name)) },
+        response: { ...e.response, cookies: [], headers: e.response.headers.filter((h) => !/^set-cookie$/i.test(h.name)) },
+      };
+    });
+  await writeFile(file, JSON.stringify(har));
+}
+
 export class AuditSession {
   /**
    * @param {{ viewport: string, site?: string, allowedHosts?: string[], providers?: string[], keepBodies?: boolean, blockPrivate?: boolean | ((ip: string) => boolean), blockHits?: boolean, headless?: boolean }} opts
@@ -196,6 +261,8 @@ export class AuditSession {
    *   providers: which tracking providers to record (default GA4, Google Ads, Meta, TikTok). Requests to the
    *     others are still answered locally, but only counted, per step, in this.skipped.
    *   site: navigation is limited to this site's host and its subdomains, plus allowedHosts.
+   *   recordHar: save every response the site sends into this .har file, so the journey can be replayed offline later.
+   *   replayHar: answer requests from a .har file saved with recordHar instead of the network; requests it lacks fail.
    *   blockPrivate: refuse loopback, private and link-local addresses (used when the MCP server is reachable over HTTP).
    *     All browser traffic then goes through the egress proxy. A function decides which addresses count as private (for tests).
    */
@@ -232,7 +299,8 @@ export class AuditSession {
       launch.args = ["--force-webrtc-ip-handling-policy=disable_non_proxied_udp"];
     }
     this.browser = await chromium.launch(launch);
-    this.context = await this.browser.newContext({ ...device, locale: "en-US" });
+    const record = this.opts.recordHar ? { recordHar: { path: this.opts.recordHar, mode: "minimal" } } : {};
+    this.context = await this.browser.newContext({ ...device, locale: "en-US", ...record });
     await this.context.exposeBinding("__atdRecord", (_source, entry) => {
       this.record("dataLayerLog", { ...this.redactEntry(entry), step: this.currentStep, t: Date.now() });
     });
@@ -307,6 +375,8 @@ export class AuditSession {
     // the page navigates away (a page_view on the next load, a beacon on unload) can be
     // cancelled before its route handler runs, but its request event has already fired.
     this.context.on("request", onRequest);
+    // Registered first, so the routes below run before it and tracking requests are still answered locally.
+    if (this.opts.replayHar) await this.context.routeFromHAR(this.opts.replayHar, { notFound: "abort" });
     if (this.opts.blockHits) {
       await this.context.route((url) => isCollectionEndpoint(url.href), async (route) => {
         await route.fulfill({ status: 204, body: "" });
@@ -354,9 +424,10 @@ export class AuditSession {
   /** The hit with user data for ad matching redacted, and userData: { fields, plain } when there was any. */
   redactHit({ userFields, ...hit }) {
     const found = { fields: [], plain: [] };
-    const url = redactUrl(hit.url, found).url;
-    const params = redactParams(hit.params, found);
-    redactParams(userFields, found); // fields from the POST body that the hit doesn't keep
+    const site = hostOf(hit.params?.page_location) ?? hostOf(this.page.url());
+    const url = redactUrl(hit.url, found, site).url;
+    const params = redactParams(hit.params, found, site);
+    redactParams(userFields, found, site); // fields from the POST body that the hit doesn't keep
     if (!found.fields.length) return hit;
     const userData = { fields: [...new Set(found.fields)], plain: [...new Set(found.plain)] };
     return this.opts.keepBodies ? { ...hit, userData } : { ...hit, url, params, userData };
@@ -364,7 +435,7 @@ export class AuditSession {
 
   /** A dataLayer entry with user data redacted, and userData: { fields, plain } when there was any. */
   redactEntry(entry) {
-    const found = redactPush(entry.value);
+    const found = redactPush(entry.value, undefined, hostOf(this.page.url()));
     if (!found.fields.length) return entry;
     const userData = { fields: [...new Set(found.fields)], plain: [...new Set(found.plain)] };
     return this.opts.keepBodies ? { ...entry, userData } : { ...entry, value: found.value, userData };
@@ -589,12 +660,14 @@ export class AuditSession {
   async act(a) {
     const describe = a.reason || `${a.action} ${a.selector ?? (a.ref != null ? `#${a.ref}` : a.text != null ? `"${a.text}"` : a.url ?? "")}`.trim();
     this.beginStep(describe, { action: a.action, acceptsConsent: !!a.accepts_consent });
+    const step = this.steps[this.currentStep];
     try {
       switch (a.action) {
         case "goto": {
           const target = new URL(a.url, this.page.url() === "about:blank" ? undefined : this.page.url()).href;
           const refused = await this.checkUrl(target);
           if (refused) return refused;
+          step.replay = { action: "goto", url: target };
           await this.page.goto(target, { waitUntil: "domcontentloaded", timeout: 30000 });
           break;
         }
@@ -608,12 +681,14 @@ export class AuditSession {
             if (found.error) return found.error;
             target = found.locator;
           } else return `${a.action} needs a ref or the element's visible text.`;
+          step.replay = { action: a.action, ...(await replayTarget(a, target)), ...(a.value !== undefined ? { value: a.value } : {}) };
           if (a.action === "click") await this.click(target);
           else if (a.action === "fill") await target.fill(a.value ?? "", { timeout: 10000 });
           else await target.selectOption(a.value ?? "", { timeout: 10000 });
           break;
         }
         case "scroll": {
+          step.replay = { action: "scroll", ...(a.value !== undefined ? { value: a.value } : {}) };
           const dy = /^up$/i.test(a.value ?? "") ? -900 : 900;
           const size = this.page.viewportSize() ?? { width: 800, height: 600 };
           const before = await this.page.evaluate(() => window.scrollY);
@@ -625,6 +700,7 @@ export class AuditSession {
           break;
         }
         case "wait":
+          step.replay = { action: "wait", ...(a.value !== undefined ? { value: a.value } : {}) };
           await this.page.waitForTimeout(Math.min(Number(a.value) || 1000, 10000));
           break;
         default:
@@ -645,6 +721,12 @@ export class AuditSession {
   }
 
   async close() {
+    if (this.opts.recordHar && this.context) {
+      // Writes the recording. If the browser already went away, there is none to clean.
+      const written = await this.context.close().then(() => true, () => false);
+      // A recording that could not be cleaned may hold cookies or form data, so it is deleted.
+      if (written) await cleanRecording(this.opts.recordHar).catch(() => rm(this.opts.recordHar, { force: true }));
+    }
     await this.browser?.close();
     await this.egress?.close();
   }

package/src/checks.js

@@ -2,9 +2,10 @@
 // They run without any API call; the AI model only explains the results.
 
 import { messages } from "./i18n.js";
+import { isOwnAddress } from "./redact.js";
 
 const NON_GOOGLE_AD_PLATFORMS = new Set(["meta", "tiktok"]);
-const EMAIL = /[A-Z0-9._%+-]+(@|%40)[A-Z0-9.-]+\.[A-Z]{2,}/i;
+const EMAIL = /[A-Z0-9._%+-]+(@|%40)[A-Z0-9.-]+\.[A-Z]{2,}/gi;
 const SEVERITY_ORDER = { broken: 0, risk: 1, warning: 2, info: 3, ok: 4 };
 
 /**
@@ -24,6 +25,24 @@ function pageOf(hit) {
   return hit.params?.page_location ?? hit.pageUrl ?? "";
 }
 
+// Just before an address: a mailto: link or a link field (link_url=..., "link_text":"...").
+const LINK_BEFORE = /(mailto:|\b(link_url|link_text|click_url|click_text)(=|":"))$/i;
+
+/**
+ * Whether text holds an email address that could be a visitor's. The site's own address in a
+ * mailto: link or a link field is a contact link it shows; anywhere else it still counts.
+ */
+function visitorEmailIn(text, hit) {
+  let site = null;
+  try {
+    site = new URL(hit.params?.page_location ?? hit.pageUrl).hostname;
+  } catch { /* no page address */ }
+  return [...text.matchAll(EMAIL)].some((m) => {
+    const link = LINK_BEFORE.test(text.slice(Math.max(0, m.index - 16), m.index));
+    return !(link && isOwnAddress(m[0].replace(/%40/i, "@"), site));
+  });
+}
+
 function expectedIn(planEvent, journeyId) {
   return !planEvent.journeys || planEvent.journeys.includes(journeyId);
 }
@@ -172,7 +191,7 @@ export function checkRun(plan, run, { lang = "en" } = {}) {
     if (hit.userData?.plain?.length) {
       piiSeen.add(key(hit.platform, hit.name));
       findings.push({ ...base, severity: "risk", check: "pii", platform: hit.platform, event: hit.name, step: stepLabel(hit.step), detail: d.piiUnhashed(hit.platform, hit.name, hit.userData.plain) });
-    } else if (EMAIL.test(haystack)) {
+    } else if (visitorEmailIn(haystack, hit)) {
       piiSeen.add(key(hit.platform, hit.name));
       findings.push({ ...base, severity: "risk", check: "pii", platform: hit.platform, event: hit.name, step: stepLabel(hit.step), detail: d.pii(hit.platform, hit.name) });
     }

package/src/cli.js

@@ -6,6 +6,7 @@ import { LANGUAGES, messages } from "./i18n.js";
 import { PROVIDERS } from "./providers.js";
 import { JOURNEY_PRESETS } from "./journeys.js";
 import { PROVIDERS as TRACKING_PROVIDERS, DEFAULT_PROVIDERS } from "./parsers.js";
+import { replaySavedSite } from "./replay.js";
 
 const USAGE = `ai-tag-debugger: an open-source AI agent that QA-tests your analytics and ad tracking.
 
@@ -13,6 +14,8 @@ Usage:
   ai-tag-debugger run --site <url> --journey purchase [options]   Audit ready-made journeys, no config needed.
   ai-tag-debugger run --site <url> --goal "<what to do>"          Audit a journey you describe in plain language.
   ai-tag-debugger run --config <file> [options]                   Run an audit from a config file.
+  ai-tag-debugger replay <dir>... [--update]      Audit sites saved with --save-site again, offline and without AI,
+                                                  and list findings that changed. Exits with code 1 when any did.
   ai-tag-debugger mcp [--http] [--port <n>]       Start the MCP server for Claude, ChatGPT, Gemini and other chat apps.
 
 Run options:
@@ -41,6 +44,12 @@ Run options:
   --keep-bodies        Store tracking requests as sent. By default, emails, phone numbers and user ids sent
                        for ad matching are redacted in the report and capture.json ("keep_bodies" in the config).
   --no-branding        Leave the Analitik İşler logo and credit line out of the reports ("branding": false in the config).
+  --save-site <dir>    Also save the site's responses and the steps taken in this folder, so "replay" can audit
+                       the same pages again offline. Its findings become the expected ones.
+
+Replay options:
+  --update             Make this replay's findings the expected ones (after checking that the changes are right).
+  --out <dir>          Where to write the replay's reports (default: <dir>/last-replay).
 
 MCP options:
   --http               Serve over Streamable HTTP instead of stdio (for apps that connect to a URL).
@@ -87,6 +96,8 @@ async function main() {
       token: { type: "string" },
       reports: { type: "string" },
       "public-url": { type: "string" },
+      "save-site": { type: "string" },
+      update: { type: "boolean", default: false },
       help: { type: "boolean", short: "h", default: false },
     },
   });
@@ -97,6 +108,22 @@ async function main() {
     await startMcp({ http: values.http, port: values.port ? Number(values.port) : undefined, host: values.host, token: values.token, reportsDir: values.reports, publicUrl: values["public-url"], allowPrivateNetwork: values["allow-private-network"] });
     return;
   }
+  if (command === "replay" && !values.help) {
+    const dirs = positionals.slice(1);
+    if (!dirs.length) throw new Error("replay needs the folder of at least one site saved with --save-site.");
+    let changed = false;
+    for (const dir of dirs) {
+      const r = await replaySavedSite(dir, { update: values.update, outDir: dirs.length === 1 ? values.out : undefined });
+      const same = !r.missing.length && !r.extra.length;
+      changed ||= !same;
+      console.log(`${same ? "✓" : values.update ? "↻" : "✗"} ${r.site} (${dir}): ${same ? "same findings" : `${r.extra.length} new, ${r.missing.length} gone${values.update ? ", saved as expected" : ""}`}`);
+      for (const k of r.extra) console.log(`    + ${k}`);
+      for (const k of r.missing) console.log(`    - ${k}`);
+      for (const note of r.incomplete) console.log(`    ! ${note}`);
+    }
+    if (changed && !values.update) process.exitCode = 1;
+    return;
+  }
   if (values.help || command !== "run" || (!values.config && !values.site)) {
     console.log(USAGE);
     process.exit(values.help ? 0 : 2);
@@ -136,6 +163,7 @@ async function main() {
     headed: values.headed,
     branding: values["no-branding"] ? false : undefined,
     outDir: values.out,
+    saveSite: values["save-site"],
     log: (msg) => console.log(msg),
   });
 

package/src/mcp.js

@@ -22,7 +22,7 @@ import { loadConfig, readJson, runAudit, writeReports } from "./audit.js";
 import { DEFAULT_PROVIDERS, PROVIDERS as TRACKING_PROVIDERS } from "./parsers.js";
 import { expectEvents, filterFindings, JOURNEY_PRESETS, onlyEvents, presetFor, withPresetEvents } from "./journeys.js";
 
-const VERSION = "0.6.2";
+const VERSION = "0.6.3";
 const MAX_OPEN_AUDITS = 4;
 const IDLE_MS = 15 * 60 * 1000; // an audit untouched this long is closed
 const MAX_BODY_BYTES = 1024 * 1024;

package/src/parsers.js

@@ -35,9 +35,10 @@ export function parseGa4(url, postData) {
   // Matched on the GA4 protocol rather than the host, so server-side GTM
   // endpoints on first-party domains are captured too.
   if (!u.pathname.endsWith("/g/collect") || u.searchParams.get("v") !== "2") return null;
-  // gtag.js resends a hit to www.google.com/g/collect (marked gaf=1) when the first send
-  // seems lost, and copies some to doubleclick.net. GA4 doesn't count these twice, so they aren't hits.
-  if (GOOGLE_COMPANION_HOST.test(u.hostname) || u.searchParams.has("gaf")) return null;
+  // gtag.js resends a hit to www.google.com/g/collect when the first send seems lost, and
+  // copies some to doubleclick.net. GA4 doesn't count these twice, so they aren't hits.
+  // (Primary hits can carry gaf too, e.g. gaf=2, so the host decides, not the parameter.)
+  if (GOOGLE_COMPANION_HOST.test(u.hostname)) return null;
   const base = new URLSearchParams(u.search);
   const lines = postData ? postData.split("\n").filter(Boolean) : [""];
   return lines.map((line) => {
@@ -135,8 +136,12 @@ export function parseTrackingRequest(url, postData) {
 // hits: Google Ads remarketing and consent pings, and the other common ad pixels.
 const OTHER_BEACONS = [
   [/(^|\.)(doubleclick\.net|googleadservices\.com|google\.[a-z.]+)$/, /^\/pagead\/(viewthroughconversion|1p-user-list|landing)\//],
-  [/(^|\.)google\.[a-z.]+$/, /^\/ccm\/collect/],
+  [/(^|\.)google\.[a-z.]+$/, /^\/rmkt\/collect\//],
   [GOOGLE_COMPANION_HOST, /\/g\/collect$/],
+  [/(^|\.)merchant-center-analytics\.goog$/, /^\/mc\/collect/],
+  // Google tag pings on any host: consent mode (ccm), the Google tag's own /g/s/collect, and
+  // Google tag gateway paths served from the site's own domain (/<id>/gs/ccm/collect).
+  [/./, /\/(ccm|ccm\/s|g\/s)\/collect$/],
   [/^td\.doubleclick\.net$/, /^\/td\//],
   [/(^|\.)facebook\.com$/, /^\/privacy_sandbox\/pixel\//],
   [/^bat\.bing\.com$/, /^\/action\//],

package/src/redact.js

@@ -14,9 +14,48 @@ const USER_KEYS = new Set([
 const PUSH_KEYS = new Set(["email", "email_address", "phone", "phone_number", "sha256_email_address", "sha256_phone_number", "sha256_email", "sha256_phone"]);
 const USER_OBJECTS = /(^|\.)(user_data|user_provided_data|enhanced_conversion_data)(\.|$)/;
 
-const EMAIL = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
-/** A value that is an email address, whatever its field is called. */
-const isEmail = (v) => typeof v === "string" && EMAIL.test(v.trim());
+const EMAIL = /^(mailto:)?[^\s@]+@([^\s@?]+\.[^\s@?]+)(\?.*)?$/i;
+
+// Second levels that are shared under a country code (shop.com.tr, shop.co.uk), and hosting
+// suffixes whose subdomains belong to different owners (shop.myshopify.com).
+const SHARED_SECOND_LEVELS = new Set(["com", "co", "org", "net", "gov", "edu", "ac", "gen", "web", "bel", "k12", "or", "ne", "go"]);
+const SHARED_SUFFIXES = ["myshopify.com", "github.io", "vercel.app", "pages.dev", "netlify.app", "herokuapp.com", "web.app", "firebaseapp.com", "appspot.com", "wixsite.com", "blogspot.com", "azurewebsites.net", "cloudfront.net", "workers.dev"];
+
+/** "www.shop.example.com.tr" -> "example.com.tr", "a.shop.myshopify.com" -> "shop.myshopify.com": the part one owner controls. */
+function siteOf(host) {
+  const labels = String(host).toLowerCase().replace(/\.$/, "").split(".");
+  const name = labels.join(".");
+  const shared = SHARED_SUFFIXES.find((s) => name === s || name.endsWith(`.${s}`));
+  let n = 2;
+  if (shared) n = shared.split(".").length + 1;
+  else if (labels.length >= 3 && labels.at(-1).length === 2 && SHARED_SECOND_LEVELS.has(labels.at(-2))) n = 3;
+  return labels.slice(-n).join(".");
+}
+
+/** Whether an email address is on the audited site's own domain, like hello@shop.com on a link at shop.com. */
+export function isOwnAddress(email, siteHost) {
+  if (!siteHost) return false;
+  const domain = String(email).trim().match(EMAIL)?.[2];
+  return Boolean(domain) && siteOf(domain) === siteOf(siteHost);
+}
+
+// Fields that hold a link or its text. The site's own address there is a contact link it shows.
+const LINK_KEYS = new Set(["link_url", "link_text", "click_url", "click_text"]);
+const isLinkKey = (key) => {
+  const k = String(key ?? "").toLowerCase();
+  return LINK_KEYS.has(k.split(".").pop()) || /(^|\.)gtm\.element(url|text)$/.test(k);
+};
+
+/**
+ * A value that is an email address (or a mailto: link to one), whatever its field is called.
+ * The site's own address in a mailto: link or a link field is a contact link it shows, not
+ * visitor data. Anywhere else even an own-domain address could be a visitor's (staff, B2B).
+ */
+const isEmail = (v, site, key) => {
+  if (typeof v !== "string" || !EMAIL.test(v.trim())) return false;
+  const link = /^mailto:/i.test(v.trim()) || isLinkKey(key);
+  return !(link && isOwnAddress(v, site));
+};
 
 /** Whether a request parameter (or dotted JSON path) carries user data for ad matching. */
 export function isUserDataKey(key) {
@@ -49,34 +88,34 @@ function note(found, key, value) {
 }
 
 /** Redacts form-encoded parameters (a URL query or a GA4 body line). Returns null when nothing needed it. */
-function redactParamString(text, found) {
+function redactParamString(text, found, site) {
   const params = new URLSearchParams(text);
-  if (![...params].some(([k, v]) => isUserDataKey(k) || isEmail(v))) return null;
+  if (![...params].some(([k, v]) => isUserDataKey(k) || isEmail(v, site, k))) return null;
   const out = new URLSearchParams();
-  for (const [k, v] of params) out.append(k, isUserDataKey(k) || isEmail(v) ? note(found, k, v) : v);
+  for (const [k, v] of params) out.append(k, isUserDataKey(k) || isEmail(v, site, k) ? note(found, k, v) : v);
   return out.toString();
 }
 
-/** Redacts user data in a URL's query. Returns the URL unchanged when there is none. */
-export function redactUrl(url, found = emptyFound()) {
+/** Redacts user data in a URL's query. Returns the URL unchanged when there is none. site: see isOwnAddress. */
+export function redactUrl(url, found = emptyFound(), site = null) {
   let u;
   try {
     u = new URL(url);
   } catch {
     return { url, ...found };
   }
-  const query = redactParamString(u.search, found);
+  const query = redactParamString(u.search, found, site);
   if (query === null) return { url, ...found };
   u.search = query;
   return { url: u.href, ...found };
 }
 
-function redactJson(value, path, found, isKey) {
-  if (Array.isArray(value)) return value.map((v) => redactJson(v, path, found, isKey));
+function redactJson(value, path, found, isKey, site = null) {
+  if (Array.isArray(value)) return value.map((v) => redactJson(v, path, found, isKey, site));
   if (value && typeof value === "object") {
-    return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, redactJson(v, path ? `${path}.${k}` : k, found, isKey)]));
+    return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, redactJson(v, path ? `${path}.${k}` : k, found, isKey, site)]));
   }
-  if ((typeof value === "string" || typeof value === "number") && ((path && isKey(path)) || isEmail(value))) return note(found, path || "(value)", value);
+  if ((typeof value === "string" || typeof value === "number") && ((path && isKey(path)) || isEmail(value, site, path))) return note(found, path || "(value)", value);
   return value;
 }
 
@@ -110,21 +149,21 @@ export function redactBody(body, found = emptyFound()) {
   return { text, ...found };
 }
 
-/** Redacts user-data entries in parsed hit parameters. */
-export function redactParams(params, found = emptyFound()) {
+/** Redacts user-data entries in parsed hit parameters. site: see isOwnAddress. */
+export function redactParams(params, found = emptyFound(), site = null) {
   const out = {};
-  for (const [k, v] of Object.entries(params ?? {})) out[k] = isUserDataKey(k) || isEmail(v) ? note(found, k, v) : v;
+  for (const [k, v] of Object.entries(params ?? {})) out[k] = isUserDataKey(k) || isEmail(v, site, k) ? note(found, k, v) : v;
   return out;
 }
 
 /**
  * Redacts user data in a dataLayer push (GTM objects or gtag.js arguments):
- * email and phone fields, anything under user_data, and email-like values.
+ * email and phone fields, anything under user_data, and email-like values not on the site's own domain.
  */
-export function redactPush(value, found = emptyFound()) {
+export function redactPush(value, found = emptyFound(), site = null) {
   // gtag("set", "user_data", {...}) has no user_data key: the objects after the name are the user data.
   if (Array.isArray(value) && value.includes("user_data")) {
     return { value: value.map((v) => (v && typeof v === "object" ? redactJson(v, "user_data", found, isPushUserKey) : v)), ...found };
   }
-  return { value: redactJson(value, "", found, isPushUserKey), ...found };
+  return { value: redactJson(value, "", found, isPushUserKey, site), ...found };
 }

package/src/replay.js

@@ -0,0 +1,97 @@
+// Saved sites: the responses a site sent during an audit and the steps taken, so a later
+// version of the tool can audit the same pages offline and show what it now reports
+// differently. Replays need no network and no AI model, so they cost nothing.
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { runAudit } from "./audit.js";
+
+export const SAVED_SITE_FILE = "site.json";
+
+/** The steps a run took, as fixed steps a scripted run can repeat. */
+export function replayableJourney(run) {
+  const steps = run.steps.filter((s) => s.index > 0);
+  const start = steps[0]?.replay?.action === "goto" ? steps[0].replay.url : undefined;
+  return {
+    id: run.journeyId,
+    viewport: run.viewport,
+    ...(start ? { start } : {}),
+    steps: (start ? steps.slice(1) : steps)
+      .filter((s) => s.replay)
+      .map((s) => ({ ...s.replay, reason: s.label, ...(s.acceptsConsent ? { accepts_consent: true } : {}) })),
+  };
+}
+
+/** One line per finding, sorted: what a replay is compared on. */
+export function findingKeys(findings) {
+  return findings.map((f) => [f.severity, f.check, f.journey, f.viewport, f.platform ?? "-", f.event ?? "-"].join(" ")).sort();
+}
+
+/** Lines in one list and not the other, counting repeats. */
+export function compareKeys(expected, actual) {
+  const left = [...actual];
+  const missing = [];
+  for (const k of expected) {
+    const i = left.indexOf(k);
+    if (i === -1) missing.push(k);
+    else left.splice(i, 1);
+  }
+  return { missing, extra: left };
+}
+
+/** Writes site.json next to the recorded responses. The findings of this run become the expected ones. */
+export async function writeSavedSite(dir, { config, plan, runs, findings }) {
+  await mkdir(dir, { recursive: true });
+  const { journeys: _journeys, plan: _plan, gtm_container: _gtm, ...site } = config;
+  const saved = {
+    version: 1,
+    saved_at: new Date().toISOString(),
+    config: site,
+    plan,
+    journeys: runs.map(replayableJourney),
+    expected: findingKeys(findings),
+  };
+  await writeFile(path.join(dir, SAVED_SITE_FILE), JSON.stringify(saved, null, 2));
+}
+
+/**
+ * Audits a saved site again, offline and without AI, and compares the findings with the expected ones.
+ * update: make this replay's findings the expected ones.
+ * @returns {Promise<{ site: string, missing: string[], extra: string[], reportPaths: string[], incomplete: string[] }>}
+ */
+export async function replaySavedSite(dir, { update = false, outDir, log = () => {} } = {}) {
+  const file = path.join(dir, SAVED_SITE_FILE);
+  let saved;
+  try {
+    saved = JSON.parse(await readFile(file, "utf8"));
+  } catch (err) {
+    throw new Error(`Could not read ${file}: ${err.message}`);
+  }
+  const viewports = [...new Set(saved.journeys.map((j) => j.viewport))];
+  const findings = [];
+  const reportPaths = [];
+  const incomplete = [];
+  for (const viewport of viewports) {
+    const journeys = saved.journeys.filter((j) => j.viewport === viewport).map(({ viewport: _v, ...j }) => j);
+    const out = path.resolve(outDir ?? path.join(dir, "last-replay"), viewports.length > 1 ? viewport : "");
+    const result = await runAudit({
+      config: { ...saved.config, journeys },
+      plan: saved.plan,
+      viewports: [viewport],
+      scripted: true,
+      ai: false,
+      replaySite: dir,
+      lang: "en",
+      outDir: out,
+      log,
+    });
+    findings.push(...result.findings);
+    reportPaths.push(result.reportPath);
+    incomplete.push(...result.runs.filter((r) => !r.completed).map((r) => `${r.journeyId} (${r.viewport}): ${r.note}`));
+  }
+  const actual = findingKeys(findings);
+  const { missing, extra } = compareKeys(saved.expected ?? [], actual);
+  if (update && (missing.length || extra.length)) {
+    await writeFile(file, JSON.stringify({ ...saved, expected: actual }, null, 2));
+  }
+  return { site: saved.config.site, missing, extra, reportPaths, incomplete };
+}