browser_review_gate 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a2e4e7dfaa70718cc4eaf691aa3f60023ae10968b34127b0b363ff15c42971c6
-  data.tar.gz: e29d1f6ba94e5d07ebba865d6faa838efe9222eba7f5ece6e286fc3d88895bbf
+  metadata.gz: bf07fc0385a222895293b09a8000db74bf153a3b87731528b3e2d51c20cfd4e0
+  data.tar.gz: 5866378bb1acbfa17f03d44d98413c6f6222903858b7ec7dd6f71a9b6ba95788
 SHA512:
-  metadata.gz: d9859bc00fe366be2568ee088c18cdd19431e80cf0f468fd8370ec9465ce7a2111cd761ea96b1c44693c5188c1c1c2073daec2de928b05265942c9babdd99cd1
-  data.tar.gz: c5a80a4802e8d9053d3f0c2fcf2f197152e6b8b9ac9d8e172c24c3fcdc57ce153dd15802afaaab8fe31c8908d26a25161caa09a922252f3313b0f2d7333e45a4
+  metadata.gz: de467b5092e7d8d3e9871e789b2c844712808f6af4916173d3ec903fb84407eac61218c5e6ee54e4c20e91e10ccdd3cb4d06010a355e2ddae869118b496c4791
+  data.tar.gz: 4ca8044e0cc223d3e269542ff1edb0ab635bdd710c549d017e48802c834eeef9b00fb2ff7ed71f9c9be37bd683185ea2d5450e95256574da7b12cf82723b7979

data/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 0.4.0
+
+- `provenance.trace` (off by default): the gem opens each case's Playwright trace and checks that a
+  response carried the tested commit in `revision_header`, that the case's page was requested, and
+  that requests fall within the run's time window. `warn` shows the result in the PR; `require`
+  counts a passing case only with a verified trace. Traces are read in memory with only zlib, with
+  caps on entries, sizes and compression ratio.
+- A HAR network log (`.har`) is accepted as a case's trace too, so the check works with any browser
+  tool that saves HAR, not only Playwright.
+- `doctor` warns when `provenance.trace` is on and no Playwright is configured in the project; `verify`
+  tells the agent a trace or HAR is needed per case.
+- `BrowserReviewGate::RevisionHeader`: Rack middleware for development that sends `X-App-Revision`
+  with the checked-out commit and follows new commits without a restart.
+- `doctor` checks that the app sends the revision header and that it matches the checkout.
+- The installer's next steps name the 0.3 settings.
+
 ## 0.3.0
 
 Breaking: the report format and the settings change, and nothing from 0.2 is read. Move `url`,

data/README.md

@@ -95,6 +95,12 @@ scenarios:
 gate:
   require_evidence: false            # enforced: passing cases need the evidence asked for above
   block_on_failure: true             # enforced: any failing case keeps review on hold
+
+provenance:
+  trace: "off"                       # enforced: off | warn | require (see "Checking that a run happened")
+  revision_header: "X-App-Revision"
+  check_pages: true
+  clock_skew_seconds: 300
 ```
 
 Other keys: `provider` and `model` (who assesses, see below), `workflow`, `label`, `waiver_label`, `waiver_by_author`, `status_context`, `command`, `skip_authors`, `skip_branch_prefixes`, `ignore_paths`.
@@ -105,11 +111,30 @@ Other keys: `provider` and `model` (who assesses, see below), `workflow`, `label
 
 **Not configurable on purpose.** Run timeouts and retries are not settings: the browser run happens in the agent, and the gem cannot enforce them. Tell the agent in `rules.md` if you have expectations.
 
+## Checking that a run happened
+
+Off by default. A report is the agent's account; `provenance.trace` lets the gem check it against a Playwright trace instead of taking it on trust.
+
+1. Make the app name the commit it serves, in development only:
+
+   ```ruby
+   # config/environments/development.rb
+   config.middleware.use BrowserReviewGate::RevisionHeader
+   ```
+
+   It adds `X-App-Revision: <HEAD SHA>` to every response and follows new commits without a restart. Any other way of sending the header works too; name it in `provenance.revision_header`.
+2. Set `provenance.trace` to `warn` (results shown in the PR) or `require` (a passing case counts only with a verified trace). `doctor` checks that the app sends the header and that it matches the checkout.
+3. The agent records a Playwright trace (`.zip`) or a HAR network log (`.har`) per case and lists it as `evidence.trace`. HAR is a standard format that Playwright, Chrome DevTools ("Save all as HAR") and many other tools write, so the check does not need Playwright itself. A browser tool that can save neither cannot satisfy `require`; `doctor` and `verify` warn about it before a run.
+
+When the run is published, the gem opens each trace (a zip, or HAR JSON; read in memory with size, entry and compression-ratio caps, no extra dependency) and checks its network log: a response carried the revision header with the tested commit, the case's page was requested, and the request times fall within `started_at`..`finished_at` (± `clock_skew_seconds`). The PR shows "trace verified" or what was wrong.
+
+What it catches: an agent that reports a run it did not do, ran against another commit or a stale server, or ran a different page. What it does not catch: someone who builds a trace by hand. It is a check on agents, not an attestation.
+
 ## Reports
 
 A report is JSON in the shape the playbook describes (`schema_version: 2`): the tested commit (full SHA), the environment, the browser tool and its capabilities, start and finish times, and one case per scenario with `result` (`pass`, `fail`, `blocked`, `skipped`), the page, what was observed, and evidence.
 
-- Screenshots (`.png`, `.jpg`, `.webp`) and traces (`.zip`) are read from the report's directory. Paths that leave it, links, wrong file types (checked by content, not just name) and files over 20 MB are refused. Accepted files are hashed and kept in the git directory; the PR shows their names, sizes and SHA-256 digests, not the files.
+- Screenshots (`.png`, `.jpg`, `.webp`) and traces (`.zip` Playwright traces or `.har` network logs) are read from the report's directory. Paths that leave it, links, wrong file types (checked by content, not just name) and files over 20 MB are refused. Accepted files are hashed and kept in the git directory; the PR shows their names, sizes and SHA-256 digests, not the files.
 - Console errors and failed requests are trimmed, query strings are dropped, and values that look like credentials are masked.
 - Evidence links are accepted only on hosts listed in `evidence.link_hosts`; use them to point reviewers at screenshots you uploaded to your own artifact store.
 - A case counts when it passes, breaks no enforced policy, and was tested on a commit of the PR.
@@ -169,7 +194,7 @@ Set `provider` to `claude-cli`, `anthropic`, `openai` or `gemini` to choose expl
 
 Residual risks, stated plainly:
 
-- **A report is the agent's account, not an attestation.** Someone with push access, or an agent that misreports, can publish a passing report for a run that did not happen. The gem raises the cost (exact scenario ids, pages, observations, evidence digests, commit checks) but cannot prove a browser did anything. Human review remains the control.
+- **A report is the agent's account, not an attestation.** Someone with push access can publish a passing report for a run that did not happen. With `provenance.trace: require` an agent cannot do so by accident or by guessing, but a hand-built trace still passes. Human review remains the control.
 - **Evidence files stay local.** Reviewers see digests, not screenshots, unless you upload them and link them through `evidence.link_hosts`.
 - **Scrubbing is pattern-based.** It catches common credential shapes, not everything. Agents are told not to record secrets; screenshots and traces are never inspected.
 - **A taken-back review request may still notify the reviewer**, because GitHub has no way to refuse a request.

data/lib/browser_review_gate/config.rb

@@ -30,6 +30,7 @@ module BrowserReviewGate
       "evidence" => nil,
       "scenarios" => nil,
       "gate" => nil,
+      "provenance" => nil,
       "skip_authors" => [ "dependabot[bot]", "github-actions[bot]" ],
       "skip_branch_prefixes" => [],
       # A PR that only touches these paths cannot change browser behavior; no model call is made.
@@ -41,8 +42,12 @@ module BrowserReviewGate
       "evidence" => { "screenshots" => "never", "trace" => "never", "console_logs" => false, "network_logs" => false,
                       "retention_days" => 14, "link_hosts" => [] },
       "scenarios" => { "max_count" => 12, "reuse_passing" => true, "mandatory" => [], "matrix" => {} },
-      "gate" => { "require_evidence" => false, "block_on_failure" => true }
+      "gate" => { "require_evidence" => false, "block_on_failure" => true },
+      # Checks a case's Playwright trace against the tested commit: off, warn (shown in the PR) or
+      # require (a passing case counts only with a verified trace).
+      "provenance" => { "trace" => "off", "revision_header" => "X-App-Revision", "check_pages" => true, "clock_skew_seconds" => 300 }
     }.freeze
+    TRACE_MODES = %w[off warn require].freeze
 
     ENVIRONMENT_KEYS = %w[url healthcheck start_command sign_in].freeze
     MANDATORY_KEYS = %w[id name why paths matrix].freeze
@@ -81,7 +86,7 @@ module BrowserReviewGate
       validate
     end
 
-    (DEFAULTS.keys - %w[environments verification evidence scenarios gate]).each do |key|
+    (DEFAULTS.keys - %w[environments verification evidence scenarios gate provenance]).each do |key|
       define_method(key) { @data[key] }
     end
 
@@ -91,6 +96,7 @@ module BrowserReviewGate
     def evidence = @sections["evidence"]
     def scenarios = @sections["scenarios"]
     def gate = @sections["gate"]
+    def provenance = @sections["provenance"]
 
     # The environment a run uses: the one named, the configured default, or the only one there is.
     def environment(name = nil)
@@ -132,10 +138,12 @@ module BrowserReviewGate
           "require_evidence" => gate["require_evidence"],
           "block_on_failure" => gate["block_on_failure"],
           "reuse_passing" => scenarios["reuse_passing"],
-          "max_scenarios" => scenarios["max_count"]
+          "max_scenarios" => scenarios["max_count"],
+          "trace_check" => provenance["trace"]
         },
         "agent" => {
-          "screenshots" => evidence["screenshots"], "trace" => evidence["trace"],
+          "screenshots" => evidence["screenshots"],
+          "trace" => provenance["trace"] == "off" ? evidence["trace"] : "always",
           "console_logs" => evidence["console_logs"], "network_logs" => evidence["network_logs"]
         }
       }
@@ -182,6 +190,7 @@ module BrowserReviewGate
       environment # the configured default must exist
       validate_flags
       validate_evidence
+      validate_provenance
       validate_scenarios
     end
 
@@ -209,6 +218,15 @@ module BrowserReviewGate
       raise Error, "#{PATH}: evidence.link_hosts must be a list of host names"
     end
 
+    def validate_provenance
+      raise Error, "#{PATH}: provenance.trace must be one of #{TRACE_MODES.join(", ")}" unless TRACE_MODES.include?(provenance["trace"])
+      raise Error, "#{PATH}: provenance.revision_header must be an HTTP header name" unless provenance["revision_header"].to_s.match?(/\A[A-Za-z0-9-]{1,64}\z/)
+      raise Error, "#{PATH}: provenance.check_pages must be true or false" unless [ true, false ].include?(provenance["check_pages"])
+
+      skew = provenance["clock_skew_seconds"]
+      raise Error, "#{PATH}: provenance.clock_skew_seconds must be a whole number from 0 to 3600" unless skew.is_a?(Integer) && skew.between?(0, 3600)
+    end
+
     def validate_scenarios
       count = scenarios["max_count"]
       raise Error, "#{PATH}: scenarios.max_count must be a whole number from 1 to #{MAX_SCENARIOS}" unless count.is_a?(Integer) && count.between?(1, MAX_SCENARIOS)

data/lib/browser_review_gate/doctor.rb

@@ -19,7 +19,9 @@ module BrowserReviewGate
     }.freeze
 
     # `probe` takes a URL and says whether something answers there.
-    def initialize(root:, shell: Shell.new, github: nil, probe: nil, config_loader: nil)
+    # `header_probe` takes a URL and a header name and returns the header's value there, or nil.
+    def initialize(root:, shell: Shell.new, github: nil, probe: nil, config_loader: nil, header_probe: nil)
+      @header_probe = header_probe || Status.method(:header_at)
       @root = root
       @shell = shell
       @github = github
@@ -37,7 +39,7 @@ module BrowserReviewGate
       repository_access if gh_ready
       environments(config) if config
       agents(config)
-      browser_tools
+      browser_tools(config)
       workflow(config) if config
       required_check(config) if config && gh_ready
       provider_secrets(config) if config && gh_ready
@@ -134,6 +136,7 @@ module BrowserReviewGate
         url = Status.healthcheck_url(settings.merge("name" => name))
         if @probe.call(url)
           add("environment #{name}", "PASS", "answers at #{url}")
+          revision_header(config, name, settings["url"])
         else
           add("environment #{name}", "WARN", "does not answer at #{url}",
             settings["start_command"] ? "Start it with #{settings["start_command"]} before a run" : "Make it reachable from where the agent runs")
@@ -141,6 +144,23 @@ module BrowserReviewGate
       end
     end
 
+    # With trace checks on, a run can only count when the app names the commit it serves.
+    def revision_header(config, name, url)
+      mode = config.provenance["trace"]
+      return if mode == "off"
+
+      header = config.provenance["revision_header"]
+      served = @header_probe.call(url, header).to_s.strip.downcase
+      head = run("git", "-C", @root, "rev-parse", "HEAD").to_s.strip
+      label = "revision header #{name}"
+      fix = "Add `config.middleware.use BrowserReviewGate::RevisionHeader` to config/environments/development.rb"
+      return add(label, mode == "require" ? "FAIL" : "WARN", "#{url} sends no #{header} header; traces cannot be checked", fix) if served.empty?
+      return add(label, "PASS", "#{header} matches the checked-out commit") if !head.empty? && head.start_with?(served)
+
+      add(label, "WARN", "#{header} is #{served[0, 12]}, the checkout is at #{head[0, 12]}",
+        "Serve the checked-out code (restart the server, or run it from this worktree)")
+    end
+
     def agents(config)
       installer = Installer.new(root: @root, log: StringIO.new)
       installer.detect_agents.each do |agent|
@@ -163,14 +183,30 @@ module BrowserReviewGate
     end
 
     # Only what the project's own files show. A tool configured elsewhere cannot be seen from here.
-    def browser_tools
+    def browser_tools(config)
       files = [ ".mcp.json", ".cursor/mcp.json", ".codex/config.toml", ".claude/settings.json", "package.json" ]
       text = files.map { |path| read(path) }.join("\n").downcase
       found = BROWSER_TOOLS.select { |key, _label| text.include?(key) }.values
-      return add("browser tool", "PASS", "configured in the project: #{found.join(", ")}") if found.any?
+      if found.any?
+        add("browser tool", "PASS", "configured in the project: #{found.join(", ")}")
+      else
+        add("browser tool", "SKIP", "none found in the project's files; one may be configured per user",
+          "Give the agent a browser tool, e.g. Playwright MCP or Chrome DevTools MCP")
+      end
+      trace_tool(config, text) if config
+    end
+
+    # With trace checks on, every case needs a Playwright trace or a HAR file. Only Playwright is known
+    # to write both; other tools may write HAR, but that cannot be told from here.
+    def trace_tool(config, text)
+      mode = config.provenance["trace"]
+      return if mode == "off"
+      return add("trace recording", "PASS", "Playwright is configured in the project; it records traces and HAR files") if text.include?("playwright")
 
-      add("browser tool", "SKIP", "none found in the project's files; one may be configured per user",
-        "Give the agent a browser tool, e.g. Playwright MCP or Chrome DevTools MCP")
+      add("trace recording", mode == "require" ? "WARN" : "SKIP",
+        "provenance.trace is #{mode}, and no Playwright is configured in the project's files. Each case needs a Playwright " \
+        "trace (.zip) or a HAR file (.har); a browser tool that cannot save either #{mode == "require" ? "gets every passing case refused" : "gets no check"}",
+        "Give the agent Playwright (e.g. Playwright MCP), or a tool that saves HAR files, or set provenance.trace to warn or off")
     end
 
     def workflow(config)

data/lib/browser_review_gate/evidence.rb

@@ -22,9 +22,11 @@ module BrowserReviewGate
       ".jpg" => [ "image/jpeg", [ "\xFF\xD8\xFF".b ] ],
       ".jpeg" => [ "image/jpeg", [ "\xFF\xD8\xFF".b ] ],
       ".webp" => [ "image/webp", [ "RIFF".b ] ],
-      ".zip" => [ "application/zip", [ "PK\x03\x04".b ] ]
+      ".zip" => [ "application/zip", [ "PK\x03\x04".b ] ],
+      # A HAR network log: JSON, so it starts with "{" after optional whitespace or a byte order mark.
+      ".har" => [ "application/json", [ "{".b ] ]
     }.freeze
-    TRACE_TYPES = %w[.zip].freeze
+    TRACE_TYPES = %w[.zip .har].freeze
     SECRET = /
       (?<key>authorization|proxy-authorization|cookie|set-cookie|x-api-key|api[_-]?key|access[_-]?token|refresh[_-]?token|
         id[_-]?token|token|secret|password|passwd|session(?:[_-]?id)?|csrf[_-]?token|auth)
@@ -67,6 +69,7 @@ module BrowserReviewGate
     def process(data)
       return data unless data.is_a?(Hash) && data["cases"].is_a?(Array)
 
+      @run = data
       data.merge("cases" => data["cases"].each_with_index.map { |test_case, index| process_case(test_case, index + 1) })
     end
 
@@ -96,6 +99,8 @@ module BrowserReviewGate
       raise Error, "case #{position} evidence lists more than #{MAX_FILES} files" if listed.size > MAX_FILES
 
       published["files"] = listed.map { |kind, path| file(kind, path, position) } if listed.any?
+      trace = @files.reverse.find { |file| file["kind"] == "trace" && published.fetch("files", []).any? { |entry| entry["sha256"] == file["sha256"] } }
+      published["trace_check"] = check_trace(trace["source"], test_case) if trace && @config.provenance["trace"] != "off"
       published["console_errors"] = entries(evidence["console_errors"], position, "console_errors") { |entry| Evidence.scrub(entry) } if evidence.key?("console_errors")
       published["network_errors"] = entries(evidence["network_errors"], position, "network_errors") { |entry| network(entry) } if evidence.key?("network_errors")
       published["links"] = links(evidence["links"], position) if evidence.key?("links")
@@ -121,7 +126,7 @@ module BrowserReviewGate
       raise Error, "case #{position} #{kind} file #{path} is larger than #{MAX_FILE_BYTES / 1024 / 1024} MB" if bytes > MAX_FILE_BYTES
 
       media_type, signatures = TYPES.fetch(extension)
-      head = File.binread(real, 16).to_s
+      head = File.binread(real, 64).to_s.delete_prefix("\xEF\xBB\xBF".b).lstrip
       raise Error, "case #{position} #{kind} file #{path} is not a #{media_type} file" unless signatures.any? { |signature| head.start_with?(signature) }
 
       entry = { "kind" => KINDS.fetch(kind), "name" => File.basename(real)[0, 100], "media_type" => media_type,
@@ -130,6 +135,14 @@ module BrowserReviewGate
       entry
     end
 
+    # A trace that cannot be read is a failed check, not an error: the PR shows why.
+    def check_trace(path, test_case)
+      TraceCheck.new(@config).call(path, tested_sha: @run["tested_sha"].to_s, url: test_case["url"],
+        started_at: @run["started_at"], finished_at: @run["finished_at"])
+    rescue Error => error
+      { "status" => "failed", "requests" => 0, "findings" => [ error.message[0, 300] ] }
+    end
+
     def relative?(path)
       path.is_a?(String) && !path.empty? && path.length <= 300 && !path.start_with?("/", "~") &&
         !path.include?("\0") && !path.split(%r{[/\\]}).include?("..")

data/lib/browser_review_gate/installer.rb

@@ -193,9 +193,10 @@ module BrowserReviewGate
       <<~TEXT
         Next:
           1. Add one secret to the repository: CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_API_KEY, OPENAI_API_KEY or GEMINI_API_KEY.
-          2. Fill start_command, url and sign_in in #{Config::PATH}.
+          2. Describe your environments (url, healthcheck, start_command, sign_in) in #{Config::PATH}.
           3. Describe what this application always checks in #{Prompts::RULES}.
           4. Commit and merge to the default branch: the workflow runs from there.
+          5. Run browser-review-gate doctor to check the setup.
       TEXT
     end
   end

data/lib/browser_review_gate/report.rb

@@ -19,7 +19,7 @@ module BrowserReviewGate
     TOP_KEYS = %w[schema_version tested_sha verified_by environment tool started_at finished_at coverage_complete
                   cases excluded runs failed_runs].freeze
     CASE_KEYS = %w[id name result url details reason evidence tested_sha verified_by environment tool].freeze
-    EVIDENCE_KEYS = %w[files console_errors network_errors links].freeze
+    EVIDENCE_KEYS = %w[files console_errors network_errors links trace_check].freeze
     FILE_KEYS = %w[kind name media_type bytes sha256].freeze
 
     attr_reader :errors
@@ -112,7 +112,7 @@ module BrowserReviewGate
         return "failed network requests were reported"
       end
 
-      missing_evidence(test_case, evidence)
+      missing_evidence(test_case, evidence) || trace_problem(evidence)
     end
 
     def failed? = cases.any? { |test_case| status_of(test_case["id"]) == "failed" }
@@ -139,7 +139,8 @@ module BrowserReviewGate
           found << "case #{position} is #{test_case["result"]} and needs a reason" unless text?(test_case["reason"], 300)
         end
         found << "case #{position} details must say what was observed" unless text?(test_case["details"], 500)
-        problem = test_case["result"] == "pass" && @config.gate["require_evidence"] ? missing_evidence(test_case, test_case["evidence"] || {}) : nil
+        evidence = test_case["evidence"].is_a?(Hash) ? test_case["evidence"] : {}
+        problem = test_case["result"] == "pass" ? (missing_evidence(test_case, evidence) || trace_problem(evidence)) : nil
         found << "case #{position} #{problem}" if problem
         found
       end
@@ -193,6 +194,17 @@ module BrowserReviewGate
       nil
     end
 
+    # With provenance.trace require, a passing case counts only when its trace showed the run.
+    def trace_problem(evidence)
+      return unless @config.provenance["trace"] == "require"
+
+      check = evidence["trace_check"]
+      return "has no Playwright trace, which provenance.trace requires" unless check.is_a?(Hash)
+      return if check["status"] == "verified"
+
+      "trace does not show the run: #{Array(check["findings"]).first || "the check failed"}"
+    end
+
     # A carried-over case keeps the tool of the run that produced it.
     def tool_of(test_case) = test_case["tool"].is_a?(Hash) ? test_case["tool"] : tool
 
@@ -305,6 +317,7 @@ module BrowserReviewGate
       unless files.nil? || (files.is_a?(Array) && files.size <= Evidence::MAX_FILES && files.all? { |file| file?(file) })
         errors << "case #{position} evidence files are malformed"
       end
+      validate_trace_check(evidence["trace_check"], position) unless evidence["trace_check"].nil?
       links = Array(evidence["links"])
       errors << "case #{position} evidence links must be https URLs" unless links.all? { |link| link.to_s.match?(%r{\Ahttps://[^\s()<>]+\z}) }
       %w[console_errors network_errors links].each do |key|
@@ -315,6 +328,14 @@ module BrowserReviewGate
       end
     end
 
+    def validate_trace_check(check, position)
+      valid = check.is_a?(Hash) && (check.keys - %w[status revision requests findings]).empty? &&
+        %w[verified failed].include?(check["status"]) && (check["revision"].nil? || check["revision"].to_s.match?(/\A\h{7,12}\z/)) &&
+        check["requests"].is_a?(Integer) && check["findings"].is_a?(Array) && check["findings"].size <= 5 &&
+        check["findings"].all? { |finding| finding.is_a?(String) && finding.length <= 300 }
+      errors << "case #{position} evidence trace_check is malformed" unless valid
+    end
+
     def file?(file)
       file.is_a?(Hash) && (file.keys - FILE_KEYS).empty? && Evidence::KINDS.value?(file["kind"]) && text?(file["name"], 100) &&
         file["bytes"].is_a?(Integer) && file["sha256"].to_s.match?(/\A\h{64}\z/) && Evidence::TYPES.values.map(&:first).include?(file["media_type"])
@@ -353,6 +374,10 @@ module BrowserReviewGate
       parts << "#{evidence["console_errors"].size} console errors" if evidence["console_errors"]
       parts << "#{evidence["network_errors"].size} failed requests" if evidence["network_errors"]
       parts.concat(Array(evidence["links"]).each_with_index.map { |link, index| "[link #{index + 1}](#{link})" })
+      if (check = evidence["trace_check"])
+        parts << (check["status"] == "verified" ? "trace verified: revision `#{check["revision"]}`, #{check["requests"]} requests" :
+                                                  "trace check failed: #{Markdown.escape(check["findings"].first)}")
+      end
       parts.empty? ? "none" : parts.join("<br>")
     end
 

data/lib/browser_review_gate/revision_header.rb

@@ -0,0 +1,72 @@
+module BrowserReviewGate
+  # Rack middleware that names the commit the app is serving in a response header, so a Playwright
+  # trace shows which code a browser run exercised (`provenance.trace`). Meant for development and
+  # test environments only: it reads the checkout's HEAD, and a production app has no business telling
+  # visitors its commit.
+  #
+  #   # config/environments/development.rb
+  #   config.middleware.use BrowserReviewGate::RevisionHeader
+  #
+  # HEAD is read again whenever .git/HEAD or the branch it points to changes, so a new commit shows up
+  # without a restart. A worktree with uncommitted changes is still reported as HEAD: publishing a run
+  # requires a clean worktree anyway.
+  class RevisionHeader
+    def initialize(app, header: "X-App-Revision", root: Dir.pwd)
+      @app = app
+      @header = header.downcase # Rack 3 wants lowercase header names
+      @git_dir = git_dir(root)
+      @cached = nil
+    end
+
+    def call(env)
+      status, headers, body = @app.call(env)
+      revision = current_revision
+      headers[@header] = revision if revision
+      [ status, headers, body ]
+    end
+
+    private
+
+    def current_revision
+      return unless @git_dir
+
+      head_path = File.join(@git_dir, "HEAD")
+      head = File.read(head_path).strip
+      ref_path = head.start_with?("ref: ") ? File.join(common_dir, head.delete_prefix("ref: ")) : nil
+      stamp = [ head, File.mtime(head_path), ref_path && File.exist?(ref_path) ? File.mtime(ref_path) : nil ]
+      return @cached.last if @cached&.first == stamp
+
+      @cached = [ stamp, resolve(head, ref_path) ]
+      @cached.last
+    rescue SystemCallError
+      nil
+    end
+
+    def resolve(head, ref_path)
+      return head[/\A\h{40}\z/] unless ref_path
+      return File.read(ref_path).strip[/\A\h{40}\z/] if File.exist?(ref_path)
+
+      # A ref that was packed: look it up in packed-refs.
+      packed = File.join(common_dir, "packed-refs")
+      ref = head.delete_prefix("ref: ")
+      File.exist?(packed) ? File.foreach(packed).find { |line| line.strip.end_with?(" #{ref}") }.to_s[/\A\h{40}/] : nil
+    end
+
+    # A worktree's .git is a file pointing at its git directory; branches live in the common one.
+    def git_dir(root)
+      dot_git = File.join(root, ".git")
+      return dot_git if File.directory?(dot_git)
+      return unless File.file?(dot_git)
+
+      path = File.read(dot_git)[/\Agitdir: (.+)$/, 1]
+      path && File.expand_path(path.strip, root)
+    end
+
+    def common_dir
+      @common_dir ||= begin
+        pointer = File.join(@git_dir, "commondir")
+        File.exist?(pointer) ? File.expand_path(File.read(pointer).strip, @git_dir) : @git_dir
+      end
+    end
+  end
+end

data/lib/browser_review_gate/status.rb

@@ -41,6 +41,17 @@ module BrowserReviewGate
       false
     end
 
+    # The value of `header` in the answer at `url`, or nil. Same limits as answers?.
+    def self.header_at(url, header)
+      uri = URI(url)
+      return unless %w[http https].include?(uri.scheme) && uri.host
+
+      Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", verify_mode: OpenSSL::SSL::VERIFY_NONE,
+        open_timeout: 2, read_timeout: 3) { |http| http.head(uri.request_uri)[header] }
+    rescue StandardError
+      nil
+    end
+
     # The URL a health check asks: an absolute `healthcheck`, or its path on the environment URL.
     def self.healthcheck_url(environment)
       check = environment["healthcheck"]

data/lib/browser_review_gate/templates/config.yml.erb

@@ -64,6 +64,17 @@ gate:
   # [enforced] Any failing case keeps review on hold, not only failing required scenarios.
   block_on_failure: true
 
+provenance:
+  # [enforced] Check each case's Playwright trace against the tested commit. off: no check. warn: the
+  # result is shown in the PR. require: a passing case counts only when its trace shows a response
+  # from the tested commit (via revision_header), a request to the case's page, and request times
+  # inside the run. The app must send the header: add BrowserReviewGate::RevisionHeader to the
+  # development middleware. With warn or require the agent is asked for a trace of every case.
+  trace: "off"
+  revision_header: "X-App-Revision"
+  check_pages: true
+  clock_skew_seconds: 300
+
 # Who assesses a PR in CI. Unset: the Claude Code CLI with a CLAUDE_CODE_OAUTH_TOKEN or
 # ANTHROPIC_API_KEY secret; with only an OPENAI_API_KEY or GEMINI_API_KEY secret, that provider's API.
 # Set `provider` to claude-cli, anthropic, openai or gemini to choose; openai and gemini need `model`.

data/lib/browser_review_gate/templates/playbook.md.erb

@@ -25,6 +25,7 @@ This file is the playbook for the AI coding agent that does the run. The person
 4. Run every scenario end to end. For each case record:
    - `url`: the page it was exercised on; `details`: what you observed;
    - evidence, as `policy.agent` in the `verify` output asks: screenshots (`never`, `on_failure`, `always`), a trace, and the console errors and failed requests you saw (`[]` when there were none). Save files in a directory outside the repository, next to the report.
+   - When `policy.enforced.trace_check` is `warn` or `require`, record a Playwright trace (`.zip`) or a HAR network log (`.har`, which Playwright and Chrome DevTools can save) for every case, one per case, and list it as `evidence.trace`. If your browser tool can save neither, stop and tell the person: with `require`, every passing case would be refused. The gem opens it and checks that the app answered with the tested commit, that the case's page was requested, and that the requests fall between `started_at` and `finished_at`. With `require`, a passing case without such a trace does not count. Record the trace while running the case; a trace from another run will not match.
    - A scenario you could not run is `blocked` (something prevented it) or `skipped` (deliberately not run), with a `reason`. Neither is a pass.
 5. Write the report as JSON next to the evidence (see the shape below). Set `coverage_complete` to true only when every scenario from step 2 ran.
 6. Publish, from the checkout that was tested, with everything committed and pushed: `browser-review-gate verify --report PATH` (add `--pr NUMBER` when the branch has several PRs). With a PR it is published; without one it is saved and published when the PR for this commit is opened.
@@ -37,6 +38,7 @@ This file is the playbook for the AI coding agent that does the run. The person
 - With `gate.require_evidence`, a passing case without the screenshots, trace or logs the project asks for does not count, unless your tool lacks that capability (say so in `tool.capabilities`).
 - With `verification.fail_on_console_errors` / `fail_on_network_errors`, a passing case that lists such errors counts as failed.
 - Only cases tested on a commit of this PR count.
+- With `provenance.trace`, it reads each case's Playwright trace and checks it against the tested commit and the case's page.
 - It cannot tell whether a browser really did what you wrote. File digests show that files existed when the run was published, not who made them.
 
 ## When a scenario cannot be run
@@ -84,7 +86,7 @@ Some scenarios cannot be exercised locally, for example a sign-in through an ext
 
 - `result`: `pass`, `fail`, `blocked` or `skipped`. `url` is an http(s) URL or a path starting with `/`; required for `pass` and `fail`. `reason` is required for `blocked` and `skipped`.
 - `tool.capabilities`: what your browser tool could record, from `screenshots`, `trace`, `console`, `network`. Claim only what it really can.
-- `evidence` paths are relative to the report file's directory. Accepted: `.png`, `.jpg`, `.jpeg`, `.webp` screenshots and a `.zip` trace, at most 10 files of 20 MB each. `network_errors` entries may be strings or `{ "method", "url", "status" }`; query strings are dropped. `links` may point to evidence kept elsewhere, on hosts listed in `evidence.link_hosts`.
+- `evidence` paths are relative to the report file's directory. Accepted: `.png`, `.jpg`, `.jpeg`, `.webp` screenshots and a `.zip` Playwright trace or `.har` network log, at most 10 files of 20 MB each. `network_errors` entries may be strings or `{ "method", "url", "status" }`; query strings are dropped. `links` may point to evidence kept elsewhere, on hosts listed in `evidence.link_hosts`.
 - Everything you write is scrubbed of values that look like credentials, but do not rely on it: never put cookies, tokens, passwords or customer data into a report, a screenshot or a trace. Keep customer and real company names out of the report.
 
 ## Rules

data/lib/browser_review_gate/trace_check.rb

@@ -0,0 +1,119 @@
+require "json"
+require "time"
+require "uri"
+
+module BrowserReviewGate
+  # Reads a Playwright trace (trace.zip) or a HAR network log (.har, which Playwright, Chrome DevTools
+  # and other tools write) and checks that it shows the case really ran against the
+  # tested code: a response carried the app's revision header with the tested commit, the case's page
+  # was requested, and the requests fall inside the run's time window.
+  #
+  # This catches an agent that reports a run it did not do. It does not stop someone who builds a
+  # trace by hand; a trace is a file like any other.
+  class TraceCheck
+    NETWORK_ENTRY = /(?:\A|\/)(?:\d+-)?trace\.network\z/
+
+    def initialize(config)
+      @settings = config.provenance
+    end
+
+    # Returns { "status" => "verified" | "failed", "revision" => ..., "requests" => n, "findings" => [...] }.
+    def call(path, tested_sha:, url:, started_at:, finished_at:)
+      requests = read(path)
+      findings = []
+      revisions = requests.filter_map { |request| request[:revision] }.uniq
+      matched = revisions.find { |revision| tested_sha.start_with?(revision) }
+      if revisions.empty?
+        findings << "no response carried the #{@settings["revision_header"]} header"
+      elsif matched.nil?
+        findings << "the app answered with revision #{revisions.first[0, 12]}, not the tested commit"
+      end
+      findings << "the case's page #{url} was not requested" if @settings["check_pages"] && url && !page_requested?(requests, url)
+      findings.concat(time_findings(requests, started_at, finished_at))
+      findings << "the trace has no network requests" if requests.empty?
+
+      { "status" => findings.empty? ? "verified" : "failed", "revision" => matched && tested_sha[0, 12],
+        "requests" => requests.size, "findings" => findings.first(5) }.compact
+    end
+
+    private
+
+    def read(path)
+      return read_har(path) if File.extname(path).casecmp?(".har")
+
+      requests = []
+      ZipReader.new(path).each(->(name) { NETWORK_ENTRY.match?(name) }) do |_name, content|
+        content.each_line do |line|
+          request = parse(line)
+          requests << request if request
+        end
+      end
+      requests
+    end
+
+    def read_har(path)
+      text = File.binread(path, ZipReader::MAX_ENTRY_BYTES + 1).to_s.force_encoding(Encoding::UTF_8).delete_prefix("\uFEFF")
+      log = JSON.parse(text)
+      entries = log.is_a?(Hash) ? log.dig("log", "entries") : nil
+      raise Error, "the HAR file has no log.entries" unless entries.is_a?(Array)
+
+      entries.filter_map { |entry| request(entry) }
+    rescue JSON::ParserError
+      raise Error, "the HAR file is not valid JSON"
+    end
+
+    # One HAR-like resource snapshot per line; anything else is skipped.
+    def parse(line)
+      event = JSON.parse(line)
+      event.is_a?(Hash) && event["type"] == "resource-snapshot" ? request(event["snapshot"]) : nil
+    rescue JSON::ParserError
+      nil
+    end
+
+    # A HAR entry, which is also what a Playwright resource snapshot is.
+    def request(snapshot)
+      return unless snapshot.is_a?(Hash) && snapshot["request"].is_a?(Hash)
+
+      headers = snapshot.dig("response", "headers")
+      header = Array(headers).find { |entry| entry.is_a?(Hash) && entry["name"].to_s.casecmp?(@settings["revision_header"]) }
+      { url: snapshot["request"]["url"].to_s, time: time(snapshot["startedDateTime"]),
+        revision: header && header["value"].to_s.strip.downcase[/\A\h{7,40}\z/] }
+    end
+
+    def time(value)
+      Time.iso8601(value.to_s)
+    rescue ArgumentError
+      nil
+    end
+
+    # The case's page counts as requested when a request has the same path, and the same host when
+    # the case names a full URL.
+    def page_requested?(requests, url)
+      wanted = URI.parse(url)
+      requests.any? do |request|
+        seen = URI.parse(request[:url])
+        seen.path.to_s.chomp("/") == wanted.path.to_s.chomp("/") && (wanted.host.nil? || wanted.host == seen.host)
+      rescue URI::InvalidURIError
+        false
+      end
+    rescue URI::InvalidURIError
+      false
+    end
+
+    def time_findings(requests, started_at, finished_at)
+      return [] unless started_at && finished_at
+
+      skew = @settings["clock_skew_seconds"]
+      from = Time.iso8601(started_at) - skew
+      to = Time.iso8601(finished_at) + skew
+      times = requests.filter_map { |request| request[:time] }
+      return [ "the trace has no request times" ] if times.empty? && requests.any?
+      return [] if times.empty?
+
+      outside = times.count { |seen| seen < from || seen > to }
+      outside.positive? ? [ "#{outside} of #{times.size} requests fall outside the run's started_at..finished_at" ] : []
+    rescue ArgumentError
+      []
+    end
+  end
+end

data/lib/browser_review_gate/verify.rb

@@ -69,9 +69,18 @@ module BrowserReviewGate
       [ "A browser run is needed#{status["pull_request"] ? " for PR ##{status["pull_request"]}" : ""}.",
         "Environment: #{environment["name"] || "not configured"}#{environment["url"] ? " at #{environment["url"]}" : ""}.",
         "Scenarios to run in your browser tool, one case per id:", *list,
+        *trace_note(status),
         "Then: browser-review-gate verify --report PATH (publishes, or saves before the PR exists)." ].join("\n")
     end
 
+    def self.trace_note(status)
+      mode = status.dig("policy", "enforced", "trace_check")
+      return [] if mode.nil? || mode == "off"
+
+      [ "Record a Playwright trace (.zip) or a HAR file (.har) for every case and list it as evidence.trace." \
+        "#{mode == "require" ? " Without one, a passing case is refused. If your browser tool can save neither, stop and tell the person." : ""}" ]
+    end
+
     private
 
     def submit(status, report)

data/lib/browser_review_gate/version.rb

@@ -1,3 +1,3 @@
 module BrowserReviewGate
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/lib/browser_review_gate/zip_reader.rb

@@ -0,0 +1,103 @@
+require "zlib"
+
+module BrowserReviewGate
+  # Reads selected entries of a zip archive into memory, using only zlib. Made for Playwright traces,
+  # which come from the agent and are untrusted: entry count, sizes and compression ratio are capped,
+  # nothing is written to disk, and entry names are only compared, never used as paths.
+  class ZipReader
+    MAX_ENTRIES = 10_000
+    MAX_ENTRY_BYTES = 50 * 1024 * 1024
+    MAX_TOTAL_BYTES = 200 * 1024 * 1024
+    MAX_RATIO = 200
+    END_OF_DIRECTORY = "PK\x05\x06".b
+    DIRECTORY_ENTRY = "PK\x01\x02".b
+    LOCAL_ENTRY = "PK\x03\x04".b
+
+    def initialize(path)
+      @path = path
+    end
+
+    # Yields the name and content of every entry whose name the block's filter accepts.
+    def each(filter)
+      total = 0
+      File.open(@path, "rb") do |file|
+        directory(file).each do |entry|
+          next unless filter.call(entry[:name])
+
+          content = read(file, entry)
+          total += content.bytesize
+          raise Error, "trace is larger than #{MAX_TOTAL_BYTES / 1024 / 1024} MB uncompressed" if total > MAX_TOTAL_BYTES
+
+          yield entry[:name], content
+        end
+      end
+    end
+
+    private
+
+    def directory(file)
+      size = file.size
+      tail_size = [ size, 65_557 ].min
+      file.seek(size - tail_size)
+      tail = file.read(tail_size)
+      at = tail.rindex(END_OF_DIRECTORY) or raise Error, "trace is not a zip archive"
+      count, _directory_size, offset = tail[at + 10, 10].unpack("vVV")
+      raise Error, "trace has too many entries" if count > MAX_ENTRIES
+
+      file.seek(offset)
+      Array.new(count) { directory_entry(file) }
+    end
+
+    def directory_entry(file)
+      header = file.read(46)
+      raise Error, "trace zip directory is damaged" unless header&.start_with?(DIRECTORY_ENTRY)
+
+      method, compressed, uncompressed, name_length, extra_length, comment_length, offset =
+        header.unpack("@10v@20VV@28vvv@42V")
+      name = file.read(name_length).to_s.force_encoding(Encoding::UTF_8)
+      file.seek(extra_length + comment_length, IO::SEEK_CUR)
+      { name: name, method: method, compressed: compressed, uncompressed: uncompressed, offset: offset }
+    end
+
+    def read(file, entry)
+      raise Error, "trace entry #{entry[:name][0, 80]} is too large" if entry[:uncompressed] > MAX_ENTRY_BYTES
+      if entry[:compressed].positive? && entry[:uncompressed] / entry[:compressed] > MAX_RATIO
+        raise Error, "trace entry #{entry[:name][0, 80]} is compressed suspiciously well"
+      end
+
+      position = file.pos
+      file.seek(entry[:offset])
+      local = file.read(30)
+      raise Error, "trace zip entry is damaged" unless local&.start_with?(LOCAL_ENTRY)
+
+      name_length, extra_length = local.unpack("@26vv")
+      file.seek(name_length + extra_length, IO::SEEK_CUR)
+      data = file.read(entry[:compressed]).to_s
+      file.seek(position)
+      content = case entry[:method]
+                when 0 then data
+                when 8 then inflate(data)
+                else raise Error, "trace entry #{entry[:name][0, 80]} uses an unsupported compression method"
+                end
+      raise Error, "trace entry #{entry[:name][0, 80]} is not the size its header says" unless content.bytesize == entry[:uncompressed]
+
+      content
+    end
+
+    # Stops as soon as the output passes the entry cap, so a lying header cannot exhaust memory.
+    def inflate(data)
+      inflater = Zlib::Inflate.new(-Zlib::MAX_WBITS)
+      out = +""
+      (0...data.bytesize).step(4096) do |offset|
+        out << inflater.inflate(data.byteslice(offset, 4096))
+        raise Error, "trace entry is larger than it says" if out.bytesize > MAX_ENTRY_BYTES
+      end
+      out << inflater.finish
+      out
+    rescue Zlib::Error
+      raise Error, "trace entry is not valid deflate data"
+    ensure
+      inflater&.close
+    end
+  end
+end

data/lib/browser_review_gate.rb

@@ -25,6 +25,9 @@ module BrowserReviewGate
   autoload :CLI, File.join(__dir__, "browser_review_gate/cli")
   autoload :Doctor, File.join(__dir__, "browser_review_gate/doctor")
   autoload :Evidence, File.join(__dir__, "browser_review_gate/evidence")
+  autoload :RevisionHeader, File.join(__dir__, "browser_review_gate/revision_header")
+  autoload :TraceCheck, File.join(__dir__, "browser_review_gate/trace_check")
+  autoload :ZipReader, File.join(__dir__, "browser_review_gate/zip_reader")
   autoload :Verify, File.join(__dir__, "browser_review_gate/verify")
   autoload :Config, File.join(__dir__, "browser_review_gate/config")
   autoload :Gate, File.join(__dir__, "browser_review_gate/gate")

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: browser_review_gate
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Islam Gagiev
@@ -55,6 +55,7 @@ files:
 - lib/browser_review_gate/prompts.rb
 - lib/browser_review_gate/publisher.rb
 - lib/browser_review_gate/report.rb
+- lib/browser_review_gate/revision_header.rb
 - lib/browser_review_gate/saved_report.rb
 - lib/browser_review_gate/shell.rb
 - lib/browser_review_gate/stats.rb
@@ -67,8 +68,10 @@ files:
 - lib/browser_review_gate/templates/policy.md
 - lib/browser_review_gate/templates/rules.md.erb
 - lib/browser_review_gate/templates/workflow.yml.erb
+- lib/browser_review_gate/trace_check.rb
 - lib/browser_review_gate/verify.rb
 - lib/browser_review_gate/version.rb
+- lib/browser_review_gate/zip_reader.rb
 - lib/generators/browser_review_gate/install/install_generator.rb
 homepage: https://github.com/Halvanhelv/browser_review_gate
 licenses: