raider_reporter 0.6.0 → 0.7.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e5bb1cd462fbac0bd6a44635be2b0e14f6da29667e6cb4cf0a5f358aed7581be
-  data.tar.gz: c814a3009d7e7e6ac0857bc34636ad1fdc913d3e3489ff64c3e5b62cd7f50dbb
+  metadata.gz: 6eb732095ea2284078a27ae18b268a3fe33a128e2ec131f95999bd278cd15cd2
+  data.tar.gz: 556f21195f523faabf7785eada01089a5537a7a4f052617152de87a0673c362b
 SHA512:
-  metadata.gz: a1064490905e372f3392d056343cc3b8304d1de127e785d122e5ef6e3c50622873583f7ceedece6f87bbf633d8fbe4e2b94a0111f3d325ff98ba6280ba84e75c
-  data.tar.gz: a91849038df0a28f54023e411d4115636ce97e1aa6a4ad83ec03a298f7f87e438e0b94583a6248e568c8c14af9c9176e813ddb0b287962e5ebd2021737a8748d
+  metadata.gz: ecf148c6450bdd03be6ed3634a89149ecd117d4b7e02cc65196d91c66cab050bedab950d3db7a2ebf9b66186cab381ff32d91e6783796f72bcb8a7d6a54877d9
+  data.tar.gz: 6dfe2cd5cb2ed735e124f580501c90ed9ef6ace13a21d7c37c7e8f8ed0bb25a6044f0ba248891b4c943898757f58bd077d7b57f9b05c0230cfce697d393f736b

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.7.0 — 2026-10-18
+
+- Add a local HTML report: every run now also writes `raider-results/index.html`, one self-contained page (inline CSS and JS, no network requests) built from every `run*.json` in the folder, so shards from one machine share a report. Failures come first with their message, exception class, backtrace and screenshots; a filter box narrows by test name or file. Open it in a browser after a run, or publish it as a CI artifact
+- `raider-reporter report [dir]` rebuilds the report from the run files on disk (default `raider-results`); exits 1 when there are no run files or the report cannot be written
+- `config.html_report = false` turns the report off. An existing `index.html` that raider_reporter did not write is never overwritten or removed; a warning is printed instead
+- Fail-open: a report error prints `[raider_reporter] HTML report failed: …` and leaves `run.json`, the upload and the exit status unchanged
+- No breaking changes
+
 ## 0.6.0 — 2026-10-14
 
 - Schema v1, additive: optional `failure.app_frame`, the first line of the raw exception backtrace that lies inside the project root (`Dir.pwd` when the reporter starts), outside `vendor/`, `.bundle/`, `Gem.path` and Ruby's library directory, and not `<internal:...>`. The path is made project-relative and the rest of the line is unchanged, e.g. `spec/pages/login_page.rb:42:in 'LoginPage#submit'`; at most 500 characters. The key is omitted when no line qualifies or computing it fails, so payloads without it are unchanged. Raider uses it to group failures by error

data/README.md

@@ -136,7 +136,8 @@ end
 | `token` | `ENV["RAIDER_TOKEN"]` | API token used for uploads. |
 | `endpoint` | `ENV["RAIDER_ENDPOINT"]` | Base URL of the Raider server to upload to. |
 | `output_dir` | `"raider-results"` | Directory for local output. |
-| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory (`run-<n>.json` and `attachments-<n>/` when sharded) inside `output_dir` before writing. Nothing else in `output_dir` is touched. |
+| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory (`run-<n>.json` and `attachments-<n>/` when sharded) inside `output_dir` before writing. Nothing else in `output_dir` is touched, except a raider_reporter-written `index.html`. |
+| `html_report` | `true` | Write `index.html`, the local HTML report, into `output_dir` after each run. See [Local report](#local-report). |
 | `environment` | `{}` | Free-form key/values stored with the run (browser, suite name, and so on). Values are JSON values: strings, booleans or numbers. |
 
 ## Local output
@@ -145,10 +146,23 @@ end
 raider-results/
   run.json          # the run in schema v1
   attachments/      # copies of attached screenshots and videos
+  index.html        # the local report, see below
 ```
 
 Add `raider-results/` to your `.gitignore`. When a run is sharded, the files are named per shard; see [Parallel runs](#parallel-runs).
 
+## Local report
+
+Every run also writes `raider-results/index.html`: one self-contained page (inline CSS and JS, no network requests, so it works offline) built from every `run*.json` in the folder. It shows totals and pass rate, then failures first with message, exception class, backtrace and screenshots, followed by the other statuses; a filter box narrows by test name or file. Open it in a browser or publish it as a CI artifact.
+
+Rebuild it from the run files on disk, for example after `parallel_rspec` has written one file per shard:
+
+```
+raider-reporter report [dir]   # default: raider-results; prints the path, exits 1 when there are no run files or the report cannot be written
+```
+
+Set `config.html_report = false` to turn it off. An `index.html` that raider_reporter did not write is never overwritten or removed; a warning is printed instead.
+
 ## Uploading to Raider
 
 If both a token and an endpoint are set, the run (plus attachments) is uploaded when the suite finishes:

data/lib/raider_reporter/cli.rb

@@ -3,8 +3,9 @@
 require "raider_reporter"
 
 module RaiderReporter
-  # `raider-reporter comment`: posts the build summary as a PR comment and sets a commit status.
-  # Fail-open: #run always returns 0, notices go to stderr, nothing goes to stdout.
+  # `raider-reporter comment` posts the build summary as a PR comment and sets a commit status
+  # (fail-open: always exits 0, notices go to stderr). `raider-reporter report [dir]` rebuilds the
+  # local HTML report (exits 1 when there are no run files or the report cannot be written).
   class CLI
     MARKER = "<!-- raider-reporter:build-comment -->"
     CONTEXT = "raider/tests"
@@ -23,6 +24,10 @@ module RaiderReporter
     }.freeze
     DEFAULT_API_URL = "https://api.github.com"
     USAGE = <<~TEXT
+      Usage: raider-reporter report [dir]
+
+      Rebuilds <dir>/index.html (default raider-results) from every run*.json in it. Prints its path.
+
       Usage: raider-reporter comment
 
       Posts (or updates) the Raider build summary on the current pull request and sets a status.
@@ -43,6 +48,8 @@ module RaiderReporter
     end
 
     def run
+      return report if @argv.first == "report" && !@argv.include?("--help") && !@argv.include?("-h")
+
       command = @argv.first
       if command == "comment" && !@argv.include?("--help") && !@argv.include?("-h")
         comment
@@ -57,6 +64,20 @@ module RaiderReporter
 
     private
 
+    def report
+      dir = File.expand_path(@argv[1] || "raider-results")
+      path = Writers::Html.new.write(dir)
+      unless path
+        @err.puts("[raider_reporter] no run*.json in #{dir}")
+        return 1
+      end
+      @out.puts(path)
+      0
+    rescue StandardError => e
+      @err.puts("[raider_reporter] report failed: #{e.class}: #{e.message}")
+      1
+    end
+
     def comment
       if @env["GITHUB_ACTIONS"] == "true"
         github_comment

data/lib/raider_reporter/configuration.rb

@@ -4,13 +4,14 @@ require "raider_reporter/azure_env"
 
 module RaiderReporter
   class Configuration
-    attr_accessor :token, :endpoint, :output_dir, :clean_output_dir, :environment, :max_attachment_bytes
+    attr_accessor :token, :endpoint, :output_dir, :clean_output_dir, :html_report, :environment, :max_attachment_bytes
 
     def initialize(env = ENV)
       @token = AzureEnv.value(env, "RAIDER_TOKEN")
       @endpoint = AzureEnv.value(env, "RAIDER_ENDPOINT")
       @output_dir = "raider-results"
       @clean_output_dir = true
+      @html_report = true
       @environment = {}
       @max_attachment_bytes = 5 * 1024 * 1024
     end

data/lib/raider_reporter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  VERSION = "0.6.0"
+  VERSION = "0.7.0"
 end

data/lib/raider_reporter/writers/html.rb

@@ -0,0 +1,291 @@
+# frozen_string_literal: true
+
+require "erb"
+require "fileutils"
+require "json"
+require "time"
+
+module RaiderReporter
+  module Writers
+    # Builds index.html from every run*.json in a results folder: one page with inline
+    # CSS and JS and no network requests, so a run can be read without Raider or Java.
+    class Html
+      class ForeignFile < StandardError; end
+
+      FILE = "index.html"
+      MARKER = '<meta name="generator" content="raider_reporter">'
+      ROW_LIMIT = 2_000
+      RUN_FILE = /\Arun(-[^\/]+)?\.json\z/
+      SECTIONS = [%w[failed Failed], %w[passed Passed], %w[pending Pending], %w[skipped Skipped]].freeze
+      PROVIDERS = { "github_actions" => "GitHub Actions", "azure_pipelines" => "Azure Pipelines" }.freeze
+
+      def self.ours?(path)
+        File.file?(path) && File.read(path, 4096).to_s.include?(MARKER)
+      rescue SystemCallError
+        false
+      end
+
+      def write(dir)
+        dir = File.expand_path(dir)
+        files = run_files(dir)
+        return nil if files.empty?
+
+        target = File.join(dir, FILE)
+        if File.exist?(target) && !self.class.ours?(target)
+          raise ForeignFile, "#{target} exists and was not written by raider_reporter; not overwritten"
+        end
+
+        runs, skipped = load(files)
+        runs, ignored = newest_run(runs)
+        tmp = "#{target}.tmp-#{Process.pid}"
+        File.write(tmp, render(runs.map { |r| r[:data] }, skipped, ignored, project_name))
+        File.rename(tmp, target)
+        target
+      ensure
+        FileUtils.rm_f(tmp) if tmp && File.exist?(tmp)
+      end
+
+      private
+
+      def run_files(dir)
+        return [] unless File.directory?(dir)
+
+        Dir.children(dir).grep(RUN_FILE).sort.map { |name| File.join(dir, name) }
+      end
+
+      def load(files)
+        runs = []
+        skipped = []
+        files.each do |file|
+          data = JSON.parse(File.read(file))
+          raise TypeError unless data.is_a?(Hash) && data["results"].is_a?(Array)
+
+          runs << { name: File.basename(file), data: data, mtime: File.mtime(file) }
+        rescue StandardError
+          skipped << File.basename(file)
+        end
+        [runs, skipped]
+      end
+
+      SHARD_WINDOW = 300
+
+      # Old files from earlier runs stay in the folder, so report only the newest run:
+      # its own file, or all the shard files that belong to it. Everything else is named in the footer.
+      def newest_run(runs)
+        return [runs, []] if runs.empty?
+
+        newest = runs.max_by { |r| time(meta_of(r)["finished_at"]) || r[:mtime] }
+        keep = shard_of(newest) ? runs.select { |r| same_run?(r, newest) } : [newest]
+        [keep, (runs - keep).map { |r| r[:name] }.sort]
+      end
+
+      def meta_of(run) = run[:data]["run"].is_a?(Hash) ? run[:data]["run"] : {}
+
+      def shard_of(run) = meta_of(run)["shard"].is_a?(Hash) ? meta_of(run)["shard"] : nil
+
+      def same_run?(run, newest)
+        return true if run.equal?(newest)
+
+        shard = shard_of(run)
+        return false unless shard
+
+        total = shard_of(newest)["total"]
+        return false if total && shard["total"] && shard["total"] != total
+
+        (started_at(run) - started_at(newest)).abs <= SHARD_WINDOW
+      end
+
+      def started_at(run) = time(meta_of(run)["started_at"]) || run[:mtime]
+
+      def project_name
+        name = File.basename(Dir.pwd)
+        name.empty? || name == "/" ? "Raider run" : name
+      end
+
+      def render(runs, skipped, ignored, project)
+        results = runs.flat_map { |run| run["results"] }.select { |r| r.is_a?(Hash) }
+        meta = runs.map { |run| run["run"].is_a?(Hash) ? run["run"] : {} }
+        counts = SECTIONS.to_h { |status, _| [status, results.count { |r| r["status"] == status }] }
+        counts["flaky"] = results.count { |r| r["status"] == "passed" && r["attempts"].to_i > 1 }
+
+        <<~HTML
+          <!doctype html>
+          <html lang="en">
+          <head>
+          <meta charset="utf-8">
+          <meta name="viewport" content="width=device-width, initial-scale=1">
+          #{MARKER}
+          <title>#{h(project)} · Raider report</title>
+          <style>#{CSS}</style>
+          </head>
+          <body>
+          <header>
+          <h1>#{h(project)}</h1>
+          <p class="totals">#{totals(counts)}</p>
+          <p class="meta">#{meta_line(meta)}</p>
+          </header>
+          <main>
+          #{results.empty? ? '<p class="empty">No results</p>' : filter_box}
+          #{SECTIONS.map { |status, label| section(status, label, results) }.join}
+          </main>
+          <footer>#{footer(skipped, ignored)}</footer>
+          <script>#{JS}</script>
+          </body>
+          </html>
+        HTML
+      end
+
+      def totals(counts)
+        passed = counts["passed"]
+        flaky = counts["flaky"]
+        decided = passed + counts["failed"]
+        rate = decided.zero? ? "—" : "#{(passed * 100.0 / decided).round}%"
+        parts = SECTIONS.map { |status, _| %(<span class="b #{status}">#{counts[status]} #{status}</span>) }
+        parts << %(<span class="b flaky">#{flaky} flaky</span>) if flaky.positive?
+        "#{parts.join(' ')} <span class=\"rate\">Pass rate #{rate}</span>"
+      end
+
+      def meta_line(meta)
+        parts = []
+        duration = run_duration(meta)
+        parts << "Duration #{h(duration)}" if duration
+        git = meta.map { |m| m["git"] }.find { |g| g.is_a?(Hash) } || {}
+        parts << "Branch #{h(git['branch'])}" if git["branch"]
+        parts << "Commit #{h(git['commit_sha'].to_s[0, 7])}" if git["commit_sha"]
+        ci = meta.map { |m| m["ci"] }.find { |c| c.is_a?(Hash) } || {}
+        parts << ci_link(ci) if ci["provider"]
+        parts << "Generated #{h(Time.now.utc.strftime('%Y-%m-%d %H:%M UTC'))}"
+        parts.join(" · ")
+      end
+
+      def ci_link(ci)
+        label = h(PROVIDERS.fetch(ci["provider"], ci["provider"]))
+        url = ci["build_url"].to_s
+        url.match?(%r{\Ahttps?://}) ? %(<a href="#{h(url)}">#{label}</a>) : label
+      end
+
+      def run_duration(meta)
+        starts = meta.filter_map { |m| time(m["started_at"]) }
+        ends = meta.filter_map { |m| time(m["finished_at"]) }
+        return nil if starts.empty? || ends.empty? || ends.max < starts.min
+
+        format_ms(((ends.max - starts.min) * 1000).round)
+      end
+
+      def time(value)
+        Time.iso8601(value.to_s)
+      rescue ArgumentError
+        nil
+      end
+
+      def filter_box
+        '<input id="q" type="search" placeholder="Filter by test name or file" aria-label="Filter tests">'
+      end
+
+      def section(status, label, results)
+        rows = results.select { |r| r["status"] == status }
+        return "" if rows.empty?
+
+        shown = rows.first(ROW_LIMIT).map { |r| row(r) }.join
+        more = rows.size > ROW_LIMIT ? %(<p class="more">#{rows.size - ROW_LIMIT} more not shown</p>) : ""
+        open = status == "failed" ? " open" : ""
+        %(<details class="s"#{open}><summary>#{label} (#{rows.size})</summary>#{shown}#{more}</details>\n)
+      end
+
+      def row(result)
+        key = "#{result['full_name']} #{result['file_path']}".downcase
+        location = [result["file_path"], result["line_number"]].compact.join(":")
+        attempts = result["attempts"].to_i > 1 ? %( <span class="att">#{result['attempts'].to_i} attempts</span>) : ""
+        <<~ROW
+          <div class="r #{h(result['status'])}" data-k="#{h(key)}">
+          <p class="n"><span class="b #{h(result['status'])}">#{h(result['status'])}</span> #{h(result['full_name'])}</p>
+          <p class="l">#{h(location)} · #{h(format_ms(result['duration_ms'].to_i))}#{attempts}</p>
+          #{failure(result['failure'])}#{attachments(result['attachments'])}
+          </div>
+        ROW
+      end
+
+      def failure(failure)
+        return "" unless failure.is_a?(Hash)
+
+        trace = Array(failure["backtrace"]).join("\n")
+        backtrace = trace.empty? ? "" : "<details><summary>Backtrace</summary><pre>#{h(trace)}</pre></details>"
+        %(<p class="x">#{h(failure['exception_class'])}</p><pre>#{h(failure['message'])}</pre>#{backtrace})
+      end
+
+      def attachments(list)
+        items = Array(list).filter_map do |attachment|
+          next unless attachment.is_a?(Hash)
+
+          href = safe_href(attachment["path"])
+          next unless href
+
+          if attachment["content_type"].to_s.start_with?("image/")
+            %(<a href="#{href}"><img loading="lazy" src="#{href}" alt="#{h(attachment['name'])}"></a>)
+          else
+            %(<a href="#{href}">#{h(attachment['name'])}</a>)
+          end
+        end
+        items.empty? ? "" : %(<div class="a">#{items.join(' ')}</div>)
+      end
+
+      # Relative paths inside the results folder only, each segment URL-encoded.
+      def safe_href(path)
+        path = path.to_s
+        segments = path.split("/")
+        return nil if path.empty? || path.start_with?("/") || segments.include?("..") || path.include?("\\")
+
+        segments.map { |segment| ERB::Util.url_encode(segment) }.join("/")
+      end
+
+      def footer(skipped, ignored)
+        note = skipped.empty? ? "" : "Skipped unreadable run files: #{h(skipped.join(', '))} · "
+        note += "Ignored older run files: #{h(ignored.join(', '))} · " unless ignored.empty?
+        "#{note}Generated by raider_reporter #{h(RaiderReporter::VERSION)}"
+      end
+
+      def format_ms(ms)
+        return "0ms" if ms.negative?
+        return "#{ms}ms" if ms < 1000
+
+        seconds = (ms / 1000.0).round
+        return format("%.1fs", ms / 1000.0) if ms < 59_950
+
+        "#{seconds / 60}m #{seconds % 60}s"
+      end
+
+      def h(value) = ERB::Util.html_escape(value.to_s)
+
+      CSS = <<~CSS.gsub(/\s*\n\s*/, "")
+        :root{--bg:#fff;--fg:#1f2328;--mute:#656d76;--bdr:#d0d7de;--card:#f6f8fa;--ruby:#C14420;--ok:#1a7f37;--warn:#9a6700}
+        @media (prefers-color-scheme:dark){:root{--bg:#0d1117;--fg:#e6edf3;--mute:#8d96a0;--bdr:#30363d;--card:#161b22;--ok:#3fb950;--warn:#d29922}}
+        *{box-sizing:border-box}
+        body{margin:0;padding:16px;background:var(--bg);color:var(--fg);font:14px/1.5 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
+        header,main,footer{max-width:1100px;margin:0 auto}
+        h1{margin:0 0 4px;font-size:20px;color:var(--ruby)}
+        .meta,.l,footer,.more{color:var(--mute);font-size:12px}
+        .b{display:inline-block;padding:0 6px;border-radius:9px;font-size:11px;font-weight:600;border:1px solid var(--bdr)}
+        .b.failed{background:var(--ruby);color:#fff;border-color:var(--ruby)}
+        .b.passed{color:var(--ok)}.b.pending,.b.skipped,.b.flaky{color:var(--warn)}
+        .rate{font-weight:600;margin-left:6px}
+        #q{width:100%;margin:12px 0;padding:8px;border:1px solid var(--bdr);border-radius:6px;background:var(--card);color:var(--fg)}
+        details.s{margin:8px 0;border:1px solid var(--bdr);border-radius:8px}
+        details.s>summary{padding:8px 12px;font-weight:600;cursor:pointer}
+        .r{padding:8px 12px;border-top:1px solid var(--bdr)}
+        .r p{margin:2px 0;overflow-wrap:anywhere}.n{font-weight:500}.x{color:var(--ruby);font-weight:600}
+        pre{margin:4px 0;padding:8px;background:var(--card);border-radius:6px;overflow-x:auto;white-space:pre-wrap;font-size:12px}
+        .a img{max-width:100%;max-height:240px;border:1px solid var(--bdr);border-radius:4px;margin-top:4px}
+        a{color:var(--ruby)}.empty{padding:24px 0;color:var(--mute)}
+      CSS
+
+      JS = <<~JS.gsub(/\s*\n\s*/, "")
+        (function(){var q=document.getElementById("q");if(!q)return;
+        q.addEventListener("input",function(){var v=q.value.toLowerCase();
+        document.querySelectorAll("details.s").forEach(function(s){var n=0;
+        s.querySelectorAll(".r").forEach(function(r){var hide=v!==""&&r.dataset.k.indexOf(v)<0;r.hidden=hide;if(!hide)n++;});
+        if(v===""){s.hidden=false;if(s.dataset.o!==undefined){s.open=s.dataset.o==="1";delete s.dataset.o;}}
+        else{if(s.dataset.o===undefined)s.dataset.o=s.open?"1":"0";s.hidden=n===0;s.open=n>0;}});});})();
+      JS
+    end
+  end
+end

data/lib/raider_reporter/writers/local.rb

@@ -2,6 +2,7 @@
 
 require "fileutils"
 require "json"
+require_relative "html"
 
 module RaiderReporter
   module Writers
@@ -28,6 +29,8 @@ module RaiderReporter
       def clean(dir, payload)
         FileUtils.rm_f(File.join(dir, payload.run_file))
         FileUtils.rm_rf(File.join(dir, payload.attachments_dir))
+        index = File.join(dir, Html::FILE)
+        FileUtils.rm_f(index) if Html.ours?(index)
       end
     end
   end

data/lib/raider_reporter.rb

@@ -11,6 +11,7 @@ require_relative "raider_reporter/configuration"
 require_relative "raider_reporter/content_types"
 require_relative "raider_reporter/collector"
 require_relative "raider_reporter/writers/local"
+require_relative "raider_reporter/writers/html"
 require_relative "raider_reporter/ci_detector"
 require_relative "raider_reporter/shard_detector"
 require_relative "raider_reporter/writers/uploader"
@@ -91,6 +92,7 @@ module RaiderReporter
       rescue StandardError => e
         warn("could not write #{configuration.output_dir}: #{e.class}: #{e.message}")
       end
+      write_html_report
       return nil unless configuration.upload?
 
       result = Writers::Uploader.new(configuration).upload(payload)
@@ -106,6 +108,14 @@ module RaiderReporter
 
     private
 
+    def write_html_report
+      return unless configuration.html_report
+
+      Writers::Html.new.write(configuration.output_dir)
+    rescue StandardError => e
+      warn("HTML report failed: #{e.class}: #{e.message}")
+    end
+
     def success_line(result)
       link = result.build_url || result.url
       return "Raider report: #{link}" if link

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: raider_reporter
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.7.0
 platform: ruby
 authors:
 - Augustin Gottlieb
@@ -44,6 +44,7 @@ files:
 - lib/raider_reporter/shard_detector.rb
 - lib/raider_reporter/text.rb
 - lib/raider_reporter/version.rb
+- lib/raider_reporter/writers/html.rb
 - lib/raider_reporter/writers/local.rb
 - lib/raider_reporter/writers/uploader.rb
 homepage: https://github.com/RaiderHQ/raider_reporter