raider_reporter 0.6.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: e5bb1cd462fbac0bd6a44635be2b0e14f6da29667e6cb4cf0a5f358aed7581be
-  data.tar.gz: c814a3009d7e7e6ac0857bc34636ad1fdc913d3e3489ff64c3e5b62cd7f50dbb
+  metadata.gz: 300255531548132ae2fedcd7f10e8e9832925a204f5790ed8accb1d272ab64c6
+  data.tar.gz: ae60aacce4ecd140c8d771b92e8d83b704ec945a3339917d79fb7aeb50d49da9
 SHA512:
-  metadata.gz: a1064490905e372f3392d056343cc3b8304d1de127e785d122e5ef6e3c50622873583f7ceedece6f87bbf633d8fbe4e2b94a0111f3d325ff98ba6280ba84e75c
-  data.tar.gz: a91849038df0a28f54023e411d4115636ce97e1aa6a4ad83ec03a298f7f87e438e0b94583a6248e568c8c14af9c9176e813ddb0b287962e5ebd2021737a8748d
+  metadata.gz: 0213a3b50dbeeedb77ac4e335afc632bf6a25ec815e875cabd4f54bed79fb4164b23770968f38911503055bdb6bc941460f379f08c7e34238f4c379740352f8e
+  data.tar.gz: 6c2c8759eee7dccfb81f93095d4a5c522b6029680ba999ab56073a91a76f5b7e7524da659c9350ea22f8185e1d9c1f294a75ee53aea4ceced898c36b699e0a8d

data/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 0.8.0 — 2026-10-24
+
+- Quarantine: with `RAIDER_QUARANTINE=1` (or `config.quarantine = true`) and a token and endpoint, the formatters fetch the project's quarantine list once per run (`GET /api/v1/quarantine`, 3-second timeout) and report matching failures as `quarantined`, keeping their message, backtrace and attachments. Fail-open: any fetch problem prints `[raider_reporter] quarantine list unavailable (<reason>); no tests quarantined` and nothing is quarantined
+- `raider-reporter exec -- <command…>` runs the suite (no shell) and exits 0 when the command exited 1 and every failure the framework counted was quarantined, with no errors outside tests; it prints `raider-reporter: N quarantined failure(s) did not fail the build`. Otherwise it exits with the command's code (128 + signal when killed, 127 when the command cannot start, 2 without a command)
+- Runs with a quarantined result are written as schema 2; every other run stays schema 1, so older Raider servers keep accepting them
+- The HTML report shows quarantined results in their own section and color; `raider-reporter comment` reads `N quarantined` from the headline
+- RSpec and Cucumber exit codes are unchanged without `exec`
+
+## 0.7.0 — 2026-10-18
+
+- Add a local HTML report: every run now also writes `raider-results/index.html`, one self-contained page (inline CSS and JS, no network requests) built from every `run*.json` in the folder, so shards from one machine share a report. Failures come first with their message, exception class, backtrace and screenshots; a filter box narrows by test name or file. Open it in a browser after a run, or publish it as a CI artifact
+- `raider-reporter report [dir]` rebuilds the report from the run files on disk (default `raider-results`); exits 1 when there are no run files or the report cannot be written
+- `config.html_report = false` turns the report off. An existing `index.html` that raider_reporter did not write is never overwritten or removed; a warning is printed instead
+- Fail-open: a report error prints `[raider_reporter] HTML report failed: …` and leaves `run.json`, the upload and the exit status unchanged
+- No breaking changes
+
 ## 0.6.0 — 2026-10-14
 
 - Schema v1, additive: optional `failure.app_frame`, the first line of the raw exception backtrace that lies inside the project root (`Dir.pwd` when the reporter starts), outside `vendor/`, `.bundle/`, `Gem.path` and Ruby's library directory, and not `<internal:...>`. The path is made project-relative and the rest of the line is unchanged, e.g. `spec/pages/login_page.rb:42:in 'LoginPage#submit'`; at most 500 characters. The key is omitted when no line qualifies or computing it fails, so payloads without it are unchanged. Raider uses it to group failures by error

data/README.md

@@ -1,6 +1,6 @@
 # raider_reporter
 
-Raider-schema test reporting for RSpec and Cucumber. `raider_reporter` is an RSpec formatter and a Cucumber reporter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
+Raider-schema test reporting for RSpec and Cucumber. `raider_reporter` is an RSpec formatter and a Cucumber reporter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1, or v2 when a run has a quarantined result), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
 
 It has no runtime dependencies and requires Ruby 3.2 or newer.
 
@@ -66,7 +66,7 @@ cucumber --format pretty --format RaiderReporter::Cucumber::Formatter --out raid
 - `identifier` is `"<file_path>:<line>"`: the scenario line, or the Examples row line for outline rows.
 - `tags` are the feature, scenario and Examples tags, merged, sorted, unique, without `@`.
 
-Statuses: passed, failed and ambiguous (reported as failed), pending and undefined (reported as pending), skipped.
+Statuses: passed, failed and ambiguous (reported as failed), pending and undefined (reported as pending), skipped. Failed (and ambiguous) scenarios on the quarantine list are reported as quarantined.
 
 ### Failures
 
@@ -136,19 +136,34 @@ end
 | `token` | `ENV["RAIDER_TOKEN"]` | API token used for uploads. |
 | `endpoint` | `ENV["RAIDER_ENDPOINT"]` | Base URL of the Raider server to upload to. |
 | `output_dir` | `"raider-results"` | Directory for local output. |
-| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory (`run-<n>.json` and `attachments-<n>/` when sharded) inside `output_dir` before writing. Nothing else in `output_dir` is touched. |
+| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory (`run-<n>.json` and `attachments-<n>/` when sharded) inside `output_dir` before writing. Nothing else in `output_dir` is touched, except a raider_reporter-written `index.html`. |
+| `html_report` | `true` | Write `index.html`, the local HTML report, into `output_dir` after each run. See [Local report](#local-report). |
+| `quarantine` | `true` when `ENV["RAIDER_QUARANTINE"]` is `1` or `true` | Fetch the quarantine list at run start and report matching failures as `quarantined`. Needs `token` and `endpoint`. See [Quarantine](#quarantine). |
 | `environment` | `{}` | Free-form key/values stored with the run (browser, suite name, and so on). Values are JSON values: strings, booleans or numbers. |
 
 ## Local output
 
 ```
 raider-results/
-  run.json          # the run in schema v1
+  run.json          # the run in schema v1 (v2 with quarantined results)
   attachments/      # copies of attached screenshots and videos
+  index.html        # the local report, see below
 ```
 
 Add `raider-results/` to your `.gitignore`. When a run is sharded, the files are named per shard; see [Parallel runs](#parallel-runs).
 
+## Local report
+
+Every run also writes `raider-results/index.html`: one self-contained page (inline CSS and JS, no network requests, so it works offline) built from every `run*.json` in the folder. It shows totals and pass rate, then failures first with message, exception class, backtrace and screenshots, followed by the other statuses; a filter box narrows by test name or file. Open it in a browser or publish it as a CI artifact.
+
+Rebuild it from the run files on disk, for example after `parallel_rspec` has written one file per shard:
+
+```
+raider-reporter report [dir]   # default: raider-results; prints the path, exits 1 when there are no run files or the report cannot be written
+```
+
+Set `config.html_report = false` to turn it off. An `index.html` that raider_reporter did not write is never overwritten or removed; a warning is printed instead.
+
 ## Uploading to Raider
 
 If both a token and an endpoint are set, the run (plus attachments) is uploaded when the suite finishes:
@@ -165,6 +180,24 @@ Reporting is fail-open: no error inside the gem, network failure, timeout or non
 
 The hosted Raider service is not public yet; uploads go to a Raider server whose address you provide in `RAIDER_ENDPOINT`.
 
+## Quarantine
+
+A test quarantined in Raider (with a reason and an expiry) still runs and is still reported, but its failure is reported as `quarantined`, and a CI build stays green when quarantined tests are its only failures.
+
+1. Turn it on in CI with `RAIDER_QUARANTINE=1` (or `config.quarantine = true`), next to `RAIDER_TOKEN` and `RAIDER_ENDPOINT`.
+2. Run the suite through `raider-reporter exec`:
+
+   ```
+   bundle exec raider-reporter exec -- bundle exec rspec
+   bundle exec raider-reporter exec -- bundle exec cucumber
+   ```
+
+When the run starts, the formatter fetches the project's quarantine list once (`GET /api/v1/quarantine`, 3-second timeout). On any error it prints `[raider_reporter] quarantine list unavailable (<reason>); no tests quarantined` and nothing is quarantined. Matching failures keep their message, backtrace and screenshots; the run is written as schema 2, and `run.json`, the upload and the HTML report show `quarantined`.
+
+`exec` runs the command without a shell and exits with its status, with one exception: when the command exits 1, every failure the framework counted is quarantined, and nothing went wrong outside tests, it prints `raider-reporter: N quarantined failure(s) did not fail the build` to stderr and exits 0. "Outside tests" means RSpec errors outside examples, a run stopped early (`--fail-fast`, an interrupt, a load error), or a Cucumber run that did not finish every scenario. RSpec `--fail-fast` counts even when the last example triggers it, and interrupts count too, so the build stays red. Under Cucumber `--strict`, a flaky scenario (failed, then passed on retry) counts as a real failure and keeps the build red. A command killed by a signal exits 128 + the signal number; a command that cannot be started exits 127. Several processes under one `exec` (for example `parallel_rspec`) never turn red into green: their summaries conflict and the command's exit code is used.
+
+Without `exec`, RSpec and Cucumber exit codes are unchanged; quarantine only changes how failures are reported.
+
 ## Pull request comments
 
 On pull requests, `raider-reporter comment` posts one comment with the build's results (and sets a `raider/tests` commit status on the PR head commit). Run it in a job that runs after your test jobs, so every shard has uploaded. Use the workflow's `GITHUB_TOKEN` (or a GitHub App token): a personal access token posts as a user, so the step adds a new comment on every run instead of updating the existing one.
@@ -305,7 +338,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
 
 ## Schema
 
-`run.json` follows schema v1, documented in [docs/schema-v1.md](docs/schema-v1.md). Statuses are exactly `passed`, `failed`, `pending` and `skipped`. In both RSpec and Cucumber, a failure may carry `app_frame` (when a project frame is found): the first backtrace line inside your project, for example `spec/pages/login_page.rb:42:in 'LoginPage#submit'`, which Raider uses to group failures with the same error.
+`run.json` follows schema v1, documented in [docs/schema-v1.md](docs/schema-v1.md). Statuses are `passed`, `failed`, `pending` and `skipped`, plus `quarantined` in schema 2 (written only when a run has a quarantined result). In both RSpec and Cucumber, a failure may carry `app_frame` (when a project frame is found): the first backtrace line inside your project, for example `spec/pages/login_page.rb:42:in 'LoginPage#submit'`, which Raider uses to group failures with the same error.
 
 ## Contributing
 

data/lib/raider_reporter/cli.rb

@@ -1,10 +1,13 @@
 # frozen_string_literal: true
 
 require "raider_reporter"
+require "raider_reporter/exec"
 
 module RaiderReporter
-  # `raider-reporter comment`: posts the build summary as a PR comment and sets a commit status.
-  # Fail-open: #run always returns 0, notices go to stderr, nothing goes to stdout.
+  # `raider-reporter exec -- <command…>` runs the test suite and decides the build's exit code (see Exec).
+  # `raider-reporter comment` posts the build summary as a PR comment and sets a commit status
+  # (fail-open: always exits 0, notices go to stderr). `raider-reporter report [dir]` rebuilds the
+  # local HTML report (exits 1 when there are no run files or the report cannot be written).
   class CLI
     MARKER = "<!-- raider-reporter:build-comment -->"
     CONTEXT = "raider/tests"
@@ -23,6 +26,12 @@ module RaiderReporter
     }.freeze
     DEFAULT_API_URL = "https://api.github.com"
     USAGE = <<~TEXT
+      #{Exec::USAGE.chomp}
+
+      Usage: raider-reporter report [dir]
+
+      Rebuilds <dir>/index.html (default raider-results) from every run*.json in it. Prints its path.
+
       Usage: raider-reporter comment
 
       Posts (or updates) the Raider build summary on the current pull request and sets a status.
@@ -43,6 +52,22 @@ module RaiderReporter
     end
 
     def run
+      return run_exec if @argv.first == "exec"
+
+      run_reporting
+    end
+
+    private
+
+    # exec decides the build's exit code; Exec owns its own failure handling.
+    def run_exec
+      command = @argv[1] == "--" ? @argv.drop(2) : []
+      Exec.new(command, err: @err).run
+    end
+
+    def run_reporting
+      return report if @argv.first == "report" && !@argv.include?("--help") && !@argv.include?("-h")
+
       command = @argv.first
       if command == "comment" && !@argv.include?("--help") && !@argv.include?("-h")
         comment
@@ -55,7 +80,19 @@ module RaiderReporter
       0
     end
 
-    private
+    def report
+      dir = File.expand_path(@argv[1] || "raider-results")
+      path = Writers::Html.new.write(dir)
+      unless path
+        @err.puts("[raider_reporter] no run*.json in #{dir}")
+        return 1
+      end
+      @out.puts(path)
+      0
+    rescue StandardError => e
+      @err.puts("[raider_reporter] report failed: #{e.class}: #{e.message}")
+      1
+    end
 
     def comment
       if @env["GITHUB_ACTIONS"] == "true"
@@ -160,7 +197,7 @@ module RaiderReporter
       url.empty? ? DEFAULT_API_URL : url
     end
 
-    # "**Raider: failed** - 2 failed, 148 passed, 3 skipped (1m 12s)" -> "2 failed, 148 passed, 3 skipped".
+    # "**Raider: failed** - 2 failed, 1 quarantined, 148 passed (1m 12s)" -> "2 failed, 1 quarantined, 148 passed".
     # Falls back to a fixed text per state when the Markdown carries no headline counts.
     def description(summary)
       counts = headline_counts(summary.markdown)
@@ -172,7 +209,7 @@ module RaiderReporter
       line = markdown.to_s.lines.find { |l| l.include?("Raider:") }
       return nil unless line
 
-      counts = line.scan(/\b\d+ (?:failed|passed|skipped|pending)\b/)
+      counts = line.scan(/\b\d+ (?:failed|quarantined|passed|skipped|pending)\b/)
       counts.empty? ? nil : counts.join(", ")
     end
 

data/lib/raider_reporter/collector.rb

@@ -3,15 +3,19 @@
 require "securerandom"
 require "tmpdir"
 require "fileutils"
+require "set"
 
 module RaiderReporter
   # Accumulates one suite run. Framework formatters call it; writers read the Run it returns.
   class Collector
-    def initialize(max_attachment_bytes:, warn:, clock: -> { Time.now.utc }, uuid: -> { SecureRandom.uuid })
+    def initialize(max_attachment_bytes:, warn:, clock: -> { Time.now.utc }, uuid: -> { SecureRandom.uuid },
+                   quarantine: -> { Quarantine::EMPTY })
       @max_attachment_bytes = max_attachment_bytes
       @warn = warn
       @clock = clock
       @uuid = uuid
+      @quarantine = quarantine
+      @quarantined = nil
       @results = []
       @pending_attachments = nil
       @staged = 0
@@ -20,6 +24,26 @@ module RaiderReporter
     def start_run
       @started_at = @clock.call
       @results = []
+      @quarantined = load_quarantine
+    end
+
+    # True when the test's key is in the list loaded at run start.
+    def quarantined?(file_path, full_name)
+      return false if @quarantined.nil? || @quarantined.empty?
+
+      @quarantined.include?(RaiderReporter.test_key(file_path, full_name))
+    end
+
+    attr_reader :framework_counts
+
+    # What the framework itself counted (spec §4.4); nil until it reports, and
+    # then no exec summary is written.
+    def record_framework_counts(failures:, errors_outside_tests:)
+      @framework_counts = { failures: Integer(failures), errors_outside_tests: Integer(errors_outside_tests) }
+    end
+
+    def quarantined_count
+      @results.count { |result| result.status == "quarantined" }
     end
 
     def start_test
@@ -81,6 +105,13 @@ module RaiderReporter
 
     private
 
+    def load_quarantine
+      @quarantine.call
+    rescue StandardError => e
+      @warn.call(Quarantine.warning("#{e.class}: #{e.message}"))
+      Quarantine::EMPTY
+    end
+
     def keep?(attachment)
       path = attachment.source_path
       unless File.file?(path)

data/lib/raider_reporter/configuration.rb

@@ -4,21 +4,32 @@ require "raider_reporter/azure_env"
 
 module RaiderReporter
   class Configuration
-    attr_accessor :token, :endpoint, :output_dir, :clean_output_dir, :environment, :max_attachment_bytes
+    TRUTHY = %w[1 true].freeze
+
+    attr_accessor :token, :endpoint, :output_dir, :clean_output_dir, :html_report, :environment, :max_attachment_bytes,
+                  :quarantine, :exec_summary_path
 
     def initialize(env = ENV)
       @token = AzureEnv.value(env, "RAIDER_TOKEN")
       @endpoint = AzureEnv.value(env, "RAIDER_ENDPOINT")
       @output_dir = "raider-results"
       @clean_output_dir = true
+      @html_report = true
       @environment = {}
       @max_attachment_bytes = 5 * 1024 * 1024
+      @quarantine = TRUTHY.include?(AzureEnv.value(env, "RAIDER_QUARANTINE").to_s.downcase)
+      @exec_summary_path = AzureEnv.value(env, "RAIDER_EXEC_SUMMARY")
     end
 
     def upload?
       present?(token) && present?(endpoint)
     end
 
+    # Quarantine needs the server: on, with both token and endpoint (spec §4.1).
+    def quarantine?
+      quarantine == true && upload?
+    end
+
     private
 
     def present?(value)

data/lib/raider_reporter/cucumber/formatter.rb

@@ -84,6 +84,10 @@ module RaiderReporter
         @buffer = {}
         @attempts = Hash.new(0)
         @attempt = nil
+        @outcomes = {}
+        @failed_before = {}
+        @outcomes_broken = false
+        @expected_cases = nil
         guard { setup(config) if self.class.claim(config) }
       end
 
@@ -95,6 +99,8 @@ module RaiderReporter
         @collector = RaiderReporter.start_collector!
         RaiderReporter.dry_run = config.dry_run?
         config.on_event(:test_run_started) { |_event| guard { @collector.start_run } }
+        config.on_event(:test_run_started) { |event| guard { expect_cases(event.test_cases) } }
+        config.on_event(:test_case_finished) { |event| count_outcome(event.test_case, event.result) }
         config.on_event(:test_case_started) { |event| guard { case_started(event.test_case) } }
         config.on_event(:test_step_finished) { |event| guard { step_finished(event.test_step, event.result) } }
         config.on_event(:test_case_finished) { |event| guard { case_finished(event.test_case, event.result) } }
@@ -124,6 +130,7 @@ module RaiderReporter
         # Always buffer: a lookup problem may lose detail, never the scenario.
         failure = status == "failed" ? safely(nil) { failure_for(problem, result, test_case) } : nil
         fields = identity_for(test_case)
+        status = "quarantined" if status == "failed" && quarantined?(fields)
         key = "#{test_case.location}|#{test_case.name}"
         @attempts[key] += 1
         # The newest attempt replaces the older one, taking its attachments with it.
@@ -135,11 +142,42 @@ module RaiderReporter
       def run_finished
         @buffer.each_value { |result| @collector.add_result(result) }
         @buffer.clear
+        safely(nil) { record_framework_counts }
         RaiderReporter.finish!($stderr)
       ensure
         self.class.release(@config)
       end
 
+      def expect_cases(test_cases)
+        @expected_cases = test_cases.map { |test_case| "#{test_case.location}|#{test_case.name}" }.uniq.size
+      end
+
+      # Its own handler, registered before case_finished: if counting ever fails, no
+      # summary is written and exec keeps cucumber's exit code. The newest attempt wins,
+      # except that under strict flaky (--strict, --strict-flaky) a scenario that failed
+      # on an earlier attempt and then passed still fails cucumber's run.
+      def count_outcome(test_case, result)
+        return if @outcomes_broken
+
+        key = "#{test_case.location}|#{test_case.name}"
+        failed = !result.ok?(strict: @config.strict)
+        flaky = !failed && @failed_before[key] && @config.strict.strict?(:flaky)
+        @failed_before[key] ||= failed
+        @outcomes[key] = failed || flaky ? true : false
+      rescue StandardError => e
+        @outcomes_broken = true
+        RaiderReporter.warn("#{e.class}: #{e.message}")
+      end
+
+      # Scenarios that make cucumber's run fail; one error outside tests when the run
+      # stopped before every scenario finished (--fail-fast) (spec §4.4).
+      def record_framework_counts
+        return if @outcomes_broken || @expected_cases.nil?
+
+        aborted = @outcomes.size < @expected_cases ? 1 : 0
+        @collector.record_framework_counts(failures: @outcomes.values.count(true), errors_outside_tests: aborted)
+      end
+
       def identity_for(test_case)
         file_path = relative_path(test_case.location.file)
         line_number = test_case.location.lines.max
@@ -152,6 +190,11 @@ module RaiderReporter
         }
       end
 
+      # A lookup problem must never turn a real failure into a quarantined one.
+      def quarantined?(fields)
+        safely(false) { @collector.quarantined?(fields[:file_path], fields[:full_name]) }
+      end
+
       def full_name(test_case)
         safely(test_case.name) do
           feature = @ast.gherkin_document(test_case.location.file)&.feature

data/lib/raider_reporter/exec.rb

@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+require "tmpdir"
+require "raider_reporter/exec_summary"
+
+module RaiderReporter
+  # `raider-reporter exec -- <command…>` (spec §4.5). Runs the command without a
+  # shell, streams its output, and keeps its exit code, except that exit 1 becomes 0
+  # when the summary says every counted failure was quarantined and nothing else
+  # went wrong. Anything uncertain keeps the command's code.
+  class Exec
+    USAGE = <<~TEXT
+      Usage: raider-reporter exec -- <command> [args...]
+
+      Runs the command (no shell) and exits with its status. When the command exits 1, every failure it
+      counted was quarantined in Raider and nothing else went wrong, exits 0 instead and says so on stderr.
+      Quarantine needs RAIDER_QUARANTINE=1 with RAIDER_TOKEN and RAIDER_ENDPOINT.
+    TEXT
+    USAGE_ERROR = 2
+    NOT_STARTED = 127
+    # A terminal Ctrl-C already reaches the command through the process group.
+    FORWARDED = %w[TERM HUP].freeze
+    IGNORED = %w[INT].freeze
+
+    def self.quarantined_only?(summary)
+      return false unless summary
+
+      failures = summary["framework_failures"]
+      failures.positive? && failures == summary["quarantined"] && summary["errors_outside_tests"].zero?
+    end
+
+    def initialize(command, err: $stderr)
+      @command = command
+      @err = err
+      @pid = nil
+      @code = nil
+      @pending = nil
+    end
+
+    def run
+      if @command.empty?
+        @err.puts(USAGE)
+        return USAGE_ERROR
+      end
+
+      Dir.mktmpdir("raider-exec") { |dir| run_with(File.join(dir, "summary.json")) }
+    rescue StandardError => e
+      @err.puts("[raider_reporter] exec failed: #{e.class}: #{e.message}")
+      @code || 1
+    end
+
+    private
+
+    def run_with(summary_path)
+      previous = install_traps
+      @pid = start(summary_path)
+      return NOT_STARTED unless @pid
+
+      forward(@pending) if @pending
+
+      @code = wait
+      summary = ExecSummary.read(summary_path)
+      return @code unless @code == 1 && self.class.quarantined_only?(summary)
+
+      @err.puts("raider-reporter: #{summary['quarantined']} quarantined failure(s) did not fail the build")
+      0
+    ensure
+      previous&.each { |signal, handler| trap(signal, handler || "DEFAULT") }
+    end
+
+    def start(summary_path)
+      program, *args = @command
+      Process.spawn({ ExecSummary::ENV_NAME => summary_path }, [program, program], *args)
+    rescue SystemCallError => e
+      @err.puts("[raider_reporter] exec: cannot run #{program}: #{e.message}")
+      nil
+    end
+
+    def wait
+      _, status = Process.wait2(@pid)
+      status.signaled? ? 128 + status.termsig : status.exitstatus
+    end
+
+    def forward(signal)
+      Process.kill(signal, @pid)
+    rescue SystemCallError
+      nil
+    end
+
+    # Installed before spawning; a signal that arrives before the child exists is kept in
+    # @pending and forwarded right after spawn. A Ruby handler (not "IGNORE") is reset to
+    # the default in the child by exec(2).
+    def install_traps
+      forwarded = FORWARDED.to_h do |signal|
+        handler = trap(signal) { @pid ? forward(signal) : @pending = signal }
+        [signal, handler]
+      end
+      forwarded.merge(IGNORED.to_h { |signal| [signal, trap(signal) {}] })
+    end
+  end
+end

data/lib/raider_reporter/exec_summary.rb

@@ -0,0 +1,46 @@
+# frozen_string_literal: true
+
+require "json"
+
+module RaiderReporter
+  # The file `raider-reporter exec` reads to decide its exit code (spec §4.4). One
+  # process writes it. A second writer (several processes under one exec) turns it
+  # into a conflict, which exec treats as no summary, so the command's code wins.
+  module ExecSummary
+    ENV_NAME = "RAIDER_EXEC_SUMMARY"
+    KEYS = %w[framework_failures quarantined errors_outside_tests].freeze
+    CONFLICT = { "conflict" => true }.freeze
+
+    module_function
+
+    def write(path, framework_failures:, quarantined:, errors_outside_tests:, pid: Process.pid)
+      summary = { "framework_failures" => framework_failures, "quarantined" => quarantined,
+                  "errors_outside_tests" => errors_outside_tests, "pid" => pid }
+      File.open(path, File::RDWR | File::CREAT, 0o600) do |file|
+        file.flock(File::LOCK_EX)
+        existing = file.read
+        same_writer = existing.empty? || parse(existing)&.fetch("pid", nil) == pid
+        file.rewind
+        file.truncate(0)
+        file.write(JSON.generate(same_writer ? summary : CONFLICT))
+      end
+    end
+
+    def read(path)
+      data = parse(File.read(path))
+      return nil if data.nil? || data.key?("conflict")
+      return nil unless KEYS.all? { |key| data[key].is_a?(Integer) && !data[key].negative? }
+
+      data.slice(*KEYS)
+    rescue SystemCallError, IOError
+      nil
+    end
+
+    def parse(text)
+      data = JSON.parse(text)
+      data.is_a?(Hash) ? data : nil
+    rescue JSON::ParserError
+      nil
+    end
+  end
+end

data/lib/raider_reporter/models.rb

@@ -1,8 +1,8 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  SCHEMA_VERSION = 1
-  STATUSES = %w[passed failed pending skipped].freeze
+  SCHEMA_VERSION = 2
+  STATUSES = %w[passed failed pending skipped quarantined].freeze
 
   Failure = Data.define(:message, :exception_class, :backtrace, :app_frame) do
     def initialize(message:, exception_class:, backtrace:, app_frame: nil)

data/lib/raider_reporter/payload.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  # Serializes a Run into schema v1. Local mode points attachments at copied
+  # Serializes a Run: schema 1, or schema 2 when a result is quarantined. Local mode points attachments at copied
   # files; upload mode points them at multipart part names.
   class Payload
     MODES = %i[local upload].freeze
@@ -17,7 +17,7 @@ module RaiderReporter
 
       entries = attachment_entries.each
       {
-        "schema_version" => SCHEMA_VERSION,
+        "schema_version" => schema_version,
         "client_run_id" => @run.client_run_id,
         "run" => run_hash,
         "results" => @run.results.map do |result|
@@ -34,6 +34,10 @@ module RaiderReporter
 
     def run_file = shard_index ? "run-#{shard_index}.json" : "run.json"
 
+    # Version 2 only adds the quarantined status; older servers keep accepting
+    # every run that has none (spec §4.3).
+    def schema_version = @run.results.any? { |result| result.status == "quarantined" } ? SCHEMA_VERSION : 1
+
     def attachment_entries
       @attachment_entries ||= @run.results.flat_map(&:attachments).each_with_index.map do |attachment, index|
         {

data/lib/raider_reporter/quarantine.rb

@@ -0,0 +1,79 @@
+# frozen_string_literal: true
+
+require "digest"
+require "json"
+require "net/http"
+require "set"
+require "timeout"
+require "uri"
+
+module RaiderReporter
+  NUL = "\u0000"
+  NUL_REPLACEMENT = "\u{FFFD}"
+
+  # The server's test identity: raider_saas TestCase.key_for over the payload's
+  # file_path and full_name, after ingest replaced NUL in each of them.
+  def self.test_key(file_path, full_name)
+    parts = [file_path, full_name].map { |part| part.to_s.tr(NUL, NUL_REPLACEMENT) }
+    Digest::SHA256.hexdigest(parts.join(NUL))
+  end
+
+  # Fetches the project's quarantine list (spec §4.2). Fail-open: any problem
+  # means one warning and an empty set, never an exception.
+  module Quarantine
+    PATH = "/api/v1/quarantine"
+    TIMEOUT = 3
+    EMPTY = Set.new.freeze
+
+    class Unavailable < StandardError; end
+
+    def self.load(config, warn:)
+      return EMPTY unless config.quarantine?
+
+      Set.new(fetch(config)).freeze
+    rescue StandardError => e
+      reason = e.is_a?(Unavailable) ? e.message : "#{e.class}: #{e.message}"
+      warn.call(warning(reason, token: config.token))
+      EMPTY
+    end
+
+    # The one place the fail-open warning text is built (used by load and the Collector).
+    def self.warning(reason, token: nil)
+      "quarantine list unavailable (#{redact(reason, token)}); no tests quarantined"
+    end
+
+    def self.fetch(config)
+      uri = URI.parse("#{config.endpoint.to_s.strip.chomp('/')}#{PATH}")
+      raise Unavailable, "RAIDER_ENDPOINT must start with http:// or https://" unless uri.is_a?(URI::HTTP)
+
+      request = Net::HTTP::Get.new(uri)
+      request["Authorization"] = "Bearer #{config.token}"
+      request["Accept"] = "application/json"
+      request["User-Agent"] = "raider_reporter/#{VERSION}"
+      # The per-step timeouts do not bound a server that trickles bytes; the deadline does.
+      response = Timeout.timeout(TIMEOUT) do
+        Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https",
+                                            open_timeout: TIMEOUT, read_timeout: TIMEOUT, max_retries: 0) { |http| http.request(request) }
+      end
+      raise Unavailable, "HTTP #{response.code}" unless response.code == "200"
+
+      keys(response.body)
+    end
+
+    def self.keys(body)
+      data = JSON.parse(body.to_s)
+      list = data.is_a?(Hash) ? data["quarantined"] : nil
+      raise Unavailable, "unexpected response" unless list.is_a?(Array)
+
+      list.filter_map { |entry| entry["test_key"] if entry.is_a?(Hash) && entry["test_key"].is_a?(String) }
+    rescue JSON::ParserError
+      raise Unavailable, "invalid JSON"
+    end
+
+    def self.redact(text, token)
+      token.to_s.empty? ? text : text.gsub(token.to_s, "[REDACTED]")
+    end
+
+    private_class_method :fetch, :keys, :redact
+  end
+end

data/lib/raider_reporter/rspec/formatter.rb

@@ -11,7 +11,7 @@ module RaiderReporter
     class Formatter
       ::RSpec::Core::Formatters.register self,
                                          :start, :example_started, :example_passed,
-                                         :example_failed, :example_pending, :close
+                                         :example_failed, :example_pending, :dump_summary, :close
 
       NON_TAG_KEYS = %i[pending skip].freeze
       MAX_MESSAGE = Text::MAX_MESSAGE
@@ -45,6 +45,13 @@ module RaiderReporter
         end
       end
 
+      def dump_summary(notification)
+        guard do
+          @collector.record_framework_counts(failures: notification.failure_count,
+                                             errors_outside_tests: errors_outside_tests(notification))
+        end
+      end
+
       def close(_notification)
         guard { RaiderReporter.finish! }
       end
@@ -54,19 +61,32 @@ module RaiderReporter
       def record(example, status)
         metadata = example.metadata
         execution = example.execution_result
+        file_path = metadata[:file_path].to_s.delete_prefix("./")
+        full_name = utf8(example.full_description)
+        failure = status == "failed" ? failure_for(execution.exception) : nil
+        # The key is computed from the payload's own file_path and full_name (spec §4.2).
+        status = "quarantined" if status == "failed" && @collector.quarantined?(file_path, full_name)
         @collector.finish_test(
           identifier: example.id,
-          full_name: utf8(example.full_description),
-          file_path: metadata[:file_path].to_s.delete_prefix("./"),
+          full_name: full_name,
+          file_path: file_path,
           line_number: metadata[:line_number],
           status: status,
           duration_ms: ((execution.run_time || 0) * 1000).round,
           attempts: metadata.fetch(:retry_attempts, 0).to_i + 1,
           tags: metadata.select { |key, value| value == true && !NON_TAG_KEYS.include?(key) }.keys.map(&:to_s).sort,
-          failure: status == "failed" ? failure_for(execution.exception) : nil
+          failure: failure
         )
       end
 
+      # Errors RSpec reported outside examples, plus one when it stopped early
+      # (--fail-fast, an interrupt, a load error): then not every example ran.
+      def errors_outside_tests(notification)
+        world = ::RSpec.world
+        quitting = world.wants_to_quit || (world.respond_to?(:rspec_is_quitting) && world.rspec_is_quitting)
+        notification.errors_outside_of_examples_count + (quitting ? 1 : 0)
+      end
+
       def failure_for(exception)
         return nil unless exception
 

data/lib/raider_reporter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  VERSION = "0.6.0"
+  VERSION = "0.8.0"
 end

data/lib/raider_reporter/writers/html.rb

@@ -0,0 +1,296 @@
+# frozen_string_literal: true
+
+require "erb"
+require "fileutils"
+require "json"
+require "time"
+
+module RaiderReporter
+  module Writers
+    # Builds index.html from every run*.json in a results folder: one page with inline
+    # CSS and JS and no network requests, so a run can be read without Raider or Java.
+    class Html
+      class ForeignFile < StandardError; end
+
+      FILE = "index.html"
+      MARKER = '<meta name="generator" content="raider_reporter">'
+      ROW_LIMIT = 2_000
+      RUN_FILE = /\Arun(-[^\/]+)?\.json\z/
+      SECTIONS = [%w[failed Failed], %w[quarantined Quarantined], %w[passed Passed], %w[pending Pending], %w[skipped Skipped]].freeze
+      PROVIDERS = { "github_actions" => "GitHub Actions", "azure_pipelines" => "Azure Pipelines" }.freeze
+
+      def self.ours?(path)
+        File.file?(path) && File.read(path, 4096).to_s.include?(MARKER)
+      rescue SystemCallError
+        false
+      end
+
+      def write(dir)
+        dir = File.expand_path(dir)
+        files = run_files(dir)
+        return nil if files.empty?
+
+        target = File.join(dir, FILE)
+        if File.exist?(target) && !self.class.ours?(target)
+          raise ForeignFile, "#{target} exists and was not written by raider_reporter; not overwritten"
+        end
+
+        runs, skipped = load(files)
+        runs, ignored = newest_run(runs)
+        tmp = "#{target}.tmp-#{Process.pid}"
+        File.write(tmp, render(runs.map { |r| r[:data] }, skipped, ignored, project_name))
+        File.rename(tmp, target)
+        target
+      ensure
+        FileUtils.rm_f(tmp) if tmp && File.exist?(tmp)
+      end
+
+      private
+
+      def run_files(dir)
+        return [] unless File.directory?(dir)
+
+        Dir.children(dir).grep(RUN_FILE).sort.map { |name| File.join(dir, name) }
+      end
+
+      def load(files)
+        runs = []
+        skipped = []
+        files.each do |file|
+          data = JSON.parse(File.read(file))
+          raise TypeError unless data.is_a?(Hash) && data["results"].is_a?(Array)
+
+          runs << { name: File.basename(file), data: data, mtime: File.mtime(file) }
+        rescue StandardError
+          skipped << File.basename(file)
+        end
+        [runs, skipped]
+      end
+
+      SHARD_WINDOW = 300
+
+      # Old files from earlier runs stay in the folder, so report only the newest run:
+      # its own file, or all the shard files that belong to it. Everything else is named in the footer.
+      def newest_run(runs)
+        return [runs, []] if runs.empty?
+
+        newest = runs.max_by { |r| time(meta_of(r)["finished_at"]) || r[:mtime] }
+        keep = shard_of(newest) ? runs.select { |r| same_run?(r, newest) } : [newest]
+        [keep, (runs - keep).map { |r| r[:name] }.sort]
+      end
+
+      def meta_of(run) = run[:data]["run"].is_a?(Hash) ? run[:data]["run"] : {}
+
+      def shard_of(run) = meta_of(run)["shard"].is_a?(Hash) ? meta_of(run)["shard"] : nil
+
+      def same_run?(run, newest)
+        return true if run.equal?(newest)
+
+        shard = shard_of(run)
+        return false unless shard
+
+        total = shard_of(newest)["total"]
+        return false if total && shard["total"] && shard["total"] != total
+
+        (started_at(run) - started_at(newest)).abs <= SHARD_WINDOW
+      end
+
+      def started_at(run) = time(meta_of(run)["started_at"]) || run[:mtime]
+
+      def project_name
+        name = File.basename(Dir.pwd)
+        name.empty? || name == "/" ? "Raider run" : name
+      end
+
+      def render(runs, skipped, ignored, project)
+        results = runs.flat_map { |run| run["results"] }.select { |r| r.is_a?(Hash) }
+        meta = runs.map { |run| run["run"].is_a?(Hash) ? run["run"] : {} }
+        counts = SECTIONS.to_h { |status, _| [status, results.count { |r| r["status"] == status }] }
+        counts["flaky"] = results.count { |r| r["status"] == "passed" && r["attempts"].to_i > 1 }
+
+        <<~HTML
+          <!doctype html>
+          <html lang="en">
+          <head>
+          <meta charset="utf-8">
+          <meta name="viewport" content="width=device-width, initial-scale=1">
+          #{MARKER}
+          <title>#{h(project)} · Raider report</title>
+          <style>#{CSS}</style>
+          </head>
+          <body>
+          <header>
+          <h1>#{h(project)}</h1>
+          <p class="totals">#{totals(counts)}</p>
+          <p class="meta">#{meta_line(meta)}</p>
+          </header>
+          <main>
+          #{results.empty? ? '<p class="empty">No results</p>' : filter_box}
+          #{SECTIONS.map { |status, label| section(status, label, results) }.join}
+          </main>
+          <footer>#{footer(skipped, ignored)}</footer>
+          <script>#{JS}</script>
+          </body>
+          </html>
+        HTML
+      end
+
+      def totals(counts)
+        passed = counts["passed"]
+        flaky = counts["flaky"]
+        decided = passed + counts["failed"]
+        rate = decided.zero? ? "—" : "#{(passed * 100.0 / decided).round}%"
+        parts = SECTIONS.filter_map do |status, _|
+          next if status == "quarantined" && counts[status].zero?
+
+          %(<span class="b #{status}">#{counts[status]} #{status}</span>)
+        end
+        parts << %(<span class="b flaky">#{flaky} flaky</span>) if flaky.positive?
+        "#{parts.join(' ')} <span class=\"rate\">Pass rate #{rate}</span>"
+      end
+
+      def meta_line(meta)
+        parts = []
+        duration = run_duration(meta)
+        parts << "Duration #{h(duration)}" if duration
+        git = meta.map { |m| m["git"] }.find { |g| g.is_a?(Hash) } || {}
+        parts << "Branch #{h(git['branch'])}" if git["branch"]
+        parts << "Commit #{h(git['commit_sha'].to_s[0, 7])}" if git["commit_sha"]
+        ci = meta.map { |m| m["ci"] }.find { |c| c.is_a?(Hash) } || {}
+        parts << ci_link(ci) if ci["provider"]
+        parts << "Generated #{h(Time.now.utc.strftime('%Y-%m-%d %H:%M UTC'))}"
+        parts.join(" · ")
+      end
+
+      def ci_link(ci)
+        label = h(PROVIDERS.fetch(ci["provider"], ci["provider"]))
+        url = ci["build_url"].to_s
+        url.match?(%r{\Ahttps?://}) ? %(<a href="#{h(url)}">#{label}</a>) : label
+      end
+
+      def run_duration(meta)
+        starts = meta.filter_map { |m| time(m["started_at"]) }
+        ends = meta.filter_map { |m| time(m["finished_at"]) }
+        return nil if starts.empty? || ends.empty? || ends.max < starts.min
+
+        format_ms(((ends.max - starts.min) * 1000).round)
+      end
+
+      def time(value)
+        Time.iso8601(value.to_s)
+      rescue ArgumentError
+        nil
+      end
+
+      def filter_box
+        '<input id="q" type="search" placeholder="Filter by test name or file" aria-label="Filter tests">'
+      end
+
+      def section(status, label, results)
+        rows = results.select { |r| r["status"] == status }
+        return "" if rows.empty?
+
+        shown = rows.first(ROW_LIMIT).map { |r| row(r) }.join
+        more = rows.size > ROW_LIMIT ? %(<p class="more">#{rows.size - ROW_LIMIT} more not shown</p>) : ""
+        open = status == "failed" ? " open" : ""
+        %(<details class="s"#{open}><summary>#{label} (#{rows.size})</summary>#{shown}#{more}</details>\n)
+      end
+
+      def row(result)
+        key = "#{result['full_name']} #{result['file_path']}".downcase
+        location = [result["file_path"], result["line_number"]].compact.join(":")
+        attempts = result["attempts"].to_i > 1 ? %( <span class="att">#{result['attempts'].to_i} attempts</span>) : ""
+        <<~ROW
+          <div class="r #{h(result['status'])}" data-k="#{h(key)}">
+          <p class="n"><span class="b #{h(result['status'])}">#{h(result['status'])}</span> #{h(result['full_name'])}</p>
+          <p class="l">#{h(location)} · #{h(format_ms(result['duration_ms'].to_i))}#{attempts}</p>
+          #{failure(result['failure'])}#{attachments(result['attachments'])}
+          </div>
+        ROW
+      end
+
+      def failure(failure)
+        return "" unless failure.is_a?(Hash)
+
+        trace = Array(failure["backtrace"]).join("\n")
+        backtrace = trace.empty? ? "" : "<details><summary>Backtrace</summary><pre>#{h(trace)}</pre></details>"
+        %(<p class="x">#{h(failure['exception_class'])}</p><pre>#{h(failure['message'])}</pre>#{backtrace})
+      end
+
+      def attachments(list)
+        items = Array(list).filter_map do |attachment|
+          next unless attachment.is_a?(Hash)
+
+          href = safe_href(attachment["path"])
+          next unless href
+
+          if attachment["content_type"].to_s.start_with?("image/")
+            %(<a href="#{href}"><img loading="lazy" src="#{href}" alt="#{h(attachment['name'])}"></a>)
+          else
+            %(<a href="#{href}">#{h(attachment['name'])}</a>)
+          end
+        end
+        items.empty? ? "" : %(<div class="a">#{items.join(' ')}</div>)
+      end
+
+      # Relative paths inside the results folder only, each segment URL-encoded.
+      def safe_href(path)
+        path = path.to_s
+        segments = path.split("/")
+        return nil if path.empty? || path.start_with?("/") || segments.include?("..") || path.include?("\\")
+
+        segments.map { |segment| ERB::Util.url_encode(segment) }.join("/")
+      end
+
+      def footer(skipped, ignored)
+        note = skipped.empty? ? "" : "Skipped unreadable run files: #{h(skipped.join(', '))} · "
+        note += "Ignored older run files: #{h(ignored.join(', '))} · " unless ignored.empty?
+        "#{note}Generated by raider_reporter #{h(RaiderReporter::VERSION)}"
+      end
+
+      def format_ms(ms)
+        return "0ms" if ms.negative?
+        return "#{ms}ms" if ms < 1000
+
+        seconds = (ms / 1000.0).round
+        return format("%.1fs", ms / 1000.0) if ms < 59_950
+
+        "#{seconds / 60}m #{seconds % 60}s"
+      end
+
+      def h(value) = ERB::Util.html_escape(value.to_s)
+
+      CSS = <<~CSS.gsub(/\s*\n\s*/, "")
+        :root{--bg:#fff;--fg:#1f2328;--mute:#656d76;--bdr:#d0d7de;--card:#f6f8fa;--ruby:#C14420;--ok:#1a7f37;--warn:#9a6700;--q:#6B4FA0}
+        @media (prefers-color-scheme:dark){:root{--bg:#0d1117;--fg:#e6edf3;--mute:#8d96a0;--bdr:#30363d;--card:#161b22;--ok:#3fb950;--warn:#d29922;--q:#a78bfa}}
+        *{box-sizing:border-box}
+        body{margin:0;padding:16px;background:var(--bg);color:var(--fg);font:14px/1.5 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif}
+        header,main,footer{max-width:1100px;margin:0 auto}
+        h1{margin:0 0 4px;font-size:20px;color:var(--ruby)}
+        .meta,.l,footer,.more{color:var(--mute);font-size:12px}
+        .b{display:inline-block;padding:0 6px;border-radius:9px;font-size:11px;font-weight:600;border:1px solid var(--bdr)}
+        .b.failed{background:var(--ruby);color:#fff;border-color:var(--ruby)}
+        .b.quarantined{color:var(--q);border-color:var(--q)}
+        .b.passed{color:var(--ok)}.b.pending,.b.skipped,.b.flaky{color:var(--warn)}
+        .rate{font-weight:600;margin-left:6px}
+        #q{width:100%;margin:12px 0;padding:8px;border:1px solid var(--bdr);border-radius:6px;background:var(--card);color:var(--fg)}
+        details.s{margin:8px 0;border:1px solid var(--bdr);border-radius:8px}
+        details.s>summary{padding:8px 12px;font-weight:600;cursor:pointer}
+        .r{padding:8px 12px;border-top:1px solid var(--bdr)}
+        .r p{margin:2px 0;overflow-wrap:anywhere}.n{font-weight:500}.x{color:var(--ruby);font-weight:600}
+        pre{margin:4px 0;padding:8px;background:var(--card);border-radius:6px;overflow-x:auto;white-space:pre-wrap;font-size:12px}
+        .a img{max-width:100%;max-height:240px;border:1px solid var(--bdr);border-radius:4px;margin-top:4px}
+        a{color:var(--ruby)}.empty{padding:24px 0;color:var(--mute)}
+      CSS
+
+      JS = <<~JS.gsub(/\s*\n\s*/, "")
+        (function(){var q=document.getElementById("q");if(!q)return;
+        q.addEventListener("input",function(){var v=q.value.toLowerCase();
+        document.querySelectorAll("details.s").forEach(function(s){var n=0;
+        s.querySelectorAll(".r").forEach(function(r){var hide=v!==""&&r.dataset.k.indexOf(v)<0;r.hidden=hide;if(!hide)n++;});
+        if(v===""){s.hidden=false;if(s.dataset.o!==undefined){s.open=s.dataset.o==="1";delete s.dataset.o;}}
+        else{if(s.dataset.o===undefined)s.dataset.o=s.open?"1":"0";s.hidden=n===0;s.open=n>0;}});});})();
+      JS
+    end
+  end
+end

data/lib/raider_reporter/writers/local.rb

@@ -2,6 +2,7 @@
 
 require "fileutils"
 require "json"
+require_relative "html"
 
 module RaiderReporter
   module Writers
@@ -28,6 +29,8 @@ module RaiderReporter
       def clean(dir, payload)
         FileUtils.rm_f(File.join(dir, payload.run_file))
         FileUtils.rm_rf(File.join(dir, payload.attachments_dir))
+        index = File.join(dir, Html::FILE)
+        FileUtils.rm_f(index) if Html.ours?(index)
       end
     end
   end

data/lib/raider_reporter.rb

@@ -8,9 +8,12 @@ require_relative "raider_reporter/models"
 require_relative "raider_reporter/app_frame"
 require_relative "raider_reporter/payload"
 require_relative "raider_reporter/configuration"
+require_relative "raider_reporter/quarantine"
+require_relative "raider_reporter/exec_summary"
 require_relative "raider_reporter/content_types"
 require_relative "raider_reporter/collector"
 require_relative "raider_reporter/writers/local"
+require_relative "raider_reporter/writers/html"
 require_relative "raider_reporter/ci_detector"
 require_relative "raider_reporter/shard_detector"
 require_relative "raider_reporter/writers/uploader"
@@ -39,6 +42,7 @@ module RaiderReporter
       @configuration = nil
       @collector = nil
       @project_root = nil
+      @quarantined_keys = nil
     end
 
     def start_collector!
@@ -46,10 +50,16 @@ module RaiderReporter
       @project_root = Dir.pwd
       @collector = Collector.new(
         max_attachment_bytes: configuration.max_attachment_bytes,
-        warn: ->(message) { warn(message) }
+        warn: ->(message) { warn(message) },
+        quarantine: -> { quarantined_keys }
       )
     end
 
+    # Fetched once per process, at the first run start (spec §4.2).
+    def quarantined_keys
+      @quarantined_keys ||= Quarantine.load(configuration, warn: ->(message) { warn(message) })
+    end
+
     def attach(name, path, type: nil)
       unless collector&.in_test?
         warn("attach(#{name.inspect}) called outside a running example; ignored")
@@ -81,6 +91,8 @@ module RaiderReporter
     def finish!(output = $stderr)
       return nil if dry_run?
 
+      write_exec_summary
+
       detector = CiDetector.new
       ci = detector.ci
       shard = ShardDetector.new(job: ci&.fetch("job", nil)).shard
@@ -91,6 +103,7 @@ module RaiderReporter
       rescue StandardError => e
         warn("could not write #{configuration.output_dir}: #{e.class}: #{e.message}")
       end
+      write_html_report
       return nil unless configuration.upload?
 
       result = Writers::Uploader.new(configuration).upload(payload)
@@ -106,6 +119,27 @@ module RaiderReporter
 
     private
 
+    # First, so a later reporting problem cannot lose it. A failure here only means
+    # no summary, and exec then keeps the command's exit code.
+    def write_exec_summary
+      path = configuration.exec_summary_path
+      counts = collector&.framework_counts
+      return if path.nil? || counts.nil?
+
+      ExecSummary.write(path, framework_failures: counts[:failures], quarantined: collector.quarantined_count,
+                              errors_outside_tests: counts[:errors_outside_tests])
+    rescue StandardError => e
+      warn("could not write the exec summary: #{e.class}: #{e.message}")
+    end
+
+    def write_html_report
+      return unless configuration.html_report
+
+      Writers::Html.new.write(configuration.output_dir)
+    rescue StandardError => e
+      warn("HTML report failed: #{e.class}: #{e.message}")
+    end
+
     def success_line(result)
       link = result.build_url || result.url
       return "Raider report: #{link}" if link

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: raider_reporter
 version: !ruby/object:Gem::Version
-  version: 0.6.0
+  version: 0.8.0
 platform: ruby
 authors:
 - Augustin Gottlieb
@@ -36,14 +36,18 @@ files:
 - lib/raider_reporter/content_types.rb
 - lib/raider_reporter/cucumber.rb
 - lib/raider_reporter/cucumber/formatter.rb
+- lib/raider_reporter/exec.rb
+- lib/raider_reporter/exec_summary.rb
 - lib/raider_reporter/github_event.rb
 - lib/raider_reporter/models.rb
 - lib/raider_reporter/payload.rb
+- lib/raider_reporter/quarantine.rb
 - lib/raider_reporter/rspec.rb
 - lib/raider_reporter/rspec/formatter.rb
 - lib/raider_reporter/shard_detector.rb
 - lib/raider_reporter/text.rb
 - lib/raider_reporter/version.rb
+- lib/raider_reporter/writers/html.rb
 - lib/raider_reporter/writers/local.rb
 - lib/raider_reporter/writers/uploader.rb
 homepage: https://github.com/RaiderHQ/raider_reporter
@@ -67,7 +71,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.6
+rubygems_version: 4.0.21
 specification_version: 4
 summary: RSpec and Cucumber formatters that write Raider reports locally and upload
   them to Raider