raider_reporter 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b1e069eeb6569206c085d042b6b3e26f0e35bbb2459b60dccfecf6d744aaef86
-  data.tar.gz: f117594302d75a2d97edfba870beb04e69d477acbb6f80ab276689e55d3c531a
+  metadata.gz: 7c1a8baed9855a5cba10100e57b9052184f6f748b7f015d630541d52905031c6
+  data.tar.gz: cc40652b897de2d3832a3d5da58d8bdb6f8e3a6873b11da927b68fd5a0478e24
 SHA512:
-  metadata.gz: ad5162cad62d6efed699f663f977540ef84f1dc640845d112c8f594bac34767aec0224a341e55e4ebb354b5a59b57bc13c5b4deacd071b8ade6bbade90b85366
-  data.tar.gz: 6b68c2e3ab889e30a2ff1e9c6cb075f7e3e8556d92557e60f36360daf232e9f9ed72d8041592f8766c8496bc5d303700222e2b96f1df685b8813ece4627c1375
+  metadata.gz: e16586146bc9d569b5cd827c5533e6aae6cdef3bec2bc4ae3d6390c230a193ad56be6cc8fd01d11e90e933522a7f2f2ae54613450acb5a95daa609364634af1e
+  data.tar.gz: 134a1d1de73ff5b93c00740996ffea7cc8cde10f7be727bbb2ab8320b3158b710ac2df413d3e68ce334bff3fc7b3d40cd5cf2150fcc4cb6cd21d9b0e79521554

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.2.0 — 2026-10-11
+
+- Report the CI build key, attempt and job so runs of one build are grouped (GitHub Actions)
+- Detect shards from `RAIDER_SHARD_INDEX`/`RAIDER_SHARD_TOTAL` or `parallel_tests` (`TEST_ENV_NUMBER`, `PARALLEL_TEST_GROUPS`) and report them per run; a matrix leg running parallel_tests is composed into unique shards
+- Write parallel-safe local output (`run-<n>.json`, `attachments-<n>/`) when sharded; fixes parallel_tests processes overwriting each other's results
+- Print the Raider build URL after upload, falling back to the run URL
+
+## 0.1.1 — 2026-10-11
+
+- Send run timestamps with millisecond precision, so durations under a second are no longer reported as 0
+
 ## 0.1.0 — 2026-10-11
 
 - RSpec formatter writing Raider schema v1 to `raider-results/run.json`

data/README.md

@@ -63,7 +63,7 @@ end
 | `token` | `ENV["RAIDER_TOKEN"]` | API token used for uploads. |
 | `endpoint` | `ENV["RAIDER_ENDPOINT"]` | Base URL of the Raider server to upload to. |
 | `output_dir` | `"raider-results"` | Directory for local output. |
-| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory inside `output_dir` before writing. Nothing else in `output_dir` is touched. |
+| `clean_output_dir` | `true` | Remove the previous `run.json` and `attachments/` directory (`run-<n>.json` and `attachments-<n>/` when sharded) inside `output_dir` before writing. Nothing else in `output_dir` is touched. |
 | `environment` | `{}` | Free-form key/values stored with the run (browser, suite name, and so on). Values are JSON values: strings, booleans or numbers. |
 
 ## Local output
@@ -74,7 +74,7 @@ raider-results/
   attachments/      # copies of attached screenshots and videos
 ```
 
-Add `raider-results/` to your `.gitignore`.
+Add `raider-results/` to your `.gitignore`. When a run is sharded, the files are named per shard; see [Parallel runs](#parallel-runs).
 
 ## Uploading to Raider
 
@@ -84,7 +84,7 @@ If both a token and an endpoint are set, the run (plus attachments) is uploaded
 RAIDER_TOKEN=... RAIDER_ENDPOINT=https://your-raider-server bundle exec rspec
 ```
 
-On success the formatter prints `Raider report: <url>` (or `Raider report uploaded (run <id>)` if the server's response has no URL). If either value is missing, nothing is uploaded and the local files are still written.
+On success the formatter prints `Raider report: <url>`, using the build URL when the server returns one and the run URL otherwise (or `Raider report uploaded (run <id>)` if the response has no URL). If either value is missing, nothing is uploaded and the local files are still written.
 
 An upload carries at most 45 MB of attachments and at most 1000 attachment files. Attachments past that budget are left out of the upload (in order, with one warning) but stay in the local `raider-results/`.
 
@@ -92,6 +92,64 @@ Reporting is fail-open: no error inside the gem, network failure, timeout or non
 
 The hosted Raider service is not public yet; uploads go to a Raider server whose address you provide in `RAIDER_ENDPOINT`.
 
+## Parallel runs
+
+When a suite is split across several processes or CI jobs, each one uploads its own run. Raider groups runs of the same CI build, so the shards show up together as one build. Each run records its shard (`index`, `total` and a `label`) and, on GitHub Actions, a build key made from the repository, run id and run attempt. A re-run attempt of a workflow is a separate build.
+
+Shard detection, in order of precedence:
+
+1. `RAIDER_SHARD_INDEX` (1 or more) and, optionally, `RAIDER_SHARD_TOTAL`.
+2. `parallel_tests` / `parallel_rspec`: `TEST_ENV_NUMBER` (empty means shard 1) and `PARALLEL_TEST_GROUPS` for the total.
+3. Otherwise the run is not sharded.
+
+A GitHub Actions matrix needs only the two variables:
+
+```yaml
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        shard: [1, 2, 3, 4]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ruby/setup-ruby@v1
+        with:
+          bundler-cache: true
+      - run: bundle exec rspec  # split the spec files by shard however you like
+        env:
+          RAIDER_TOKEN: ${{ secrets.RAIDER_TOKEN }}
+          RAIDER_ENDPOINT: ${{ vars.RAIDER_ENDPOINT }}
+          RAIDER_SHARD_INDEX: ${{ matrix.shard }}
+          RAIDER_SHARD_TOTAL: 4
+```
+
+`parallel_rspec` works with no configuration, because it sets `TEST_ENV_NUMBER` and `PARALLEL_TEST_GROUPS` for each process.
+
+When a run is sharded, local output is written per shard so that processes sharing one `raider-results/` do not overwrite each other. Cleaning only removes the current shard's own files:
+
+```
+raider-results/
+  run-1.json
+  attachments-1/
+  run-2.json
+  attachments-2/
+```
+
+Unsharded runs still write `run.json` and `attachments/`. Tools that read local results should load `run*.json`.
+
+### Matrix and parallel_tests together
+
+A GitHub matrix leg can also run `parallel_rspec -n 2`. When `RAIDER_SHARD_INDEX` and `TEST_ENV_NUMBER` are both present and `PARALLEL_TEST_GROUPS` is a positive integer `g`, the two are composed so every process gets a unique shard:
+
+- `index = (RAIDER_SHARD_INDEX - 1) * g + parallel_index`
+- `total = RAIDER_SHARD_TOTAL * g`
+- `label = "<job>#<RAIDER_SHARD_INDEX>.<parallel_index>"`
+
+For example, matrix leg 2 of 2 with `parallel_rspec -n 2` yields shards 3 and 4 of 4, labelled `#2.1` and `#2.2`. `RAIDER_SHARD_TOTAL` must match the matrix size. Without it, `total` is null. If `PARALLEL_TEST_GROUPS` is missing or invalid, only `RAIDER_SHARD_INDEX` is used. Every matrix leg must use the same parallel_tests process count (`-n`) and set `RAIDER_SHARD_TOTAL`; mixing different `-n` values, or composed and plain legs, can repeat shard numbers.
+
+Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans its own files, so files from an earlier run with more processes stay behind.
+
 ## Examples
 
 - `examples/rspec_basic` is a runnable RSpec project with a passing, a failing (with an attachment) and a skipped example. Run it with `cd examples/rspec_basic && bundle install && bundle exec rspec`; the failure is deliberate.

data/lib/raider_reporter/ci_detector.rb

@@ -16,7 +16,11 @@ module RaiderReporter
       build_url = if @env["GITHUB_SERVER_URL"] && @env["GITHUB_REPOSITORY"] && run_id
                     "#{@env['GITHUB_SERVER_URL']}/#{@env['GITHUB_REPOSITORY']}/actions/runs/#{run_id}"
                   end
-      { "provider" => "github_actions", "build_url" => build_url, "build_id" => run_id }.compact
+      attempt = @env["GITHUB_RUN_ATTEMPT"].to_s.empty? ? "1" : @env["GITHUB_RUN_ATTEMPT"]
+      repo = @env["GITHUB_REPOSITORY"]
+      build_key = ("github_actions:#{repo}:#{run_id}:#{attempt}" if repo && run_id)
+      { "provider" => "github_actions", "build_url" => build_url, "build_id" => run_id,
+        "attempt" => attempt, "job" => @env["GITHUB_JOB"], "build_key" => build_key }.compact
     end
 
     def git

data/lib/raider_reporter/collector.rb

@@ -61,10 +61,10 @@ module RaiderReporter
       result
     end
 
-    def finish_run(ci:, git:, environment:)
+    def finish_run(ci:, git:, environment:, shard: nil)
       Run.new(
         client_run_id: @uuid.call, started_at: @started_at, finished_at: @clock.call,
-        ci: ci, git: git, environment: environment,
+        ci: ci, git: git, environment: environment, shard: shard,
         results: @results.map { |r| r.with(attachments: r.attachments.select { |a| keep?(a) }) }
       )
     end

data/lib/raider_reporter/models.rb

@@ -10,5 +10,5 @@ module RaiderReporter
     :identifier, :full_name, :file_path, :line_number, :status,
     :duration_ms, :attempts, :tags, :failure, :attachments
   )
-  Run = Data.define(:client_run_id, :started_at, :finished_at, :ci, :git, :environment, :results)
+  Run = Data.define(:client_run_id, :started_at, :finished_at, :ci, :git, :environment, :results, :shard)
 end

data/lib/raider_reporter/payload.rb

@@ -28,12 +28,18 @@ module RaiderReporter
       }
     end
 
+    def shard_index = @run.shard&.fetch("index")
+
+    def attachments_dir = shard_index ? "attachments-#{shard_index}" : "attachments"
+
+    def run_file = shard_index ? "run-#{shard_index}.json" : "run.json"
+
     def attachment_entries
       @attachment_entries ||= @run.results.flat_map(&:attachments).each_with_index.map do |attachment, index|
         {
           attachment: attachment,
           part: "att_#{index}",
-          path: "attachments/#{index}-#{File.basename(attachment.source_path)}"
+          path: "#{attachments_dir}/#{index}-#{File.basename(attachment.source_path)}"
         }
       end
     end
@@ -42,11 +48,12 @@ module RaiderReporter
 
     def run_hash
       {
-        "started_at" => @run.started_at&.utc&.iso8601,
-        "finished_at" => @run.finished_at&.utc&.iso8601,
+        "started_at" => @run.started_at&.utc&.iso8601(3),
+        "finished_at" => @run.finished_at&.utc&.iso8601(3),
         "ci" => @run.ci,
         "git" => @run.git,
-        "environment" => @run.environment
+        "environment" => @run.environment,
+        "shard" => @run.shard
       }.compact
     end
 

data/lib/raider_reporter/shard_detector.rb

@@ -0,0 +1,49 @@
+# frozen_string_literal: true
+
+module RaiderReporter
+  # Identifies which slice of a parallel suite this process is.
+  class ShardDetector
+    def initialize(env: ENV, job: nil)
+      @env = env
+      @job = job.to_s.strip.empty? ? nil : job.to_s
+    end
+
+    def shard
+      index, total, suffix = composed || explicit || parallel_tests
+      return nil unless index
+
+      { "index" => index, "total" => total, "label" => "#{@job || "local"}##{suffix || index}" }
+    end
+
+    private
+
+    # A matrix leg that also runs parallel_tests: every process gets a unique index.
+    def composed
+      explicit_index, explicit_total = explicit
+      parallel_index, = parallel_tests
+      groups = positive(@env["PARALLEL_TEST_GROUPS"])
+      return nil unless explicit_index && parallel_index && groups
+
+      [(explicit_index - 1) * groups + parallel_index,
+       explicit_total && explicit_total * groups,
+       "#{explicit_index}.#{parallel_index}"]
+    end
+
+    def explicit
+      index = positive(@env["RAIDER_SHARD_INDEX"])
+      [index, positive(@env["RAIDER_SHARD_TOTAL"])] if index
+    end
+
+    def parallel_tests
+      return nil unless @env.key?("TEST_ENV_NUMBER")
+
+      raw = @env["TEST_ENV_NUMBER"].to_s
+      index = raw.empty? ? 1 : positive(raw)
+      [index, positive(@env["PARALLEL_TEST_GROUPS"])] if index
+    end
+
+    def positive(value)
+      Integer(value.to_s, exception: false)&.then { |n| n.positive? ? n : nil }
+    end
+  end
+end

data/lib/raider_reporter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  VERSION = "0.1.0"
+  VERSION = "0.2.0"
 end

data/lib/raider_reporter/writers/local.rb

@@ -12,12 +12,12 @@ module RaiderReporter
 
       def write(payload)
         dir = File.expand_path(@config.output_dir)
-        clean(dir) if @config.clean_output_dir
-        FileUtils.mkdir_p(File.join(dir, "attachments"))
+        clean(dir, payload) if @config.clean_output_dir
+        FileUtils.mkdir_p(File.join(dir, payload.attachments_dir))
         payload.attachment_entries.each do |entry|
           FileUtils.cp(entry[:attachment].source_path, File.join(dir, entry[:path]))
         end
-        run_json = File.join(dir, "run.json")
+        run_json = File.join(dir, payload.run_file)
         File.write(run_json, JSON.pretty_generate(payload.to_h(mode: :local)))
         run_json
       end
@@ -25,9 +25,9 @@ module RaiderReporter
       private
 
       # Remove only what this writer creates, so output_dir "." or "spec" is safe.
-      def clean(dir)
-        FileUtils.rm_f(File.join(dir, "run.json"))
-        FileUtils.rm_rf(File.join(dir, "attachments"))
+      def clean(dir, payload)
+        FileUtils.rm_f(File.join(dir, payload.run_file))
+        FileUtils.rm_rf(File.join(dir, payload.attachments_dir))
       end
     end
   end

data/lib/raider_reporter/writers/uploader.rb

@@ -8,7 +8,7 @@ require "securerandom"
 module RaiderReporter
   module Writers
     class Uploader
-      Result = Data.define(:ok, :url, :run_id, :error)
+      Result = Data.define(:ok, :url, :run_id, :build_url, :error)
       # Keeps uploads under the server's 50 MB request limit and multipart part limit.
       MAX_UPLOAD_ATTACHMENT_BYTES = 45 * 1024 * 1024
       MAX_UPLOAD_ATTACHMENTS = 1000
@@ -42,7 +42,7 @@ module RaiderReporter
         end
         to_result(response)
       rescue StandardError => e
-        Result.new(ok: false, url: nil, run_id: nil, error: "#{e.class}: #{e.message}")
+        Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: "#{e.class}: #{e.message}")
       end
 
       private
@@ -93,9 +93,11 @@ module RaiderReporter
       def to_result(response)
         if response.is_a?(Net::HTTPSuccess)
           body = parse_body(response.body)
-          Result.new(ok: true, url: present(body["url"]), run_id: present(body["run_id"]), error: nil)
+          build = body["build"].is_a?(Hash) ? body["build"] : {}
+          Result.new(ok: true, url: present(body["url"]), run_id: present(body["run_id"]),
+                     build_url: present(build["url"]), error: nil)
         else
-          Result.new(ok: false, url: nil, run_id: nil, error: "HTTP #{response.code}: #{response.body.to_s[0, 200]}")
+          Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: "HTTP #{response.code}: #{response.body.to_s[0, 200]}")
         end
       end
 

data/lib/raider_reporter.rb

@@ -10,6 +10,7 @@ require_relative "raider_reporter/content_types"
 require_relative "raider_reporter/collector"
 require_relative "raider_reporter/writers/local"
 require_relative "raider_reporter/ci_detector"
+require_relative "raider_reporter/shard_detector"
 require_relative "raider_reporter/writers/uploader"
 
 module RaiderReporter
@@ -69,7 +70,9 @@ module RaiderReporter
       return nil if dry_run?
 
       detector = CiDetector.new
-      run = collector.finish_run(ci: detector.ci, git: detector.git, environment: default_environment)
+      ci = detector.ci
+      shard = ShardDetector.new(job: ci&.fetch("job", nil)).shard
+      run = collector.finish_run(ci: ci, git: detector.git, environment: default_environment, shard: shard)
       payload = Payload.new(run)
       begin
         Writers::Local.new(configuration).write(payload)
@@ -92,7 +95,8 @@ module RaiderReporter
     private
 
     def success_line(result)
-      return "Raider report: #{result.url}" if result.url
+      link = result.build_url || result.url
+      return "Raider report: #{link}" if link
       return "Raider report uploaded (run #{result.run_id})" if result.run_id
 
       "Raider report uploaded"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: raider_reporter
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - Augustin Gottlieb
@@ -29,6 +29,7 @@ files:
 - lib/raider_reporter/payload.rb
 - lib/raider_reporter/rspec.rb
 - lib/raider_reporter/rspec/formatter.rb
+- lib/raider_reporter/shard_detector.rb
 - lib/raider_reporter/version.rb
 - lib/raider_reporter/writers/local.rb
 - lib/raider_reporter/writers/uploader.rb