raider_reporter 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7c1a8baed9855a5cba10100e57b9052184f6f748b7f015d630541d52905031c6
-  data.tar.gz: cc40652b897de2d3832a3d5da58d8bdb6f8e3a6873b11da927b68fd5a0478e24
+  metadata.gz: c4d38649c7323d93145079d6de1aac3a00f4724a0d906a2f004ddf331e201b77
+  data.tar.gz: 8f5ad9717d74c2c626ba0ab7507e2ebc413f9f80337d0ecfd02343c69bc52051
 SHA512:
-  metadata.gz: e16586146bc9d569b5cd827c5533e6aae6cdef3bec2bc4ae3d6390c230a193ad56be6cc8fd01d11e90e933522a7f2f2ae54613450acb5a95daa609364634af1e
-  data.tar.gz: 134a1d1de73ff5b93c00740996ffea7cc8cde10f7be727bbb2ab8320b3158b710ac2df413d3e68ce334bff3fc7b3d40cd5cf2150fcc4cb6cd21d9b0e79521554
+  metadata.gz: d43a79f1ff81eff80f4fa8a9afbff388df98671e65920013b88cdb8cdd24ab3313266646e27e72bbebfbad4213231855c6d7470c28177ba601d0db1ddb1aea76
+  data.tar.gz: 61cd6025ab3d9c713fb075d5288eec3ce46c219eb317995129a6bc41fa139ecccf4026eca160cafeeeae0f25f8d4da290c9e661429cb1f21e8ad8520da007c94

data/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.3.0 — 2026-10-11
+
+- Report pull request identity: `run.ci.repository`, `run.git.default_branch` and `run.git.pr` (number, base/head branch, head SHA) on `pull_request` and `pull_request_target` events. Requires a Raider server that accepts `git.pr`
+- Add `raider-reporter comment`: posts (and updates) one results comment on the pull request and sets the `raider/tests` commit status; fail-open, always exits 0, skips fork PRs
+- Print the `Raider report: <url>` line to stderr instead of stdout, so stdout-parsing tools (such as `--format json`) stay clean
+- Report the PR head commit's SHA and message on pull request events instead of the merge commit's
+- Harden the GitHub client: only a bot-authored comment starting with the marker is updated, any number of trailing slashes in `GITHUB_API_URL` are accepted, and the token is redacted from error messages
+
 ## 0.2.0 — 2026-10-11
 
 - Report the CI build key, attempt and job so runs of one build are grouped (GitHub Actions)

data/README.md

@@ -84,7 +84,7 @@ If both a token and an endpoint are set, the run (plus attachments) is uploaded
 RAIDER_TOKEN=... RAIDER_ENDPOINT=https://your-raider-server bundle exec rspec
 ```
 
-On success the formatter prints `Raider report: <url>`, using the build URL when the server returns one and the run URL otherwise (or `Raider report uploaded (run <id>)` if the response has no URL). If either value is missing, nothing is uploaded and the local files are still written.
+On success the formatter prints `Raider report: <url>` to stderr (so tools that parse RSpec's stdout, such as `--format json`, stay clean), using the build URL when the server returns one and the run URL otherwise (or `Raider report uploaded (run <id>)` if the response has no URL). If either value is missing, nothing is uploaded and the local files are still written.
 
 An upload carries at most 45 MB of attachments and at most 1000 attachment files. Attachments past that budget are left out of the upload (in order, with one warning) but stay in the local `raider-results/`.
 
@@ -92,6 +92,36 @@ Reporting is fail-open: no error inside the gem, network failure, timeout or non
 
 The hosted Raider service is not public yet; uploads go to a Raider server whose address you provide in `RAIDER_ENDPOINT`.
 
+## Pull request comments
+
+On pull requests, `raider-reporter comment` posts one comment with the build's results (and sets a `raider/tests` commit status on the PR head commit). Run it in a job that runs after your test jobs, so every shard has uploaded. Use the workflow's `GITHUB_TOKEN` (or a GitHub App token): a personal access token posts as a user, so the step adds a new comment on every run instead of updating the existing one.
+
+```yaml
+  raider-comment:
+    needs: [test]   # every test job id
+    if: always() && github.event_name == 'pull_request'
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+      statuses: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ruby/setup-ruby@v1
+        with: { ruby-version: "3.4", bundler-cache: true }
+      - name: Comment Raider results on the pull request
+        run: bundle exec raider-reporter comment
+        env:
+          RAIDER_TOKEN: ${{ secrets.RAIDER_TOKEN }}
+          RAIDER_ENDPOINT: ${{ vars.RAIDER_ENDPOINT }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+```
+
+- The comment is rendered by the Raider server (`GET /api/v1/builds/summary`); the gem only fetches the Markdown and posts it. The same comment is updated on every run instead of adding a new one.
+- Pull requests from forks are skipped: GitHub gives their workflows a read-only token, so the command prints a notice and exits 0.
+- The command never fails your CI: it always exits 0 and prints notices to stderr, prefixed `[raider_reporter]`. It needs `GITHUB_TOKEN`, `GITHUB_REPOSITORY`, `GITHUB_EVENT_PATH`, `RAIDER_TOKEN` and `RAIDER_ENDPOINT`; if one is missing, or the event is not a pull request, it does nothing.
+- It requires a Raider server that supports the summary endpoint and the `git.pr` field. Upgrade the server before using it.
+
 ## Parallel runs
 
 When a suite is split across several processes or CI jobs, each one uploads its own run. Raider groups runs of the same CI build, so the shards show up together as one build. Each run records its shard (`index`, `total` and a `label`) and, on GitHub Actions, a build key made from the repository, run id and run attempt. A re-run attempt of a workflow is a separate build.

data/exe/raider-reporter

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+$LOAD_PATH.unshift(File.expand_path("../lib", __dir__))
+require "raider_reporter"
+require "raider_reporter/cli"
+
+exit RaiderReporter::CLI.new(ARGV).run

data/lib/raider_reporter/ci_detector.rb

@@ -1,12 +1,16 @@
 # frozen_string_literal: true
 
 require "open3"
+require "raider_reporter/github_event"
 
 module RaiderReporter
   class CiDetector
-    def initialize(env: ENV, git: method(:git_command))
+    PR_EVENTS = %w[pull_request pull_request_target].freeze
+
+    def initialize(env: ENV, git: method(:git_command), event_reader: nil)
       @env = env
       @git = git
+      @event_reader = event_reader || -> { GithubEvent.load(@env) }
     end
 
     def ci
@@ -20,17 +24,69 @@ module RaiderReporter
       repo = @env["GITHUB_REPOSITORY"]
       build_key = ("github_actions:#{repo}:#{run_id}:#{attempt}" if repo && run_id)
       { "provider" => "github_actions", "build_url" => build_url, "build_id" => run_id,
-        "attempt" => attempt, "job" => @env["GITHUB_JOB"], "build_key" => build_key }.compact
+        "attempt" => attempt, "job" => @env["GITHUB_JOB"], "build_key" => build_key,
+        "repository" => (repo unless repo.to_s.empty?) }.compact
     end
 
     def git
-      info = { "branch" => branch, "commit_sha" => sha, "commit_message" => @git.call("log", "-1", "--pretty=%s") }
+      pr = pull_request
+      info = { "branch" => branch, "commit_sha" => pr ? pr["head_sha"] : sha, "commit_message" => commit_message(pr),
+               "default_branch" => default_branch, "pr" => pr }
       info = info.compact
       info.empty? ? nil : info
     end
 
     private
 
+    def event
+      return @event if defined?(@event)
+
+      loaded = begin
+        @event_reader.call
+      rescue StandardError
+        nil
+      end
+      @event = loaded.is_a?(Hash) ? loaded : nil
+    end
+
+    def default_branch
+      value = event&.dig("repository", "default_branch")
+      value.is_a?(String) && !value.empty? ? value : nil
+    rescue StandardError
+      nil
+    end
+
+    def pull_request
+      return nil unless event && PR_EVENTS.include?(@env["GITHUB_EVENT_NAME"])
+
+      pr = event["pull_request"]
+      return nil unless pr.is_a?(Hash)
+
+      number = pr["number"] || event["number"]
+      base = pr.dig("base", "ref")
+      head = pr.dig("head", "ref")
+      sha = pr.dig("head", "sha")
+      return nil unless number.is_a?(Integer) && number >= 1
+      return nil unless [base, head, sha].all? { |v| v.is_a?(String) && !v.empty? }
+
+      { "number" => number, "base_branch" => base, "head_branch" => head, "head_sha" => sha }
+    rescue StandardError
+      nil
+    end
+
+    # On a PR, GitHub checks out a synthetic merge commit; its message is not the author's.
+    def commit_message(pr)
+      if pr && merge_commit?
+        message = @git.call("log", "-1", "--format=%s", "HEAD^2")
+        return message unless message.to_s.empty?
+      end
+      @git.call("log", "-1", "--pretty=%s")
+    end
+
+    def merge_commit?
+      @git.call("rev-list", "--parents", "-n", "1", "HEAD").to_s.split.size == 3
+    end
+
     def branch
       head_ref = @env["GITHUB_HEAD_REF"].to_s
       return head_ref unless head_ref.empty?

data/lib/raider_reporter/cli.rb

@@ -0,0 +1,109 @@
+# frozen_string_literal: true
+
+require "raider_reporter"
+
+module RaiderReporter
+  # `raider-reporter comment`: posts the build summary as a PR comment and sets a commit status.
+  # Fail-open: #run always returns 0, notices go to stderr, nothing goes to stdout.
+  class CLI
+    MARKER = "<!-- raider-reporter:build-comment -->"
+    CONTEXT = "raider/tests"
+    REQUIRED_ENV = %w[GITHUB_TOKEN GITHUB_REPOSITORY GITHUB_EVENT_PATH RAIDER_TOKEN RAIDER_ENDPOINT].freeze
+    STATUS_STATES = { "passed" => "success", "failed" => "failure", "incomplete" => "pending" }.freeze
+    FALLBACK_DESCRIPTIONS = {
+      "passed" => "Raider: tests passed",
+      "failed" => "Raider: tests failed",
+      "incomplete" => "Raider: build incomplete"
+    }.freeze
+    DEFAULT_API_URL = "https://api.github.com"
+    USAGE = <<~TEXT
+      Usage: raider-reporter comment
+
+      Posts (or updates) the Raider build summary as a comment on the current pull request
+      and sets the "#{CONTEXT}" commit status. Intended for GitHub Actions; always exits 0.
+
+      Environment: #{REQUIRED_ENV.join(', ')} (GITHUB_API_URL optional).
+    TEXT
+
+    def initialize(argv, env: ENV, out: $stdout, err: $stderr)
+      @argv = argv
+      @env = env
+      @out = out
+      @err = err
+    end
+
+    def run
+      command = @argv.first
+      if command == "comment" && !@argv.include?("--help") && !@argv.include?("-h")
+        comment
+      else
+        @err.puts(USAGE)
+      end
+      0
+    rescue StandardError => e
+      notice("comment failed: #{e.class}: #{e.message}")
+      0
+    end
+
+    private
+
+    def comment
+      missing = REQUIRED_ENV.select { |name| @env[name].to_s.strip.empty? }
+      return notice("skipped: missing #{missing.join(', ')}") unless missing.empty?
+
+      event = GithubEvent.load(@env)
+      detector = CiDetector.new(env: @env, event_reader: -> { event })
+      pr = detector.git&.dig("pr")
+      return notice("skipped: not a pull request event") unless pr
+
+      build_key = detector.ci&.dig("build_key")
+      return notice("skipped: could not determine the build key") unless build_key
+
+      publish(pr, build_key)
+    rescue Comment::GithubClient::Forbidden
+      notice("skipped: token cannot write to this PR")
+    rescue StandardError => e
+      notice("comment failed: #{e.class}: #{e.message}")
+    end
+
+    def publish(pr, build_key)
+      summaries = Comment::SummaryClient.new(endpoint: @env["RAIDER_ENDPOINT"], token: @env["RAIDER_TOKEN"])
+      summary = summaries.fetch(build_key)
+      return notice("summary unavailable: #{summaries.last_error}") unless summary
+
+      github = Comment::GithubClient.new(token: @env["GITHUB_TOKEN"], repository: @env["GITHUB_REPOSITORY"],
+                                         api_url: api_url)
+      html_url = github.upsert_comment(pr["number"], summary.markdown, MARKER)
+      github.set_status(pr["head_sha"], state: STATUS_STATES.fetch(summary.state, "pending"),
+                                        target_url: summary.url, description: description(summary),
+                                        context: CONTEXT)
+      notice("PR comment updated: #{html_url}")
+    end
+
+    def api_url
+      url = @env["GITHUB_API_URL"].to_s.strip
+      url.empty? ? DEFAULT_API_URL : url
+    end
+
+    # "**Raider: failed** - 2 failed, 148 passed, 3 skipped (1m 12s)" -> "2 failed, 148 passed, 3 skipped".
+    # Falls back to a fixed text per state when the Markdown carries no headline counts.
+    def description(summary)
+      counts = headline_counts(summary.markdown)
+      text = counts || FALLBACK_DESCRIPTIONS.fetch(summary.state, FALLBACK_DESCRIPTIONS["incomplete"])
+      text[0, Comment::GithubClient::DESCRIPTION_LIMIT]
+    end
+
+    def headline_counts(markdown)
+      line = markdown.to_s.lines.find { |l| l.include?("Raider:") }
+      return nil unless line
+
+      counts = line.scan(/\b\d+ (?:failed|passed|skipped|pending)\b/)
+      counts.empty? ? nil : counts.join(", ")
+    end
+
+    def notice(message)
+      @err.puts("[raider_reporter] #{message}")
+      nil
+    end
+  end
+end

data/lib/raider_reporter/comment/github_client.rb

@@ -0,0 +1,112 @@
+# frozen_string_literal: true
+
+require "net/http"
+require "uri"
+require "json"
+
+module RaiderReporter
+  module Comment
+    # Minimal GitHub REST client: upsert one marked PR comment and set a commit status.
+    class GithubClient
+      class Error < StandardError; end
+      # 403/404: the token cannot write here (e.g. a fork's read-only token).
+      class Forbidden < Error; end
+
+      MAX_PAGES = 10
+      PER_PAGE = 100
+      DESCRIPTION_LIMIT = 140
+
+      def initialize(token:, repository:, api_url: "https://api.github.com", timeout: 30)
+        @token = token
+        @repository = repository
+        @api_url = api_url.to_s.strip.sub(%r{/+\z}, "")
+        @timeout = timeout
+      end
+
+      def upsert_comment(pr_number, body, marker)
+        existing = find_comment(pr_number, marker)
+        response = if existing
+                     request(:patch, "/repos/#{@repository}/issues/comments/#{existing['id']}", body: body)
+                   else
+                     request(:post, "/repos/#{@repository}/issues/#{pr_number}/comments", body: body)
+                   end
+        html_url = response.is_a?(Hash) ? response["html_url"] : nil
+        raise Error, "GitHub response had no html_url" if html_url.to_s.empty?
+
+        html_url
+      end
+
+      def set_status(sha, state:, target_url:, description:, context:)
+        return nil if sha.to_s.strip.empty?
+
+        payload = { state: state, target_url: target_url, context: context,
+                    description: description.to_s[0, DESCRIPTION_LIMIT] }
+        request(:post, "/repos/#{@repository}/statuses/#{sha}", payload: payload)
+        nil
+      end
+
+      private
+
+      def find_comment(pr_number, marker)
+        (1..MAX_PAGES).each do |page|
+          comments = request(:get, "/repos/#{@repository}/issues/#{pr_number}/comments?per_page=#{PER_PAGE}&page=#{page}")
+          raise Error, "unexpected comments response" unless comments.is_a?(Array)
+
+          found = comments.find { |c| ours?(c, marker) }
+          return found if found
+          return nil if comments.size < PER_PAGE
+        end
+        nil
+      end
+
+      # Only our own (bot-authored) marked comment is ever edited.
+      def ours?(comment, marker)
+        comment.is_a?(Hash) && comment["body"].to_s.start_with?(marker) &&
+          comment.dig("user", "type") == "Bot"
+      end
+
+      def redact(text)
+        @token.to_s.empty? ? text.to_s : text.to_s.gsub(@token.to_s, "[REDACTED]")
+      end
+
+      def request(verb, path, body: nil, payload: nil)
+        uri = URI.parse(@api_url + path)
+        raise Error, "GITHUB_API_URL must start with http:// or https://" unless uri.is_a?(URI::HTTP)
+
+        req = build_request(verb, uri, payload || ({ body: body } if body))
+        response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https",
+                                                       open_timeout: @timeout, read_timeout: @timeout,
+                                                       write_timeout: @timeout) { |http| http.request(req) }
+        handle(response)
+      rescue Error
+        raise
+      rescue StandardError => e
+        raise Error, redact("#{e.class}: #{e.message}")
+      end
+
+      def build_request(verb, uri, json)
+        klass = { get: Net::HTTP::Get, post: Net::HTTP::Post, patch: Net::HTTP::Patch }.fetch(verb)
+        req = klass.new(uri)
+        req["Authorization"] = "Bearer #{@token}"
+        req["Accept"] = "application/vnd.github+json"
+        req["X-GitHub-Api-Version"] = "2022-11-28"
+        req["User-Agent"] = "raider_reporter/#{VERSION}"
+        if json
+          req["Content-Type"] = "application/json"
+          req.body = JSON.generate(json)
+        end
+        req
+      end
+
+      def handle(response)
+        code = response.code.to_i
+        raise Forbidden, "HTTP #{code}" if [403, 404].include?(code)
+        raise Error, redact("HTTP #{code}: #{response.body.to_s[0, 200]}") unless (200..299).cover?(code)
+
+        JSON.parse(response.body.to_s)
+      rescue JSON::ParserError
+        raise Error, "GitHub returned invalid JSON"
+      end
+    end
+  end
+end

data/lib/raider_reporter/comment/summary_client.rb

@@ -0,0 +1,74 @@
+# frozen_string_literal: true
+
+require "net/http"
+require "uri"
+require "json"
+require "cgi/escape"
+
+module RaiderReporter
+  module Comment
+    # Fetches the rendered build summary from the Raider API. Never raises:
+    # failures return nil and set #last_error to a printable reason.
+    class SummaryClient
+      Summary = Data.define(:state, :markdown, :url)
+
+      attr_reader :last_error
+
+      def initialize(endpoint:, token:, timeout: 30)
+        @endpoint = endpoint
+        @token = token
+        @timeout = timeout
+        @last_error = nil
+      end
+
+      def fetch(build_key)
+        @last_error = nil
+        uri = summary_uri(build_key)
+        request = Net::HTTP::Get.new(uri)
+        request["Authorization"] = "Bearer #{@token}"
+        request["Accept"] = "application/json"
+        request["User-Agent"] = "raider_reporter/#{VERSION}"
+        response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https",
+                                                       open_timeout: @timeout, read_timeout: @timeout,
+                                                       write_timeout: @timeout) { |http| http.request(request) }
+        parse(response)
+      rescue StandardError => e
+        fail_with("#{e.class}: #{e.message}")
+      end
+
+      private
+
+      def summary_uri(build_key)
+        uri = URI.parse("#{@endpoint.to_s.strip.chomp('/')}/api/v1/builds/summary?build_key=#{CGI.escape(build_key.to_s)}")
+        raise ArgumentError, "endpoint must start with http:// or https://" unless uri.is_a?(URI::HTTP)
+
+        uri
+      end
+
+      def parse(response)
+        case response.code.to_i
+        when 200..299 then build_summary(response.body)
+        when 401 then fail_with("HTTP 401: unauthorized (check RAIDER_TOKEN)")
+        when 404 then fail_with("HTTP 404: build not found for this project")
+        else fail_with("HTTP #{response.code}: #{response.body.to_s[0, 200]}")
+        end
+      end
+
+      def build_summary(text)
+        data = JSON.parse(text.to_s)
+        unless data.is_a?(Hash) && %w[state markdown url].all? { |k| data[k].is_a?(String) }
+          return fail_with("invalid summary response")
+        end
+
+        Summary.new(state: data["state"], markdown: data["markdown"], url: data["url"])
+      rescue JSON::ParserError
+        fail_with("invalid summary response (not JSON)")
+      end
+
+      def fail_with(reason)
+        @last_error = @token.to_s.empty? ? reason : reason.gsub(@token.to_s, "[REDACTED]")
+        nil
+      end
+    end
+  end
+end

data/lib/raider_reporter/github_event.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require "json"
+
+module RaiderReporter
+  # Reads the webhook payload GitHub Actions leaves at GITHUB_EVENT_PATH.
+  module GithubEvent
+    # Returns the parsed event Hash, or nil when missing, unreadable or invalid.
+    def self.load(env = ENV)
+      path = env["GITHUB_EVENT_PATH"].to_s
+      return nil if path.empty?
+
+      data = JSON.parse(File.read(path))
+      data.is_a?(Hash) ? data : nil
+    rescue StandardError
+      nil
+    end
+  end
+end

data/lib/raider_reporter/rspec/formatter.rb

@@ -45,7 +45,7 @@ module RaiderReporter
       end
 
       def close(_notification)
-        guard { RaiderReporter.finish!(@output) }
+        guard { RaiderReporter.finish! }
       end
 
       private

data/lib/raider_reporter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/raider_reporter.rb

@@ -12,6 +12,8 @@ require_relative "raider_reporter/writers/local"
 require_relative "raider_reporter/ci_detector"
 require_relative "raider_reporter/shard_detector"
 require_relative "raider_reporter/writers/uploader"
+require_relative "raider_reporter/comment/summary_client"
+require_relative "raider_reporter/comment/github_client"
 
 module RaiderReporter
   class << self
@@ -66,7 +68,7 @@ module RaiderReporter
       $stderr.puts("[raider_reporter] #{message}")
     end
 
-    def finish!(output = $stdout)
+    def finish!(output = $stderr)
       return nil if dry_run?
 
       detector = CiDetector.new

metadata

@@ -1,30 +1,36 @@
 --- !ruby/object:Gem::Specification
 name: raider_reporter
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Augustin Gottlieb
-bindir: bin
+bindir: exe
 cert_chain: []
-date: 1980-01-02 00:00:00.000000000 Z
+date: 2026-10-11 00:00:00.000000000 Z
 dependencies: []
 description: RSpec formatter that records results, screenshots and videos in the Raider
   schema, writes them locally, and optionally uploads them to a Raider server.
 email:
 - 33221555+aguspe@users.noreply.github.com
-executables: []
+executables:
+- raider-reporter
 extensions: []
 extra_rdoc_files: []
 files:
 - CHANGELOG.md
 - LICENSE
 - README.md
+- exe/raider-reporter
 - lib/raider_reporter.rb
 - lib/raider_reporter/ci_detector.rb
+- lib/raider_reporter/cli.rb
 - lib/raider_reporter/collector.rb
+- lib/raider_reporter/comment/github_client.rb
+- lib/raider_reporter/comment/summary_client.rb
 - lib/raider_reporter/configuration.rb
 - lib/raider_reporter/content_types.rb
+- lib/raider_reporter/github_event.rb
 - lib/raider_reporter/models.rb
 - lib/raider_reporter/payload.rb
 - lib/raider_reporter/rspec.rb
@@ -54,7 +60,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 4.0.6
+rubygems_version: 3.6.2
 specification_version: 4
 summary: 'Test reporting for Ruby Raider: local run.json and Raider uploads'
 test_files: []