raider_reporter 0.5.0 → 0.6.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: bcf65fcd172398a1040a5b15c3bff03150bde91ea1c80ddb9cb1a2ca731f1496
-  data.tar.gz: bfdaf38608619f2831d108b0a170a0da7aa8aa7eb65a1cf8d0b42703e44c58ad
+  metadata.gz: e5bb1cd462fbac0bd6a44635be2b0e14f6da29667e6cb4cf0a5f358aed7581be
+  data.tar.gz: c814a3009d7e7e6ac0857bc34636ad1fdc913d3e3489ff64c3e5b62cd7f50dbb
 SHA512:
-  metadata.gz: c635eb065857b147b25758e304e9a2efdf9e5885bfc1767beac062b3b9ea1dd6e344ca3463265f70f78ebb2ae3aca3f1388f8ed6f27bf425b8692e9eb846ed34
-  data.tar.gz: 8851d8bf5aa69eda7d96f42f6b8a72c893ba306fd82906b66dda19ab21dba0d8c122de1be2df94106454db4e27b1fbef06e6d8a8d4a659509763c2b4b53bee72
+  metadata.gz: a1064490905e372f3392d056343cc3b8304d1de127e785d122e5ef6e3c50622873583f7ceedece6f87bbf633d8fbe4e2b94a0111f3d325ff98ba6280ba84e75c
+  data.tar.gz: a91849038df0a28f54023e411d4115636ce97e1aa6a4ad83ec03a298f7f87e438e0b94583a6248e568c8c14af9c9176e813ddb0b287962e5ebd2021737a8748d

data/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.6.0 — 2026-10-14
+
+- Schema v1, additive: optional `failure.app_frame`, the first line of the raw exception backtrace that lies inside the project root (`Dir.pwd` when the reporter starts), outside `vendor/`, `.bundle/`, `Gem.path` and Ruby's library directory, and not `<internal:...>`. The path is made project-relative and the rest of the line is unchanged, e.g. `spec/pages/login_page.rb:42:in 'LoginPage#submit'`; at most 500 characters. The key is omitted when no line qualifies or computing it fails, so payloads without it are unchanged. Raider uses it to group failures by error
+- Reported by both the RSpec and Cucumber reporters
+- No breaking changes
+
+## 0.5.1 — 2026-10-13
+
+- Fix: a failed upload now reports one short line. An HTML error page (for example a server 500) is summarised as `the server returned an HTML error page` instead of being printed, other bodies are collapsed to one line of at most 200 characters, and the token is redacted from upload errors, including exception messages
+- Docs: the Azure Pipelines snippet publishes `raider-results-$(System.JobPositionInPhase)-$(System.JobAttempt)`, so re-running failed jobs or `strategy: parallel` no longer collides on the artifact name
+
 ## 0.5.0 — 2026-10-12
 
 - Add Azure Pipelines support: detect the build (`provider: azure_pipelines`, build URL, attempt, job, branch, commit), report Azure Repos pull requests in `run.git.pr`, and derive a stable id-based build key. `strategy: parallel` jobs are detected as shards of one build

data/README.md

@@ -214,7 +214,7 @@ jobs:
           RAIDER_ENDPOINT: $(RAIDER_ENDPOINT)
       - task: PublishPipelineArtifact@1
         condition: always()
-        inputs: { targetPath: raider-results, artifact: raider-results }
+        inputs: { targetPath: raider-results, artifact: raider-results-$(System.JobPositionInPhase)-$(System.JobAttempt) }
   - job: raider_comment
     dependsOn: tests
     condition: and(always(), eq(variables['Build.Reason'], 'PullRequest'))
@@ -234,6 +234,7 @@ jobs:
 - **Map secrets explicitly:** Azure does not expose secret variables to scripts automatically. Map `RAIDER_TOKEN` and `System.AccessToken` with `env:` as shown. If a mapped variable is undefined, Azure passes the literal `$(...)`; the gem treats it as missing: results are still written locally but not uploaded, and `raider-reporter comment` prints a skip notice.
 - **Pull request builds:** for Azure Repos, YAML `pr:` triggers are ignored. PR builds (and therefore the comment job) run only when the pipeline is added as a **Build validation** branch policy on the target branch.
 - **Same stage:** keep the comment job in the same stage as the test jobs; the build key includes `System.StageAttempt`.
+- **Artifact names:** pipeline artifacts cannot be overwritten within a run, so the name includes the job position and attempt. A fixed name would fail when you re-run failed jobs or use `strategy: parallel`.
 - **Azure Repos only:** the PR comment works for repositories hosted in Azure Repos. A GitHub-hosted repository built by Azure Pipelines still uploads results, but `comment` prints `skipped: not an Azure Repos pull request`.
 - **Parallel jobs:** `strategy: parallel: N` shards are detected automatically and grouped into one build, like GitHub matrix jobs.
 - The command never fails your pipeline: it always exits 0 and prints notices to stderr, prefixed `[raider_reporter]`. It needs `SYSTEM_ACCESSTOKEN`, `RAIDER_TOKEN` and `RAIDER_ENDPOINT`, and requires a Raider server that supports the summary endpoint.
@@ -304,7 +305,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
 
 ## Schema
 
-`run.json` follows schema v1, documented in [docs/schema-v1.md](docs/schema-v1.md). Statuses are exactly `passed`, `failed`, `pending` and `skipped`.
+`run.json` follows schema v1, documented in [docs/schema-v1.md](docs/schema-v1.md). Statuses are exactly `passed`, `failed`, `pending` and `skipped`. In both RSpec and Cucumber, a failure may carry `app_frame` (when a project frame is found): the first backtrace line inside your project, for example `spec/pages/login_page.rb:42:in 'LoginPage#submit'`, which Raider uses to group failures with the same error.
 
 ## Contributing
 

data/lib/raider_reporter/app_frame.rb

@@ -0,0 +1,57 @@
+# frozen_string_literal: true
+
+require "rbconfig"
+
+module RaiderReporter
+  # Finds the first project frame of a raw backtrace, so the server can group
+  # failures by where the project's own code raised them.
+  module AppFrame
+    MAX_LENGTH = 500
+
+    # Returns "relative/path.rb:42:in 'Class#method'" or nil when no line
+    # of the backtrace lies in project code.
+    def self.from(backtrace, root:)
+      return nil if backtrace.nil? || backtrace.empty? || root.nil?
+
+      base = "#{File.expand_path(root.to_s).chomp("/")}/"
+      excluded = excluded_prefixes
+      Array(backtrace).each do |line|
+        line = line.to_s
+        path = line[/\A(.+?):\d+/, 1]
+        next if path.nil? || path.start_with?("<internal:", "(", "-")
+
+        absolute = File.expand_path(path, base)
+        next unless File.file?(absolute)
+
+        absolute = inside_root(absolute, base)
+        next unless absolute
+        next if excluded.any? { |prefix| absolute.start_with?(prefix) }
+
+        relative = absolute.delete_prefix(base)
+        next if relative.start_with?("vendor/", ".bundle/") || relative.include?("/.bundle/")
+
+        return Text.utf8("#{relative}#{line[path.length..]}")[0, MAX_LENGTH]
+      end
+      nil
+    end
+
+    # Returns the path (rewritten under base when only the real paths match,
+    # e.g. a symlinked tmpdir) or nil when it lies outside the root.
+    def self.inside_root(absolute, base)
+      return absolute if absolute.start_with?(base)
+
+      real_base = "#{File.realpath(base)}/"
+      real = File.realpath(absolute)
+      real.start_with?(real_base) ? "#{base}#{real.delete_prefix(real_base)}" : nil
+    rescue SystemCallError
+      nil
+    end
+    private_class_method :inside_root
+
+    def self.excluded_prefixes
+      dirs = Array(Gem.path) + [RbConfig::CONFIG["rubylibdir"]]
+      dirs.compact.map(&:to_s).reject(&:empty?).map { |dir| "#{File.expand_path(dir).chomp("/")}/" }
+    end
+    private_class_method :excluded_prefixes
+  end
+end

data/lib/raider_reporter/cucumber/formatter.rb

@@ -199,10 +199,17 @@ module RaiderReporter
         Failure.new(
           message: Text.utf8("#{step_label(step_info, test_case)}\n\n#{exception.message}")[0, Text::MAX_MESSAGE],
           exception_class: Text.utf8(exception.class.name),
-          backtrace: (exception.backtrace || []).first(Text::MAX_BACKTRACE).map { |line| Text.utf8(line) }
+          backtrace: (exception.backtrace || []).first(Text::MAX_BACKTRACE).map { |line| Text.utf8(line) },
+          app_frame: app_frame_for(exception)
         )
       end
 
+      def app_frame_for(exception)
+        AppFrame.from(exception.backtrace, root: RaiderReporter.project_root || Dir.pwd)
+      rescue StandardError
+        nil
+      end
+
       def step_label(step_info, test_case)
         return "Step" unless step_info
 

data/lib/raider_reporter/models.rb

@@ -4,7 +4,11 @@ module RaiderReporter
   SCHEMA_VERSION = 1
   STATUSES = %w[passed failed pending skipped].freeze
 
-  Failure = Data.define(:message, :exception_class, :backtrace)
+  Failure = Data.define(:message, :exception_class, :backtrace, :app_frame) do
+    def initialize(message:, exception_class:, backtrace:, app_frame: nil)
+      super
+    end
+  end
   Attachment = Data.define(:name, :source_path, :content_type)
   TestResult = Data.define(
     :identifier, :full_name, :file_path, :line_number, :status,

data/lib/raider_reporter/payload.rb

@@ -73,11 +73,13 @@ module RaiderReporter
     end
 
     def failure_hash(failure)
-      {
+      hash = {
         "message" => failure.message,
         "exception_class" => failure.exception_class,
         "backtrace" => failure.backtrace
       }
+      hash["app_frame"] = failure.app_frame if failure.app_frame
+      hash
     end
 
     def attachment_hash(entry, mode)

data/lib/raider_reporter/rspec/formatter.rb

@@ -74,10 +74,26 @@ module RaiderReporter
         Failure.new(
           message: utf8(exception.message)[0, MAX_MESSAGE],
           exception_class: utf8(exception.class.name),
-          backtrace: backtrace.first(MAX_BACKTRACE).map { |line| utf8(line) }
+          backtrace: backtrace.first(MAX_BACKTRACE).map { |line| utf8(line) },
+          app_frame: app_frame_for(exception)
         )
       end
 
+      # From the raw backtrace: RSpec's formatter strips gem and project-support frames.
+      # MultipleExceptionError (aggregate_failures) has no backtrace of its own.
+      def raw_backtrace(exception)
+        return exception.backtrace if exception.backtrace
+        return nil unless exception.respond_to?(:all_exceptions)
+
+        exception.all_exceptions.first&.backtrace
+      end
+
+      def app_frame_for(exception)
+        AppFrame.from(raw_backtrace(exception), root: RaiderReporter.project_root || Dir.pwd)
+      rescue StandardError
+        nil
+      end
+
       def utf8(value)
         Text.utf8(value)
       end

data/lib/raider_reporter/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module RaiderReporter
-  VERSION = "0.5.0"
+  VERSION = "0.6.0"
 end

data/lib/raider_reporter/writers/uploader.rb

@@ -12,6 +12,7 @@ module RaiderReporter
       # Keeps uploads under the server's 50 MB request limit and multipart part limit.
       MAX_UPLOAD_ATTACHMENT_BYTES = 45 * 1024 * 1024
       MAX_UPLOAD_ATTACHMENTS = 1000
+      ERROR_DETAIL_LIMIT = 200
 
       def initialize(config, timeout: 30, warn: RaiderReporter.method(:warn))
         @config = config
@@ -42,7 +43,7 @@ module RaiderReporter
         end
         to_result(response)
       rescue StandardError => e
-        Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: "#{e.class}: #{e.message}")
+        Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: redact("#{e.class}: #{e.message}"))
       end
 
       private
@@ -97,10 +98,31 @@ module RaiderReporter
           Result.new(ok: true, url: present(body["url"]), run_id: present(body["run_id"]),
                      build_url: present(build["url"]), error: nil)
         else
-          Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: "HTTP #{response.code}: #{response.body.to_s[0, 200]}")
+          Result.new(ok: false, url: nil, run_id: nil, build_url: nil, error: error_message(response))
         end
       end
 
+      # One short line: an HTML error page (e.g. a Rails 500) is summarised, anything
+      # else is collapsed to one line, redacted, then cut to ERROR_DETAIL_LIMIT.
+      def error_message(response)
+        text = response.body.to_s.dup.force_encoding(Encoding::UTF_8).scrub
+        detail = if html?(response, text)
+                   "the server returned an HTML error page"
+                 else
+                   redact(text).gsub(/\s+/, " ").strip[0, ERROR_DETAIL_LIMIT]
+                 end
+        detail.empty? ? "HTTP #{response.code}" : "HTTP #{response.code}: #{detail}"
+      end
+
+      def html?(response, text)
+        response["Content-Type"].to_s.include?("html") || text.lstrip.start_with?("<")
+      end
+
+      def redact(text)
+        token = @config.token.to_s
+        token.empty? ? text.to_s : text.to_s.gsub(token, "[REDACTED]")
+      end
+
       def parse_body(text)
         parsed = JSON.parse(text.to_s)
         parsed.is_a?(Hash) ? parsed : {}

data/lib/raider_reporter.rb

@@ -5,6 +5,7 @@ require "rbconfig"
 require_relative "raider_reporter/version"
 require_relative "raider_reporter/text"
 require_relative "raider_reporter/models"
+require_relative "raider_reporter/app_frame"
 require_relative "raider_reporter/payload"
 require_relative "raider_reporter/configuration"
 require_relative "raider_reporter/content_types"
@@ -19,7 +20,7 @@ require_relative "raider_reporter/comment/azure_client"
 
 module RaiderReporter
   class << self
-    attr_reader :collector
+    attr_reader :collector, :project_root
 
     def configuration
       @configuration ||= Configuration.new
@@ -37,10 +38,12 @@ module RaiderReporter
       @collector&.cleanup
       @configuration = nil
       @collector = nil
+      @project_root = nil
     end
 
     def start_collector!
       @dry_run = false
+      @project_root = Dir.pwd
       @collector = Collector.new(
         max_attachment_bytes: configuration.max_attachment_bytes,
         warn: ->(message) { warn(message) }

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: raider_reporter
 version: !ruby/object:Gem::Version
-  version: 0.5.0
+  version: 0.6.0
 platform: ruby
 authors:
 - Augustin Gottlieb
@@ -24,6 +24,7 @@ files:
 - README.md
 - exe/raider-reporter
 - lib/raider_reporter.rb
+- lib/raider_reporter/app_frame.rb
 - lib/raider_reporter/azure_env.rb
 - lib/raider_reporter/ci_detector.rb
 - lib/raider_reporter/cli.rb