raider_reporter 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c4d38649c7323d93145079d6de1aac3a00f4724a0d906a2f004ddf331e201b77
4
- data.tar.gz: 8f5ad9717d74c2c626ba0ab7507e2ebc413f9f80337d0ecfd02343c69bc52051
3
+ metadata.gz: bcf65fcd172398a1040a5b15c3bff03150bde91ea1c80ddb9cb1a2ca731f1496
4
+ data.tar.gz: bfdaf38608619f2831d108b0a170a0da7aa8aa7eb65a1cf8d0b42703e44c58ad
5
5
  SHA512:
6
- metadata.gz: d43a79f1ff81eff80f4fa8a9afbff388df98671e65920013b88cdb8cdd24ab3313266646e27e72bbebfbad4213231855c6d7470c28177ba601d0db1ddb1aea76
7
- data.tar.gz: 61cd6025ab3d9c713fb075d5288eec3ce46c219eb317995129a6bc41fa139ecccf4026eca160cafeeeae0f25f8d4da290c9e661429cb1f21e8ad8520da007c94
6
+ metadata.gz: c635eb065857b147b25758e304e9a2efdf9e5885bfc1767beac062b3b9ea1dd6e344ca3463265f70f78ebb2ae3aca3f1388f8ed6f27bf425b8692e9eb846ed34
7
+ data.tar.gz: 8851d8bf5aa69eda7d96f42f6b8a72c893ba306fd82906b66dda19ab21dba0d8c122de1be2df94106454db4e27b1fbef06e6d8a8d4a659509763c2b4b53bee72
data/CHANGELOG.md CHANGED
@@ -1,5 +1,26 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.5.0 — 2026-10-12
4
+
5
+ - Add Azure Pipelines support: detect the build (`provider: azure_pipelines`, build URL, attempt, job, branch, commit), report Azure Repos pull requests in `run.git.pr`, and derive a stable id-based build key. `strategy: parallel` jobs are detected as shards of one build
6
+ - `raider-reporter comment` now works on Azure Pipelines: it creates or updates one tagged comment thread on the pull request (closed when the build passed) and sets a `raider/tests` PR status. Uses `System.AccessToken` with the PR statuses API version 7.1; the build service needs "Contribute to pull requests" on the repository. GitHub-hosted repositories built by Azure Pipelines are skipped with a notice
7
+ - Schema v1, additive: optional `run.git.pr.url` (set by GitHub Actions and Azure Pipelines)
8
+ - Unexpanded Azure macros such as `$(RAIDER_TOKEN)` or `$(System.AccessToken)` are treated as missing instead of being sent as a token
9
+ - Fix: redact the token before truncating error bodies in the GitHub and Azure clients, so a token straddling the 200-character cut can no longer leak partially
10
+ - Notices: the missing-permission notice uses `<project>` when the project name is unknown, and the success notice prints `PR #<n>` when the PR URL is unknown
11
+ - `raider-reporter comment` now runs only on GitHub Actions (`GITHUB_ACTIONS=true`) or Azure Pipelines; elsewhere it prints `skipped: unsupported CI` and exits 0
12
+ - No breaking changes
13
+
14
+ ## 0.4.0 — 2026-10-11
15
+
16
+ - Add a Cucumber reporter (Cucumber >= 9, < 12; not a gem dependency). `require "raider_reporter/cucumber"` from `features/support/` registers it in plugin mode (recommended, works with any `--format` and no `--out`); `--format RaiderReporter::Cucumber::Formatter --out <file>` also works. Using both registers one listener
17
+ - Identity: `file_path` is project-relative (`features/login.feature`), `full_name` is `"<Feature>: <Scenario>"` (outline rows add `(<Examples name>, row <n>)`; when several Examples tables share a label, each is numbered: `(Examples 1, row 1)`, `(Examples 2, row 1)`), `identifier` is `<file>:<line>`
18
+ - Report the failing step (or Before/After hook) with its location in the failure message; ambiguous maps to failed, undefined to pending
19
+ - Retries: only the final attempt is kept, with `attempts` set and earlier attachments dropped
20
+ - Attachments: `RaiderReporter.attach` and Cucumber's `attach(path, media_type)` with an existing file path are captured; inline data and `log` messages are ignored
21
+ - Dry runs (`cucumber --dry-run`) write and upload nothing; `parallel_tests` shards write `run-<n>.json`
22
+ - No breaking changes: RSpec behaviour is unchanged
23
+
3
24
  ## 0.3.0 — 2026-10-11
4
25
 
5
26
  - Report pull request identity: `run.ci.repository`, `run.git.default_branch` and `run.git.pr` (number, base/head branch, head SHA) on `pull_request` and `pull_request_target` events. Requires a Raider server that accepts `git.pr`
data/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # raider_reporter
2
2
 
3
- Raider-schema test reporting for RSpec. `raider_reporter` is an RSpec formatter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
3
+ Raider-schema test reporting for RSpec and Cucumber. `raider_reporter` is an RSpec formatter and a Cucumber reporter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
4
4
 
5
5
  It has no runtime dependencies and requires Ruby 3.2 or newer.
6
6
 
@@ -26,7 +26,80 @@ Enable the formatter in `.rspec`:
26
26
  --format RaiderReporter::RSpec::Formatter
27
27
  ```
28
28
 
29
- Run `bundle exec rspec` as usual. Results land in `raider-results/run.json`. A dry run (`rspec --dry-run`) writes and uploads nothing.
29
+ Run `bundle exec rspec` as usual. (For Cucumber, see [Cucumber](#cucumber).) Results land in `raider-results/run.json`. A dry run (`rspec --dry-run`) writes and uploads nothing.
30
+
31
+ ## Cucumber
32
+
33
+ Requirements: Cucumber 9, 10 or 11 in your own Gemfile (`gem "cucumber"`). It is not a dependency of this gem; requiring `raider_reporter/cucumber` without Cucumber raises a `LoadError` that says so. Cucumber 9.2.1 on Ruby 4 also needs `gem "logger"` in your Gemfile, because Cucumber requires it without declaring it.
34
+
35
+ ### Plugin mode (recommended)
36
+
37
+ Require the reporter from a file under `features/support/`, for example `features/support/raider.rb`:
38
+
39
+ ```ruby
40
+ require "raider_reporter/cucumber"
41
+ ```
42
+
43
+ Then run Cucumber with any normal formatter and no `--out`:
44
+
45
+ ```
46
+ # cucumber.yml
47
+ default: --format pretty
48
+ ```
49
+
50
+ The require registers one listener on Cucumber's event bus, so it sits next to `pretty`, `progress` or any other formatter. Results land in `raider-results/run.json`, and a dry run (`cucumber --dry-run`) writes and uploads nothing. Plugin mode registers once per Ruby process, because `require` is cached: a second in-process `Cucumber::Cli::Main` run in the same process does not register again.
51
+
52
+ ### Formatter mode
53
+
54
+ You can also name the formatter explicitly:
55
+
56
+ ```
57
+ cucumber --format pretty --format RaiderReporter::Cucumber::Formatter --out raider-results/cucumber.log
58
+ ```
59
+
60
+ `--out` is required here. Cucumber rejects a second formatter on the same output stream ("All but one formatter must use --out", also in 9.2.1), and this formatter writes nothing to its stream, so give it any file path (it is never created). If both modes are active, only one listener is registered.
61
+
62
+ ### How tests are identified
63
+
64
+ - `file_path` is relative to the project root (`features/login.feature`), even when Cucumber is given `./features/login.feature` or an absolute path.
65
+ - `full_name` is `"<Feature name>: <Scenario name>"`. Scenario Outline rows are `"<Feature name>: <Scenario name> (<Examples name or 'Examples'>, row <n>)"`, where `n` is the 1-based data row of that Examples table. When two or more Examples tables of one outline share a label (for example two unnamed ones), every table with that label is numbered by its position among them, `(Examples 1, row 1)`, `(Examples 2, row 1)`, so each row keeps a distinct name; tables with a unique label are unchanged.
66
+ - `identifier` is `"<file_path>:<line>"`: the scenario line, or the Examples row line for outline rows.
67
+ - `tags` are the feature, scenario and Examples tags, merged, sorted, unique, without `@`.
68
+
69
+ Statuses: passed, failed and ambiguous (reported as failed), pending and undefined (reported as pending), skipped.
70
+
71
+ ### Failures
72
+
73
+ A failed test reports the step that failed, not the whole scenario:
74
+
75
+ ```
76
+ When I fail on purpose (features/login.feature:9)
77
+
78
+ expected 5, got 4
79
+ ```
80
+
81
+ A failing `Before` or `After` hook is named `Before hook` or `After hook` in place of the step, so an `After` hook that raises after every step passed is still a failed test. Messages are capped at 10,000 characters and backtraces at 50 lines.
82
+
83
+ ### Retries
84
+
85
+ With `--retry n`, only the final attempt of each scenario is reported, and `attempts` is the number of attempts. Each attempt starts with fresh attachments; attachments from earlier attempts are dropped.
86
+
87
+ ### Attachments
88
+
89
+ Both of these are captured while a scenario runs:
90
+
91
+ ```ruby
92
+ After do |scenario|
93
+ attach(path, "image/png") if scenario.failed? # Cucumber's own attach, with an existing file path (a Pathname works too)
94
+ RaiderReporter.attach("screenshot", path, type: :png) # or the Raider helper, as in RSpec
95
+ end
96
+ ```
97
+
98
+ Cucumber's `attach` is captured only when given the path of an existing file. Inline data (`attach("some text", "text/plain")`) and `log` messages are ignored.
99
+
100
+ ### Parallel runs with Cucumber
101
+
102
+ `parallel_tests` works with no configuration for Cucumber too: each process writes `run-<n>.json`. See [Parallel runs](#parallel-runs).
30
103
 
31
104
  ## Attaching screenshots and videos
32
105
 
@@ -122,6 +195,49 @@ On pull requests, `raider-reporter comment` posts one comment with the build's r
122
195
  - The command never fails your CI: it always exits 0 and prints notices to stderr, prefixed `[raider_reporter]`. It needs `GITHUB_TOKEN`, `GITHUB_REPOSITORY`, `GITHUB_EVENT_PATH`, `RAIDER_TOKEN` and `RAIDER_ENDPOINT`; if one is missing, or the event is not a pull request, it does nothing.
123
196
  - It requires a Raider server that supports the summary endpoint and the `git.pr` field. Upgrade the server before using it.
124
197
 
198
+ ## Azure Pipelines
199
+
200
+ The gem detects Azure Pipelines when `TF_BUILD` is `true`. It reports the build (`provider: azure_pipelines`, build URL, attempt, job), the branch and commit, the pull request and a build key, so shards of one build group together. On a pull request from Azure Repos, `raider-reporter comment` posts one comment thread with the results and sets a `raider/tests` status on the PR.
201
+
202
+ ```yaml
203
+ jobs:
204
+ - job: tests
205
+ steps:
206
+ - task: UseRubyVersion@0
207
+ inputs: { versionSpec: '3.4' }
208
+ - script: bundle install
209
+ displayName: Install gems
210
+ - script: bundle exec rspec spec/ --format documentation
211
+ displayName: Run tests
212
+ env:
213
+ RAIDER_TOKEN: $(RAIDER_TOKEN)
214
+ RAIDER_ENDPOINT: $(RAIDER_ENDPOINT)
215
+ - task: PublishPipelineArtifact@1
216
+ condition: always()
217
+ inputs: { targetPath: raider-results, artifact: raider-results }
218
+ - job: raider_comment
219
+ dependsOn: tests
220
+ condition: and(always(), eq(variables['Build.Reason'], 'PullRequest'))
221
+ steps:
222
+ - task: UseRubyVersion@0
223
+ inputs: { versionSpec: '3.4' }
224
+ - script: bundle install
225
+ - script: bundle exec raider-reporter comment
226
+ displayName: Comment Raider results on the pull request
227
+ env:
228
+ RAIDER_TOKEN: $(RAIDER_TOKEN)
229
+ RAIDER_ENDPOINT: $(RAIDER_ENDPOINT)
230
+ SYSTEM_ACCESSTOKEN: $(System.AccessToken)
231
+ ```
232
+
233
+ - **One-time grant:** give "Contribute to pull requests" to `<Project> Build Service (<org>)` on the repository (Project settings > Repositories > Security). Without it the command prints a notice naming the exact identity and exits 0.
234
+ - **Map secrets explicitly:** Azure does not expose secret variables to scripts automatically. Map `RAIDER_TOKEN` and `System.AccessToken` with `env:` as shown. If a mapped variable is undefined, Azure passes the literal `$(...)`; the gem treats it as missing: results are still written locally but not uploaded, and `raider-reporter comment` prints a skip notice.
235
+ - **Pull request builds:** for Azure Repos, YAML `pr:` triggers are ignored. PR builds (and therefore the comment job) run only when the pipeline is added as a **Build validation** branch policy on the target branch.
236
+ - **Same stage:** keep the comment job in the same stage as the test jobs; the build key includes `System.StageAttempt`.
237
+ - **Azure Repos only:** the PR comment works for repositories hosted in Azure Repos. A GitHub-hosted repository built by Azure Pipelines still uploads results, but `comment` prints `skipped: not an Azure Repos pull request`.
238
+ - **Parallel jobs:** `strategy: parallel: N` shards are detected automatically and grouped into one build, like GitHub matrix jobs.
239
+ - The command never fails your pipeline: it always exits 0 and prints notices to stderr, prefixed `[raider_reporter]`. It needs `SYSTEM_ACCESSTOKEN`, `RAIDER_TOKEN` and `RAIDER_ENDPOINT`, and requires a Raider server that supports the summary endpoint.
240
+
125
241
  ## Parallel runs
126
242
 
127
243
  When a suite is split across several processes or CI jobs, each one uploads its own run. Raider groups runs of the same CI build, so the shards show up together as one build. Each run records its shard (`index`, `total` and a `label`) and, on GitHub Actions, a build key made from the repository, run id and run attempt. A re-run attempt of a workflow is a separate build.
@@ -183,6 +299,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
183
299
  ## Examples
184
300
 
185
301
  - `examples/rspec_basic` is a runnable RSpec project with a passing, a failing (with an attachment) and a skipped example. Run it with `cd examples/rspec_basic && bundle install && bundle exec rspec`; the failure is deliberate.
302
+ - `examples/cucumber_basic` is the same for Cucumber (plugin mode): a passing scenario, a failing one with an attachment, and an outline. Run it with `cd examples/cucumber_basic && bundle install && bundle exec cucumber`; the failure is deliberate.
186
303
  - `examples/github_actions.yml` shows the workflow steps for CI.
187
304
 
188
305
  ## Schema
@@ -191,7 +308,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
191
308
 
192
309
  ## Contributing
193
310
 
194
- Bug reports and pull requests are welcome at https://github.com/RaiderHQ/raider_reporter. Run `bundle install && bundle exec rake` to run the specs.
311
+ Bug reports and pull requests are welcome at https://github.com/RaiderHQ/raider_reporter. Run `bundle install && bundle exec rake` to run the specs. To test against another Cucumber, set `CUCUMBER_VERSION` (for example `CUCUMBER_VERSION="~> 9.0" bundle update cucumber && bundle exec rspec`).
195
312
 
196
313
  ## License
197
314
 
@@ -0,0 +1,36 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RaiderReporter
4
+ # Reads Azure Pipelines variables. An undefined macro is left literal by Azure
5
+ # ("$(System.AccessToken)"), so such values count as missing.
6
+ module AzureEnv
7
+ MACRO = /\A\$\(.+\)\z/
8
+
9
+ module_function
10
+
11
+ def value(env, name)
12
+ raw = env[name].to_s.strip
13
+ raw.empty? || raw.match?(MACRO) ? nil : raw
14
+ end
15
+
16
+ def azure?(env)
17
+ env["GITHUB_ACTIONS"] != "true" && value(env, "TF_BUILD")&.casecmp?("true") == true
18
+ end
19
+
20
+ def collection_base(env)
21
+ uri = value(env, "SYSTEM_COLLECTIONURI")
22
+ uri && "#{uri.sub(%r{/+\z}, '')}/"
23
+ end
24
+
25
+ def azure_repos_pr?(env)
26
+ value(env, "BUILD_REASON") == "PullRequest" && value(env, "BUILD_REPOSITORY_PROVIDER") == "TfsGit"
27
+ end
28
+
29
+ def strip_heads(ref)
30
+ return nil if ref.nil?
31
+ return ref.delete_prefix("refs/heads/") if ref.start_with?("refs/heads/")
32
+
33
+ ref.start_with?("refs/") ? nil : ref
34
+ end
35
+ end
36
+ end
@@ -1,6 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "erb"
3
4
  require "open3"
5
+ require "raider_reporter/azure_env"
4
6
  require "raider_reporter/github_event"
5
7
 
6
8
  module RaiderReporter
@@ -14,6 +16,7 @@ module RaiderReporter
14
16
  end
15
17
 
16
18
  def ci
19
+ return azure_ci if AzureEnv.azure?(@env)
17
20
  return nil unless @env["GITHUB_ACTIONS"] == "true"
18
21
 
19
22
  run_id = @env["GITHUB_RUN_ID"]
@@ -38,6 +41,40 @@ module RaiderReporter
38
41
 
39
42
  private
40
43
 
44
+ def azure_ci
45
+ v = ->(name) { AzureEnv.value(@env, name) }
46
+ base = AzureEnv.collection_base(@env)
47
+ project = v.("SYSTEM_TEAMPROJECT")
48
+ build_id = v.("BUILD_BUILDID")
49
+ attempt = v.("SYSTEM_STAGEATTEMPT") || "1"
50
+ build_url = ("#{base}#{ERB::Util.url_encode(project)}/_build/results?buildId=#{build_id}" if base && project && build_id)
51
+ ids = [base&.sub(%r{\Ahttps?://}i, "")&.chomp("/"), v.("SYSTEM_TEAMPROJECTID"), v.("BUILD_REPOSITORY_ID")]
52
+ build_key = ("azure_pipelines:#{ids.join('/')}:#{build_id}:#{attempt}" if ids.all? && build_id)
53
+ repo = v.("BUILD_REPOSITORY_NAME")
54
+ { "provider" => "azure_pipelines", "build_url" => build_url, "build_id" => build_id, "attempt" => attempt,
55
+ "job" => v.("SYSTEM_JOBDISPLAYNAME"), "build_key" => build_key,
56
+ "repository" => (project && repo ? "#{project}/#{repo}" : nil) }.compact
57
+ end
58
+
59
+ def azure_pull_request
60
+ return nil unless AzureEnv.azure_repos_pr?(@env)
61
+
62
+ v = ->(name) { AzureEnv.value(@env, name) }
63
+ number = Integer(v.("SYSTEM_PULLREQUEST_PULLREQUESTID").to_s, exception: false)
64
+ base = AzureEnv.strip_heads(v.("SYSTEM_PULLREQUEST_TARGETBRANCH"))
65
+ head = AzureEnv.strip_heads(v.("SYSTEM_PULLREQUEST_SOURCEBRANCH"))
66
+ sha = v.("SYSTEM_PULLREQUEST_SOURCECOMMITID")
67
+ return nil unless number&.positive? && [base, head, sha].all? { |s| s.is_a?(String) && !s.empty? }
68
+
69
+ collection = AzureEnv.collection_base(@env)
70
+ project = v.("SYSTEM_TEAMPROJECT")
71
+ repo = v.("BUILD_REPOSITORY_NAME")
72
+ url = if collection && project && repo
73
+ "#{collection}#{ERB::Util.url_encode(project)}/_git/#{ERB::Util.url_encode(repo)}/pullrequest/#{number}"
74
+ end
75
+ { "number" => number, "base_branch" => base, "head_branch" => head, "head_sha" => sha, "url" => url }.compact
76
+ end
77
+
41
78
  def event
42
79
  return @event if defined?(@event)
43
80
 
@@ -57,6 +94,7 @@ module RaiderReporter
57
94
  end
58
95
 
59
96
  def pull_request
97
+ return azure_pull_request if AzureEnv.azure?(@env)
60
98
  return nil unless event && PR_EVENTS.include?(@env["GITHUB_EVENT_NAME"])
61
99
 
62
100
  pr = event["pull_request"]
@@ -69,7 +107,10 @@ module RaiderReporter
69
107
  return nil unless number.is_a?(Integer) && number >= 1
70
108
  return nil unless [base, head, sha].all? { |v| v.is_a?(String) && !v.empty? }
71
109
 
72
- { "number" => number, "base_branch" => base, "head_branch" => head, "head_sha" => sha }
110
+ server = @env["GITHUB_SERVER_URL"].to_s
111
+ repo = @env["GITHUB_REPOSITORY"].to_s
112
+ url = ("#{server}/#{repo}/pull/#{number}" unless server.empty? || repo.empty?)
113
+ { "number" => number, "base_branch" => base, "head_branch" => head, "head_sha" => sha, "url" => url }.compact
73
114
  rescue StandardError
74
115
  nil
75
116
  end
@@ -88,6 +129,8 @@ module RaiderReporter
88
129
  end
89
130
 
90
131
  def branch
132
+ return azure_branch if AzureEnv.azure?(@env)
133
+
91
134
  head_ref = @env["GITHUB_HEAD_REF"].to_s
92
135
  return head_ref unless head_ref.empty?
93
136
 
@@ -97,7 +140,18 @@ module RaiderReporter
97
140
  local == "HEAD" ? nil : local # "HEAD" means detached, not a branch name
98
141
  end
99
142
 
143
+ def azure_branch
144
+ pr = pull_request
145
+ name = pr ? pr["head_branch"] : AzureEnv.strip_heads(AzureEnv.value(@env, "BUILD_SOURCEBRANCH"))
146
+ return name if name
147
+
148
+ local = @git.call("rev-parse", "--abbrev-ref", "HEAD")
149
+ local == "HEAD" ? nil : local
150
+ end
151
+
100
152
  def sha
153
+ return AzureEnv.value(@env, "BUILD_SOURCEVERSION") || @git.call("rev-parse", "HEAD") if AzureEnv.azure?(@env)
154
+
101
155
  @env["GITHUB_SHA"] || @git.call("rev-parse", "HEAD")
102
156
  end
103
157
 
@@ -8,7 +8,13 @@ module RaiderReporter
8
8
  class CLI
9
9
  MARKER = "<!-- raider-reporter:build-comment -->"
10
10
  CONTEXT = "raider/tests"
11
- REQUIRED_ENV = %w[GITHUB_TOKEN GITHUB_REPOSITORY GITHUB_EVENT_PATH RAIDER_TOKEN RAIDER_ENDPOINT].freeze
11
+ GITHUB_REQUIRED_ENV = %w[GITHUB_TOKEN GITHUB_REPOSITORY GITHUB_EVENT_PATH RAIDER_TOKEN RAIDER_ENDPOINT].freeze
12
+ REQUIRED_ENV = GITHUB_REQUIRED_ENV
13
+ AZURE_REQUIRED_ENV = %w[SYSTEM_ACCESSTOKEN SYSTEM_COLLECTIONURI SYSTEM_TEAMPROJECTID BUILD_REPOSITORY_ID
14
+ RAIDER_TOKEN RAIDER_ENDPOINT].freeze
15
+ THREAD_STATUS = { "passed" => "closed" }.freeze
16
+ AZURE_STATES = { "passed" => "succeeded", "failed" => "failed", "incomplete" => "pending" }.freeze
17
+ TOKEN_HINT = "map it with env: SYSTEM_ACCESSTOKEN: $(System.AccessToken)"
12
18
  STATUS_STATES = { "passed" => "success", "failed" => "failure", "incomplete" => "pending" }.freeze
13
19
  FALLBACK_DESCRIPTIONS = {
14
20
  "passed" => "Raider: tests passed",
@@ -19,10 +25,14 @@ module RaiderReporter
19
25
  USAGE = <<~TEXT
20
26
  Usage: raider-reporter comment
21
27
 
22
- Posts (or updates) the Raider build summary as a comment on the current pull request
23
- and sets the "#{CONTEXT}" commit status. Intended for GitHub Actions; always exits 0.
28
+ Posts (or updates) the Raider build summary on the current pull request and sets a status.
29
+ Intended for GitHub Actions and Azure Pipelines (Azure Repos); always exits 0.
24
30
 
25
- Environment: #{REQUIRED_ENV.join(', ')} (GITHUB_API_URL optional).
31
+ GitHub Actions: comment + "#{CONTEXT}" commit status.
32
+ Environment: #{GITHUB_REQUIRED_ENV.join(', ')} (GITHUB_API_URL optional).
33
+ Azure Pipelines: one PR thread updated in place + a "raider/tests" PR status.
34
+ Environment: #{AZURE_REQUIRED_ENV.join(', ')}.
35
+ Map the token in the pipeline: env: SYSTEM_ACCESSTOKEN: $(System.AccessToken)
26
36
  TEXT
27
37
 
28
38
  def initialize(argv, env: ENV, out: $stdout, err: $stderr)
@@ -48,7 +58,17 @@ module RaiderReporter
48
58
  private
49
59
 
50
60
  def comment
51
- missing = REQUIRED_ENV.select { |name| @env[name].to_s.strip.empty? }
61
+ if @env["GITHUB_ACTIONS"] == "true"
62
+ github_comment
63
+ elsif AzureEnv.azure?(@env)
64
+ azure_comment
65
+ else
66
+ notice("skipped: unsupported CI")
67
+ end
68
+ end
69
+
70
+ def github_comment
71
+ missing = GITHUB_REQUIRED_ENV.select { |name| @env[name].to_s.strip.empty? }
52
72
  return notice("skipped: missing #{missing.join(', ')}") unless missing.empty?
53
73
 
54
74
  event = GithubEvent.load(@env)
@@ -66,6 +86,61 @@ module RaiderReporter
66
86
  notice("comment failed: #{e.class}: #{e.message}")
67
87
  end
68
88
 
89
+ def azure_comment
90
+ missing = AZURE_REQUIRED_ENV.select { |name| AzureEnv.value(@env, name).nil? }
91
+ return notice(azure_missing_message(missing)) unless missing.empty?
92
+
93
+ detector = CiDetector.new(env: @env, git: ->(*) {})
94
+ pr = detector.git&.dig("pr")
95
+ return notice("skipped: not an Azure Repos pull request") unless pr
96
+
97
+ build_key = detector.ci&.dig("build_key")
98
+ return notice("skipped: could not determine the build key") unless build_key
99
+
100
+ publish_azure(pr, build_key)
101
+ rescue Comment::AzureClient::Forbidden
102
+ notice(azure_forbidden_message)
103
+ rescue StandardError => e
104
+ notice("comment failed: #{e.class}: #{e.message}")
105
+ end
106
+
107
+ def azure_missing_message(missing)
108
+ message = "skipped: missing #{missing.join(', ')}"
109
+ message += "; #{TOKEN_HINT}" if missing.include?("SYSTEM_ACCESSTOKEN")
110
+ message
111
+ end
112
+
113
+ def publish_azure(pr, build_key)
114
+ summaries = Comment::SummaryClient.new(endpoint: AzureEnv.value(@env, "RAIDER_ENDPOINT"),
115
+ token: AzureEnv.value(@env, "RAIDER_TOKEN"))
116
+ summary = summaries.fetch(build_key)
117
+ return notice("summary unavailable: #{summaries.last_error}") unless summary
118
+
119
+ azure = Comment::AzureClient.new(token: AzureEnv.value(@env, "SYSTEM_ACCESSTOKEN"), base_url: azure_pr_url(pr))
120
+ result = azure.upsert_thread(summary.markdown, MARKER, status: THREAD_STATUS.fetch(summary.state, "active"))
121
+ azure.set_status(state: AZURE_STATES.fetch(summary.state, "pending"), description: description(summary),
122
+ target_url: summary.url)
123
+ notice("PR thread #{result}: #{pr['url'] || "PR ##{pr['number']}"}")
124
+ end
125
+
126
+ def azure_pr_url(pr)
127
+ "#{AzureEnv.collection_base(@env)}#{AzureEnv.value(@env, 'SYSTEM_TEAMPROJECTID')}/_apis/git/repositories/" \
128
+ "#{AzureEnv.value(@env, 'BUILD_REPOSITORY_ID')}/pullRequests/#{pr['number']}"
129
+ end
130
+
131
+ def azure_forbidden_message
132
+ project = AzureEnv.value(@env, "SYSTEM_TEAMPROJECT") || "<project>"
133
+ "skipped: the build service cannot write to this PR — grant \"Contribute to pull requests\" to " \
134
+ "\"#{project} Build Service (#{azure_org})\""
135
+ end
136
+
137
+ def azure_org
138
+ uri = URI.parse(AzureEnv.collection_base(@env).to_s)
139
+ uri.host.to_s.end_with?("visualstudio.com") ? uri.host.split(".").first : uri.path.split("/").reject(&:empty?).first
140
+ rescue URI::InvalidURIError
141
+ nil
142
+ end
143
+
69
144
  def publish(pr, build_key)
70
145
  summaries = Comment::SummaryClient.new(endpoint: @env["RAIDER_ENDPOINT"], token: @env["RAIDER_TOKEN"])
71
146
  summary = summaries.fetch(build_key)
@@ -55,8 +55,18 @@ module RaiderReporter
55
55
  end
56
56
 
57
57
  def finish_test(**fields)
58
+ add_result(build_test(**fields))
59
+ end
60
+
61
+ # Closes the current test and returns its result without recording it, so a
62
+ # framework that retries can hold one result per test and replace it per attempt.
63
+ def build_test(**fields)
58
64
  result = TestResult.new(**fields, attachments: @pending_attachments || [])
59
65
  @pending_attachments = nil
66
+ result
67
+ end
68
+
69
+ def add_result(result)
60
70
  @results << result
61
71
  result
62
72
  end
@@ -0,0 +1,110 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "net/http"
4
+ require "uri"
5
+ require "json"
6
+
7
+ module RaiderReporter
8
+ module Comment
9
+ # Minimal Azure DevOps REST client: one Raider thread per pull request, plus a PR status.
10
+ class AzureClient
11
+ class Error < StandardError; end
12
+ # 401/403/404: the build service identity cannot write to this PR.
13
+ class Forbidden < Error; end
14
+
15
+ API_VERSION = "7.1"
16
+ STATUS_API_VERSION = "7.1" # confirmed in the 7.1 Pull Request Statuses doc; see spec header
17
+ PROPERTY = "Raider.BuildComment"
18
+ DESCRIPTION_LIMIT = 140
19
+
20
+ def initialize(token:, base_url:, timeout: 30)
21
+ @token = token
22
+ @base_url = base_url.to_s.sub(%r{/+\z}, "")
23
+ @timeout = timeout
24
+ end
25
+
26
+ def upsert_thread(markdown, marker, status:)
27
+ thread = find_thread(marker)
28
+ if thread
29
+ comment_id = thread["comments"].first["id"]
30
+ request(:patch, "/threads/#{thread['id']}/comments/#{comment_id}", { content: markdown })
31
+ request(:patch, "/threads/#{thread['id']}", { status: status })
32
+ :updated
33
+ else
34
+ request(:post, "/threads", { comments: [{ parentCommentId: 0, content: markdown, commentType: 1 }],
35
+ status: status,
36
+ properties: { PROPERTY => { "$type" => "System.Int32", "$value" => 1 } } })
37
+ :created
38
+ end
39
+ end
40
+
41
+ def set_status(state:, description:, target_url:)
42
+ request(:post, "/statuses", { state: state, description: description.to_s[0, DESCRIPTION_LIMIT],
43
+ targetUrl: target_url, context: { genre: "raider", name: "tests" } },
44
+ version: STATUS_API_VERSION)
45
+ nil
46
+ end
47
+
48
+ private
49
+
50
+ def find_thread(marker)
51
+ list = request(:get, "/threads")
52
+ threads = list.is_a?(Hash) ? list["value"] : nil
53
+ raise Error, "unexpected threads response" unless threads.is_a?(Array)
54
+
55
+ threads.find { |t| ours?(t, marker) }
56
+ end
57
+
58
+ # Only a property-tagged thread whose live first comment starts with the marker is ours.
59
+ def ours?(thread, marker)
60
+ return false unless thread.is_a?(Hash) && thread["properties"].is_a?(Hash) && thread["properties"].key?(PROPERTY)
61
+ return false unless thread["comments"].is_a?(Array)
62
+
63
+ first = thread["comments"].first
64
+ first.is_a?(Hash) && !first["isDeleted"] && first["content"].to_s.start_with?(marker)
65
+ end
66
+
67
+ def redact(text)
68
+ @token.to_s.empty? ? text.to_s : text.to_s.gsub(@token.to_s, "[REDACTED]")
69
+ end
70
+
71
+ def request(verb, path, payload = nil, version: nil)
72
+ uri = URI.parse("#{@base_url}#{path}?api-version=#{version || API_VERSION}")
73
+ raise Error, "Azure DevOps URL must start with http:// or https://" unless uri.is_a?(URI::HTTP)
74
+
75
+ req = build_request(verb, uri, payload)
76
+ response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https",
77
+ open_timeout: @timeout, read_timeout: @timeout,
78
+ write_timeout: @timeout) { |http| http.request(req) }
79
+ handle(response)
80
+ rescue Error
81
+ raise
82
+ rescue StandardError => e
83
+ raise Error, redact("#{e.class}: #{e.message}")
84
+ end
85
+
86
+ def build_request(verb, uri, json)
87
+ klass = { get: Net::HTTP::Get, post: Net::HTTP::Post, patch: Net::HTTP::Patch }.fetch(verb)
88
+ req = klass.new(uri)
89
+ req["Authorization"] = "Bearer #{@token}"
90
+ req["Accept"] = "application/json"
91
+ req["User-Agent"] = "raider_reporter/#{VERSION}"
92
+ if json
93
+ req["Content-Type"] = "application/json"
94
+ req.body = JSON.generate(json)
95
+ end
96
+ req
97
+ end
98
+
99
+ def handle(response)
100
+ code = response.code.to_i
101
+ raise Forbidden, "HTTP #{code}" if [401, 403, 404].include?(code)
102
+ raise Error, "HTTP #{code}: #{redact(response.body.to_s)[0, 200]}" unless (200..299).cover?(code)
103
+
104
+ JSON.parse(response.body.to_s)
105
+ rescue JSON::ParserError
106
+ raise Error, "Azure DevOps returned invalid JSON"
107
+ end
108
+ end
109
+ end
110
+ end
@@ -101,7 +101,7 @@ module RaiderReporter
101
101
  def handle(response)
102
102
  code = response.code.to_i
103
103
  raise Forbidden, "HTTP #{code}" if [403, 404].include?(code)
104
- raise Error, redact("HTTP #{code}: #{response.body.to_s[0, 200]}") unless (200..299).cover?(code)
104
+ raise Error, "HTTP #{code}: #{redact(response.body.to_s)[0, 200]}" unless (200..299).cover?(code)
105
105
 
106
106
  JSON.parse(response.body.to_s)
107
107
  rescue JSON::ParserError
@@ -1,12 +1,14 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "raider_reporter/azure_env"
4
+
3
5
  module RaiderReporter
4
6
  class Configuration
5
7
  attr_accessor :token, :endpoint, :output_dir, :clean_output_dir, :environment, :max_attachment_bytes
6
8
 
7
9
  def initialize(env = ENV)
8
- @token = env["RAIDER_TOKEN"]
9
- @endpoint = env["RAIDER_ENDPOINT"]
10
+ @token = AzureEnv.value(env, "RAIDER_TOKEN")
11
+ @endpoint = AzureEnv.value(env, "RAIDER_ENDPOINT")
10
12
  @output_dir = "raider-results"
11
13
  @clean_output_dir = true
12
14
  @environment = {}
@@ -0,0 +1,258 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "pathname"
4
+ require "cucumber"
5
+ require "cucumber/formatter/ast_lookup"
6
+ require "raider_reporter"
7
+ require "raider_reporter/text"
8
+
9
+ module RaiderReporter
10
+ module Cucumber
11
+ # Translates Cucumber events into Collector calls. Every handler is guarded:
12
+ # a reporting bug must never fail or crash the user's run. The formatter
13
+ # writes nothing to its IO; the report goes to RaiderReporter's own writers.
14
+ #
15
+ # Inside this namespace `Cucumber` means RaiderReporter::Cucumber, so the
16
+ # gem's constants are always written ::Cucumber.
17
+ class Formatter
18
+ STATUSES = {
19
+ passed: "passed", failed: "failed", ambiguous: "failed",
20
+ pending: "pending", undefined: "pending", skipped: "skipped"
21
+ }.freeze
22
+
23
+ # Passed to the World's `attach`; reports an existing file as an attachment.
24
+ module AttachCapture
25
+ # Takes whatever Cucumber passes (file, media_type, filename, and any
26
+ # argument a later release adds) and forwards it untouched.
27
+ ruby2_keywords def attach(*args, &block)
28
+ result = super
29
+ Formatter.record_attachment(args[0], args[1])
30
+ result
31
+ end
32
+ end
33
+
34
+ @claimed = []
35
+
36
+ class << self
37
+ # One listener per run, whether it arrives as a --format or as a plugin.
38
+ # Cucumber copies the configuration for --out, but the event bus is shared.
39
+ def claim(config)
40
+ bus = config.event_bus
41
+ return false if @claimed.any? { |claimed| claimed.equal?(bus) }
42
+
43
+ @claimed << bus
44
+ true
45
+ end
46
+
47
+ # Inline text, IO objects and missing paths are ignored: only a file that
48
+ # exists can be copied. Never raises into the user's step.
49
+ def record_attachment(file, media_type)
50
+ # `log` is attach with this media type; a logged message is not a file.
51
+ return if media_type.to_s.start_with?("text/x.cucumber.log")
52
+
53
+ file = file.to_path if file.respond_to?(:to_path)
54
+ return unless file.is_a?(String) && RaiderReporter.collector&.in_test? && File.file?(file)
55
+
56
+ RaiderReporter.attach(File.basename(file), file, type: media_type)
57
+ rescue StandardError => e
58
+ RaiderReporter.warn("#{e.class}: #{e.message}")
59
+ end
60
+
61
+ # Called when the run ends, so the list does not outlive the run.
62
+ def release(config)
63
+ bus = config.event_bus
64
+ @claimed.reject! { |claimed| claimed.equal?(bus) }
65
+ RaiderReporter.dry_run = false
66
+ end
67
+
68
+ # Cucumber hands steps a World, not the formatter, so `attach` is observed
69
+ # by prepending to ProtoWorld. Installed once, at the first formatter.
70
+ # Event-bus alternatives were checked in cucumber 9.2.1, 10.2.0 and 11.1.1:
71
+ # there is no :attach_called event before 11, and even there (and in the
72
+ # :envelope attachment message) a file path has already been read into
73
+ # bytes with no name, so the basename the report needs is only available
74
+ # in ProtoWorld#attach, which exists unchanged in all three versions.
75
+ def capture_attach
76
+ return if @capturing
77
+
78
+ ::Cucumber::Glue::ProtoWorld.prepend(AttachCapture)
79
+ @capturing = true
80
+ end
81
+ end
82
+
83
+ def initialize(config)
84
+ @buffer = {}
85
+ @attempts = Hash.new(0)
86
+ @attempt = nil
87
+ guard { setup(config) if self.class.claim(config) }
88
+ end
89
+
90
+ private
91
+
92
+ def setup(config)
93
+ @config = config
94
+ @ast = ::Cucumber::Formatter::AstLookup.new(config)
95
+ @collector = RaiderReporter.start_collector!
96
+ RaiderReporter.dry_run = config.dry_run?
97
+ config.on_event(:test_run_started) { |_event| guard { @collector.start_run } }
98
+ config.on_event(:test_case_started) { |event| guard { case_started(event.test_case) } }
99
+ config.on_event(:test_step_finished) { |event| guard { step_finished(event.test_step, event.result) } }
100
+ config.on_event(:test_case_finished) { |event| guard { case_finished(event.test_case, event.result) } }
101
+ config.on_event(:test_run_finished) { |_event| guard { run_finished } }
102
+ # Last, and on its own: a broken attach hook may only lose attachments.
103
+ safely(nil) { self.class.capture_attach }
104
+ end
105
+
106
+ # Every attempt starts with an empty attachment list.
107
+ def case_started(test_case)
108
+ @collector.start_test
109
+ @attempt = { test_case: test_case }
110
+ end
111
+
112
+ # The first non-passed step explains why the scenario did not pass.
113
+ def step_finished(test_step, result)
114
+ return unless @attempt
115
+ return if @attempt[:problem] || result.passed?
116
+
117
+ @attempt[:problem] = { step: test_step, result: result }
118
+ end
119
+
120
+ def case_finished(test_case, result)
121
+ problem = @attempt && @attempt[:problem]
122
+ @attempt = nil
123
+ status = STATUSES.fetch(result.to_sym, "pending")
124
+ # Always buffer: a lookup problem may lose detail, never the scenario.
125
+ failure = status == "failed" ? safely(nil) { failure_for(problem, result, test_case) } : nil
126
+ fields = identity_for(test_case)
127
+ key = "#{test_case.location}|#{test_case.name}"
128
+ @attempts[key] += 1
129
+ # The newest attempt replaces the older one, taking its attachments with it.
130
+ @buffer[key] = @collector.build_test(
131
+ **fields, status: status, duration_ms: duration_ms(result), attempts: @attempts[key], failure: failure
132
+ )
133
+ end
134
+
135
+ def run_finished
136
+ @buffer.each_value { |result| @collector.add_result(result) }
137
+ @buffer.clear
138
+ RaiderReporter.finish!($stderr)
139
+ ensure
140
+ self.class.release(@config)
141
+ end
142
+
143
+ def identity_for(test_case)
144
+ file_path = relative_path(test_case.location.file)
145
+ line_number = test_case.location.lines.max
146
+ {
147
+ file_path: file_path,
148
+ line_number: line_number,
149
+ full_name: Text.utf8(full_name(test_case)),
150
+ identifier: "#{file_path}:#{line_number}",
151
+ tags: test_case.tags.map { |tag| tag.name.to_s.delete_prefix("@") }.uniq.sort
152
+ }
153
+ end
154
+
155
+ def full_name(test_case)
156
+ safely(test_case.name) do
157
+ feature = @ast.gherkin_document(test_case.location.file)&.feature
158
+ name = scenario_name(@ast.scenario_source(test_case))
159
+ next test_case.name unless name
160
+
161
+ feature ? "#{feature.name}: #{name}" : name
162
+ end
163
+ end
164
+
165
+ # nil when the scenario cannot be located in the AST.
166
+ def scenario_name(source)
167
+ case source&.type
168
+ when :ScenarioOutline
169
+ examples = source.examples
170
+ row = examples.table_body.index { |candidate| candidate.location.line == source.row.location.line }
171
+ row && "#{source.scenario_outline.name} (#{examples_label(source)}, row #{row + 1})"
172
+ when :Scenario
173
+ source.scenario.name
174
+ end
175
+ end
176
+
177
+ # "Examples" or the table's name. When several tables of one outline share a
178
+ # label (two unnamed ones, say), each gets its 1-based position among them,
179
+ # so rows of different tables never collide on full_name.
180
+ def examples_label(source)
181
+ tables = source.scenario_outline.examples
182
+ label = examples_name(source.examples)
183
+ same = tables.select { |table| examples_name(table) == label }
184
+ return label if same.size < 2
185
+
186
+ "#{label} #{same.index { |table| table.equal?(source.examples) || table.location.line == source.examples.location.line } + 1}"
187
+ end
188
+
189
+ def examples_name(table)
190
+ table.name.to_s.strip.empty? ? "Examples" : table.name.strip
191
+ end
192
+
193
+ # step_info is { step:, result: } for the first non-passed step, if any.
194
+ def failure_for(step_info, result, test_case)
195
+ exception = result.respond_to?(:exception) ? result.exception : nil
196
+ exception ||= step_info && step_info[:result].respond_to?(:exception) ? step_info[:result].exception : nil
197
+ return nil unless exception
198
+
199
+ Failure.new(
200
+ message: Text.utf8("#{step_label(step_info, test_case)}\n\n#{exception.message}")[0, Text::MAX_MESSAGE],
201
+ exception_class: Text.utf8(exception.class.name),
202
+ backtrace: (exception.backtrace || []).first(Text::MAX_BACKTRACE).map { |line| Text.utf8(line) }
203
+ )
204
+ end
205
+
206
+ def step_label(step_info, test_case)
207
+ return "Step" unless step_info
208
+
209
+ test_step = step_info[:step]
210
+ location = "(#{relative_path(test_step.location.file)}:#{test_step.location.lines.min})"
211
+ if test_step.hook?
212
+ "#{hook_kind(test_step, test_case)} hook #{location}"
213
+ else
214
+ "#{step_keyword(test_step)}#{test_step.text} #{location}"
215
+ end
216
+ end
217
+
218
+ def step_keyword(test_step)
219
+ safely("") { @ast.step_source(test_step).step.keyword.to_s }
220
+ end
221
+
222
+ # A hook with a real step before it runs after the scenario started.
223
+ def hook_kind(test_step, test_case)
224
+ steps = test_case.test_steps
225
+ index = steps.index { |candidate| candidate.equal?(test_step) }
226
+ index && steps.first(index).any? { |candidate| !candidate.hook? } ? "After" : "Before"
227
+ end
228
+
229
+ def duration_ms(result)
230
+ (result.duration.nanoseconds / 1_000_000.0).round
231
+ rescue StandardError
232
+ 0
233
+ end
234
+
235
+ def relative_path(path)
236
+ absolute = Pathname.new(File.expand_path(path.to_s, Dir.pwd))
237
+ relative = absolute.relative_path_from(Pathname.new(Dir.pwd)).to_s
238
+ relative.start_with?("..") ? path.to_s.delete_prefix("./") : relative
239
+ rescue ArgumentError
240
+ # Windows: a path on another drive has no relative form.
241
+ path.to_s.delete_prefix("./")
242
+ end
243
+
244
+ def safely(fallback)
245
+ yield
246
+ rescue StandardError => e
247
+ RaiderReporter.warn("#{e.class}: #{e.message}")
248
+ fallback
249
+ end
250
+
251
+ def guard
252
+ yield
253
+ rescue StandardError => e
254
+ RaiderReporter.warn("#{e.class}: #{e.message}")
255
+ end
256
+ end
257
+ end
258
+ end
@@ -0,0 +1,40 @@
1
+ # frozen_string_literal: true
2
+
3
+ begin
4
+ require "cucumber"
5
+ rescue LoadError => e
6
+ raise unless e.path == "cucumber"
7
+
8
+ raise LoadError, "raider_reporter/cucumber needs the cucumber gem (>= 9, < 12) in your Gemfile"
9
+ end
10
+ require "raider_reporter"
11
+ require_relative "cucumber/formatter"
12
+
13
+ module RaiderReporter
14
+ module Cucumber
15
+ # Plugin mode: requiring this file from features/support registers the
16
+ # listener through Cucumber's InstallPlugin hook, so no --format is needed
17
+ # (and none of the "one formatter per stream" juggling). Outside a Cucumber
18
+ # run there is no step-definition registry and this is a no-op.
19
+ def self.install_plugin
20
+ # Cucumber's public InstallPlugin DSL method, called on a throwaway object
21
+ # (it is only an instance method of Dsl; main is extended with it).
22
+ Object.new.extend(::Cucumber::Glue::Dsl).InstallPlugin(name: "raider_reporter") do |config, *|
23
+ RaiderReporter::Cucumber::Formatter.new(config)
24
+ end
25
+ true
26
+ rescue StandardError => e
27
+ # Outside a run there is no registry (Dsl has no rb_language) and nothing to
28
+ # report. Inside one, losing the listener must not be silent.
29
+ RaiderReporter.warn("could not register the Cucumber plugin: #{e.class}: #{e.message}") if run_active?
30
+ false
31
+ end
32
+
33
+ def self.run_active?
34
+ !::Cucumber::Glue::Dsl.instance_variable_get(:@rb_language).nil?
35
+ end
36
+ private_class_method :run_active?
37
+ end
38
+ end
39
+
40
+ RaiderReporter::Cucumber.install_plugin
@@ -2,6 +2,7 @@
2
2
 
3
3
  require "rspec/core"
4
4
  require "raider_reporter"
5
+ require "raider_reporter/text"
5
6
 
6
7
  module RaiderReporter
7
8
  module RSpec
@@ -13,8 +14,8 @@ module RaiderReporter
13
14
  :example_failed, :example_pending, :close
14
15
 
15
16
  NON_TAG_KEYS = %i[pending skip].freeze
16
- MAX_MESSAGE = 10_000
17
- MAX_BACKTRACE = 50
17
+ MAX_MESSAGE = Text::MAX_MESSAGE
18
+ MAX_BACKTRACE = Text::MAX_BACKTRACE
18
19
 
19
20
  def initialize(output)
20
21
  @output = output
@@ -77,14 +78,8 @@ module RaiderReporter
77
78
  )
78
79
  end
79
80
 
80
- # JSON generation raises on invalid UTF-8, which would lose the whole run.
81
81
  def utf8(value)
82
- str = value.to_s
83
- if [Encoding::UTF_8, Encoding::BINARY, Encoding::US_ASCII].include?(str.encoding)
84
- str.dup.force_encoding(Encoding::UTF_8).scrub("?")
85
- else
86
- str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "?")
87
- end
82
+ Text.utf8(value)
88
83
  end
89
84
 
90
85
  def guard
@@ -1,5 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "raider_reporter/azure_env"
4
+
3
5
  module RaiderReporter
4
6
  # Identifies which slice of a parallel suite this process is.
5
7
  class ShardDetector
@@ -9,7 +11,7 @@ module RaiderReporter
9
11
  end
10
12
 
11
13
  def shard
12
- index, total, suffix = composed || explicit || parallel_tests
14
+ index, total, suffix = composed || outer || parallel_tests
13
15
  return nil unless index
14
16
 
15
17
  { "index" => index, "total" => total, "label" => "#{@job || "local"}##{suffix || index}" }
@@ -19,7 +21,7 @@ module RaiderReporter
19
21
 
20
22
  # A matrix leg that also runs parallel_tests: every process gets a unique index.
21
23
  def composed
22
- explicit_index, explicit_total = explicit
24
+ explicit_index, explicit_total = outer
23
25
  parallel_index, = parallel_tests
24
26
  groups = positive(@env["PARALLEL_TEST_GROUPS"])
25
27
  return nil unless explicit_index && parallel_index && groups
@@ -29,6 +31,18 @@ module RaiderReporter
29
31
  "#{explicit_index}.#{parallel_index}"]
30
32
  end
31
33
 
34
+ def outer
35
+ explicit || azure_parallel
36
+ end
37
+
38
+ def azure_parallel
39
+ return nil unless AzureEnv.azure?(@env)
40
+
41
+ total = positive(AzureEnv.value(@env, "SYSTEM_TOTALJOBSINPHASE"))
42
+ index = positive(AzureEnv.value(@env, "SYSTEM_JOBPOSITIONINPHASE"))
43
+ [index, total] if index && total && total > 1
44
+ end
45
+
32
46
  def explicit
33
47
  index = positive(@env["RAIDER_SHARD_INDEX"])
34
48
  [index, positive(@env["RAIDER_SHARD_TOTAL"])] if index
@@ -0,0 +1,19 @@
1
+ # frozen_string_literal: true
2
+
3
+ module RaiderReporter
4
+ # Text limits and cleaning shared by the framework formatters.
5
+ module Text
6
+ MAX_MESSAGE = 10_000
7
+ MAX_BACKTRACE = 50
8
+
9
+ # JSON generation raises on invalid UTF-8, which would lose the whole run.
10
+ def self.utf8(value)
11
+ str = value.to_s
12
+ if [Encoding::UTF_8, Encoding::BINARY, Encoding::US_ASCII].include?(str.encoding)
13
+ str.dup.force_encoding(Encoding::UTF_8).scrub("?")
14
+ else
15
+ str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "?")
16
+ end
17
+ end
18
+ end
19
+ end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module RaiderReporter
4
- VERSION = "0.3.0"
4
+ VERSION = "0.5.0"
5
5
  end
@@ -3,6 +3,7 @@
3
3
  require "time"
4
4
  require "rbconfig"
5
5
  require_relative "raider_reporter/version"
6
+ require_relative "raider_reporter/text"
6
7
  require_relative "raider_reporter/models"
7
8
  require_relative "raider_reporter/payload"
8
9
  require_relative "raider_reporter/configuration"
@@ -14,6 +15,7 @@ require_relative "raider_reporter/shard_detector"
14
15
  require_relative "raider_reporter/writers/uploader"
15
16
  require_relative "raider_reporter/comment/summary_client"
16
17
  require_relative "raider_reporter/comment/github_client"
18
+ require_relative "raider_reporter/comment/azure_client"
17
19
 
18
20
  module RaiderReporter
19
21
  class << self
@@ -27,13 +29,18 @@ module RaiderReporter
27
29
  yield configuration
28
30
  end
29
31
 
32
+ # Set by the Cucumber formatter when the run is a --dry-run.
33
+ attr_writer :dry_run
34
+
30
35
  def reset!
36
+ @dry_run = false
31
37
  @collector&.cleanup
32
38
  @configuration = nil
33
39
  @collector = nil
34
40
  end
35
41
 
36
42
  def start_collector!
43
+ @dry_run = false
37
44
  @collector = Collector.new(
38
45
  max_attachment_bytes: configuration.max_attachment_bytes,
39
46
  warn: ->(message) { warn(message) }
@@ -104,8 +111,10 @@ module RaiderReporter
104
111
  "Raider report uploaded"
105
112
  end
106
113
 
107
- # `rspec --dry-run` reports every example as passed without running it.
114
+ # `rspec --dry-run` and `cucumber --dry-run` report every example as passed without running it.
108
115
  def dry_run?
116
+ return true if @dry_run
117
+
109
118
  defined?(::RSpec.configuration) && ::RSpec.configuration.respond_to?(:dry_run?) &&
110
119
  ::RSpec.configuration.dry_run?
111
120
  end
metadata CHANGED
@@ -1,16 +1,17 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: raider_reporter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 0.5.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Augustin Gottlieb
8
8
  bindir: exe
9
9
  cert_chain: []
10
- date: 2026-10-11 00:00:00.000000000 Z
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
11
  dependencies: []
12
- description: RSpec formatter that records results, screenshots and videos in the Raider
13
- schema, writes them locally, and optionally uploads them to a Raider server.
12
+ description: RSpec and Cucumber formatters that record results, screenshots and videos
13
+ in the Raider schema, write them locally, and optionally upload them to a Raider
14
+ server.
14
15
  email:
15
16
  - 33221555+aguspe@users.noreply.github.com
16
17
  executables:
@@ -23,19 +24,24 @@ files:
23
24
  - README.md
24
25
  - exe/raider-reporter
25
26
  - lib/raider_reporter.rb
27
+ - lib/raider_reporter/azure_env.rb
26
28
  - lib/raider_reporter/ci_detector.rb
27
29
  - lib/raider_reporter/cli.rb
28
30
  - lib/raider_reporter/collector.rb
31
+ - lib/raider_reporter/comment/azure_client.rb
29
32
  - lib/raider_reporter/comment/github_client.rb
30
33
  - lib/raider_reporter/comment/summary_client.rb
31
34
  - lib/raider_reporter/configuration.rb
32
35
  - lib/raider_reporter/content_types.rb
36
+ - lib/raider_reporter/cucumber.rb
37
+ - lib/raider_reporter/cucumber/formatter.rb
33
38
  - lib/raider_reporter/github_event.rb
34
39
  - lib/raider_reporter/models.rb
35
40
  - lib/raider_reporter/payload.rb
36
41
  - lib/raider_reporter/rspec.rb
37
42
  - lib/raider_reporter/rspec/formatter.rb
38
43
  - lib/raider_reporter/shard_detector.rb
44
+ - lib/raider_reporter/text.rb
39
45
  - lib/raider_reporter/version.rb
40
46
  - lib/raider_reporter/writers/local.rb
41
47
  - lib/raider_reporter/writers/uploader.rb
@@ -60,7 +66,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
60
66
  - !ruby/object:Gem::Version
61
67
  version: '0'
62
68
  requirements: []
63
- rubygems_version: 3.6.2
69
+ rubygems_version: 4.0.6
64
70
  specification_version: 4
65
- summary: 'Test reporting for Ruby Raider: local run.json and Raider uploads'
71
+ summary: RSpec and Cucumber formatters that write Raider reports locally and upload
72
+ them to Raider
66
73
  test_files: []