raider_reporter 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +10 -0
- data/README.md +77 -3
- data/lib/raider_reporter/collector.rb +10 -0
- data/lib/raider_reporter/cucumber/formatter.rb +258 -0
- data/lib/raider_reporter/cucumber.rb +40 -0
- data/lib/raider_reporter/rspec/formatter.rb +4 -9
- data/lib/raider_reporter/text.rb +19 -0
- data/lib/raider_reporter/version.rb +1 -1
- data/lib/raider_reporter.rb +9 -1
- metadata +11 -6
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 61be0da46b6e8038ccb7da397ce371e20e398792aaa42aff9e93e2e765efd73b
|
|
4
|
+
data.tar.gz: 9d2859ced0b58e73ccd0f8d90fece82b3d8f379bca46ef958edb8089e81ae30e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a805ca67f914f9413cb95913b2015c7a56020564bf709a45aadd96c25f183a374927fd572b6dab80cf2969adca7b2d6305edd04b617d0414f1af2caba108ddc4
|
|
7
|
+
data.tar.gz: 1f9ba4407c3677d5166ec09aced32f3d2c43ac4a599468eda510f3e7327ce417d6eea25af8de47ea308bacde60e45b9c8ba36414434bd0bd0d938bb2ecdd1bfc
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.0 — 2026-10-11
|
|
4
|
+
|
|
5
|
+
- Add a Cucumber reporter (Cucumber >= 9, < 12; not a gem dependency). `require "raider_reporter/cucumber"` from `features/support/` registers it in plugin mode (recommended, works with any `--format` and no `--out`); `--format RaiderReporter::Cucumber::Formatter --out <file>` also works. Using both registers one listener
|
|
6
|
+
- Identity: `file_path` is project-relative (`features/login.feature`), `full_name` is `"<Feature>: <Scenario>"` (outline rows add `(<Examples name>, row <n>)`; when several Examples tables share a label, each is numbered: `(Examples 1, row 1)`, `(Examples 2, row 1)`), `identifier` is `<file>:<line>`
|
|
7
|
+
- Report the failing step (or Before/After hook) with its location in the failure message; ambiguous maps to failed, undefined to pending
|
|
8
|
+
- Retries: only the final attempt is kept, with `attempts` set and earlier attachments dropped
|
|
9
|
+
- Attachments: `RaiderReporter.attach` and Cucumber's `attach(path, media_type)` with an existing file path are captured; inline data and `log` messages are ignored
|
|
10
|
+
- Dry runs (`cucumber --dry-run`) write and upload nothing; `parallel_tests` shards write `run-<n>.json`
|
|
11
|
+
- No breaking changes: RSpec behaviour is unchanged
|
|
12
|
+
|
|
3
13
|
## 0.3.0 — 2026-10-11
|
|
4
14
|
|
|
5
15
|
- Report pull request identity: `run.ci.repository`, `run.git.default_branch` and `run.git.pr` (number, base/head branch, head SHA) on `pull_request` and `pull_request_target` events. Requires a Raider server that accepts `git.pr`
|
data/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# raider_reporter
|
|
2
2
|
|
|
3
|
-
Raider-schema test reporting for RSpec. `raider_reporter` is an RSpec formatter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
|
|
3
|
+
Raider-schema test reporting for RSpec and Cucumber. `raider_reporter` is an RSpec formatter and a Cucumber reporter that records every example's result, failure details, screenshots and videos, writes them to a local `run.json` (schema v1), and can optionally upload the run to a Raider server. It replaces Allure for Ruby Raider projects.
|
|
4
4
|
|
|
5
5
|
It has no runtime dependencies and requires Ruby 3.2 or newer.
|
|
6
6
|
|
|
@@ -26,7 +26,80 @@ Enable the formatter in `.rspec`:
|
|
|
26
26
|
--format RaiderReporter::RSpec::Formatter
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
Run `bundle exec rspec` as usual. Results land in `raider-results/run.json`. A dry run (`rspec --dry-run`) writes and uploads nothing.
|
|
29
|
+
Run `bundle exec rspec` as usual. (For Cucumber, see [Cucumber](#cucumber).) Results land in `raider-results/run.json`. A dry run (`rspec --dry-run`) writes and uploads nothing.
|
|
30
|
+
|
|
31
|
+
## Cucumber
|
|
32
|
+
|
|
33
|
+
Requirements: Cucumber 9, 10 or 11 in your own Gemfile (`gem "cucumber"`). It is not a dependency of this gem; requiring `raider_reporter/cucumber` without Cucumber raises a `LoadError` that says so. Cucumber 9.2.1 on Ruby 4 also needs `gem "logger"` in your Gemfile, because Cucumber requires it without declaring it.
|
|
34
|
+
|
|
35
|
+
### Plugin mode (recommended)
|
|
36
|
+
|
|
37
|
+
Require the reporter from a file under `features/support/`, for example `features/support/raider.rb`:
|
|
38
|
+
|
|
39
|
+
```ruby
|
|
40
|
+
require "raider_reporter/cucumber"
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Then run Cucumber with any normal formatter and no `--out`:
|
|
44
|
+
|
|
45
|
+
```
|
|
46
|
+
# cucumber.yml
|
|
47
|
+
default: --format pretty
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
The require registers one listener on Cucumber's event bus, so it sits next to `pretty`, `progress` or any other formatter. Results land in `raider-results/run.json`, and a dry run (`cucumber --dry-run`) writes and uploads nothing. Plugin mode registers once per Ruby process, because `require` is cached: a second in-process `Cucumber::Cli::Main` run in the same process does not register again.
|
|
51
|
+
|
|
52
|
+
### Formatter mode
|
|
53
|
+
|
|
54
|
+
You can also name the formatter explicitly:
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
cucumber --format pretty --format RaiderReporter::Cucumber::Formatter --out raider-results/cucumber.log
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
`--out` is required here. Cucumber rejects a second formatter on the same output stream ("All but one formatter must use --out", also in 9.2.1), and this formatter writes nothing to its stream, so give it any file path (it is never created). If both modes are active, only one listener is registered.
|
|
61
|
+
|
|
62
|
+
### How tests are identified
|
|
63
|
+
|
|
64
|
+
- `file_path` is relative to the project root (`features/login.feature`), even when Cucumber is given `./features/login.feature` or an absolute path.
|
|
65
|
+
- `full_name` is `"<Feature name>: <Scenario name>"`. Scenario Outline rows are `"<Feature name>: <Scenario name> (<Examples name or 'Examples'>, row <n>)"`, where `n` is the 1-based data row of that Examples table. When two or more Examples tables of one outline share a label (for example two unnamed ones), every table with that label is numbered by its position among them, `(Examples 1, row 1)`, `(Examples 2, row 1)`, so each row keeps a distinct name; tables with a unique label are unchanged.
|
|
66
|
+
- `identifier` is `"<file_path>:<line>"`: the scenario line, or the Examples row line for outline rows.
|
|
67
|
+
- `tags` are the feature, scenario and Examples tags, merged, sorted, unique, without `@`.
|
|
68
|
+
|
|
69
|
+
Statuses: passed, failed and ambiguous (reported as failed), pending and undefined (reported as pending), skipped.
|
|
70
|
+
|
|
71
|
+
### Failures
|
|
72
|
+
|
|
73
|
+
A failed test reports the step that failed, not the whole scenario:
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
When I fail on purpose (features/login.feature:9)
|
|
77
|
+
|
|
78
|
+
expected 5, got 4
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
A failing `Before` or `After` hook is named `Before hook` or `After hook` in place of the step, so an `After` hook that raises after every step passed is still a failed test. Messages are capped at 10,000 characters and backtraces at 50 lines.
|
|
82
|
+
|
|
83
|
+
### Retries
|
|
84
|
+
|
|
85
|
+
With `--retry n`, only the final attempt of each scenario is reported, and `attempts` is the number of attempts. Each attempt starts with fresh attachments; attachments from earlier attempts are dropped.
|
|
86
|
+
|
|
87
|
+
### Attachments
|
|
88
|
+
|
|
89
|
+
Both of these are captured while a scenario runs:
|
|
90
|
+
|
|
91
|
+
```ruby
|
|
92
|
+
After do |scenario|
|
|
93
|
+
attach(path, "image/png") if scenario.failed? # Cucumber's own attach, with an existing file path (a Pathname works too)
|
|
94
|
+
RaiderReporter.attach("screenshot", path, type: :png) # or the Raider helper, as in RSpec
|
|
95
|
+
end
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Cucumber's `attach` is captured only when given the path of an existing file. Inline data (`attach("some text", "text/plain")`) and `log` messages are ignored.
|
|
99
|
+
|
|
100
|
+
### Parallel runs with Cucumber
|
|
101
|
+
|
|
102
|
+
`parallel_tests` works with no configuration for Cucumber too: each process writes `run-<n>.json`. See [Parallel runs](#parallel-runs).
|
|
30
103
|
|
|
31
104
|
## Attaching screenshots and videos
|
|
32
105
|
|
|
@@ -183,6 +256,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
|
|
|
183
256
|
## Examples
|
|
184
257
|
|
|
185
258
|
- `examples/rspec_basic` is a runnable RSpec project with a passing, a failing (with an attachment) and a skipped example. Run it with `cd examples/rspec_basic && bundle install && bundle exec rspec`; the failure is deliberate.
|
|
259
|
+
- `examples/cucumber_basic` is the same for Cucumber (plugin mode): a passing scenario, a failing one with an attachment, and an outline. Run it with `cd examples/cucumber_basic && bundle install && bundle exec cucumber`; the failure is deliberate.
|
|
186
260
|
- `examples/github_actions.yml` shows the workflow steps for CI.
|
|
187
261
|
|
|
188
262
|
## Schema
|
|
@@ -191,7 +265,7 @@ Tip: run `rm -rf raider-results` before a parallel run. Each process only cleans
|
|
|
191
265
|
|
|
192
266
|
## Contributing
|
|
193
267
|
|
|
194
|
-
Bug reports and pull requests are welcome at https://github.com/RaiderHQ/raider_reporter. Run `bundle install && bundle exec rake` to run the specs.
|
|
268
|
+
Bug reports and pull requests are welcome at https://github.com/RaiderHQ/raider_reporter. Run `bundle install && bundle exec rake` to run the specs. To test against another Cucumber, set `CUCUMBER_VERSION` (for example `CUCUMBER_VERSION="~> 9.0" bundle update cucumber && bundle exec rspec`).
|
|
195
269
|
|
|
196
270
|
## License
|
|
197
271
|
|
|
@@ -55,8 +55,18 @@ module RaiderReporter
|
|
|
55
55
|
end
|
|
56
56
|
|
|
57
57
|
def finish_test(**fields)
|
|
58
|
+
add_result(build_test(**fields))
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Closes the current test and returns its result without recording it, so a
|
|
62
|
+
# framework that retries can hold one result per test and replace it per attempt.
|
|
63
|
+
def build_test(**fields)
|
|
58
64
|
result = TestResult.new(**fields, attachments: @pending_attachments || [])
|
|
59
65
|
@pending_attachments = nil
|
|
66
|
+
result
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def add_result(result)
|
|
60
70
|
@results << result
|
|
61
71
|
result
|
|
62
72
|
end
|
|
@@ -0,0 +1,258 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "pathname"
|
|
4
|
+
require "cucumber"
|
|
5
|
+
require "cucumber/formatter/ast_lookup"
|
|
6
|
+
require "raider_reporter"
|
|
7
|
+
require "raider_reporter/text"
|
|
8
|
+
|
|
9
|
+
module RaiderReporter
|
|
10
|
+
module Cucumber
|
|
11
|
+
# Translates Cucumber events into Collector calls. Every handler is guarded:
|
|
12
|
+
# a reporting bug must never fail or crash the user's run. The formatter
|
|
13
|
+
# writes nothing to its IO; the report goes to RaiderReporter's own writers.
|
|
14
|
+
#
|
|
15
|
+
# Inside this namespace `Cucumber` means RaiderReporter::Cucumber, so the
|
|
16
|
+
# gem's constants are always written ::Cucumber.
|
|
17
|
+
class Formatter
|
|
18
|
+
STATUSES = {
|
|
19
|
+
passed: "passed", failed: "failed", ambiguous: "failed",
|
|
20
|
+
pending: "pending", undefined: "pending", skipped: "skipped"
|
|
21
|
+
}.freeze
|
|
22
|
+
|
|
23
|
+
# Passed to the World's `attach`; reports an existing file as an attachment.
|
|
24
|
+
module AttachCapture
|
|
25
|
+
# Takes whatever Cucumber passes (file, media_type, filename, and any
|
|
26
|
+
# argument a later release adds) and forwards it untouched.
|
|
27
|
+
ruby2_keywords def attach(*args, &block)
|
|
28
|
+
result = super
|
|
29
|
+
Formatter.record_attachment(args[0], args[1])
|
|
30
|
+
result
|
|
31
|
+
end
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
@claimed = []
|
|
35
|
+
|
|
36
|
+
class << self
|
|
37
|
+
# One listener per run, whether it arrives as a --format or as a plugin.
|
|
38
|
+
# Cucumber copies the configuration for --out, but the event bus is shared.
|
|
39
|
+
def claim(config)
|
|
40
|
+
bus = config.event_bus
|
|
41
|
+
return false if @claimed.any? { |claimed| claimed.equal?(bus) }
|
|
42
|
+
|
|
43
|
+
@claimed << bus
|
|
44
|
+
true
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# Inline text, IO objects and missing paths are ignored: only a file that
|
|
48
|
+
# exists can be copied. Never raises into the user's step.
|
|
49
|
+
def record_attachment(file, media_type)
|
|
50
|
+
# `log` is attach with this media type; a logged message is not a file.
|
|
51
|
+
return if media_type.to_s.start_with?("text/x.cucumber.log")
|
|
52
|
+
|
|
53
|
+
file = file.to_path if file.respond_to?(:to_path)
|
|
54
|
+
return unless file.is_a?(String) && RaiderReporter.collector&.in_test? && File.file?(file)
|
|
55
|
+
|
|
56
|
+
RaiderReporter.attach(File.basename(file), file, type: media_type)
|
|
57
|
+
rescue StandardError => e
|
|
58
|
+
RaiderReporter.warn("#{e.class}: #{e.message}")
|
|
59
|
+
end
|
|
60
|
+
|
|
61
|
+
# Called when the run ends, so the list does not outlive the run.
|
|
62
|
+
def release(config)
|
|
63
|
+
bus = config.event_bus
|
|
64
|
+
@claimed.reject! { |claimed| claimed.equal?(bus) }
|
|
65
|
+
RaiderReporter.dry_run = false
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
# Cucumber hands steps a World, not the formatter, so `attach` is observed
|
|
69
|
+
# by prepending to ProtoWorld. Installed once, at the first formatter.
|
|
70
|
+
# Event-bus alternatives were checked in cucumber 9.2.1, 10.2.0 and 11.1.1:
|
|
71
|
+
# there is no :attach_called event before 11, and even there (and in the
|
|
72
|
+
# :envelope attachment message) a file path has already been read into
|
|
73
|
+
# bytes with no name, so the basename the report needs is only available
|
|
74
|
+
# in ProtoWorld#attach, which exists unchanged in all three versions.
|
|
75
|
+
def capture_attach
|
|
76
|
+
return if @capturing
|
|
77
|
+
|
|
78
|
+
::Cucumber::Glue::ProtoWorld.prepend(AttachCapture)
|
|
79
|
+
@capturing = true
|
|
80
|
+
end
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
def initialize(config)
|
|
84
|
+
@buffer = {}
|
|
85
|
+
@attempts = Hash.new(0)
|
|
86
|
+
@attempt = nil
|
|
87
|
+
guard { setup(config) if self.class.claim(config) }
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
private
|
|
91
|
+
|
|
92
|
+
def setup(config)
|
|
93
|
+
@config = config
|
|
94
|
+
@ast = ::Cucumber::Formatter::AstLookup.new(config)
|
|
95
|
+
@collector = RaiderReporter.start_collector!
|
|
96
|
+
RaiderReporter.dry_run = config.dry_run?
|
|
97
|
+
config.on_event(:test_run_started) { |_event| guard { @collector.start_run } }
|
|
98
|
+
config.on_event(:test_case_started) { |event| guard { case_started(event.test_case) } }
|
|
99
|
+
config.on_event(:test_step_finished) { |event| guard { step_finished(event.test_step, event.result) } }
|
|
100
|
+
config.on_event(:test_case_finished) { |event| guard { case_finished(event.test_case, event.result) } }
|
|
101
|
+
config.on_event(:test_run_finished) { |_event| guard { run_finished } }
|
|
102
|
+
# Last, and on its own: a broken attach hook may only lose attachments.
|
|
103
|
+
safely(nil) { self.class.capture_attach }
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# Every attempt starts with an empty attachment list.
|
|
107
|
+
def case_started(test_case)
|
|
108
|
+
@collector.start_test
|
|
109
|
+
@attempt = { test_case: test_case }
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
# The first non-passed step explains why the scenario did not pass.
|
|
113
|
+
def step_finished(test_step, result)
|
|
114
|
+
return unless @attempt
|
|
115
|
+
return if @attempt[:problem] || result.passed?
|
|
116
|
+
|
|
117
|
+
@attempt[:problem] = { step: test_step, result: result }
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
def case_finished(test_case, result)
|
|
121
|
+
problem = @attempt && @attempt[:problem]
|
|
122
|
+
@attempt = nil
|
|
123
|
+
status = STATUSES.fetch(result.to_sym, "pending")
|
|
124
|
+
# Always buffer: a lookup problem may lose detail, never the scenario.
|
|
125
|
+
failure = status == "failed" ? safely(nil) { failure_for(problem, result, test_case) } : nil
|
|
126
|
+
fields = identity_for(test_case)
|
|
127
|
+
key = "#{test_case.location}|#{test_case.name}"
|
|
128
|
+
@attempts[key] += 1
|
|
129
|
+
# The newest attempt replaces the older one, taking its attachments with it.
|
|
130
|
+
@buffer[key] = @collector.build_test(
|
|
131
|
+
**fields, status: status, duration_ms: duration_ms(result), attempts: @attempts[key], failure: failure
|
|
132
|
+
)
|
|
133
|
+
end
|
|
134
|
+
|
|
135
|
+
def run_finished
|
|
136
|
+
@buffer.each_value { |result| @collector.add_result(result) }
|
|
137
|
+
@buffer.clear
|
|
138
|
+
RaiderReporter.finish!($stderr)
|
|
139
|
+
ensure
|
|
140
|
+
self.class.release(@config)
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
def identity_for(test_case)
|
|
144
|
+
file_path = relative_path(test_case.location.file)
|
|
145
|
+
line_number = test_case.location.lines.max
|
|
146
|
+
{
|
|
147
|
+
file_path: file_path,
|
|
148
|
+
line_number: line_number,
|
|
149
|
+
full_name: Text.utf8(full_name(test_case)),
|
|
150
|
+
identifier: "#{file_path}:#{line_number}",
|
|
151
|
+
tags: test_case.tags.map { |tag| tag.name.to_s.delete_prefix("@") }.uniq.sort
|
|
152
|
+
}
|
|
153
|
+
end
|
|
154
|
+
|
|
155
|
+
def full_name(test_case)
|
|
156
|
+
safely(test_case.name) do
|
|
157
|
+
feature = @ast.gherkin_document(test_case.location.file)&.feature
|
|
158
|
+
name = scenario_name(@ast.scenario_source(test_case))
|
|
159
|
+
next test_case.name unless name
|
|
160
|
+
|
|
161
|
+
feature ? "#{feature.name}: #{name}" : name
|
|
162
|
+
end
|
|
163
|
+
end
|
|
164
|
+
|
|
165
|
+
# nil when the scenario cannot be located in the AST.
|
|
166
|
+
def scenario_name(source)
|
|
167
|
+
case source&.type
|
|
168
|
+
when :ScenarioOutline
|
|
169
|
+
examples = source.examples
|
|
170
|
+
row = examples.table_body.index { |candidate| candidate.location.line == source.row.location.line }
|
|
171
|
+
row && "#{source.scenario_outline.name} (#{examples_label(source)}, row #{row + 1})"
|
|
172
|
+
when :Scenario
|
|
173
|
+
source.scenario.name
|
|
174
|
+
end
|
|
175
|
+
end
|
|
176
|
+
|
|
177
|
+
# "Examples" or the table's name. When several tables of one outline share a
|
|
178
|
+
# label (two unnamed ones, say), each gets its 1-based position among them,
|
|
179
|
+
# so rows of different tables never collide on full_name.
|
|
180
|
+
def examples_label(source)
|
|
181
|
+
tables = source.scenario_outline.examples
|
|
182
|
+
label = examples_name(source.examples)
|
|
183
|
+
same = tables.select { |table| examples_name(table) == label }
|
|
184
|
+
return label if same.size < 2
|
|
185
|
+
|
|
186
|
+
"#{label} #{same.index { |table| table.equal?(source.examples) || table.location.line == source.examples.location.line } + 1}"
|
|
187
|
+
end
|
|
188
|
+
|
|
189
|
+
def examples_name(table)
|
|
190
|
+
table.name.to_s.strip.empty? ? "Examples" : table.name.strip
|
|
191
|
+
end
|
|
192
|
+
|
|
193
|
+
# step_info is { step:, result: } for the first non-passed step, if any.
|
|
194
|
+
def failure_for(step_info, result, test_case)
|
|
195
|
+
exception = result.respond_to?(:exception) ? result.exception : nil
|
|
196
|
+
exception ||= step_info && step_info[:result].respond_to?(:exception) ? step_info[:result].exception : nil
|
|
197
|
+
return nil unless exception
|
|
198
|
+
|
|
199
|
+
Failure.new(
|
|
200
|
+
message: Text.utf8("#{step_label(step_info, test_case)}\n\n#{exception.message}")[0, Text::MAX_MESSAGE],
|
|
201
|
+
exception_class: Text.utf8(exception.class.name),
|
|
202
|
+
backtrace: (exception.backtrace || []).first(Text::MAX_BACKTRACE).map { |line| Text.utf8(line) }
|
|
203
|
+
)
|
|
204
|
+
end
|
|
205
|
+
|
|
206
|
+
def step_label(step_info, test_case)
|
|
207
|
+
return "Step" unless step_info
|
|
208
|
+
|
|
209
|
+
test_step = step_info[:step]
|
|
210
|
+
location = "(#{relative_path(test_step.location.file)}:#{test_step.location.lines.min})"
|
|
211
|
+
if test_step.hook?
|
|
212
|
+
"#{hook_kind(test_step, test_case)} hook #{location}"
|
|
213
|
+
else
|
|
214
|
+
"#{step_keyword(test_step)}#{test_step.text} #{location}"
|
|
215
|
+
end
|
|
216
|
+
end
|
|
217
|
+
|
|
218
|
+
def step_keyword(test_step)
|
|
219
|
+
safely("") { @ast.step_source(test_step).step.keyword.to_s }
|
|
220
|
+
end
|
|
221
|
+
|
|
222
|
+
# A hook with a real step before it runs after the scenario started.
|
|
223
|
+
def hook_kind(test_step, test_case)
|
|
224
|
+
steps = test_case.test_steps
|
|
225
|
+
index = steps.index { |candidate| candidate.equal?(test_step) }
|
|
226
|
+
index && steps.first(index).any? { |candidate| !candidate.hook? } ? "After" : "Before"
|
|
227
|
+
end
|
|
228
|
+
|
|
229
|
+
def duration_ms(result)
|
|
230
|
+
(result.duration.nanoseconds / 1_000_000.0).round
|
|
231
|
+
rescue StandardError
|
|
232
|
+
0
|
|
233
|
+
end
|
|
234
|
+
|
|
235
|
+
def relative_path(path)
|
|
236
|
+
absolute = Pathname.new(File.expand_path(path.to_s, Dir.pwd))
|
|
237
|
+
relative = absolute.relative_path_from(Pathname.new(Dir.pwd)).to_s
|
|
238
|
+
relative.start_with?("..") ? path.to_s.delete_prefix("./") : relative
|
|
239
|
+
rescue ArgumentError
|
|
240
|
+
# Windows: a path on another drive has no relative form.
|
|
241
|
+
path.to_s.delete_prefix("./")
|
|
242
|
+
end
|
|
243
|
+
|
|
244
|
+
def safely(fallback)
|
|
245
|
+
yield
|
|
246
|
+
rescue StandardError => e
|
|
247
|
+
RaiderReporter.warn("#{e.class}: #{e.message}")
|
|
248
|
+
fallback
|
|
249
|
+
end
|
|
250
|
+
|
|
251
|
+
def guard
|
|
252
|
+
yield
|
|
253
|
+
rescue StandardError => e
|
|
254
|
+
RaiderReporter.warn("#{e.class}: #{e.message}")
|
|
255
|
+
end
|
|
256
|
+
end
|
|
257
|
+
end
|
|
258
|
+
end
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
begin
|
|
4
|
+
require "cucumber"
|
|
5
|
+
rescue LoadError => e
|
|
6
|
+
raise unless e.path == "cucumber"
|
|
7
|
+
|
|
8
|
+
raise LoadError, "raider_reporter/cucumber needs the cucumber gem (>= 9, < 12) in your Gemfile"
|
|
9
|
+
end
|
|
10
|
+
require "raider_reporter"
|
|
11
|
+
require_relative "cucumber/formatter"
|
|
12
|
+
|
|
13
|
+
module RaiderReporter
|
|
14
|
+
module Cucumber
|
|
15
|
+
# Plugin mode: requiring this file from features/support registers the
|
|
16
|
+
# listener through Cucumber's InstallPlugin hook, so no --format is needed
|
|
17
|
+
# (and none of the "one formatter per stream" juggling). Outside a Cucumber
|
|
18
|
+
# run there is no step-definition registry and this is a no-op.
|
|
19
|
+
def self.install_plugin
|
|
20
|
+
# Cucumber's public InstallPlugin DSL method, called on a throwaway object
|
|
21
|
+
# (it is only an instance method of Dsl; main is extended with it).
|
|
22
|
+
Object.new.extend(::Cucumber::Glue::Dsl).InstallPlugin(name: "raider_reporter") do |config, *|
|
|
23
|
+
RaiderReporter::Cucumber::Formatter.new(config)
|
|
24
|
+
end
|
|
25
|
+
true
|
|
26
|
+
rescue StandardError => e
|
|
27
|
+
# Outside a run there is no registry (Dsl has no rb_language) and nothing to
|
|
28
|
+
# report. Inside one, losing the listener must not be silent.
|
|
29
|
+
RaiderReporter.warn("could not register the Cucumber plugin: #{e.class}: #{e.message}") if run_active?
|
|
30
|
+
false
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def self.run_active?
|
|
34
|
+
!::Cucumber::Glue::Dsl.instance_variable_get(:@rb_language).nil?
|
|
35
|
+
end
|
|
36
|
+
private_class_method :run_active?
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
RaiderReporter::Cucumber.install_plugin
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
require "rspec/core"
|
|
4
4
|
require "raider_reporter"
|
|
5
|
+
require "raider_reporter/text"
|
|
5
6
|
|
|
6
7
|
module RaiderReporter
|
|
7
8
|
module RSpec
|
|
@@ -13,8 +14,8 @@ module RaiderReporter
|
|
|
13
14
|
:example_failed, :example_pending, :close
|
|
14
15
|
|
|
15
16
|
NON_TAG_KEYS = %i[pending skip].freeze
|
|
16
|
-
MAX_MESSAGE =
|
|
17
|
-
MAX_BACKTRACE =
|
|
17
|
+
MAX_MESSAGE = Text::MAX_MESSAGE
|
|
18
|
+
MAX_BACKTRACE = Text::MAX_BACKTRACE
|
|
18
19
|
|
|
19
20
|
def initialize(output)
|
|
20
21
|
@output = output
|
|
@@ -77,14 +78,8 @@ module RaiderReporter
|
|
|
77
78
|
)
|
|
78
79
|
end
|
|
79
80
|
|
|
80
|
-
# JSON generation raises on invalid UTF-8, which would lose the whole run.
|
|
81
81
|
def utf8(value)
|
|
82
|
-
|
|
83
|
-
if [Encoding::UTF_8, Encoding::BINARY, Encoding::US_ASCII].include?(str.encoding)
|
|
84
|
-
str.dup.force_encoding(Encoding::UTF_8).scrub("?")
|
|
85
|
-
else
|
|
86
|
-
str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "?")
|
|
87
|
-
end
|
|
82
|
+
Text.utf8(value)
|
|
88
83
|
end
|
|
89
84
|
|
|
90
85
|
def guard
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module RaiderReporter
|
|
4
|
+
# Text limits and cleaning shared by the framework formatters.
|
|
5
|
+
module Text
|
|
6
|
+
MAX_MESSAGE = 10_000
|
|
7
|
+
MAX_BACKTRACE = 50
|
|
8
|
+
|
|
9
|
+
# JSON generation raises on invalid UTF-8, which would lose the whole run.
|
|
10
|
+
def self.utf8(value)
|
|
11
|
+
str = value.to_s
|
|
12
|
+
if [Encoding::UTF_8, Encoding::BINARY, Encoding::US_ASCII].include?(str.encoding)
|
|
13
|
+
str.dup.force_encoding(Encoding::UTF_8).scrub("?")
|
|
14
|
+
else
|
|
15
|
+
str.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: "?")
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
data/lib/raider_reporter.rb
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
require "time"
|
|
4
4
|
require "rbconfig"
|
|
5
5
|
require_relative "raider_reporter/version"
|
|
6
|
+
require_relative "raider_reporter/text"
|
|
6
7
|
require_relative "raider_reporter/models"
|
|
7
8
|
require_relative "raider_reporter/payload"
|
|
8
9
|
require_relative "raider_reporter/configuration"
|
|
@@ -27,13 +28,18 @@ module RaiderReporter
|
|
|
27
28
|
yield configuration
|
|
28
29
|
end
|
|
29
30
|
|
|
31
|
+
# Set by the Cucumber formatter when the run is a --dry-run.
|
|
32
|
+
attr_writer :dry_run
|
|
33
|
+
|
|
30
34
|
def reset!
|
|
35
|
+
@dry_run = false
|
|
31
36
|
@collector&.cleanup
|
|
32
37
|
@configuration = nil
|
|
33
38
|
@collector = nil
|
|
34
39
|
end
|
|
35
40
|
|
|
36
41
|
def start_collector!
|
|
42
|
+
@dry_run = false
|
|
37
43
|
@collector = Collector.new(
|
|
38
44
|
max_attachment_bytes: configuration.max_attachment_bytes,
|
|
39
45
|
warn: ->(message) { warn(message) }
|
|
@@ -104,8 +110,10 @@ module RaiderReporter
|
|
|
104
110
|
"Raider report uploaded"
|
|
105
111
|
end
|
|
106
112
|
|
|
107
|
-
# `rspec --dry-run`
|
|
113
|
+
# `rspec --dry-run` and `cucumber --dry-run` report every example as passed without running it.
|
|
108
114
|
def dry_run?
|
|
115
|
+
return true if @dry_run
|
|
116
|
+
|
|
109
117
|
defined?(::RSpec.configuration) && ::RSpec.configuration.respond_to?(:dry_run?) &&
|
|
110
118
|
::RSpec.configuration.dry_run?
|
|
111
119
|
end
|
metadata
CHANGED
|
@@ -1,16 +1,17 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: raider_reporter
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.4.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Augustin Gottlieb
|
|
8
8
|
bindir: exe
|
|
9
9
|
cert_chain: []
|
|
10
|
-
date:
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
11
|
dependencies: []
|
|
12
|
-
description: RSpec
|
|
13
|
-
schema,
|
|
12
|
+
description: RSpec and Cucumber formatters that record results, screenshots and videos
|
|
13
|
+
in the Raider schema, write them locally, and optionally upload them to a Raider
|
|
14
|
+
server.
|
|
14
15
|
email:
|
|
15
16
|
- 33221555+aguspe@users.noreply.github.com
|
|
16
17
|
executables:
|
|
@@ -30,12 +31,15 @@ files:
|
|
|
30
31
|
- lib/raider_reporter/comment/summary_client.rb
|
|
31
32
|
- lib/raider_reporter/configuration.rb
|
|
32
33
|
- lib/raider_reporter/content_types.rb
|
|
34
|
+
- lib/raider_reporter/cucumber.rb
|
|
35
|
+
- lib/raider_reporter/cucumber/formatter.rb
|
|
33
36
|
- lib/raider_reporter/github_event.rb
|
|
34
37
|
- lib/raider_reporter/models.rb
|
|
35
38
|
- lib/raider_reporter/payload.rb
|
|
36
39
|
- lib/raider_reporter/rspec.rb
|
|
37
40
|
- lib/raider_reporter/rspec/formatter.rb
|
|
38
41
|
- lib/raider_reporter/shard_detector.rb
|
|
42
|
+
- lib/raider_reporter/text.rb
|
|
39
43
|
- lib/raider_reporter/version.rb
|
|
40
44
|
- lib/raider_reporter/writers/local.rb
|
|
41
45
|
- lib/raider_reporter/writers/uploader.rb
|
|
@@ -60,7 +64,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
60
64
|
- !ruby/object:Gem::Version
|
|
61
65
|
version: '0'
|
|
62
66
|
requirements: []
|
|
63
|
-
rubygems_version:
|
|
67
|
+
rubygems_version: 4.0.6
|
|
64
68
|
specification_version: 4
|
|
65
|
-
summary:
|
|
69
|
+
summary: RSpec and Cucumber formatters that write Raider reports locally and upload
|
|
70
|
+
them to Raider
|
|
66
71
|
test_files: []
|