rspec_logfmt_formatter 0.0.2 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +74 -4
- data/lib/rspec_logfmt_formatter.rb +75 -10
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4df1ac2b2c23ebc734fa5d044e6016723b201fed2f1e8c2b184e9b64f9798ec4
|
|
4
|
+
data.tar.gz: 7386ec904d072ca6ec3ab5d8e2fbec2d86d2ee76103718c5fbc19fbb2f063ea0
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8981e2e7bd01f6ce1d8423c2ff07f9130e38cc5d36f6e31540005cbf20a58dd504a1af3cfc3ccb647055a65160cc46bc96093592118e1ea86116bb40f6d75ab2
|
|
7
|
+
data.tar.gz: e6dd81dcc72434330c668166eb9307ade3827787667bf67f5c6232cee445eec4a5944ab4d310a463297680456ee72b5bcf7fef03f528bd3dcfefa96a9522034b
|
data/README.md
CHANGED
|
@@ -52,23 +52,93 @@ Put the same arguments as the commands above in [your `.rspec`][rspec-file]:
|
|
|
52
52
|
[bundler]: https://bundler.io
|
|
53
53
|
[rspec-file]: https://relishapp.com/rspec/rspec-core/v/3-6/docs/configuration/read-command-line-configuration-options-from-files
|
|
54
54
|
|
|
55
|
+
### Output
|
|
56
|
+
|
|
57
|
+
Each run writes one field per line:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
tests.slowest_file="./spec/models/deal_spec.rb"
|
|
61
|
+
tests.slowest_file_ms="48213"
|
|
62
|
+
tests.count="6"
|
|
63
|
+
tests.skipped="1"
|
|
64
|
+
tests.failures="3"
|
|
65
|
+
tests.errors="0"
|
|
66
|
+
tests.retries="1"
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`tests.retries` is only populated when using [rspec-retry][rspec-retry].
|
|
70
|
+
|
|
71
|
+
[rspec-retry]: https://github.com/NoRedInk/rspec-retry
|
|
72
|
+
|
|
73
|
+
When running in parallel, every field is prefixed with where it ran, so each
|
|
74
|
+
worker's results stay distinct in a shared file:
|
|
75
|
+
|
|
76
|
+
* the parallel worker, from `TEST_ENV_NUMBER` (set by [flatware][flatware] or [parallel_tests][parallel_tests]).
|
|
77
|
+
An empty `TEST_ENV_NUMBER`, which parallel_tests uses for its first worker, is treated as `1`
|
|
78
|
+
* the CircleCI container, from `CIRCLE_NODE_INDEX`, joined to the worker as `<node>-<worker>`.
|
|
79
|
+
It is only used together with `TEST_ENV_NUMBER`
|
|
80
|
+
|
|
81
|
+
```
|
|
82
|
+
tests.3.count="6" # TEST_ENV_NUMBER=3
|
|
83
|
+
tests.1-3.count="6" # CIRCLE_NODE_INDEX=1 TEST_ENV_NUMBER=3
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
[flatware]: https://github.com/briandunn/flatware
|
|
87
|
+
[parallel_tests]: https://github.com/grosser/parallel_tests
|
|
88
|
+
|
|
89
|
+
### Slow spec files
|
|
90
|
+
|
|
91
|
+
The formatter times each top-level example group (wall clock, including
|
|
92
|
+
`before(:all)` hooks) and sums them per spec file. It reports the slowest one:
|
|
93
|
+
|
|
94
|
+
* `tests.slowest_file`: the path of the slowest spec file
|
|
95
|
+
* `tests.slowest_file_ms`: its duration in milliseconds
|
|
96
|
+
|
|
97
|
+
These fields are omitted when no example groups ran.
|
|
98
|
+
|
|
55
99
|
### Parallel tests
|
|
56
100
|
|
|
57
|
-
You can
|
|
101
|
+
You can combine the output of several runs in one file using `rspec_logfmt_formatter_collate rspec.txt`.
|
|
102
|
+
`tests.count`, `tests.errors`, `tests.failures`, `tests.retries` and
|
|
103
|
+
`tests.skipped` are summed, and the slowest file is the slowest across all runs,
|
|
104
|
+
reported with its own duration. The worker and node prefixes are dropped. Each
|
|
105
|
+
CircleCI container collates its own file, so with `parallelism` each
|
|
106
|
+
container's span carries its own results; combine them in Honeycomb (e.g.
|
|
107
|
+
`MAX(tests.slowest_file_ms)` per trace).
|
|
58
108
|
|
|
59
109
|
It will rewrite a file like:
|
|
60
110
|
|
|
61
111
|
```
|
|
62
|
-
tests.
|
|
63
|
-
tests.
|
|
112
|
+
tests.1.slowest_file="./spec/a_spec.rb"
|
|
113
|
+
tests.1.slowest_file_ms="900"
|
|
114
|
+
tests.1.count="3"
|
|
115
|
+
tests.1.skipped="1"
|
|
116
|
+
tests.1.failures="0"
|
|
117
|
+
tests.1.errors="0"
|
|
118
|
+
tests.1.retries="0"
|
|
119
|
+
tests.2.slowest_file="./spec/b_spec.rb"
|
|
120
|
+
tests.2.slowest_file_ms="1500"
|
|
121
|
+
tests.2.count="1"
|
|
122
|
+
tests.2.skipped="0"
|
|
123
|
+
tests.2.failures="1"
|
|
124
|
+
tests.2.errors="0"
|
|
125
|
+
tests.2.retries="0"
|
|
64
126
|
```
|
|
65
127
|
|
|
66
128
|
into:
|
|
67
129
|
|
|
68
130
|
```
|
|
69
131
|
tests.count="4"
|
|
132
|
+
tests.errors="0"
|
|
133
|
+
tests.failures="1"
|
|
134
|
+
tests.retries="0"
|
|
135
|
+
tests.skipped="1"
|
|
136
|
+
tests.slowest_file="./spec/b_spec.rb"
|
|
137
|
+
tests.slowest_file_ms="1500"
|
|
70
138
|
```
|
|
71
139
|
|
|
140
|
+
`tests.slowest_file` and `tests.slowest_file_ms` are left out when no run reported a slowest file.
|
|
141
|
+
|
|
72
142
|
### How to push to Honeycomb
|
|
73
143
|
|
|
74
144
|
There are many ways you can push this to honeycomb. One way is to use the [buildevents][buildevents] binary. Then output your spec output to `BUILDEVENT_FILE` e.g.
|
|
@@ -89,7 +159,7 @@ Currently this is a manual process and can only be run by a carwow developer.
|
|
|
89
159
|
|
|
90
160
|
1. Ensure your `~/.gem/credentials` are correct
|
|
91
161
|
2. Update the [CHANGELOG.md](./CHANGELOG.md)
|
|
92
|
-
3. Update the [
|
|
162
|
+
3. Update the version in the [gemspec](./rspec_logfmt_formatter.gemspec)
|
|
93
163
|
4. Run `bundle exec rake upload`
|
|
94
164
|
5. Check the tag was pushed and the new version was released
|
|
95
165
|
|
|
@@ -2,21 +2,60 @@ require "rspec/core"
|
|
|
2
2
|
require "rspec/core/formatters/base_formatter"
|
|
3
3
|
|
|
4
4
|
class RspecLogfmtFormatter < RSpec::Core::Formatters::BaseFormatter
|
|
5
|
-
RSpec::Core::Formatters.register self,
|
|
5
|
+
RSpec::Core::Formatters.register self,
|
|
6
|
+
:dump_summary,
|
|
7
|
+
:example_group_started,
|
|
8
|
+
:example_group_finished
|
|
9
|
+
|
|
10
|
+
COUNTERS = %w[count errors failures retries skipped].freeze
|
|
11
|
+
SLOWEST_FIELDS = %w[slowest_file slowest_file_ms].freeze
|
|
6
12
|
|
|
7
13
|
def self.collate(input)
|
|
8
|
-
|
|
14
|
+
totals = COUNTERS.to_h { |counter| [counter, 0] }
|
|
15
|
+
pending = Hash.new { |hash, worker| hash[worker] = {} }
|
|
16
|
+
slowest = []
|
|
17
|
+
|
|
18
|
+
input.scan(/tests((?:\.[\w-]+)?)\.(\w+)="([^"]+)"/) do |worker, field, value|
|
|
19
|
+
if totals.key?(field)
|
|
20
|
+
totals[field] += value.to_i
|
|
21
|
+
elsif SLOWEST_FIELDS.include?(field)
|
|
22
|
+
# Pair a worker's file and duration in whichever order they arrive
|
|
23
|
+
pair = pending[worker]
|
|
24
|
+
pair[field] = value
|
|
25
|
+
next unless pair.size == 2
|
|
9
26
|
|
|
10
|
-
|
|
11
|
-
|
|
27
|
+
pending.delete(worker)
|
|
28
|
+
slowest << [pair["slowest_file"], pair["slowest_file_ms"].to_i]
|
|
29
|
+
end
|
|
12
30
|
end
|
|
13
31
|
|
|
14
|
-
|
|
32
|
+
file, ms = slowest.max_by(&:last) # first wins on ties
|
|
33
|
+
fields = totals.map { |counter, total| %(tests.#{counter}="#{total}") }
|
|
34
|
+
fields.push(%(tests.slowest_file="#{file}"), %(tests.slowest_file_ms="#{ms}")) if file
|
|
35
|
+
fields.join("\n")
|
|
15
36
|
end
|
|
16
37
|
|
|
17
38
|
def initialize(...)
|
|
18
39
|
super
|
|
19
40
|
@retries = []
|
|
41
|
+
@file_durations_ms = Hash.new(0)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def example_group_started(notification)
|
|
45
|
+
return unless notification.group.top_level?
|
|
46
|
+
|
|
47
|
+
@group_started_at = RSpec::Core::Time.now
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
def example_group_finished(notification)
|
|
51
|
+
group = notification.group
|
|
52
|
+
|
|
53
|
+
return unless group.top_level? && @group_started_at
|
|
54
|
+
|
|
55
|
+
elapsed_ms = ((RSpec::Core::Time.now - @group_started_at) * 1000).round
|
|
56
|
+
|
|
57
|
+
@file_durations_ms[group.metadata[:file_path]] += elapsed_ms
|
|
58
|
+
@group_started_at = nil
|
|
20
59
|
end
|
|
21
60
|
|
|
22
61
|
def dump_summary(notification)
|
|
@@ -32,11 +71,37 @@ class RspecLogfmtFormatter < RSpec::Core::Formatters::BaseFormatter
|
|
|
32
71
|
|
|
33
72
|
def logfmt_dump
|
|
34
73
|
output.puts [
|
|
35
|
-
|
|
36
|
-
%(
|
|
37
|
-
%(
|
|
38
|
-
%(
|
|
39
|
-
%(
|
|
74
|
+
slow_file_fields,
|
|
75
|
+
%(#{prefix}.count="#{@summary.example_count}"),
|
|
76
|
+
%(#{prefix}.skipped="#{@summary.pending_count}"),
|
|
77
|
+
%(#{prefix}.failures="#{@summary.failure_count}"),
|
|
78
|
+
%(#{prefix}.errors="#{@summary.errors_outside_of_examples_count}"),
|
|
79
|
+
%(#{prefix}.retries="#{@retries.count}")
|
|
40
80
|
].join("\n")
|
|
41
81
|
end
|
|
82
|
+
|
|
83
|
+
def slow_file_fields
|
|
84
|
+
return [] if @file_durations_ms.empty?
|
|
85
|
+
|
|
86
|
+
slowest_file, slowest_ms = @file_durations_ms.max_by { |_file, duration| duration }
|
|
87
|
+
|
|
88
|
+
[
|
|
89
|
+
%(#{prefix}.slowest_file="#{slowest_file}"),
|
|
90
|
+
%(#{prefix}.slowest_file_ms="#{slowest_ms}")
|
|
91
|
+
]
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
def prefix
|
|
95
|
+
@prefix ||= begin
|
|
96
|
+
worker = ENV["TEST_ENV_NUMBER"]
|
|
97
|
+
if worker
|
|
98
|
+
# parallel_tests leaves TEST_ENV_NUMBER empty for the first worker
|
|
99
|
+
worker = "1" if worker.empty?
|
|
100
|
+
node = ENV["CIRCLE_NODE_INDEX"]
|
|
101
|
+
node ? "tests.#{node}-#{worker}" : "tests.#{worker}"
|
|
102
|
+
else
|
|
103
|
+
"tests"
|
|
104
|
+
end
|
|
105
|
+
end
|
|
106
|
+
end
|
|
42
107
|
end
|
metadata
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: rspec_logfmt_formatter
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.0
|
|
4
|
+
version: 0.1.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Richard Vickerstaff
|
|
8
8
|
bindir: exe
|
|
9
9
|
cert_chain: []
|
|
10
|
-
date:
|
|
10
|
+
date: 2026-11-18 00:00:00.000000000 Z
|
|
11
11
|
dependencies:
|
|
12
12
|
- !ruby/object:Gem::Dependency
|
|
13
13
|
name: rspec-core
|
|
@@ -59,7 +59,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
59
59
|
- !ruby/object:Gem::Version
|
|
60
60
|
version: '0'
|
|
61
61
|
requirements: []
|
|
62
|
-
rubygems_version: 3.6.
|
|
62
|
+
rubygems_version: 3.6.2
|
|
63
63
|
specification_version: 4
|
|
64
64
|
summary: RSpec formatter that honeycomb can read
|
|
65
65
|
test_files: []
|