rspec_logfmt_formatter 0.0.2 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. checksums.yaml +4 -4
  2. data/README.md +74 -4
  3. data/lib/rspec_logfmt_formatter.rb +75 -10
  4. metadata +3 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c2379f5938b480ca7b7f9506060f99a482b07d3144644750b6d976a665f5c774
4
- data.tar.gz: 18029228ddc06f01d38dbe2e89c480b9411ba8e6b4bf2bc474ff359df6256273
3
+ metadata.gz: 4df1ac2b2c23ebc734fa5d044e6016723b201fed2f1e8c2b184e9b64f9798ec4
4
+ data.tar.gz: 7386ec904d072ca6ec3ab5d8e2fbec2d86d2ee76103718c5fbc19fbb2f063ea0
5
5
  SHA512:
6
- metadata.gz: 55f9cede488a1b1ac881bd7e788217bca1f7e5914e28d5dc7a8ea415e923665a5509195cd96065b2ff1a06eb374dbead8c47c87b35fc029da0da0c96cdd3a3c8
7
- data.tar.gz: 05b94d45666d2ac8b9c0955754741db5c3064b9ece162e2f502fed9c7f7b44174d91590f51c469ba5752e0aad865efc044fefe39029deea4097c2b52588ca615
6
+ metadata.gz: 8981e2e7bd01f6ce1d8423c2ff07f9130e38cc5d36f6e31540005cbf20a58dd504a1af3cfc3ccb647055a65160cc46bc96093592118e1ea86116bb40f6d75ab2
7
+ data.tar.gz: e6dd81dcc72434330c668166eb9307ade3827787667bf67f5c6232cee445eec4a5944ab4d310a463297680456ee72b5bcf7fef03f528bd3dcfefa96a9522034b
data/README.md CHANGED
@@ -52,23 +52,93 @@ Put the same arguments as the commands above in [your `.rspec`][rspec-file]:
52
52
  [bundler]: https://bundler.io
53
53
  [rspec-file]: https://relishapp.com/rspec/rspec-core/v/3-6/docs/configuration/read-command-line-configuration-options-from-files
54
54
 
55
+ ### Output
56
+
57
+ Each run writes one field per line:
58
+
59
+ ```
60
+ tests.slowest_file="./spec/models/deal_spec.rb"
61
+ tests.slowest_file_ms="48213"
62
+ tests.count="6"
63
+ tests.skipped="1"
64
+ tests.failures="3"
65
+ tests.errors="0"
66
+ tests.retries="1"
67
+ ```
68
+
69
+ `tests.retries` is only populated when using [rspec-retry][rspec-retry].
70
+
71
+ [rspec-retry]: https://github.com/NoRedInk/rspec-retry
72
+
73
+ When running in parallel, every field is prefixed with where it ran, so each
74
+ worker's results stay distinct in a shared file:
75
+
76
+ * the parallel worker, from `TEST_ENV_NUMBER` (set by [flatware][flatware] or [parallel_tests][parallel_tests]).
77
+ An empty `TEST_ENV_NUMBER`, which parallel_tests uses for its first worker, is treated as `1`
78
+ * the CircleCI container, from `CIRCLE_NODE_INDEX`, joined to the worker as `<node>-<worker>`.
79
+ It is only used together with `TEST_ENV_NUMBER`
80
+
81
+ ```
82
+ tests.3.count="6" # TEST_ENV_NUMBER=3
83
+ tests.1-3.count="6" # CIRCLE_NODE_INDEX=1 TEST_ENV_NUMBER=3
84
+ ```
85
+
86
+ [flatware]: https://github.com/briandunn/flatware
87
+ [parallel_tests]: https://github.com/grosser/parallel_tests
88
+
89
+ ### Slow spec files
90
+
91
+ The formatter times each top-level example group (wall clock, including
92
+ `before(:all)` hooks) and sums them per spec file. It reports the slowest one:
93
+
94
+ * `tests.slowest_file`: the path of the slowest spec file
95
+ * `tests.slowest_file_ms`: its duration in milliseconds
96
+
97
+ These fields are omitted when no example groups ran.
98
+
55
99
  ### Parallel tests
56
100
 
57
- You can sum repeated values in one file using `rspec_logfmt_formatter_collate rspec.txt`.
101
+ You can combine the output of several runs in one file using `rspec_logfmt_formatter_collate rspec.txt`.
102
+ `tests.count`, `tests.errors`, `tests.failures`, `tests.retries` and
103
+ `tests.skipped` are summed, and the slowest file is the slowest across all runs,
104
+ reported with its own duration. The worker and node prefixes are dropped. Each
105
+ CircleCI container collates its own file, so with `parallelism` each
106
+ container's span carries its own results; combine them in Honeycomb (e.g.
107
+ `MAX(tests.slowest_file_ms)` per trace).
58
108
 
59
109
  It will rewrite a file like:
60
110
 
61
111
  ```
62
- tests.count="3"
63
- tests.count="1"
112
+ tests.1.slowest_file="./spec/a_spec.rb"
113
+ tests.1.slowest_file_ms="900"
114
+ tests.1.count="3"
115
+ tests.1.skipped="1"
116
+ tests.1.failures="0"
117
+ tests.1.errors="0"
118
+ tests.1.retries="0"
119
+ tests.2.slowest_file="./spec/b_spec.rb"
120
+ tests.2.slowest_file_ms="1500"
121
+ tests.2.count="1"
122
+ tests.2.skipped="0"
123
+ tests.2.failures="1"
124
+ tests.2.errors="0"
125
+ tests.2.retries="0"
64
126
  ```
65
127
 
66
128
  into:
67
129
 
68
130
  ```
69
131
  tests.count="4"
132
+ tests.errors="0"
133
+ tests.failures="1"
134
+ tests.retries="0"
135
+ tests.skipped="1"
136
+ tests.slowest_file="./spec/b_spec.rb"
137
+ tests.slowest_file_ms="1500"
70
138
  ```
71
139
 
140
+ `tests.slowest_file` and `tests.slowest_file_ms` are left out when no run reported a slowest file.
141
+
72
142
  ### How to push to Honeycomb
73
143
 
74
144
  There are many ways you can push this to honeycomb. One way is to use the [buildevents][buildevents] binary. Then output your spec output to `BUILDEVENT_FILE` e.g.
@@ -89,7 +159,7 @@ Currently this is a manual process and can only be run by a carwow developer.
89
159
 
90
160
  1. Ensure your `~/.gem/credentials` are correct
91
161
  2. Update the [CHANGELOG.md](./CHANGELOG.md)
92
- 3. Update the [version.rb](./lib/rspec_logfmt_formatter/version.rb)
162
+ 3. Update the version in the [gemspec](./rspec_logfmt_formatter.gemspec)
93
163
  4. Run `bundle exec rake upload`
94
164
  5. Check the tag was pushed and the new version was released
95
165
 
@@ -2,21 +2,60 @@ require "rspec/core"
2
2
  require "rspec/core/formatters/base_formatter"
3
3
 
4
4
  class RspecLogfmtFormatter < RSpec::Core::Formatters::BaseFormatter
5
- RSpec::Core::Formatters.register self, :dump_summary
5
+ RSpec::Core::Formatters.register self,
6
+ :dump_summary,
7
+ :example_group_started,
8
+ :example_group_finished
9
+
10
+ COUNTERS = %w[count errors failures retries skipped].freeze
11
+ SLOWEST_FIELDS = %w[slowest_file slowest_file_ms].freeze
6
12
 
7
13
  def self.collate(input)
8
- collated = {}
14
+ totals = COUNTERS.to_h { |counter| [counter, 0] }
15
+ pending = Hash.new { |hash, worker| hash[worker] = {} }
16
+ slowest = []
17
+
18
+ input.scan(/tests((?:\.[\w-]+)?)\.(\w+)="([^"]+)"/) do |worker, field, value|
19
+ if totals.key?(field)
20
+ totals[field] += value.to_i
21
+ elsif SLOWEST_FIELDS.include?(field)
22
+ # Pair a worker's file and duration in whichever order they arrive
23
+ pair = pending[worker]
24
+ pair[field] = value
25
+ next unless pair.size == 2
9
26
 
10
- input.scan(/(tests\.\w+)="(\d+)"/) do |key, value|
11
- collated[key] = collated[key].to_i + value.to_i
27
+ pending.delete(worker)
28
+ slowest << [pair["slowest_file"], pair["slowest_file_ms"].to_i]
29
+ end
12
30
  end
13
31
 
14
- collated.map { |key, value| "#{key}=\"#{value}\"" }.join("\n")
32
+ file, ms = slowest.max_by(&:last) # first wins on ties
33
+ fields = totals.map { |counter, total| %(tests.#{counter}="#{total}") }
34
+ fields.push(%(tests.slowest_file="#{file}"), %(tests.slowest_file_ms="#{ms}")) if file
35
+ fields.join("\n")
15
36
  end
16
37
 
17
38
  def initialize(...)
18
39
  super
19
40
  @retries = []
41
+ @file_durations_ms = Hash.new(0)
42
+ end
43
+
44
+ def example_group_started(notification)
45
+ return unless notification.group.top_level?
46
+
47
+ @group_started_at = RSpec::Core::Time.now
48
+ end
49
+
50
+ def example_group_finished(notification)
51
+ group = notification.group
52
+
53
+ return unless group.top_level? && @group_started_at
54
+
55
+ elapsed_ms = ((RSpec::Core::Time.now - @group_started_at) * 1000).round
56
+
57
+ @file_durations_ms[group.metadata[:file_path]] += elapsed_ms
58
+ @group_started_at = nil
20
59
  end
21
60
 
22
61
  def dump_summary(notification)
@@ -32,11 +71,37 @@ class RspecLogfmtFormatter < RSpec::Core::Formatters::BaseFormatter
32
71
 
33
72
  def logfmt_dump
34
73
  output.puts [
35
- %(tests.count="#{@summary.example_count}"),
36
- %(tests.skipped="#{@summary.pending_count}"),
37
- %(tests.failures="#{@summary.failure_count}"),
38
- %(tests.errors="#{@summary.errors_outside_of_examples_count}"),
39
- %(tests.retries="#{@retries.count}")
74
+ slow_file_fields,
75
+ %(#{prefix}.count="#{@summary.example_count}"),
76
+ %(#{prefix}.skipped="#{@summary.pending_count}"),
77
+ %(#{prefix}.failures="#{@summary.failure_count}"),
78
+ %(#{prefix}.errors="#{@summary.errors_outside_of_examples_count}"),
79
+ %(#{prefix}.retries="#{@retries.count}")
40
80
  ].join("\n")
41
81
  end
82
+
83
+ def slow_file_fields
84
+ return [] if @file_durations_ms.empty?
85
+
86
+ slowest_file, slowest_ms = @file_durations_ms.max_by { |_file, duration| duration }
87
+
88
+ [
89
+ %(#{prefix}.slowest_file="#{slowest_file}"),
90
+ %(#{prefix}.slowest_file_ms="#{slowest_ms}")
91
+ ]
92
+ end
93
+
94
+ def prefix
95
+ @prefix ||= begin
96
+ worker = ENV["TEST_ENV_NUMBER"]
97
+ if worker
98
+ # parallel_tests leaves TEST_ENV_NUMBER empty for the first worker
99
+ worker = "1" if worker.empty?
100
+ node = ENV["CIRCLE_NODE_INDEX"]
101
+ node ? "tests.#{node}-#{worker}" : "tests.#{worker}"
102
+ else
103
+ "tests"
104
+ end
105
+ end
106
+ end
42
107
  end
metadata CHANGED
@@ -1,13 +1,13 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rspec_logfmt_formatter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Richard Vickerstaff
8
8
  bindir: exe
9
9
  cert_chain: []
10
- date: 1980-01-02 00:00:00.000000000 Z
10
+ date: 2026-11-18 00:00:00.000000000 Z
11
11
  dependencies:
12
12
  - !ruby/object:Gem::Dependency
13
13
  name: rspec-core
@@ -59,7 +59,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
59
59
  - !ruby/object:Gem::Version
60
60
  version: '0'
61
61
  requirements: []
62
- rubygems_version: 3.6.7
62
+ rubygems_version: 3.6.2
63
63
  specification_version: 4
64
64
  summary: RSpec formatter that honeycomb can read
65
65
  test_files: []