rspec_logfmt_formatter 0.0.1 → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 211c7cbe090b13a9762b825d0cfeb2aecb4e43f2d44037147efd158883144565
4
- data.tar.gz: 46c20ba3d51965bdd6f8ceee5bdf4a3f14d9efaf3af5efb73f154ac675a55057
3
+ metadata.gz: 4df1ac2b2c23ebc734fa5d044e6016723b201fed2f1e8c2b184e9b64f9798ec4
4
+ data.tar.gz: 7386ec904d072ca6ec3ab5d8e2fbec2d86d2ee76103718c5fbc19fbb2f063ea0
5
5
  SHA512:
6
- metadata.gz: 46ae8acd77714be862c4da0455cb5c553dbcc1fb05747e158f633323aa5246d46e72154ec0a346e042b735a5b1d645503a0ce5103242997116205ecf2bb86cf6
7
- data.tar.gz: a95b1e04e094ae0067baeacffa3cce3a34a8097014c9066ebedbc6575999d22a957d0c7484981c73c3bb11bdc478134ea179f5386b6d357a6a430528e9fb1c67
6
+ metadata.gz: 8981e2e7bd01f6ce1d8423c2ff07f9130e38cc5d36f6e31540005cbf20a58dd504a1af3cfc3ccb647055a65160cc46bc96093592118e1ea86116bb40f6d75ab2
7
+ data.tar.gz: e6dd81dcc72434330c668166eb9307ade3827787667bf67f5c6232cee445eec4a5944ab4d310a463297680456ee72b5bcf7fef03f528bd3dcfefa96a9522034b
data/README.md CHANGED
@@ -1,8 +1,9 @@
1
1
  # RSpec logfmt Formatter
2
2
 
3
3
  [![CircleCI](https://circleci.com/gh/carwow/rspec_logfmt_formatter/tree/master.svg?style=svg)](https://circleci.com/gh/carwow/rspec_logfmt_formatter/tree/master)
4
+ [![Gem Version](https://badge.fury.io/rb/rspec_logfmt_formatter.svg)](https://badge.fury.io/rb/rspec_logfmt_formatter)
4
5
 
5
- [RSpec][rspec] 3 results in the [logfmt][logfmt] format that you can push to you observability tool such as [Honeycomb][honeycomb]
6
+ [RSpec][rspec] 3 results in the [logfmt][logfmt] format that you can push to your observability tool such as [Honeycomb][honeycomb]
6
7
 
7
8
  [rspec]: http://rspec.info/
8
9
  [logfmt]: https://brandur.org/logfmt
@@ -11,19 +12,15 @@
11
12
  ## Usage
12
13
 
13
14
  Install the gem:
14
- <details>
15
- <summary>WIP</summary>
16
15
 
17
16
  ```sh
18
17
  gem install rspec_logfmt_formatter
19
18
  ```
20
19
 
21
- </details>
22
-
23
20
  Use it:
24
21
 
25
22
  ```sh
26
- rspec --format RspecLogfmtFormatter::Formatter --out rspec.txt
23
+ rspec --format RspecLogfmtFormatter --out rspec.txt
27
24
  ```
28
25
 
29
26
  You'll get a file `rspec.txt` with your results in it.
@@ -31,7 +28,7 @@ You'll get a file `rspec.txt` with your results in it.
31
28
  You can use it in combination with other [formatters][rspec-formatters], too:
32
29
 
33
30
  ```sh
34
- rspec --format progress --format RspecLogfmtFormatter::Formatter --out rspec.txt
31
+ rspec --format progress --format RspecLogfmtFormatter --out rspec.txt
35
32
  ```
36
33
 
37
34
  [rspec-formatters]: https://relishapp.com/rspec/rspec-core/v/3-6/docs/formatters
@@ -50,32 +47,121 @@ end
50
47
  Put the same arguments as the commands above in [your `.rspec`][rspec-file]:
51
48
 
52
49
  ```sh
53
- --format RspecLogfmtFormatter::Formatter
54
- --out rspec.txt
50
+ --format RspecLogfmtFormatter --out rspec.txt
55
51
  ```
56
52
  [bundler]: https://bundler.io
57
53
  [rspec-file]: https://relishapp.com/rspec/rspec-core/v/3-6/docs/configuration/read-command-line-configuration-options-from-files
58
54
 
55
+ ### Output
56
+
57
+ Each run writes one field per line:
58
+
59
+ ```
60
+ tests.slowest_file="./spec/models/deal_spec.rb"
61
+ tests.slowest_file_ms="48213"
62
+ tests.count="6"
63
+ tests.skipped="1"
64
+ tests.failures="3"
65
+ tests.errors="0"
66
+ tests.retries="1"
67
+ ```
68
+
69
+ `tests.retries` is only populated when using [rspec-retry][rspec-retry].
70
+
71
+ [rspec-retry]: https://github.com/NoRedInk/rspec-retry
72
+
73
+ When running in parallel, every field is prefixed with where it ran, so each
74
+ worker's results stay distinct in a shared file:
75
+
76
+ * the parallel worker, from `TEST_ENV_NUMBER` (set by [flatware][flatware] or [parallel_tests][parallel_tests]).
77
+ An empty `TEST_ENV_NUMBER`, which parallel_tests uses for its first worker, is treated as `1`
78
+ * the CircleCI container, from `CIRCLE_NODE_INDEX`, joined to the worker as `<node>-<worker>`.
79
+ It is only used together with `TEST_ENV_NUMBER`
80
+
81
+ ```
82
+ tests.3.count="6" # TEST_ENV_NUMBER=3
83
+ tests.1-3.count="6" # CIRCLE_NODE_INDEX=1 TEST_ENV_NUMBER=3
84
+ ```
85
+
86
+ [flatware]: https://github.com/briandunn/flatware
87
+ [parallel_tests]: https://github.com/grosser/parallel_tests
88
+
89
+ ### Slow spec files
90
+
91
+ The formatter times each top-level example group (wall clock, including
92
+ `before(:all)` hooks) and sums them per spec file. It reports the slowest one:
93
+
94
+ * `tests.slowest_file`: the path of the slowest spec file
95
+ * `tests.slowest_file_ms`: its duration in milliseconds
96
+
97
+ These fields are omitted when no example groups ran.
98
+
59
99
  ### Parallel tests
60
100
 
61
- For use with `parallel_tests`, add `$TEST_ENV_NUMBER` in the output file option (in `.rspec` or `.rspec_parallel`) to avoid concurrent process write conflicts.
101
+ You can combine the output of several runs in one file using `rspec_logfmt_formatter_collate rspec.txt`.
102
+ `tests.count`, `tests.errors`, `tests.failures`, `tests.retries` and
103
+ `tests.skipped` are summed, and the slowest file is the slowest across all runs,
104
+ reported with its own duration. The worker and node prefixes are dropped. Each
105
+ CircleCI container collates its own file, so with `parallelism` each
106
+ container's span carries its own results; combine them in Honeycomb (e.g.
107
+ `MAX(tests.slowest_file_ms)` per trace).
108
+
109
+ It will rewrite a file like:
62
110
 
63
- ```sh
64
- --format RspecLogfmtFormatter::Formatter
65
- --out tmp/rspec<%= ENV["TEST_ENV_NUMBER"] %>.txt
66
111
  ```
112
+ tests.1.slowest_file="./spec/a_spec.rb"
113
+ tests.1.slowest_file_ms="900"
114
+ tests.1.count="3"
115
+ tests.1.skipped="1"
116
+ tests.1.failures="0"
117
+ tests.1.errors="0"
118
+ tests.1.retries="0"
119
+ tests.2.slowest_file="./spec/b_spec.rb"
120
+ tests.2.slowest_file_ms="1500"
121
+ tests.2.count="1"
122
+ tests.2.skipped="0"
123
+ tests.2.failures="1"
124
+ tests.2.errors="0"
125
+ tests.2.retries="0"
126
+ ```
127
+
128
+ into:
129
+
130
+ ```
131
+ tests.count="4"
132
+ tests.errors="0"
133
+ tests.failures="1"
134
+ tests.retries="0"
135
+ tests.skipped="1"
136
+ tests.slowest_file="./spec/b_spec.rb"
137
+ tests.slowest_file_ms="1500"
138
+ ```
139
+
140
+ `tests.slowest_file` and `tests.slowest_file_ms` are left out when no run reported a slowest file.
67
141
 
68
142
  ### How to push to Honeycomb
69
143
 
70
- TODO
144
+ There are many ways you can push this to honeycomb. One way is to use the [buildevents][buildevents] binary. Then output your spec output to `BUILDEVENT_FILE` e.g.
145
+
146
+ ```sh
147
+ bundle exec rspec --format RspecLogfmtFormatter --out $BUILDEVENT_FILE
148
+ ```
149
+
150
+ [buildevents]: https://github.com/honeycombio/buildevents
71
151
 
72
152
  ## Development
73
153
 
74
- TODO
154
+ Pull the repo and run `bundle install`, the specs are run with `bundle exec rspec` and make sure to read the [CONTRIBUTING.md](./CONTRIBUTING.md).
75
155
 
76
156
  ## Releasing
77
157
 
78
- TODO
158
+ Currently this is a manual process and can only be run by a carwow developer.
159
+
160
+ 1. Ensure your `~/.gem/credentials` are correct
161
+ 2. Update the [CHANGELOG.md](./CHANGELOG.md)
162
+ 3. Update the version in the [gemspec](./rspec_logfmt_formatter.gemspec)
163
+ 4. Run `bundle exec rake upload`
164
+ 5. Check the tag was pushed and the new version was released
79
165
 
80
166
  ## License
81
167
 
@@ -0,0 +1,10 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "rspec_logfmt_formatter"
4
+
5
+ file = ARGV.first
6
+ out = -> { file ? File.new(file, "w") : $stdout }
7
+
8
+ ARGF.read
9
+ .then { |input| RspecLogfmtFormatter.collate input }
10
+ .then { |collated| out.call.puts collated }
@@ -1,2 +1,107 @@
1
- require 'rspec_logfmt_formatter/formatter'
2
- require 'rspec_logfmt_formatter/version'
1
+ require "rspec/core"
2
+ require "rspec/core/formatters/base_formatter"
3
+
4
+ class RspecLogfmtFormatter < RSpec::Core::Formatters::BaseFormatter
5
+ RSpec::Core::Formatters.register self,
6
+ :dump_summary,
7
+ :example_group_started,
8
+ :example_group_finished
9
+
10
+ COUNTERS = %w[count errors failures retries skipped].freeze
11
+ SLOWEST_FIELDS = %w[slowest_file slowest_file_ms].freeze
12
+
13
+ def self.collate(input)
14
+ totals = COUNTERS.to_h { |counter| [counter, 0] }
15
+ pending = Hash.new { |hash, worker| hash[worker] = {} }
16
+ slowest = []
17
+
18
+ input.scan(/tests((?:\.[\w-]+)?)\.(\w+)="([^"]+)"/) do |worker, field, value|
19
+ if totals.key?(field)
20
+ totals[field] += value.to_i
21
+ elsif SLOWEST_FIELDS.include?(field)
22
+ # Pair a worker's file and duration in whichever order they arrive
23
+ pair = pending[worker]
24
+ pair[field] = value
25
+ next unless pair.size == 2
26
+
27
+ pending.delete(worker)
28
+ slowest << [pair["slowest_file"], pair["slowest_file_ms"].to_i]
29
+ end
30
+ end
31
+
32
+ file, ms = slowest.max_by(&:last) # first wins on ties
33
+ fields = totals.map { |counter, total| %(tests.#{counter}="#{total}") }
34
+ fields.push(%(tests.slowest_file="#{file}"), %(tests.slowest_file_ms="#{ms}")) if file
35
+ fields.join("\n")
36
+ end
37
+
38
+ def initialize(...)
39
+ super
40
+ @retries = []
41
+ @file_durations_ms = Hash.new(0)
42
+ end
43
+
44
+ def example_group_started(notification)
45
+ return unless notification.group.top_level?
46
+
47
+ @group_started_at = RSpec::Core::Time.now
48
+ end
49
+
50
+ def example_group_finished(notification)
51
+ group = notification.group
52
+
53
+ return unless group.top_level? && @group_started_at
54
+
55
+ elapsed_ms = ((RSpec::Core::Time.now - @group_started_at) * 1000).round
56
+
57
+ @file_durations_ms[group.metadata[:file_path]] += elapsed_ms
58
+ @group_started_at = nil
59
+ end
60
+
61
+ def dump_summary(notification)
62
+ @summary = notification
63
+ logfmt_dump
64
+ end
65
+
66
+ def retry(example)
67
+ @retries << example
68
+ end
69
+
70
+ private
71
+
72
+ def logfmt_dump
73
+ output.puts [
74
+ slow_file_fields,
75
+ %(#{prefix}.count="#{@summary.example_count}"),
76
+ %(#{prefix}.skipped="#{@summary.pending_count}"),
77
+ %(#{prefix}.failures="#{@summary.failure_count}"),
78
+ %(#{prefix}.errors="#{@summary.errors_outside_of_examples_count}"),
79
+ %(#{prefix}.retries="#{@retries.count}")
80
+ ].join("\n")
81
+ end
82
+
83
+ def slow_file_fields
84
+ return [] if @file_durations_ms.empty?
85
+
86
+ slowest_file, slowest_ms = @file_durations_ms.max_by { |_file, duration| duration }
87
+
88
+ [
89
+ %(#{prefix}.slowest_file="#{slowest_file}"),
90
+ %(#{prefix}.slowest_file_ms="#{slowest_ms}")
91
+ ]
92
+ end
93
+
94
+ def prefix
95
+ @prefix ||= begin
96
+ worker = ENV["TEST_ENV_NUMBER"]
97
+ if worker
98
+ # parallel_tests leaves TEST_ENV_NUMBER empty for the first worker
99
+ worker = "1" if worker.empty?
100
+ node = ENV["CIRCLE_NODE_INDEX"]
101
+ node ? "tests.#{node}-#{worker}" : "tests.#{worker}"
102
+ else
103
+ "tests"
104
+ end
105
+ end
106
+ end
107
+ end
metadata CHANGED
@@ -1,106 +1,21 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rspec_logfmt_formatter
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Richard Vickerstaff
8
- autorequire:
9
- bindir: bin
8
+ bindir: exe
10
9
  cert_chain: []
11
- date: 2022-01-19 00:00:00.000000000 Z
10
+ date: 2026-11-18 00:00:00.000000000 Z
12
11
  dependencies:
13
- - !ruby/object:Gem::Dependency
14
- name: bundler
15
- requirement: !ruby/object:Gem::Requirement
16
- requirements:
17
- - - ">="
18
- - !ruby/object:Gem::Version
19
- version: '0'
20
- type: :development
21
- prerelease: false
22
- version_requirements: !ruby/object:Gem::Requirement
23
- requirements:
24
- - - ">="
25
- - !ruby/object:Gem::Version
26
- version: '0'
27
- - !ruby/object:Gem::Dependency
28
- name: carwow_rubocop
29
- requirement: !ruby/object:Gem::Requirement
30
- requirements:
31
- - - '='
32
- - !ruby/object:Gem::Version
33
- version: 3.4.4
34
- type: :development
35
- prerelease: false
36
- version_requirements: !ruby/object:Gem::Requirement
37
- requirements:
38
- - - '='
39
- - !ruby/object:Gem::Version
40
- version: 3.4.4
41
- - !ruby/object:Gem::Dependency
42
- name: pry
43
- requirement: !ruby/object:Gem::Requirement
44
- requirements:
45
- - - ">="
46
- - !ruby/object:Gem::Version
47
- version: '0'
48
- type: :development
49
- prerelease: false
50
- version_requirements: !ruby/object:Gem::Requirement
51
- requirements:
52
- - - ">="
53
- - !ruby/object:Gem::Version
54
- version: '0'
55
- - !ruby/object:Gem::Dependency
56
- name: rake
57
- requirement: !ruby/object:Gem::Requirement
58
- requirements:
59
- - - ">="
60
- - !ruby/object:Gem::Version
61
- version: '0'
62
- type: :development
63
- prerelease: false
64
- version_requirements: !ruby/object:Gem::Requirement
65
- requirements:
66
- - - ">="
67
- - !ruby/object:Gem::Version
68
- version: '0'
69
- - !ruby/object:Gem::Dependency
70
- name: rspec_junit_formatter
71
- requirement: !ruby/object:Gem::Requirement
72
- requirements:
73
- - - ">="
74
- - !ruby/object:Gem::Version
75
- version: '0'
76
- type: :development
77
- prerelease: false
78
- version_requirements: !ruby/object:Gem::Requirement
79
- requirements:
80
- - - ">="
81
- - !ruby/object:Gem::Version
82
- version: '0'
83
- - !ruby/object:Gem::Dependency
84
- name: rspec-retry
85
- requirement: !ruby/object:Gem::Requirement
86
- requirements:
87
- - - ">="
88
- - !ruby/object:Gem::Version
89
- version: '0'
90
- type: :development
91
- prerelease: false
92
- version_requirements: !ruby/object:Gem::Requirement
93
- requirements:
94
- - - ">="
95
- - !ruby/object:Gem::Version
96
- version: '0'
97
12
  - !ruby/object:Gem::Dependency
98
13
  name: rspec-core
99
14
  requirement: !ruby/object:Gem::Requirement
100
15
  requirements:
101
16
  - - ">="
102
17
  - !ruby/object:Gem::Version
103
- version: '2'
18
+ version: '3.6'
104
19
  - - "<"
105
20
  - !ruby/object:Gem::Version
106
21
  version: '4'
@@ -110,27 +25,26 @@ dependencies:
110
25
  requirements:
111
26
  - - ">="
112
27
  - !ruby/object:Gem::Version
113
- version: '2'
28
+ version: '3.6'
114
29
  - - "<"
115
30
  - !ruby/object:Gem::Version
116
31
  version: '4'
117
32
  description: ''
118
33
  email:
119
34
  - developers@carwow.co.uk
120
- executables: []
35
+ executables:
36
+ - rspec_logfmt_formatter_collate
121
37
  extensions: []
122
38
  extra_rdoc_files: []
123
39
  files:
124
40
  - LICENSE
125
41
  - README.md
42
+ - exe/rspec_logfmt_formatter_collate
126
43
  - lib/rspec_logfmt_formatter.rb
127
- - lib/rspec_logfmt_formatter/formatter.rb
128
- - lib/rspec_logfmt_formatter/version.rb
129
44
  homepage: https://github.com/carwow/rspec_logfmt_formatter
130
45
  licenses:
131
46
  - MIT
132
47
  metadata: {}
133
- post_install_message:
134
48
  rdoc_options: []
135
49
  require_paths:
136
50
  - lib
@@ -138,15 +52,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
138
52
  requirements:
139
53
  - - ">="
140
54
  - !ruby/object:Gem::Version
141
- version: 2.7.0
55
+ version: '3.2'
142
56
  required_rubygems_version: !ruby/object:Gem::Requirement
143
57
  requirements:
144
58
  - - ">="
145
59
  - !ruby/object:Gem::Version
146
- version: 2.0.0
60
+ version: '0'
147
61
  requirements: []
148
- rubygems_version: 3.1.4
149
- signing_key:
62
+ rubygems_version: 3.6.2
150
63
  specification_version: 4
151
64
  summary: RSpec formatter that honeycomb can read
152
65
  test_files: []
@@ -1,72 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require 'rspec/core'
4
- require 'rspec/core/formatters/base_formatter'
5
-
6
- module RspecLogfmtFormatter
7
- class Formatter < RSpec::Core::Formatters::BaseFormatter
8
- RSpec::Core::Formatters.register self,
9
- :start,
10
- :stop,
11
- :dump_summary
12
-
13
- def start(notification)
14
- @start_notification = notification
15
- @started = Time.now
16
- @retries = []
17
- super
18
- end
19
-
20
- def stop(notification)
21
- @examples_notification = notification
22
- end
23
-
24
- def dump_summary(notification)
25
- @summary_notification = notification
26
- fmt_dump
27
- end
28
-
29
- def retry(example)
30
- @retries << example
31
- end
32
-
33
- private
34
-
35
- attr_reader :started, :retries
36
-
37
- def example_count
38
- @summary_notification.example_count
39
- end
40
-
41
- def pending_count
42
- @summary_notification.pending_count
43
- end
44
-
45
- def failure_count
46
- @summary_notification.failure_count
47
- end
48
-
49
- def duration
50
- @summary_notification.duration
51
- end
52
-
53
- def error_count
54
- if @summary_notification.respond_to?(:errors_outside_of_examples_count)
55
- @summary_notification.errors_outside_of_examples_count
56
- else
57
- 0
58
- end
59
- end
60
-
61
- # rubocop:disable Metrics/AbcSize
62
- def fmt_dump
63
- output << %( tests.name="rspec#{ENV['TEST_ENV_NUMBER']}")
64
- output << %( tests.count="#{example_count}")
65
- output << %( tests.skipped="#{pending_count}")
66
- output << %( tests.failures="#{failure_count}")
67
- output << %( tests.errors="#{error_count}")
68
- output << %( tests.retries.count="#{retries.size}")
69
- end
70
- # rubocop:enable Metrics/AbcSize
71
- end
72
- end
@@ -1,3 +0,0 @@
1
- module RspecLogfmtFormatter
2
- VERSION = '0.0.1'.freeze
3
- end