buildkite-test_collector 1.1.0 → 1.1.3

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0f136ef927e80dbc0b442d01addc8f66041a79824508ea19158071594e78f7b2
4
- data.tar.gz: 92136cc0d244357f96e5372af85b8923678cde70eece3582b6c02437e3a06146
3
+ metadata.gz: 84d13e5fb256a07aa96a3f19b17ee2f27c7c362d4cbbaa9e7f0efb6d4af8420d
4
+ data.tar.gz: e5fe4f34fb6fc3a0c7494e65e6b43a9ffe9dd094a8f589ad3efa85e2b7e7849f
5
5
  SHA512:
6
- metadata.gz: 3f53e8887efbb7f8ee62df7693c8061950dc77b1168e8abe784352a6e168938a8ee530c739746722b26955f22547c7bddb809e1345633afe06a9605bd2cfad2c
7
- data.tar.gz: 42e3568a83bb4d37dbb815fb27320f121af273c60d968a8fa4f75688708b3994f91dce02203e1a45ebf67630e27288665029e38b591f50e349e792b0e881b112
6
+ metadata.gz: 0cfff8a4aef5369cd83075425b233c109f92bb55c790daea12d9bea9d6ce75e6860d3e12c922dae9b676cf7791f12d41f2d17d463a4d561f888ba3df0fca9bc9
7
+ data.tar.gz: 7b01fdf26eed8c739685a78b135db2ea0ef964810b435a2897b0f1c61bb5e75ff695d344df5d9c89d1d04d37188f6125e1062c590d748ef7bec80a1790a7f15f
data/CHANGELOG.md CHANGED
@@ -1,5 +1,26 @@
1
1
  # CHANGELOG
2
2
 
3
+ ## v1.1.3
4
+ - Use a private reference to JSON.parse to prevent it being mocked #149 - @ghcan
5
+
6
+ ## v1.1.2
7
+
8
+ - Remove branch reference prefix in Github Actions #136 - @gchan
9
+ - Make ruby collector work better for non-web app #128 - @JuanitoFatas
10
+ - Avoid linters complaining about double quotes #137 - @SocalNick
11
+ - Revert "Allow specifying run name prefix and suffix" #139 - @JuanitoFatas
12
+ - Update Code of Conduct contact email address #143 - @JuanitoFatas
13
+ - Suppress errors when connecting to Buildkite #145 - @swebb
14
+ - Add some design documentation to understand how the gem works #146 - @swebb
15
+ - Fix logging level from debug to error #147 - @gchan
16
+
17
+ ## v1.1.1
18
+
19
+ - Strip CR/LF from token input #127 - @gchan
20
+ - Allow specify run name prefix and suffix #130 - @juanitofatas
21
+ - Improve Minitest support by using after_teardown #131 - @davidstosik
22
+ - Avoid breaking test suite stubs `Process.clock_gettime` #132 - @juanitofatas (thanks @ChrisBR)
23
+
3
24
  ## v1.1.0
4
25
 
5
26
  - Remove an internal debugging change #115 — @juanitofatas
data/CODE_OF_CONDUCT.md CHANGED
@@ -61,7 +61,7 @@ representative at an online or offline event.
61
61
 
62
62
  Instances of abusive, harassing, or otherwise unacceptable behavior may be
63
63
  reported to the community leaders responsible for enforcement at
64
- `support+analytics@buildkite.com`.
64
+ [coc@buildkite.com](mailto:coc@buildkite.com).
65
65
  All complaints will be reviewed and investigated promptly and fairly.
66
66
 
67
67
  All community leaders are obligated to respect the privacy and security of the
data/DESIGN.md ADDED
@@ -0,0 +1,19 @@
1
+ # Design
2
+
3
+ ## Threads
4
+
5
+ The Buildkite ruby collector uses websockets and ActionCable to send and
6
+ receive data with Buildkite. Execution information starts transmitting as soon
7
+ as possible, without waiting for the test suite to finish running.
8
+
9
+ This gem uses 3 ruby threads:
10
+
11
+ * main thread: acts as the producer. It collects span data from the
12
+ test suite and enqueues it into the send queue.
13
+ * write thread: acts as the consumer. Removes data from the send queue and
14
+ sends it to Buildkite.
15
+ * read thread: receives and processes messages from Buildkite.
16
+
17
+ ## Data
18
+
19
+ Trace data is stored in spans. See [Buildkite::TestCollector::Tracer](lib/buildkite/test_collector/tracer.rb) for more information.
data/Gemfile.lock CHANGED
@@ -1,7 +1,7 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- buildkite-test_collector (1.1.0)
4
+ buildkite-test_collector (1.1.1)
5
5
  activesupport (>= 5.2, < 8)
6
6
  websocket (~> 1.2)
7
7
 
@@ -17,7 +17,7 @@ GEM
17
17
  diff-lcs (1.4.4)
18
18
  i18n (1.10.0)
19
19
  concurrent-ruby (~> 1.0)
20
- minitest (5.15.0)
20
+ minitest (5.16.1)
21
21
  rake (13.0.6)
22
22
  rspec (3.10.0)
23
23
  rspec-core (~> 3.10.0)
data/README.md CHANGED
@@ -22,7 +22,7 @@ Or add this to your Gemfile’s test group:
22
22
 
23
23
  ```ruby
24
24
  group :test do
25
- gem "buildkite-test_collector"
25
+ gem 'buildkite-test_collector'
26
26
  end
27
27
  ```
28
28
 
@@ -34,7 +34,7 @@ Add the following code to your RSpec setup file:
34
34
 
35
35
  ```ruby
36
36
  # spec/spec_helper.rb
37
- require "buildkite/test_collector"
37
+ require 'buildkite/test_collector'
38
38
  Buildkite::TestCollector.configure(hook: :rspec)
39
39
  ```
40
40
 
@@ -50,7 +50,7 @@ Add the following code to your Minitest setup file:
50
50
 
51
51
  ```ruby
52
52
  # test/test_helper.rb
53
- require "buildkite/test_collector"
53
+ require 'buildkite/test_collector'
54
54
  Buildkite::TestCollector.configure(hook: :minitest)
55
55
  ```
56
56
 
@@ -94,6 +94,8 @@ bundle exec rspec
94
94
 
95
95
  Useful resources for developing collectors include the [Buildkite Test Analytics docs](https://buildkite.com/docs/test-analytics).
96
96
 
97
+ See [DESIGN.md](DESIGN.md) for an overview of the design of this gem.
98
+
97
99
  ## 👩‍💻 Contributing
98
100
 
99
101
  Bug reports and pull requests are welcome on GitHub at https://github.com/buildkite/test-collector-ruby
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "securerandom"
4
-
5
3
  class Buildkite::TestCollector::CI
6
4
  def self.env
7
5
  new.env
@@ -67,7 +65,7 @@ class Buildkite::TestCollector::CI
67
65
  "CI" => "github_actions",
68
66
  "key" => "#{ENV["GITHUB_ACTION"]}-#{ENV["GITHUB_RUN_NUMBER"]}-#{ENV["GITHUB_RUN_ATTEMPT"]}",
69
67
  "url" => File.join("https://github.com", ENV["GITHUB_REPOSITORY"], "actions/runs", ENV["GITHUB_RUN_ID"]),
70
- "branch" => ENV["GITHUB_REF"],
68
+ "branch" => ENV["GITHUB_REF_NAME"],
71
69
  "commit_sha" => ENV["GITHUB_SHA"],
72
70
  "number" => ENV["GITHUB_RUN_NUMBER"],
73
71
  }
@@ -0,0 +1,4 @@
1
+ module Buildkite::TestCollector
2
+ class Error < StandardError; end
3
+ class TimeoutError < ::Timeout::Error; end
4
+ end
@@ -2,7 +2,6 @@
2
2
 
3
3
  require "minitest"
4
4
 
5
- require_relative "../uploader"
6
5
  require_relative "../minitest_plugin"
7
6
 
8
7
  Buildkite::TestCollector.uploader = Buildkite::TestCollector::Uploader
@@ -13,4 +12,4 @@ end
13
12
 
14
13
  Buildkite::TestCollector.enable_tracing!
15
14
 
16
- Buildkite::TestCollector::Uploader.configure
15
+ Buildkite::TestCollector.safe { Buildkite::TestCollector::Uploader.configure }
@@ -3,7 +3,6 @@
3
3
  require "rspec/core"
4
4
  require "rspec/expectations"
5
5
 
6
- require_relative "../uploader"
7
6
  require_relative "../rspec_plugin/reporter"
8
7
  require_relative "../rspec_plugin/trace"
9
8
 
@@ -13,7 +12,7 @@ RSpec.configure do |config|
13
12
  config.before(:suite) do
14
13
  config.add_formatter Buildkite::TestCollector::RSpecPlugin::Reporter
15
14
 
16
- Buildkite::TestCollector::Uploader.configure
15
+ Buildkite::TestCollector.safe { Buildkite::TestCollector::Uploader.configure }
17
16
  end
18
17
 
19
18
  config.around(:each) do |example|
@@ -1,8 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "logger"
4
- require "time"
5
-
6
3
  module Buildkite::TestCollector
7
4
  class Logger < ::Logger
8
5
  class Formatter < ::Logger::Formatter
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "active_support/core_ext/hash/indifferent_access"
4
-
5
3
  module Buildkite::TestCollector::MinitestPlugin
6
4
  class Trace
7
5
  attr_accessor :example
@@ -1,5 +1,8 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ # Minitest finds this file before setup code
4
+ require_relative "tracer"
5
+
3
6
  require_relative "minitest_plugin/reporter"
4
7
  require_relative "minitest_plugin/trace"
5
8
 
@@ -12,9 +15,7 @@ module Buildkite::TestCollector::MinitestPlugin
12
15
  Thread.current[:_buildkite_tracer] = tracer
13
16
  end
14
17
 
15
- def before_teardown
16
- super
17
-
18
+ def after_teardown
18
19
  tracer = Thread.current[:_buildkite_tracer]
19
20
  if !tracer.nil?
20
21
  Thread.current[:_buildkite_tracer] = nil
@@ -23,5 +24,7 @@ module Buildkite::TestCollector::MinitestPlugin
23
24
  trace = Buildkite::TestCollector::MinitestPlugin::Trace.new(self, history: tracer.history)
24
25
  Buildkite::TestCollector.uploader.traces[trace.source_location] = trace
25
26
  end
27
+
28
+ super
26
29
  end
27
30
  end
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "time"
4
-
5
3
  module Buildkite::TestCollector::RSpecPlugin
6
4
  class Reporter
7
5
  RSpec::Core::Formatters.register self, :example_passed, :example_failed, :example_pending, :dump_summary
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "active_support/core_ext/hash/indifferent_access"
4
-
5
3
  module Buildkite::TestCollector::RSpecPlugin
6
4
  class Trace
7
5
  attr_accessor :example, :failure_reason, :failure_expanded
@@ -1,7 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require_relative "socket_connection"
4
-
5
3
  module Buildkite::TestCollector
6
4
  class Session
7
5
  # Picked 75 as the magic timeout number as it's longer than the TCP timeout of 60s 🤷‍♀️
@@ -9,15 +7,19 @@ module Buildkite::TestCollector
9
7
  MAX_RECONNECTION_ATTEMPTS = ENV.fetch("BUILDKITE_ANALYTICS_RECONNECTION_ATTEMPTS") { 3 }.to_i
10
8
  WAIT_BETWEEN_RECONNECTIONS = ENV.fetch("BUILDKITE_ANALYTICS_RECONNECTION_WAIT") { 5 }.to_i
11
9
 
10
+ # We keep a private reference so that mocking libraries won't break JSON
11
+ JSON_PARSE = JSON.method(:parse)
12
+ private_constant :JSON_PARSE
13
+
12
14
  class RejectedSubscription < StandardError; end
13
15
  class InitialConnectionFailure < StandardError; end
14
16
 
15
17
  DISCONNECTED_EXCEPTIONS = [
16
- SocketConnection::HandshakeError,
18
+ Buildkite::TestCollector::SocketConnection::HandshakeError,
19
+ Buildkite::TestCollector::TimeoutError,
20
+ Buildkite::TestCollector::SocketConnection::SocketError,
17
21
  RejectedSubscription,
18
- TimeoutError,
19
22
  InitialConnectionFailure,
20
- SocketConnection::SocketError
21
23
  ]
22
24
 
23
25
  def initialize(url, authorization_header, channel)
@@ -41,7 +43,7 @@ module Buildkite::TestCollector
41
43
  begin
42
44
  reconnection_count += 1
43
45
  connect
44
- rescue TimeoutError, InitialConnectionFailure => e
46
+ rescue Buildkite::TestCollector::TimeoutError, InitialConnectionFailure => e
45
47
  Buildkite::TestCollector.logger.warn("buildkite-test_collector could not establish an initial connection with Buildkite due to #{e}. Attempting retry #{reconnection_count} of #{MAX_RECONNECTION_ATTEMPTS}...")
46
48
  if reconnection_count > MAX_RECONNECTION_ATTEMPTS
47
49
  Buildkite::TestCollector.logger.error "buildkite-test_collector could not establish an initial connection with Buildkite due to #{e.message} after #{MAX_RECONNECTION_ATTEMPTS} attempts. You may be missing some data for this test suite, please contact support if this issue persists."
@@ -122,7 +124,7 @@ module Buildkite::TestCollector
122
124
  end
123
125
 
124
126
  def handle(_connection, data)
125
- data = JSON.parse(data)
127
+ data = JSON_PARSE.call(data)
126
128
  case data["type"]
127
129
  when "ping"
128
130
  # In absence of other message, the server sends us a ping every 3 seconds
@@ -1,9 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "socket"
4
- require "openssl"
5
- require "json"
6
-
7
3
  module Buildkite::TestCollector
8
4
  class SocketConnection
9
5
  class HandshakeError < StandardError; end
@@ -65,7 +61,7 @@ module Buildkite::TestCollector
65
61
 
66
62
  # Setting up a new thread that listens on the socket, and processes incoming
67
63
  # comms from the server
68
- @thread = Thread.new do
64
+ @read_thread = Thread.new do
69
65
  Buildkite::TestCollector.logger.debug("listening in on socket")
70
66
  frame = WebSocket::Frame::Incoming::Client.new
71
67
 
@@ -82,7 +78,7 @@ module Buildkite::TestCollector
82
78
  rescue EOFError => e
83
79
  Buildkite::TestCollector.logger.warn("#{e}")
84
80
  if @socket
85
- Buildkite::TestCollector.logger.warn("attempting disconnected flow")
81
+ Buildkite::TestCollector.logger.error("attempting disconnected flow")
86
82
  @session.disconnected(self)
87
83
  disconnect
88
84
  end
@@ -151,7 +147,7 @@ module Buildkite::TestCollector
151
147
  socket = @socket
152
148
  @socket = nil
153
149
  socket&.close
154
- @thread&.join unless @thread == Thread.current
150
+ @read_thread&.join unless @read_thread == Thread.current
155
151
  end
156
152
  end
157
153
  end
@@ -1,9 +1,29 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "active_support/core_ext/hash/indifferent_access"
4
-
5
3
  module Buildkite::TestCollector
4
+ # Traces the execution of an application by creating and storing spans of information.
5
+ #
6
+ # This class contains two data structures:
7
+ #
8
+ # - A stack (called @stack) that traces the entering & leaving of each part of the application.
9
+ # - A tree made up of many Span nodes. Each Span is a node in the tree. Each
10
+ # span is also stored in the stack. The root of the tree is called @top and
11
+ # is stored at @stack[0].
12
+ #
13
+ # When the trace is complete the stack MUST contain a single node @top, which
14
+ # is the root of the tree (see #finalize). The tree is converted into a hash
15
+ # in #as_json which recursively calls #as_json on all of it's children.
6
16
  class Tracer
17
+ # https://github.com/buildkite/test-collector-ruby/issues/131
18
+ class MonotonicTime
19
+ GET_TIME = Process.method(:clock_gettime)
20
+ private_constant :GET_TIME
21
+
22
+ def self.call
23
+ GET_TIME.call Process::CLOCK_MONOTONIC
24
+ end
25
+ end
26
+
7
27
  class Span
8
28
  attr_accessor :section, :start_at, :end_at, :detail, :children
9
29
 
@@ -28,23 +48,23 @@ module Buildkite::TestCollector
28
48
  end
29
49
 
30
50
  def initialize
31
- @top = Span.new(:top, Concurrent.monotonic_time, nil, {})
51
+ @top = Span.new(:top, MonotonicTime.call, nil, {})
32
52
  @stack = [@top]
33
53
  end
34
54
 
35
55
  def enter(section, **detail)
36
- new_entry = Span.new(section, Concurrent.monotonic_time, nil, detail)
56
+ new_entry = Span.new(section, MonotonicTime.call, nil, detail)
37
57
  current_span.children << new_entry
38
58
  @stack << new_entry
39
59
  end
40
60
 
41
61
  def leave
42
- current_span.end_at = Concurrent.monotonic_time
62
+ current_span.end_at = MonotonicTime.call
43
63
  @stack.pop
44
64
  end
45
65
 
46
66
  def backfill(section, duration, **detail)
47
- new_entry = Span.new(section, Concurrent.monotonic_time - duration, Concurrent.monotonic_time, detail)
67
+ new_entry = Span.new(section, MonotonicTime.call - duration, MonotonicTime.call, detail)
48
68
  current_span.children << new_entry
49
69
  end
50
70
 
@@ -54,7 +74,7 @@ module Buildkite::TestCollector
54
74
 
55
75
  def finalize
56
76
  raise "Stack not empty" unless @stack.size == 1
57
- @top.end_at = Concurrent.monotonic_time
77
+ @top.end_at = MonotonicTime.call
58
78
  self
59
79
  end
60
80
 
@@ -1,20 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "openssl"
4
- require "websocket"
5
-
6
- require_relative "tracer"
7
- require_relative "network"
8
- require_relative "object"
9
- require_relative "session"
10
- require_relative "ci"
11
- require_relative "http_client"
12
-
13
- require "active_support"
14
- require "active_support/notifications"
15
-
16
- require "securerandom"
17
-
18
3
  module Buildkite::TestCollector
19
4
  class Uploader
20
5
  def self.traces
@@ -2,7 +2,7 @@
2
2
 
3
3
  module Buildkite
4
4
  module TestCollector
5
- VERSION = "1.1.0"
5
+ VERSION = "1.1.3"
6
6
  NAME = "buildkite-test_collector"
7
7
  end
8
8
  end
@@ -1,16 +1,39 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ module Buildkite
4
+ module TestCollector
5
+ end
6
+ end
7
+
8
+ require "json"
9
+ require "logger"
10
+ require "net/http"
11
+ require "openssl"
12
+ require "time"
3
13
  require "timeout"
4
14
  require "tmpdir"
15
+ require "securerandom"
16
+ require "socket"
17
+ require "websocket"
18
+
19
+ require "active_support/core_ext/object/blank"
20
+ require "active_support/core_ext/hash/indifferent_access"
21
+ require "active_support/notifications"
5
22
 
6
23
  require_relative "test_collector/version"
24
+ require_relative "test_collector/error"
7
25
  require_relative "test_collector/logger"
26
+ require_relative "test_collector/ci"
27
+ require_relative "test_collector/http_client"
28
+ require_relative "test_collector/uploader"
29
+ require_relative "test_collector/network"
30
+ require_relative "test_collector/object"
31
+ require_relative "test_collector/tracer"
32
+ require_relative "test_collector/socket_connection"
33
+ require_relative "test_collector/session"
8
34
 
9
35
  module Buildkite
10
36
  module TestCollector
11
- class Error < StandardError; end
12
- class TimeoutError < ::Timeout::Error; end
13
-
14
37
  DEFAULT_URL = "https://analytics-api.buildkite.com/v1/uploads"
15
38
 
16
39
  class << self
@@ -23,7 +46,7 @@ module Buildkite
23
46
  end
24
47
 
25
48
  def self.configure(hook:, token: nil, url: nil, debug_enabled: false, tracing_enabled: true)
26
- self.api_token = token || ENV["BUILDKITE_ANALYTICS_TOKEN"]
49
+ self.api_token = (token || ENV["BUILDKITE_ANALYTICS_TOKEN"])&.strip
27
50
  self.url = url || DEFAULT_URL
28
51
  self.debug_enabled = debug_enabled || !!(ENV["BUILDKITE_ANALYTICS_DEBUG_ENABLED"])
29
52
  self.tracing_enabled = tracing_enabled
@@ -78,5 +101,12 @@ module Buildkite
78
101
  Buildkite::TestCollector::Uploader.tracer&.backfill(:sql, finish - start, **{ query: payload[:sql] })
79
102
  end
80
103
  end
104
+
105
+ def self.safe(&block)
106
+ block.call
107
+ rescue StandardError => e
108
+ logger.error("Buildkite::TestCollector received exception: #{e}")
109
+ logger.error("Backtrace:\n#{e.backtrace.join("\n")}")
110
+ end
81
111
  end
82
112
  end
@@ -1,3 +1,9 @@
1
+ # frozen_string_literal: true
2
+
3
+ # Minitest finds this file before setup code
4
+ require_relative "../buildkite/test_collector"
5
+ require_relative "../buildkite/test_collector/minitest_plugin/reporter"
6
+
1
7
  module Minitest
2
8
  def self.plugin_buildkite_collector_init(options)
3
9
  if defined?(Buildkite::TestCollector::MinitestPlugin) && Buildkite::TestCollector.respond_to?(:uploader)
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: buildkite-test_collector
3
3
  version: !ruby/object:Gem::Version
4
- version: 1.1.0
4
+ version: 1.1.3
5
5
  platform: ruby
6
6
  authors:
7
7
  - Buildkite
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-06-21 00:00:00.000000000 Z
11
+ date: 2022-08-10 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: activesupport
@@ -84,6 +84,7 @@ files:
84
84
  - CHANGELOG.md
85
85
  - CODEOWNERS
86
86
  - CODE_OF_CONDUCT.md
87
+ - DESIGN.md
87
88
  - Gemfile
88
89
  - Gemfile.lock
89
90
  - LICENSE.txt
@@ -95,6 +96,7 @@ files:
95
96
  - buildkite.yaml
96
97
  - lib/buildkite/test_collector.rb
97
98
  - lib/buildkite/test_collector/ci.rb
99
+ - lib/buildkite/test_collector/error.rb
98
100
  - lib/buildkite/test_collector/http_client.rb
99
101
  - lib/buildkite/test_collector/library_hooks/minitest.rb
100
102
  - lib/buildkite/test_collector/library_hooks/rspec.rb
@@ -133,7 +135,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
133
135
  - !ruby/object:Gem::Version
134
136
  version: '0'
135
137
  requirements: []
136
- rubygems_version: 3.3.3
138
+ rubygems_version: 3.1.6
137
139
  signing_key:
138
140
  specification_version: 4
139
141
  summary: Track test executions and report to Buildkite Test Analytics