faulty 0.4.0 → 0.7.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 7e68341d29ccefb2a4fa4c265f3f2d5ec37371d37cb41d5a186b3f25d2130b46
4
- data.tar.gz: 56e659d66871738e8818c4874102de04ef97873cb0d1c444d58259c875b21378
3
+ metadata.gz: e07febd816231284cfdc3f2f75e05381a26fb933af81e036c24bff019dcb1f2d
4
+ data.tar.gz: 3804c68b52a5fca7053a94bfb533444a3bb4250fbffe5ab7e966f1246629feaa
5
5
  SHA512:
6
- metadata.gz: 7db7b915923132bd1e5d234245cc5a3adb5988013f4baf8fadc59f89bacfde6828310954f0d6ea9da59ed18c97b9224e3d0f9e02ce2700ef9f6240e8b14d3706
7
- data.tar.gz: 578d4f0473ff58b1a9f6aeb3d2eab4be158de92943d6147e1521212068578ecd190094e17b02838919561d43b477b197344804478aa8288e70d0c30b26f85fd9
6
+ metadata.gz: c8753deac6a63980050cb5a2a016515f45dafadcbebe356c8409e984ffd6d01d65f7c2b1b11c9c77bae2ff422601fd45aea7b92c77e0b82733d32cdab11ec9e2
7
+ data.tar.gz: 7d92e8f081b5902909709f2777d075e224700495259747701ac9a210d334b64a4c89abaf37cfd5546763216eacccb7bb4c6aa3b177e71c6cc0d852d5a245a25a
@@ -25,12 +25,19 @@ jobs:
25
25
  steps:
26
26
  - uses: actions/checkout@v2
27
27
  - uses: ruby/setup-ruby@v1
28
+ env:
29
+ REDIS_VERSION: ${{ matrix.redis }}
28
30
  with:
29
31
  ruby-version: ${{ matrix.ruby }}
30
32
  bundler-cache: true
31
33
  - run: bundle exec rubocop
32
34
  if: matrix.ruby == '2.7'
35
+ - name: start MySQL
36
+ run: sudo /etc/init.d/mysql start
33
37
  - run: bundle exec rspec --format doc
38
+ env:
39
+ MYSQL_USER: root
40
+ MYSQL_PASSWORD: root
34
41
  - name: Run codacy-coverage-reporter
35
42
  uses: codacy/codacy-coverage-reporter-action@master
36
43
  with:
data/.rubocop.yml CHANGED
@@ -8,6 +8,9 @@ AllCops:
8
8
  Layout/ArgumentAlignment:
9
9
  EnforcedStyle: with_fixed_indentation
10
10
 
11
+ Layout/CaseIndentation:
12
+ EnforcedStyle: end
13
+
11
14
  Layout/ParameterAlignment:
12
15
  EnforcedStyle: with_fixed_indentation
13
16
 
data/CHANGELOG.md CHANGED
@@ -1,3 +1,37 @@
1
+ ## Release v0.7.0
2
+
3
+ * Add initial benchmarks and performance improvements #36 justinhoward
4
+
5
+ ### Breaking Changes
6
+
7
+ The `circuit_success` event no longer contains the status value. Computing this
8
+ value was causing performance problems.
9
+
10
+ ## Release v0.6.0
11
+
12
+ * docs, use correct state in description for skipped event #27 senny
13
+ * Fix CI to set REDIS_VERSION correctly #31 justinhoward
14
+ * Fix a potential memory leak in patches #32 justinhoward
15
+ * Capture an error for BUSY redis backend when patched #30 justinhoward
16
+ * Add a patch for mysql2 #28 justinhoward
17
+
18
+ ## Release v0.5.1
19
+
20
+ * Fix Storage::FaultTolerantProxy to return empty history on entries fail #26 justinhoward
21
+
22
+ ## Release v0.5.0
23
+
24
+ * Allow creating a new Faulty instance in Faulty#register #24 justinhoward
25
+ * Add support for patches to core dependencies starting with redis #14 justinhoward
26
+ * Improve storage #entries performance by returning entries #23 justinhoward
27
+
28
+ ### Breaking Changes
29
+
30
+ * Faulty #[] no longer differentiates between symbols and strings when accessing
31
+ Faulty instances
32
+ * Faulty::Storage::Interface must now return a history array instead of a
33
+ circuit status object. Custom storage backends must be updated.
34
+
1
35
  ## Release v0.4.0
2
36
 
3
37
  * Switch from Travis CI to GitHub actions #11 justinhoward
data/Gemfile CHANGED
@@ -13,7 +13,10 @@ not_jruby = %i[ruby mingw x64_mingw].freeze
13
13
  gem 'activesupport', '>= 4.2'
14
14
  gem 'bundler', '>= 1.17', '< 3'
15
15
  gem 'byebug', platforms: not_jruby
16
+ gem 'honeybadger', '>= 2.0'
16
17
  gem 'irb', '~> 1.0'
18
+ # Minimum of 0.5.0 for specific error classes
19
+ gem 'mysql2', '>= 0.5.0', platforms: not_jruby
17
20
  gem 'redcarpet', '~> 3.5', platforms: not_jruby
18
21
  gem 'rspec_junit_formatter', '~> 0.4'
19
22
  gem 'simplecov', '>= 0.17.1'
data/README.md CHANGED
@@ -81,6 +81,9 @@ Also see "Release It!: Design and Deploy Production-Ready Software" by
81
81
  + [Circuit Options](#circuit-options)
82
82
  + [Listing Circuits](#listing-circuits)
83
83
  + [Locking Circuits](#locking-circuits)
84
+ * [Patches](#patches)
85
+ + [Patch::Redis](#patchredis)
86
+ + [Patch::Mysql2](#patchmysql2)
84
87
  * [Event Handling](#event-handling)
85
88
  + [CallbackListener](#callbacklistener)
86
89
  + [Other Built-in Listeners](#other-built-in-listeners)
@@ -221,6 +224,10 @@ users = Faulty.circuit(:api).try_run do
221
224
  end.or_default([])
222
225
  ```
223
226
 
227
+ If you want to globally wrap your core dependencies, like your cache or
228
+ database, you may want to look at [Patches](#patches), which can automatically
229
+ wrap your connections in a Faulty circuit.
230
+
224
231
  See [Running a Circuit](#running-a-circuit) for more in-depth examples. Also,
225
232
  make sure you have proper [Event Handlers](#event-handling) setup so that you
226
233
  can monitor your circuits for failures.
@@ -580,6 +587,14 @@ principal to any other registered Faulty instance:
580
587
  Faulty[:api].circuit('api_circuit').run { 'ok' }
581
588
  ```
582
589
 
590
+ You can also create and register a Faulty instance in one step:
591
+
592
+ ```ruby
593
+ Faulty.register(:api) do |config|
594
+ # This accepts the same options as Faulty.init
595
+ end
596
+ ```
597
+
583
598
  #### Standalone Instances
584
599
 
585
600
  If you choose, you can use Faulty instances without registering them globally by
@@ -915,6 +930,121 @@ Locking or unlocking a circuit has no concurrency guarantees, so it's not
915
930
  recommended to lock or unlock circuits from production code. Instead, locks are
916
931
  intended as an emergency tool for troubleshooting and debugging.
917
932
 
933
+ ## Patches
934
+
935
+ For certain core dependencies like a cache or a database connection, it is
936
+ inconvenient to wrap every call in its own circuit. Faulty provides some patches
937
+ to wrap these calls in a circuit automatically. To use a patch, it first needs
938
+ to be loaded. Since patches modify third-party code, they are not automatically
939
+ required with the Faulty gem, so they need to be required individually.
940
+
941
+ ```ruby
942
+ require 'faulty'
943
+ require 'faulty/patch/redis'
944
+ ```
945
+
946
+ Or require them in your `Gemfile`
947
+
948
+ ```ruby
949
+ gem 'faulty', require: %w[faulty faulty/patch/redis]
950
+ ```
951
+
952
+ For core dependencies you'll most likely want to use the in-memory circuit
953
+ storage adapter and not the Redis storage adapter. That way if Redis fails, your
954
+ circuit storage doesn't also fail, causing cascading failures.
955
+
956
+ For example, you can use a separate Faulty instance to manage your Mysql2
957
+ circuit:
958
+
959
+ ```ruby
960
+ # Setup your default config. This can use the Redis backend if you prefer
961
+ Faulty.init do |config|
962
+ # ...
963
+ end
964
+
965
+ Faulty.register(:mysql) do |config|
966
+ # Here we decide to set some circuit defaults more useful for
967
+ # frequent database calls
968
+ config.circuit_defaults = {
969
+ cool_down: 20.0,
970
+ evaluation_window: 40,
971
+ sample_threshold: 25
972
+ }
973
+ end
974
+
975
+ # Now we can use our "mysql" faulty instance when constructing a Mysql2 client
976
+ Mysql2::Client.new(host: '127.0.0.1', faulty: { instance: 'mysql2' })
977
+ ```
978
+
979
+ ### Patch::Redis
980
+
981
+ [`Faulty::Patch::Redis`](https://www.rubydoc.info/gems/faulty/Faulty/Patch/Redis)
982
+ protects a Redis client with an internal circuit. Pass a `:faulty` key along
983
+ with your connection options to enable the circuit breaker.
984
+
985
+ The Redis patch supports the Redis gem versions 3 and 4.
986
+
987
+ ```ruby
988
+ require 'faulty/patch/redis'
989
+
990
+ redis = Redis.new(url: 'redis://localhost:6379', faulty: {
991
+ # The name for the redis circuit
992
+ name: 'redis'
993
+
994
+ # The faulty instance to use
995
+ # This can also be a registered faulty instance or a constant name. See API
996
+ # docs for more details
997
+ instance: Faulty.default
998
+
999
+ # By default, circuit errors will be subclasses of Redis::BaseError
1000
+ # To disable this behavior, set patch_errors to false and Faulty
1001
+ # will raise its default errors
1002
+ patch_errors: true
1003
+ })
1004
+ redis.connect # raises Faulty::CircuitError if connection fails
1005
+
1006
+ # If the faulty key is not given, no circuit is used
1007
+ redis = Redis.new(url: 'redis://localhost:6379')
1008
+ redis.connect # not protected by a circuit
1009
+ ```
1010
+
1011
+ ### Patch::Mysql2
1012
+
1013
+ [`Faulty::Patch::Mysql2`](https://www.rubydoc.info/gems/faulty/Faulty/Patch/Mysql2)
1014
+ protects a `Mysql2::Client` with an internal circuit. Pass a `:faulty` key along
1015
+ with your connection options to enable the circuit breaker.
1016
+
1017
+ Faulty supports the mysql2 gem versions 0.5 and greater.
1018
+
1019
+ Note: Although Faulty supports Ruby 2.3 in general, the Mysql2 patch is not
1020
+ fully supported on Ruby 2.3. It may work for you, but use it at your own risk.
1021
+
1022
+ ```ruby
1023
+ require 'faulty/patch/mysql2'
1024
+
1025
+ mysql = Mysql2::Client.new(host: '127.0.0.1', faulty: {
1026
+ # The name for the Mysql2 circuit
1027
+ name: 'mysql2'
1028
+
1029
+ # The faulty instance to use
1030
+ # This can also be a registered faulty instance or a constant name. See API
1031
+ # docs for more details
1032
+ instance: Faulty.default
1033
+
1034
+ # By default, circuit errors will be subclasses of
1035
+ # Mysql2::Error::ConnectionError
1036
+ # To disable this behavior, set patch_errors to false and Faulty
1037
+ # will raise its default errors
1038
+ patch_errors: true
1039
+ })
1040
+
1041
+ mysql.query('SELECT * FROM users') # raises Faulty::CircuitError if connection fails
1042
+
1043
+ # If the faulty key is not given, no circuit is used
1044
+ mysql = Mysql2::Client.new(host: '127.0.0.1')
1045
+ mysql.query('SELECT * FROM users') # not protected by a circuit
1046
+ ```
1047
+
918
1048
  ## Event Handling
919
1049
 
920
1050
  Faulty uses an event-dispatching model to deliver notifications of internal
@@ -933,9 +1063,8 @@ events. The full list of events is available from
933
1063
  - `circuit_reopened` - A circuit execution cause the circuit to reopen from
934
1064
  half-open. Payload: `circuit`, `error`.
935
1065
  - `circuit_skipped` - A circuit execution was skipped because the circuit is
936
- closed. Payload: `circuit`
937
- - `circuit_success` - A circuit execution was successful. Payload: `circuit`,
938
- `status`
1066
+ open. Payload: `circuit`
1067
+ - `circuit_success` - A circuit execution was successful. Payload: `circuit`
939
1068
  - `storage_failure` - A storage backend raised an error. Payload `circuit` (can
940
1069
  be nil), `action`, `error`
941
1070
 
data/bin/benchmark ADDED
@@ -0,0 +1,52 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require 'bundler/setup'
5
+ require 'benchmark'
6
+ require 'faulty'
7
+
8
+ n = 100_000
9
+
10
+ puts "Starting circuit benchmarks with #{n} iterations each\n\n"
11
+
12
+ Benchmark.bm(25) do |b|
13
+ in_memory = Faulty.new(listeners: [])
14
+ b.report('memory storage') do
15
+ n.times { in_memory.circuit(:memory).run { true } }
16
+ end
17
+
18
+ b.report('memory storage failures') do
19
+ n.times do
20
+ begin
21
+ in_memory.circuit(:memory_fail, sample_threshold: n + 1).run { raise 'fail' }
22
+ rescue StandardError
23
+ # Expected to raise here
24
+ end
25
+ end
26
+ end
27
+
28
+ in_memory_large = Faulty.new(listeners: [], storage: Faulty::Storage::Memory.new(max_sample_size: 1000))
29
+ b.report('large memory storage') do
30
+ n.times { in_memory_large.circuit(:memory_large).run { true } }
31
+ end
32
+ end
33
+
34
+ n = 1_000_000
35
+
36
+ puts "\n\nStarting extra benchmarks with #{n} iterations each\n\n"
37
+
38
+ Benchmark.bm(25) do |b|
39
+ in_memory = Faulty.new(listeners: [])
40
+
41
+ log_listener = Faulty::Events::LogListener.new(Logger.new(File::NULL))
42
+ log_circuit = in_memory.circuit(:log_listener)
43
+ log_status = log_circuit.status
44
+ b.report('log listener success') do
45
+ n.times { log_listener.handle(:circuit_success, circuit: log_circuit, status: log_status) }
46
+ end
47
+
48
+ log_error = StandardError.new('test error')
49
+ b.report('log listener failure') do
50
+ n.times { log_listener.handle(:circuit_failure, error: log_error, circuit: log_circuit, status: log_status) }
51
+ end
52
+ end
data/faulty.gemspec CHANGED
@@ -26,7 +26,6 @@ Gem::Specification.new do |spec|
26
26
  # Only essential development tools and dependencies go here.
27
27
  # Other non-essential development dependencies go in the Gemfile.
28
28
  spec.add_development_dependency 'connection_pool', '~> 2.0'
29
- spec.add_development_dependency 'honeybadger', '>= 2.0'
30
29
  spec.add_development_dependency 'redis', '>= 3.0'
31
30
  spec.add_development_dependency 'rspec', '~> 3.8'
32
31
  # 0.81 is the last rubocop version with Ruby 2.3 support
@@ -50,6 +50,9 @@ class Faulty
50
50
  # @!attribute [r] cool_down
51
51
  # @return [Integer] The number of seconds the circuit will
52
52
  # stay open after it is tripped. Default 300.
53
+ # @!attribute [r] error_module
54
+ # @return [Module] Used by patches to set the namespace module for
55
+ # the faulty errors that will be raised. Default `Faulty`
53
56
  # @!attribute [r] evaluation_window
54
57
  # @return [Integer] The number of seconds of history that
55
58
  # will be evaluated to determine the failure rate for a circuit.
@@ -88,6 +91,7 @@ class Faulty
88
91
  :rate_threshold,
89
92
  :sample_threshold,
90
93
  :errors,
94
+ :error_module,
91
95
  :exclude,
92
96
  :cache,
93
97
  :notifier,
@@ -103,6 +107,7 @@ class Faulty
103
107
  cache_refreshes_after: 900,
104
108
  cool_down: 300,
105
109
  errors: [StandardError],
110
+ error_module: Faulty,
106
111
  exclude: [],
107
112
  evaluation_window: 60,
108
113
  rate_threshold: 0.5,
@@ -115,6 +120,7 @@ class Faulty
115
120
  cache
116
121
  cool_down
117
122
  errors
123
+ error_module
118
124
  exclude
119
125
  evaluation_window
120
126
  rate_threshold
@@ -213,9 +219,11 @@ class Faulty
213
219
  cached_value = cache_read(cache)
214
220
  # return cached unless cached.nil?
215
221
  return cached_value if !cached_value.nil? && !cache_should_refresh?(cache)
216
- return run_skipped(cached_value) unless status.can_run?
217
222
 
218
- run_exec(cached_value, cache, &block)
223
+ current_status = status
224
+ return run_skipped(cached_value) unless current_status.can_run?
225
+
226
+ run_exec(current_status, cached_value, cache, &block)
219
227
  end
220
228
 
221
229
  # Force the circuit to stay open until unlocked
@@ -282,7 +290,7 @@ class Faulty
282
290
  # @return The result from cache if available
283
291
  def run_skipped(cached_value)
284
292
  skipped!
285
- raise OpenCircuitError.new(nil, self) if cached_value.nil?
293
+ raise options.error_module::OpenCircuitError.new(nil, self) if cached_value.nil?
286
294
 
287
295
  cached_value
288
296
  end
@@ -292,36 +300,36 @@ class Faulty
292
300
  # @param cached_value The cached value if one is available
293
301
  # @param cache_key [String, nil] The cache key if one is given
294
302
  # @return The run result
295
- def run_exec(cached_value, cache_key)
303
+ def run_exec(status, cached_value, cache_key)
296
304
  result = yield
297
- success!
305
+ success!(status)
298
306
  cache_write(cache_key, result)
299
307
  result
300
308
  rescue *options.errors => e
301
309
  raise if options.exclude.any? { |ex| e.is_a?(ex) }
302
310
 
303
311
  if cached_value.nil?
304
- raise CircuitTrippedError.new(nil, self) if failure!(e)
312
+ raise options.error_module::CircuitTrippedError.new(nil, self) if failure!(status, e)
305
313
 
306
- raise CircuitFailureError.new(nil, self)
314
+ raise options.error_module::CircuitFailureError.new(nil, self)
307
315
  else
308
316
  cached_value
309
317
  end
310
318
  end
311
319
 
312
320
  # @return [Boolean] True if the circuit transitioned to closed
313
- def success!
314
- status = storage.entry(self, Faulty.current_time, true)
315
- closed = false
316
- closed = close! if should_close?(status)
321
+ def success!(status)
322
+ storage.entry(self, Faulty.current_time, true)
323
+ closed = close! if status.half_open?
317
324
 
318
- options.notifier.notify(:circuit_success, circuit: self, status: status)
325
+ options.notifier.notify(:circuit_success, circuit: self)
319
326
  closed
320
327
  end
321
328
 
322
329
  # @return [Boolean] True if the circuit transitioned to open
323
- def failure!(error)
324
- status = storage.entry(self, Faulty.current_time, false)
330
+ def failure!(status, error)
331
+ entries = storage.entry(self, Faulty.current_time, false)
332
+ status = Status.from_entries(entries, **status.to_h)
325
333
  options.notifier.notify(:circuit_failure, circuit: self, status: status, error: error)
326
334
 
327
335
  opened = if status.half_open?
@@ -360,16 +368,6 @@ class Faulty
360
368
  closed
361
369
  end
362
370
 
363
- # Test whether we should close after a successful run
364
- #
365
- # Currently this is always true if the circuit is half-open, which is the
366
- # traditional behavior for a circuit-breaker
367
- #
368
- # @return [Boolean] True if we should close the circuit from half-open
369
- def should_close?(status)
370
- status.half_open?
371
- end
372
-
373
371
  # Read from the cache if it is configured
374
372
  #
375
373
  # @param key The key to read from the cache
data/lib/faulty/error.rb CHANGED
@@ -28,11 +28,13 @@ class Faulty
28
28
  end
29
29
  end
30
30
 
31
- # The base error for all errors raised during circuit runs
32
- #
33
- class CircuitError < FaultyError
31
+ # Included in faulty circuit errors to provide common features for
32
+ # native and patched errors
33
+ module CircuitErrorBase
34
34
  attr_reader :circuit
35
35
 
36
+ # @param message [String]
37
+ # @param circuit [Circuit] The circuit that raised the error
36
38
  def initialize(message, circuit)
37
39
  message ||= %(circuit error for "#{circuit.name}")
38
40
  @circuit = circuit
@@ -41,6 +43,12 @@ class Faulty
41
43
  end
42
44
  end
43
45
 
46
+ # The base error for all errors raised during circuit runs
47
+ #
48
+ class CircuitError < FaultyError
49
+ include CircuitErrorBase
50
+ end
51
+
44
52
  # Raised when running a circuit that is already open
45
53
  class OpenCircuitError < CircuitError; end
46
54
 
@@ -23,7 +23,7 @@ class Faulty
23
23
  # @param (see ListenerInterface#handle)
24
24
  # @return [void]
25
25
  def handle(event, payload)
26
- return unless EVENTS.include?(event)
26
+ return unless EVENT_SET.include?(event)
27
27
  return unless @handlers.key?(event)
28
28
 
29
29
  @handlers[event].each do |handler|
@@ -8,11 +8,19 @@ class Faulty
8
8
  #
9
9
  # The honeybadger gem must be available.
10
10
  class HoneybadgerListener
11
+ HONEYBADGER_EVENTS = Set[
12
+ :circuit_failure,
13
+ :circuit_opened,
14
+ :circuit_reopened,
15
+ :cache_failure,
16
+ :storage_failure
17
+ ].freeze
18
+
11
19
  # (see ListenerInterface#handle)
12
20
  def handle(event, payload)
13
- return unless EVENTS.include?(event)
21
+ return unless HONEYBADGER_EVENTS.include?(event)
14
22
 
15
- send(event, payload) if respond_to?(event, true)
23
+ send(event, payload)
16
24
  end
17
25
 
18
26
  private
@@ -16,9 +16,9 @@ class Faulty
16
16
 
17
17
  # (see ListenerInterface#handle)
18
18
  def handle(event, payload)
19
- return unless EVENTS.include?(event)
19
+ return unless EVENT_SET.include?(event)
20
20
 
21
- send(event, payload) if respond_to?(event, true)
21
+ send(event, payload)
22
22
  end
23
23
 
24
24
  private
@@ -79,8 +79,10 @@ class Faulty
79
79
  end
80
80
 
81
81
  def log(level, msg, action, extra = {})
82
- extra_str = extra.map { |k, v| "#{k}=#{v}" }.join(' ')
83
- logger.public_send(level, "#{msg}: #{action} #{extra_str}")
82
+ @logger.public_send(level) do
83
+ extra_str = extra.map { |k, v| "#{k}=#{v}" }.join(' ')
84
+ "#{msg}: #{action} #{extra_str}"
85
+ end
84
86
  end
85
87
  end
86
88
  end
data/lib/faulty/events.rb CHANGED
@@ -17,6 +17,8 @@ class Faulty
17
17
  circuit_success
18
18
  storage_failure
19
19
  ].freeze
20
+
21
+ EVENT_SET = Set.new(EVENTS)
20
22
  end
21
23
  end
22
24
 
@@ -0,0 +1,46 @@
1
+ # frozen_string_literal: true
2
+
3
+ class Faulty
4
+ module Patch
5
+ # Can be included in patch modules to provide common functionality
6
+ #
7
+ # The patch needs to set `@faulty_circuit`
8
+ #
9
+ # @example
10
+ # module ThingPatch
11
+ # include Faulty::Patch::Base
12
+ #
13
+ # def initialize(options = {})
14
+ # @faulty_circuit = Faulty::Patch.circuit_from_hash('thing', options[:faulty])
15
+ # end
16
+ #
17
+ # def do_something
18
+ # faulty_run { super }
19
+ # end
20
+ # end
21
+ #
22
+ # Thing.prepend(ThingPatch)
23
+ module Base
24
+ # Run a block wrapped by `@faulty_circuit`
25
+ #
26
+ # If `@faulty_circuit` is not set, the block will be run with no
27
+ # circuit.
28
+ #
29
+ # Nested calls to this method will only cause the circuit to be triggered
30
+ # once.
31
+ #
32
+ # @yield A block to run inside the circuit
33
+ # @return The block return value
34
+ def faulty_run
35
+ faulty_running_key = "faulty_running_#{object_id}"
36
+ return yield unless @faulty_circuit
37
+ return yield if Thread.current[faulty_running_key]
38
+
39
+ Thread.current[faulty_running_key] = true
40
+ @faulty_circuit.run { yield }
41
+ ensure
42
+ Thread.current[faulty_running_key] = nil
43
+ end
44
+ end
45
+ end
46
+ end
@@ -0,0 +1,81 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'mysql2'
4
+
5
+ if Gem::Version.new(Mysql2::VERSION) < Gem::Version.new('0.5.0')
6
+ raise NotImplementedError, 'The faulty mysql2 patch requires mysql2 0.5.0 or later'
7
+ end
8
+
9
+ class Faulty
10
+ module Patch
11
+ # Patch Mysql2 to run connections and queries in a circuit
12
+ #
13
+ # This module is not required by default
14
+ #
15
+ # Pass a `:faulty` key into your MySQL connection options to enable
16
+ # circuit protection. See {Patch.circuit_from_hash} for the available
17
+ # options.
18
+ #
19
+ # COMMIT, ROLLBACK, and RELEASE SAVEPOINT queries are intentionally not
20
+ # protected by the circuit. This is to allow open transactions to be closed
21
+ # if possible.
22
+ #
23
+ # @example
24
+ # require 'faulty/patch/mysql2'
25
+ #
26
+ # mysql = Mysql2::Client.new(host: '127.0.0.1', faulty: {})
27
+ # mysql.query('SELECT * FROM users') # raises Faulty::CircuitError if connection fails
28
+ #
29
+ # # If the faulty key is not given, no circuit is used
30
+ # mysql = Mysql2::Client.new(host: '127.0.0.1')
31
+ # mysql.query('SELECT * FROM users') # not protected by a circuit
32
+ #
33
+ # @see Patch.circuit_from_hash
34
+ module Mysql2
35
+ include Base
36
+
37
+ Patch.define_circuit_errors(self, ::Mysql2::Error::ConnectionError)
38
+
39
+ QUERY_WHITELIST = [
40
+ %r{\A(?:/\*.*?\*/)?\s*ROLLBACK}i,
41
+ %r{\A(?:/\*.*?\*/)?\s*COMMIT}i,
42
+ %r{\A(?:/\*.*?\*/)?\s*RELEASE\s+SAVEPOINT}i
43
+ ].freeze
44
+
45
+ def initialize(opts = {})
46
+ @faulty_circuit = Patch.circuit_from_hash(
47
+ 'mysql2',
48
+ opts[:faulty],
49
+ errors: [
50
+ ::Mysql2::Error::ConnectionError,
51
+ ::Mysql2::Error::TimeoutError
52
+ ],
53
+ patched_error_module: Faulty::Patch::Mysql2
54
+ )
55
+
56
+ super
57
+ end
58
+
59
+ # Protect manual connection pings
60
+ def ping
61
+ faulty_run { super }
62
+ rescue Faulty::Patch::Mysql2::FaultyError
63
+ false
64
+ end
65
+
66
+ # Protect the initial connnection
67
+ def connect(*args)
68
+ faulty_run { super }
69
+ end
70
+
71
+ # Protect queries unless they are whitelisted
72
+ def query(*args)
73
+ return super if QUERY_WHITELIST.any? { |r| !r.match(args.first).nil? }
74
+
75
+ faulty_run { super }
76
+ end
77
+ end
78
+ end
79
+ end
80
+
81
+ ::Mysql2::Client.prepend(Faulty::Patch::Mysql2)
@@ -0,0 +1,93 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'redis'
4
+
5
+ class Faulty
6
+ module Patch
7
+ # Patch Redis to run all network IO in a circuit
8
+ #
9
+ # This module is not required by default
10
+ #
11
+ # Pass a `:faulty` key into your MySQL connection options to enable
12
+ # circuit protection. See {Patch.circuit_from_hash} for the available
13
+ # options.
14
+ #
15
+ # @example
16
+ # require 'faulty/patch/redis'
17
+ #
18
+ # redis = Redis.new(url: 'redis://localhost:6379', faulty: {})
19
+ # redis.connect # raises Faulty::CircuitError if connection fails
20
+ #
21
+ # # If the faulty key is not given, no circuit is used
22
+ # redis = Redis.new(url: 'redis://localhost:6379')
23
+ # redis.connect # not protected by a circuit
24
+ #
25
+ # @see Patch.circuit_from_hash
26
+ module Redis
27
+ include Base
28
+
29
+ Patch.define_circuit_errors(self, ::Redis::BaseConnectionError)
30
+
31
+ class BusyError < ::Redis::CommandError
32
+ end
33
+
34
+ # Patches Redis to add the `:faulty` key
35
+ def initialize(options = {})
36
+ @faulty_circuit = Patch.circuit_from_hash(
37
+ 'redis',
38
+ options[:faulty],
39
+ errors: [
40
+ ::Redis::BaseConnectionError,
41
+ BusyError
42
+ ],
43
+ patched_error_module: Faulty::Patch::Redis
44
+ )
45
+
46
+ super
47
+ end
48
+
49
+ # The initial connection is protected by a circuit
50
+ def connect
51
+ faulty_run { super }
52
+ end
53
+
54
+ # Protect command calls
55
+ def call(command)
56
+ faulty_run { super }
57
+ end
58
+
59
+ # Protect command_loop calls
60
+ def call_loop(command, timeout = 0)
61
+ faulty_run { super }
62
+ end
63
+
64
+ # Protect pipelined commands
65
+ def call_pipelined(commands)
66
+ faulty_run { super }
67
+ end
68
+
69
+ # Inject specific error classes if client is patched
70
+ #
71
+ # This method does not raise errors, it returns them
72
+ # as exception objects, so we simply modify that error if necessary and
73
+ # return it.
74
+ #
75
+ # The call* methods above will then raise that error, so we are able to
76
+ # capture it with faulty_run.
77
+ def io(&block)
78
+ return super unless @faulty_circuit
79
+
80
+ reply = super
81
+ if reply.is_a?(::Redis::CommandError)
82
+ if reply.message.start_with?('BUSY')
83
+ reply = BusyError.new(reply.message)
84
+ end
85
+ end
86
+
87
+ reply
88
+ end
89
+ end
90
+ end
91
+ end
92
+
93
+ ::Redis::Client.prepend(Faulty::Patch::Redis)
@@ -0,0 +1,154 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'faulty/patch/base'
4
+
5
+ class Faulty
6
+ # Automatic wrappers for common core dependencies like database connections
7
+ # or caches
8
+ module Patch
9
+ class << self
10
+ # Create a circuit from a configuration hash
11
+ #
12
+ # This is intended to be used in contexts where the user passes in
13
+ # something like a connection hash to a third-party library. For example
14
+ # the Redis patch hooks into the normal Redis connection options to add
15
+ # a `:faulty` key, which is a hash of faulty circuit options. This is
16
+ # slightly different from the normal Faulty circuit options because
17
+ # we also accept an `:instance` key which is a faulty instance.
18
+ #
19
+ # @example
20
+ # # We pass in a faulty instance along with some circuit options
21
+ # Patch.circuit_from_hash(
22
+ # :mysql,
23
+ # { host: 'localhost', faulty: {
24
+ # name: 'my_mysql', # A custom circuit name can be included
25
+ # instance: Faulty.new,
26
+ # sample_threshold: 5
27
+ # }
28
+ # }
29
+ # )
30
+ #
31
+ # @example
32
+ # # instance can be a registered faulty instance referenced by a string
33
+ # or symbol
34
+ # Faulty.register(:db_faulty, Faulty.new)
35
+ # Patch.circuit_from_hash(
36
+ # :mysql,
37
+ # { host: 'localhost', faulty: { instance: :db_faulty } }
38
+ # )
39
+ # @example
40
+ # # If instance is a hash with the key :constant, the value can be
41
+ # # a global constant name containing a Faulty instance
42
+ # DB_FAULTY = Faulty.new
43
+ # Patch.circuit_from_hash(
44
+ # :mysql,
45
+ # { host: 'localhost', faulty: { instance: { constant: 'DB_FAULTY' } } }
46
+ # )
47
+ #
48
+ # @example
49
+ # # Certain patches may want to enforce certain options like :errors
50
+ # # This can be done via hash or the usual block syntax
51
+ # Patch.circuit_from_hash(:mysql,
52
+ # { host: 'localhost', faulty: {} }
53
+ # errors: [Mysql2::Error]
54
+ # )
55
+ #
56
+ # Patch.circuit_from_hash(:mysql,
57
+ # { host: 'localhost', faulty: {} }
58
+ # ) do |conf|
59
+ # conf.errors = [Mysql2::Error]
60
+ # end
61
+ #
62
+ # @param default_name [String] The default name for the circuit
63
+ # @param hash [Hash] A hash of user-provided options. Supports any circuit
64
+ # option and these additional options
65
+ # @option hash [String] :name The circuit name. Defaults to `default_name`
66
+ # @option hash [Boolean] :patch_errors By default, circuit errors will be
67
+ # subclasses of `options[:patched_error_module]`. The user can disable
68
+ # this by setting this option to false.
69
+ # @option hash [Faulty, String, Symbol, Hash{ constant: String }] :instance
70
+ # A reference to a faulty instance. See examples.
71
+ # @param options [Hash] Additional override options. Supports any circuit
72
+ # option and these additional ones.
73
+ # @option options [Module] :patched_error_module The namespace module
74
+ # for patched errors
75
+ # @yield [Circuit::Options] For setting override options in a block
76
+ # @return [Circuit, nil] The circuit if one was created
77
+ def circuit_from_hash(default_name, hash, **options, &block)
78
+ return unless hash
79
+
80
+ hash = symbolize_keys(hash)
81
+ name = hash.delete(:name) || default_name
82
+ patch_errors = hash.delete(:patch_errors) != false
83
+ error_module = options.delete(:patched_error_module)
84
+ hash[:error_module] ||= error_module if error_module && patch_errors
85
+ faulty = resolve_instance(hash.delete(:instance))
86
+ faulty.circuit(name, **hash, **options, &block)
87
+ end
88
+
89
+ # Create a full set of {CircuitError}s with a given base error class
90
+ #
91
+ # For patches that need their errors to be subclasses of a common base.
92
+ #
93
+ # @param namespace [Module] The module to define the error classes in
94
+ # @param base [Class] The base class for the error classes
95
+ # @return [void]
96
+ def define_circuit_errors(namespace, base)
97
+ circuit_error = Class.new(base) { include CircuitErrorBase }
98
+ namespace.const_set('CircuitError', circuit_error)
99
+ namespace.const_set('OpenCircuitError', Class.new(circuit_error))
100
+ namespace.const_set('CircuitFailureError', Class.new(circuit_error))
101
+ namespace.const_set('CircuitTrippedError', Class.new(circuit_error))
102
+ end
103
+
104
+ private
105
+
106
+ # Resolves a constant from a constant name or returns a default
107
+ #
108
+ # - If value is a string or symbol, gets a registered Faulty instance with that name
109
+ # - If value is a Hash with a key `:constant`, resolves the value to a global constant
110
+ # - If value is nil, gets Faulty.default
111
+ # - Otherwise, return value directly
112
+ #
113
+ # @param value [String, Symbol, Faulty, nil] The object or constant name to resolve
114
+ # @return [Object] The resolved Faulty instance
115
+ def resolve_instance(value)
116
+ case value
117
+ when String, Symbol
118
+ result = Faulty[value]
119
+ raise NameError, "No Faulty instance for #{value}" unless result
120
+
121
+ result
122
+ when Hash
123
+ const_name = value[:constant]
124
+ raise ArgumentError 'Missing hash key :constant for Faulty instance' unless const_name
125
+
126
+ Kernel.const_get(const_name)
127
+ when nil
128
+ Faulty.default
129
+ else
130
+ value
131
+ end
132
+ end
133
+
134
+ # Some config files may not suport symbol keys, so we convert the hash
135
+ # to use symbols so that users can pass in strings
136
+ #
137
+ # We cannot use transform_keys since we support Ruby < 2.5
138
+ #
139
+ # @param hash [Hash] A hash to convert
140
+ # @return [Hash] The hash with keys as symbols
141
+ def symbolize_keys(hash)
142
+ result = {}
143
+ hash.each do |key, val|
144
+ result[key.to_sym] = if val.is_a?(Hash)
145
+ symbolize_keys(val)
146
+ else
147
+ val
148
+ end
149
+ end
150
+ result
151
+ end
152
+ end
153
+ end
154
+ end
data/lib/faulty/status.rb CHANGED
@@ -64,10 +64,17 @@ class Faulty
64
64
  # sample_size
65
65
  # @return [Status]
66
66
  def self.from_entries(entries, **hash)
67
+ window_start = Faulty.current_time - hash[:options].evaluation_window
68
+ size = entries.size
69
+ i = 0
67
70
  failures = 0
68
71
  sample_size = 0
69
- entries.each do |(time, success)|
70
- next unless time > Faulty.current_time - hash[:options].evaluation_window
72
+
73
+ # This is a hot loop, and while is slightly faster than each
74
+ while i < size
75
+ time, success = entries[i]
76
+ i += 1
77
+ next unless time > window_start
71
78
 
72
79
  sample_size += 1
73
80
  failures += 1 unless success
@@ -94,7 +94,7 @@ class Faulty
94
94
  @storage.entry(circuit, time, success)
95
95
  rescue StandardError => e
96
96
  options.notifier.notify(:storage_failure, circuit: circuit, action: :entry, error: e)
97
- stub_status(circuit)
97
+ []
98
98
  end
99
99
 
100
100
  # Safely mark a circuit as open
@@ -14,7 +14,8 @@ class Faulty
14
14
  # @param circuit [Circuit] The circuit that ran
15
15
  # @param time [Integer] The unix timestamp for the run
16
16
  # @param success [Boolean] True if the run succeeded
17
- # @return [Status] The circuit status after the run is added
17
+ # @return [Array<Array>] An array of the new history tuples after adding
18
+ # the new entry, see {#history}
18
19
  def entry(circuit, time, success)
19
20
  raise NotImplementedError
20
21
  end
@@ -89,7 +89,7 @@ class Faulty
89
89
  runs.push([time, success])
90
90
  runs.shift if runs.size > options.max_sample_size
91
91
  end
92
- memory.status(circuit.options)
92
+ memory.runs.value
93
93
  end
94
94
 
95
95
  # Mark a circuit as open
@@ -95,15 +95,15 @@ class Faulty
95
95
  # @return (see Interface#entry)
96
96
  def entry(circuit, time, success)
97
97
  key = entries_key(circuit)
98
- pipe do |r|
98
+ result = pipe do |r|
99
99
  r.sadd(list_key, circuit.name)
100
100
  r.expire(list_key, options.circuit_ttl + options.list_granularity) if options.circuit_ttl
101
101
  r.lpush(key, "#{time}#{ENTRY_SEPARATOR}#{success ? 1 : 0}")
102
102
  r.ltrim(key, 0, options.max_sample_size - 1)
103
103
  r.expire(key, options.sample_ttl) if options.sample_ttl
104
+ r.lrange(key, 0, -1)
104
105
  end
105
-
106
- status(circuit)
106
+ map_entries(result.last)
107
107
  end
108
108
 
109
109
  # Mark a circuit as open
@@ -3,6 +3,6 @@
3
3
  class Faulty
4
4
  # The current Faulty version
5
5
  def self.version
6
- Gem::Version.new('0.4.0')
6
+ Gem::Version.new('0.7.0')
7
7
  end
8
8
  end
data/lib/faulty.rb CHANGED
@@ -9,6 +9,7 @@ require 'faulty/cache'
9
9
  require 'faulty/circuit'
10
10
  require 'faulty/error'
11
11
  require 'faulty/events'
12
+ require 'faulty/patch'
12
13
  require 'faulty/result'
13
14
  require 'faulty/status'
14
15
  require 'faulty/storage'
@@ -66,7 +67,7 @@ class Faulty
66
67
  def [](name)
67
68
  raise UninitializedError unless @instances
68
69
 
69
- @instances[name]
70
+ @instances[name.to_s]
70
71
  end
71
72
 
72
73
  # Register an instance to the global Faulty state
@@ -75,13 +76,22 @@ class Faulty
75
76
  # return value if you need to know whether the instance already existed.
76
77
  #
77
78
  # @param name [Symbol] The name of the instance to register
78
- # @param instance [Faulty] The instance to register
79
+ # @param instance [Faulty] The instance to register. If nil, a new instance
80
+ # will be created from the given options or block.
81
+ # @param config [Hash] Attributes for {Faulty::Options}
82
+ # @yield [Faulty::Options] For setting options in a block
79
83
  # @return [Faulty, nil] The previously-registered instance of that name if
80
84
  # it already existed, otherwise nil.
81
- def register(name, instance)
85
+ def register(name, instance = nil, **config, &block)
82
86
  raise UninitializedError unless @instances
83
87
 
84
- @instances.put_if_absent(name, instance)
88
+ if instance
89
+ raise ArgumentError, 'Do not give config options if an instance is given' if !config.empty? || block
90
+ else
91
+ instance = new(**config, &block)
92
+ end
93
+
94
+ @instances.put_if_absent(name.to_s, instance)
85
95
  end
86
96
 
87
97
  # Get the options for the default instance
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: faulty
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.0
4
+ version: 0.7.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Justin Howard
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2021-02-19 00:00:00.000000000 Z
11
+ date: 2021-09-02 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: concurrent-ruby
@@ -38,20 +38,6 @@ dependencies:
38
38
  - - "~>"
39
39
  - !ruby/object:Gem::Version
40
40
  version: '2.0'
41
- - !ruby/object:Gem::Dependency
42
- name: honeybadger
43
- requirement: !ruby/object:Gem::Requirement
44
- requirements:
45
- - - ">="
46
- - !ruby/object:Gem::Version
47
- version: '2.0'
48
- type: :development
49
- prerelease: false
50
- version_requirements: !ruby/object:Gem::Requirement
51
- requirements:
52
- - - ">="
53
- - !ruby/object:Gem::Version
54
- version: '2.0'
55
41
  - !ruby/object:Gem::Dependency
56
42
  name: redis
57
43
  requirement: !ruby/object:Gem::Requirement
@@ -138,6 +124,7 @@ files:
138
124
  - Gemfile
139
125
  - LICENSE.txt
140
126
  - README.md
127
+ - bin/benchmark
141
128
  - bin/check-version
142
129
  - bin/console
143
130
  - bin/rspec
@@ -165,6 +152,10 @@ files:
165
152
  - lib/faulty/events/log_listener.rb
166
153
  - lib/faulty/events/notifier.rb
167
154
  - lib/faulty/immutable_options.rb
155
+ - lib/faulty/patch.rb
156
+ - lib/faulty/patch/base.rb
157
+ - lib/faulty/patch/mysql2.rb
158
+ - lib/faulty/patch/redis.rb
168
159
  - lib/faulty/result.rb
169
160
  - lib/faulty/status.rb
170
161
  - lib/faulty/storage.rb
@@ -195,8 +186,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
195
186
  - !ruby/object:Gem::Version
196
187
  version: '0'
197
188
  requirements: []
198
- rubyforge_project:
199
- rubygems_version: 2.7.6
189
+ rubygems_version: 3.1.2
200
190
  signing_key:
201
191
  specification_version: 4
202
192
  summary: Fault-tolerance tools for ruby based on circuit-breakers