faulty 0.7.1 → 0.8.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +35 -1
- data/README.md +27 -9
- data/bin/benchmark +30 -7
- data/faulty.gemspec +1 -0
- data/lib/faulty/cache/auto_wire.rb +0 -2
- data/lib/faulty/cache/circuit_proxy.rb +1 -3
- data/lib/faulty/cache/fault_tolerant_proxy.rb +0 -2
- data/lib/faulty/circuit.rb +104 -8
- data/lib/faulty/circuit_registry.rb +49 -0
- data/lib/faulty/error.rb +4 -3
- data/lib/faulty/events/filter_notifier.rb +31 -0
- data/lib/faulty/events.rb +1 -0
- data/lib/faulty/immutable_options.rb +21 -7
- data/lib/faulty/status.rb +0 -2
- data/lib/faulty/storage/auto_wire.rb +0 -2
- data/lib/faulty/storage/circuit_proxy.rb +15 -4
- data/lib/faulty/storage/fallback_chain.rb +18 -2
- data/lib/faulty/storage/fault_tolerant_proxy.rb +24 -2
- data/lib/faulty/storage/interface.rb +24 -1
- data/lib/faulty/storage/memory.rb +19 -3
- data/lib/faulty/storage/null.rb +94 -0
- data/lib/faulty/storage/redis.rb +35 -3
- data/lib/faulty/storage.rb +1 -0
- data/lib/faulty/version.rb +1 -1
- data/lib/faulty.rb +31 -6
- metadata +19 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: ff7d5c217cd03c4b1a752cef13a7547d0623f9810cb8c7602e91e8aa6a941358
|
4
|
+
data.tar.gz: c6df02b5ac50a0078fb43cc03ee7fad9275f5fe41c471a4a649eda05e6fa1782
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0bbd345bbfe4d1acd1ba941ae73ba51fdf7a954fe1ca7a87e214dcbb78f8b44de643cb8cc05d7eb7dbd95de9b767039dbe24535717712dcb4c22d641333a9e33
|
7
|
+
data.tar.gz: afa9ccb28dec10f811c3076def8722323fb491b8e8e6ebf6f6917023a6c2a92eec515f0aaf86b74f9fd99f7947ce7e0e859358d6a16412675066a90e547d62ec
|
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,40 @@
|
|
1
|
+
## Releae v0.8.2
|
2
|
+
|
3
|
+
* Fix crash for older versions of concurrent-ruby #42 justinhoward
|
4
|
+
|
5
|
+
## Releae v0.8.1
|
6
|
+
|
7
|
+
* Add cause message to CircuitTrippedError #40 justinhoward
|
8
|
+
* Record failures for cache hits #41 justinhoward
|
9
|
+
|
10
|
+
|
11
|
+
## Release v0.8.0
|
12
|
+
|
13
|
+
* Store circuit options in the backend when run #34 justinhoward
|
14
|
+
|
15
|
+
### Breaking Changes
|
16
|
+
|
17
|
+
* Added #get_options and #set_options to Faulty::Storage::Interface.
|
18
|
+
These will need to be added to any custom backends
|
19
|
+
* Faulty::Storage::Interface#reset now requires removing options in
|
20
|
+
addition to other stored values
|
21
|
+
* Circuit options will now be supplemented by stored options until they
|
22
|
+
are run. This is technically a breaking change in behavior, although
|
23
|
+
in most cases this should cause the expected result.
|
24
|
+
* Circuits are not memoized until they are run. Subsequent calls
|
25
|
+
to Faulty#circuit can return different instances if the circuit is
|
26
|
+
not run. However, once run, options are synchronized between
|
27
|
+
instances, so likely this will not be a breaking change for most
|
28
|
+
cases.
|
29
|
+
|
30
|
+
## Release v0.7.2
|
31
|
+
|
32
|
+
* Add Faulty.disable! for disabling globally #38 justinhoward
|
33
|
+
* Suppress circuit_success for proxy circuits #39 justinhoward
|
34
|
+
|
1
35
|
## Release v0.7.1
|
2
36
|
|
3
|
-
|
37
|
+
* Fix success event crash in log listener #37 justinhoward
|
4
38
|
|
5
39
|
## Release v0.7.0
|
6
40
|
|
data/README.md
CHANGED
@@ -88,6 +88,7 @@ Also see "Release It!: Design and Deploy Production-Ready Software" by
|
|
88
88
|
+ [CallbackListener](#callbacklistener)
|
89
89
|
+ [Other Built-in Listeners](#other-built-in-listeners)
|
90
90
|
+ [Custom Listeners](#custom-listeners)
|
91
|
+
* [Disabling Faulty Globally](#disabling-faulty-globally)
|
91
92
|
* [How it Works](#how-it-works)
|
92
93
|
+ [Caching](#caching)
|
93
94
|
+ [Fault Tolerance](#fault-tolerance)
|
@@ -607,7 +608,7 @@ faulty.circuit('standalone_circuit')
|
|
607
608
|
```
|
608
609
|
|
609
610
|
Calling `#circuit` on the instance still has the same memoization behavior that
|
610
|
-
`Faulty.circuit` has, so subsequent
|
611
|
+
`Faulty.circuit` has, so subsequent runs for the same circuit will use a
|
611
612
|
memoized circuit object.
|
612
613
|
|
613
614
|
|
@@ -732,7 +733,7 @@ Both options can even be specified together.
|
|
732
733
|
```ruby
|
733
734
|
Faulty.circuit(
|
734
735
|
'api',
|
735
|
-
errors: [ActiveRecord::ActiveRecordError]
|
736
|
+
errors: [ActiveRecord::ActiveRecordError],
|
736
737
|
exclude: [ActiveRecord::RecordNotFound, ActiveRecord::RecordNotUnique]
|
737
738
|
).run do
|
738
739
|
# This only captures ActiveRecord::ActiveRecordError errors, but not
|
@@ -812,7 +813,7 @@ the options are retained within the context of each instance. All options given
|
|
812
813
|
after the first call to `Faulty.circuit` (or `Faulty#circuit`) are ignored.
|
813
814
|
|
814
815
|
```ruby
|
815
|
-
Faulty.circuit('api', rate_threshold: 0.7)
|
816
|
+
Faulty.circuit('api', rate_threshold: 0.7).run { api.call }
|
816
817
|
|
817
818
|
# These options are ignored since with already initialized the circuit
|
818
819
|
circuit = Faulty.circuit('api', rate_threshold: 0.3)
|
@@ -820,7 +821,7 @@ circuit.options.rate_threshold # => 0.7
|
|
820
821
|
```
|
821
822
|
|
822
823
|
This is because the circuit objects themselves are internally memoized, and are
|
823
|
-
read-only once
|
824
|
+
read-only once they are run.
|
824
825
|
|
825
826
|
The following example represents the defaults for a new circuit:
|
826
827
|
|
@@ -1124,6 +1125,21 @@ Faulty.init do |config|
|
|
1124
1125
|
end
|
1125
1126
|
```
|
1126
1127
|
|
1128
|
+
## Disabling Faulty Globally
|
1129
|
+
|
1130
|
+
For testing or for some environments, you may wish to disable Faulty circuits
|
1131
|
+
at a global level.
|
1132
|
+
|
1133
|
+
```ruby
|
1134
|
+
Faulty.disable!
|
1135
|
+
```
|
1136
|
+
|
1137
|
+
This only affects the process where you run the `#disable!` method and it does
|
1138
|
+
not affect the stored state of circuits.
|
1139
|
+
|
1140
|
+
Faulty will **still use the cache** even when disabled. If you also want to
|
1141
|
+
disable the cache, configure Faulty to use a `Faulty::Cache::Null` cache.
|
1142
|
+
|
1127
1143
|
## How it Works
|
1128
1144
|
|
1129
1145
|
Faulty implements a version of circuit breakers inspired by "Release It!: Design
|
@@ -1272,11 +1288,12 @@ but there are and have been many other options:
|
|
1272
1288
|
### Currently Active
|
1273
1289
|
|
1274
1290
|
- [semian](https://github.com/Shopify/semian): A resiliency toolkit that
|
1275
|
-
includes circuit breakers. It
|
1276
|
-
only in-memory storage by design.
|
1277
|
-
|
1278
|
-
|
1279
|
-
to
|
1291
|
+
includes circuit breakers. It auto-wires circuits for MySQL, Net::HTTP, and
|
1292
|
+
Redis. It has only in-memory storage by design. Its core components are
|
1293
|
+
written in C, which allows it to be faster than pure ruby.
|
1294
|
+
- [circuitbox](https://github.com/yammer/circuitbox): Also uses a block syntax
|
1295
|
+
to manually define circuits. It uses Moneta to abstract circuit storage to
|
1296
|
+
allow any key-value store.
|
1280
1297
|
|
1281
1298
|
### Previous Work
|
1282
1299
|
|
@@ -1293,6 +1310,7 @@ but there are and have been many other options:
|
|
1293
1310
|
|
1294
1311
|
- Simple API but configurable for advanced users
|
1295
1312
|
- Pluggable storage backends (circuitbox also has this)
|
1313
|
+
- Patches for common core dependencies (semian also has this)
|
1296
1314
|
- Protected storage access with fallback to safe storage
|
1297
1315
|
- Global, or object-oriented configuration with multiple instances
|
1298
1316
|
- Integrated caching support tailored for fault-tolerance
|
data/bin/benchmark
CHANGED
@@ -4,12 +4,13 @@
|
|
4
4
|
require 'bundler/setup'
|
5
5
|
require 'benchmark'
|
6
6
|
require 'faulty'
|
7
|
+
require 'redis'
|
8
|
+
require 'json'
|
7
9
|
|
8
10
|
n = 100_000
|
9
|
-
|
10
|
-
puts "
|
11
|
-
|
12
|
-
Benchmark.bm(25) do |b|
|
11
|
+
width = 25
|
12
|
+
puts "In memory circuits x#{n}"
|
13
|
+
Benchmark.bm(width) do |b|
|
13
14
|
in_memory = Faulty.new(listeners: [])
|
14
15
|
b.report('memory storage') do
|
15
16
|
n.times { in_memory.circuit(:memory).run { true } }
|
@@ -31,11 +32,33 @@ Benchmark.bm(25) do |b|
|
|
31
32
|
end
|
32
33
|
end
|
33
34
|
|
34
|
-
n =
|
35
|
+
n = 1000
|
36
|
+
puts "\n\Redis circuits x#{n}"
|
37
|
+
Benchmark.bm(width) do |b|
|
38
|
+
redis = Faulty.new(listeners: [], storage: Faulty::Storage::Redis.new)
|
39
|
+
b.report('redis storage') do
|
40
|
+
n.times { redis.circuit(:memory).run { true } }
|
41
|
+
end
|
35
42
|
|
36
|
-
|
43
|
+
b.report('redis storage failures') do
|
44
|
+
n.times do
|
45
|
+
begin
|
46
|
+
redis.circuit(:memory_fail, sample_threshold: n + 1).run { raise 'fail' }
|
47
|
+
rescue StandardError
|
48
|
+
# Expected to raise here
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
37
52
|
|
38
|
-
|
53
|
+
redis_large = Faulty.new(listeners: [], storage: Faulty::Storage::Redis.new(max_sample_size: 1000))
|
54
|
+
b.report('large redis storage') do
|
55
|
+
n.times { redis_large.circuit(:memory).run { true } }
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
n = 1_000_000
|
60
|
+
puts "\n\nExtra x#{n}"
|
61
|
+
Benchmark.bm(width) do |b|
|
39
62
|
in_memory = Faulty.new(listeners: [])
|
40
63
|
|
41
64
|
log_listener = Faulty::Events::LogListener.new(Logger.new(File::NULL))
|
data/faulty.gemspec
CHANGED
@@ -26,6 +26,7 @@ Gem::Specification.new do |spec|
|
|
26
26
|
# Only essential development tools and dependencies go here.
|
27
27
|
# Other non-essential development dependencies go in the Gemfile.
|
28
28
|
spec.add_development_dependency 'connection_pool', '~> 2.0'
|
29
|
+
spec.add_development_dependency 'json'
|
29
30
|
spec.add_development_dependency 'redis', '>= 3.0'
|
30
31
|
spec.add_development_dependency 'rspec', '~> 3.8'
|
31
32
|
# 0.81 is the last rubocop version with Ruby 2.3 support
|
@@ -26,14 +26,12 @@ class Faulty
|
|
26
26
|
) do
|
27
27
|
include ImmutableOptions
|
28
28
|
|
29
|
-
private
|
30
|
-
|
31
29
|
def finalize
|
32
30
|
raise ArgumentError, 'The circuit or notifier option must be given' unless notifier || circuit
|
33
31
|
|
34
32
|
self.circuit ||= Circuit.new(
|
35
33
|
Faulty::Storage::CircuitProxy.name,
|
36
|
-
notifier: notifier,
|
34
|
+
notifier: Events::FilterNotifier.new(notifier, exclude: %i[circuit_success]),
|
37
35
|
cache: Cache::Null.new
|
38
36
|
)
|
39
37
|
end
|
data/lib/faulty/circuit.rb
CHANGED
@@ -27,7 +27,6 @@ class Faulty
|
|
27
27
|
CACHE_REFRESH_SUFFIX = '.faulty_refresh'
|
28
28
|
|
29
29
|
attr_reader :name
|
30
|
-
attr_reader :options
|
31
30
|
|
32
31
|
# Options for {Circuit}
|
33
32
|
#
|
@@ -82,6 +81,9 @@ class Faulty
|
|
82
81
|
# @return [Storage::Interface] The storage backend. Default
|
83
82
|
# `Storage::Memory.new`. Unlike {Faulty#initialize}, this is not wrapped
|
84
83
|
# in {Storage::AutoWire} by default.
|
84
|
+
# @!attribute [r] registry
|
85
|
+
# @return [CircuitRegistry] For use by {Faulty} instances to facilitate
|
86
|
+
# memoization of circuits.
|
85
87
|
Options = Struct.new(
|
86
88
|
:cache_expires_in,
|
87
89
|
:cache_refreshes_after,
|
@@ -95,11 +97,22 @@ class Faulty
|
|
95
97
|
:exclude,
|
96
98
|
:cache,
|
97
99
|
:notifier,
|
98
|
-
:storage
|
100
|
+
:storage,
|
101
|
+
:registry
|
99
102
|
) do
|
100
103
|
include ImmutableOptions
|
101
104
|
|
102
|
-
|
105
|
+
# Get the options stored in the storage backend
|
106
|
+
#
|
107
|
+
# @return [Hash] A hash of stored options
|
108
|
+
def for_storage
|
109
|
+
{
|
110
|
+
cool_down: cool_down,
|
111
|
+
evaluation_window: evaluation_window,
|
112
|
+
rate_threshold: rate_threshold,
|
113
|
+
sample_threshold: sample_threshold
|
114
|
+
}
|
115
|
+
end
|
103
116
|
|
104
117
|
def defaults
|
105
118
|
{
|
@@ -150,7 +163,59 @@ class Faulty
|
|
150
163
|
raise ArgumentError, 'name must be a String' unless name.is_a?(String)
|
151
164
|
|
152
165
|
@name = name
|
153
|
-
@
|
166
|
+
@given_options = Options.new(options, &block)
|
167
|
+
@pulled_options = nil
|
168
|
+
@options_pushed = false
|
169
|
+
end
|
170
|
+
|
171
|
+
# Get the options for this circuit
|
172
|
+
#
|
173
|
+
# If this circuit has been run, these will the options exactly as given
|
174
|
+
# to {.new}. However, if this circuit has not yet been run, these options
|
175
|
+
# will be supplemented by the last-known options from the circuit storage.
|
176
|
+
#
|
177
|
+
# Once a circuit is run, the given options are pushed to circuit storage to
|
178
|
+
# be persisted.
|
179
|
+
#
|
180
|
+
# This is to allow circuit objects to behave as expected in contexts where
|
181
|
+
# the exact options for a circuit are not known such as an admin dashboard
|
182
|
+
# or in a debug console.
|
183
|
+
#
|
184
|
+
# Note that this distinction isn't usually important unless using
|
185
|
+
# distributed circuit storage like the Redis storage backend.
|
186
|
+
#
|
187
|
+
# @example
|
188
|
+
# Faulty.circuit('api', cool_down: 5).run { api.users }
|
189
|
+
# # This status will be calculated using the cool_down of 5 because
|
190
|
+
# # the circuit was already run
|
191
|
+
# Faulty.circuit('api').status
|
192
|
+
#
|
193
|
+
# @example
|
194
|
+
# # This status will be calculated using the cool_down in circuit storage
|
195
|
+
# # if it is available instead of using the default value.
|
196
|
+
# Faulty.circuit('api').status
|
197
|
+
#
|
198
|
+
# @example
|
199
|
+
# # For typical usage, this behaves as expected, but note that it's
|
200
|
+
# # possible to run into some unexpected behavior when creating circuits
|
201
|
+
# # in unusual ways.
|
202
|
+
#
|
203
|
+
# # For example, this status will be calculated using the cool_down in
|
204
|
+
# # circuit storage if it is available despite the given value of 5.
|
205
|
+
# Faulty.circuit('api', cool_down: 5).status
|
206
|
+
# Faulty.circuit('api').run { api.users }
|
207
|
+
# # However now, after the circuit is run, status will be calculated
|
208
|
+
# # using the given cool_down of 5 and the value of 5 will be pushed
|
209
|
+
# # permanently to circuit storage
|
210
|
+
# Faulty.circuit('api').status
|
211
|
+
#
|
212
|
+
# @return [Options] The resolved options
|
213
|
+
def options
|
214
|
+
return @given_options if @options_pushed
|
215
|
+
return @pulled_options if @pulled_options
|
216
|
+
|
217
|
+
stored = @given_options.storage.get_options(self)
|
218
|
+
@pulled_options = stored ? @given_options.dup_with(stored) : @given_options
|
154
219
|
end
|
155
220
|
|
156
221
|
# Run the circuit as with {#run}, but return a {Result}
|
@@ -204,6 +269,8 @@ class Faulty
|
|
204
269
|
# a second cool down period. However, if the circuit completes successfully,
|
205
270
|
# the circuit will be closed and reset to its initial state.
|
206
271
|
#
|
272
|
+
# When this is run, the given options are persisted to the storage backend.
|
273
|
+
#
|
207
274
|
# @param cache [String, nil] A cache key, or nil if caching is not desired
|
208
275
|
# @yield The block to protect with this circuit
|
209
276
|
# @raise If the block raises an error not in the error list, or if the error
|
@@ -216,6 +283,7 @@ class Faulty
|
|
216
283
|
# circuit to trip
|
217
284
|
# @return The return value of the block
|
218
285
|
def run(cache: nil, &block)
|
286
|
+
push_options
|
219
287
|
cached_value = cache_read(cache)
|
220
288
|
# return cached unless cached.nil?
|
221
289
|
return cached_value if !cached_value.nil? && !cache_should_refresh?(cache)
|
@@ -256,6 +324,8 @@ class Faulty
|
|
256
324
|
#
|
257
325
|
# @return [self]
|
258
326
|
def reset!
|
327
|
+
@options_pushed = false
|
328
|
+
@pulled_options = nil
|
259
329
|
storage.reset(self)
|
260
330
|
self
|
261
331
|
end
|
@@ -284,6 +354,25 @@ class Faulty
|
|
284
354
|
|
285
355
|
private
|
286
356
|
|
357
|
+
# Push the given options to circuit storage and set those as the current
|
358
|
+
# options
|
359
|
+
#
|
360
|
+
# @return [void]
|
361
|
+
def push_options
|
362
|
+
return if @options_pushed
|
363
|
+
|
364
|
+
@pulled_options = nil
|
365
|
+
@options_pushed = true
|
366
|
+
resolved = options.registry&.resolve(self)
|
367
|
+
if resolved
|
368
|
+
# If another circuit instance was resolved, don't store these options
|
369
|
+
# Instead, copy the options from that circuit as if we were given those
|
370
|
+
@given_options = resolved.options
|
371
|
+
else
|
372
|
+
storage.set_options(self, @given_options.for_storage)
|
373
|
+
end
|
374
|
+
end
|
375
|
+
|
287
376
|
# Process a skipped run
|
288
377
|
#
|
289
378
|
# @param cached_value The cached value if one is available
|
@@ -308,10 +397,13 @@ class Faulty
|
|
308
397
|
rescue *options.errors => e
|
309
398
|
raise if options.exclude.any? { |ex| e.is_a?(ex) }
|
310
399
|
|
400
|
+
opened = failure!(status, e)
|
311
401
|
if cached_value.nil?
|
312
|
-
|
313
|
-
|
314
|
-
|
402
|
+
if opened
|
403
|
+
raise options.error_module::CircuitTrippedError.new(e.message, self)
|
404
|
+
else
|
405
|
+
raise options.error_module::CircuitFailureError.new(e.message, self)
|
406
|
+
end
|
315
407
|
else
|
316
408
|
cached_value
|
317
409
|
end
|
@@ -431,9 +523,13 @@ class Faulty
|
|
431
523
|
|
432
524
|
# Alias to the storage engine from options
|
433
525
|
#
|
526
|
+
# Always returns the value from the given options
|
527
|
+
#
|
434
528
|
# @return [Storage::Interface]
|
435
529
|
def storage
|
436
|
-
|
530
|
+
return Faulty::Storage::Null.new if Faulty.disabled?
|
531
|
+
|
532
|
+
@given_options.storage
|
437
533
|
end
|
438
534
|
end
|
439
535
|
end
|
@@ -0,0 +1,49 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
class Faulty
|
4
|
+
# Used by Faulty instances to track and memoize Circuits
|
5
|
+
#
|
6
|
+
# Whenever a circuit is requested by `Faulty#circuit`, it calls
|
7
|
+
# `#retrieve`. That will return a resolved circuit if there is one, or
|
8
|
+
# otherwise, it will create a new circuit instance.
|
9
|
+
#
|
10
|
+
# Once any circuit is run, the circuit calls `#resolve`. That saves
|
11
|
+
# the instance into the registry. Any calls to `#retrieve` after
|
12
|
+
# the circuit is resolved will result in the same instance being returned.
|
13
|
+
#
|
14
|
+
# However, before a circuit is resolved, calling `Faulty#circuit` will result
|
15
|
+
# in a new Circuit instance being created for every call. If multiples of
|
16
|
+
# these call `resolve`, only the first one will "win" and be memoized.
|
17
|
+
class CircuitRegistry
|
18
|
+
def initialize(circuit_options)
|
19
|
+
@circuit_options = circuit_options
|
20
|
+
@circuit_options[:registry] = self
|
21
|
+
@circuits = Concurrent::Map.new
|
22
|
+
end
|
23
|
+
|
24
|
+
# Retrieve a memoized circuit with the same name, or if none is yet
|
25
|
+
# resolved, create a new one.
|
26
|
+
#
|
27
|
+
# @param name [String] The name of the circuit
|
28
|
+
# @param options [Hash] Options for {Circuit::Options}
|
29
|
+
# @yield [Circuit::Options] For setting options in a block
|
30
|
+
# @return [Circuit] The new or memoized circuit
|
31
|
+
def retrieve(name, options, &block)
|
32
|
+
@circuits.fetch(name) do
|
33
|
+
options = @circuit_options.merge(options)
|
34
|
+
Circuit.new(name, **options, &block)
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
# Save and memoize the given circuit as the "canonical" instance for
|
39
|
+
# the circuit name
|
40
|
+
#
|
41
|
+
# If the name is already resolved, this will be ignored
|
42
|
+
#
|
43
|
+
# @return [Circuit, nil] If this circuit name is already resolved, the
|
44
|
+
# already-resolved circuit
|
45
|
+
def resolve(circuit)
|
46
|
+
@circuits.put_if_absent(circuit.name, circuit)
|
47
|
+
end
|
48
|
+
end
|
49
|
+
end
|
data/lib/faulty/error.rb
CHANGED
@@ -36,10 +36,11 @@ class Faulty
|
|
36
36
|
# @param message [String]
|
37
37
|
# @param circuit [Circuit] The circuit that raised the error
|
38
38
|
def initialize(message, circuit)
|
39
|
-
|
40
|
-
|
39
|
+
full_message = %(circuit error for "#{circuit.name}")
|
40
|
+
full_message = %(#{full_message}: #{message}) if message
|
41
41
|
|
42
|
-
|
42
|
+
@circuit = circuit
|
43
|
+
super(full_message)
|
43
44
|
end
|
44
45
|
end
|
45
46
|
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
class Faulty
|
4
|
+
module Events
|
5
|
+
# Wraps a Notifier and filters events by name
|
6
|
+
class FilterNotifier
|
7
|
+
# @param notifier [Notifier] The internal notifier to filter events for
|
8
|
+
# @param events [Array, nil] An array of events to allow. If nil, all
|
9
|
+
# {EVENTS} will be used
|
10
|
+
# @param exclude [Array, nil] An array of events to disallow. If nil,
|
11
|
+
# no events will be disallowed. Takes priority over `events`.
|
12
|
+
def initialize(notifier, events: nil, exclude: nil)
|
13
|
+
@notifier = notifier
|
14
|
+
@events = Set.new(events || EVENTS)
|
15
|
+
exclude&.each { |e| @events.delete(e) }
|
16
|
+
end
|
17
|
+
|
18
|
+
# Notify all listeners of an event
|
19
|
+
#
|
20
|
+
# If a listener raises an error while handling an event, that error will
|
21
|
+
# be captured and written to STDERR.
|
22
|
+
#
|
23
|
+
# @param (see Notifier)
|
24
|
+
def notify(event, payload)
|
25
|
+
return unless @events.include?(event)
|
26
|
+
|
27
|
+
@notifier.notify(event, payload)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
data/lib/faulty/events.rb
CHANGED
@@ -5,18 +5,23 @@ class Faulty
|
|
5
5
|
module ImmutableOptions
|
6
6
|
# @param hash [Hash] A hash of attributes to initialize with
|
7
7
|
# @yield [self] Yields itself to the block to set options before freezing
|
8
|
-
def initialize(hash)
|
9
|
-
defaults.merge(hash)
|
8
|
+
def initialize(hash, &block)
|
9
|
+
setup(defaults.merge(hash), &block)
|
10
|
+
end
|
11
|
+
|
12
|
+
def dup_with(hash, &block)
|
13
|
+
dup.setup(hash, &block)
|
14
|
+
end
|
15
|
+
|
16
|
+
def setup(hash)
|
17
|
+
hash&.each { |key, value| self[key] = value }
|
10
18
|
yield self if block_given?
|
11
19
|
finalize
|
12
|
-
|
13
|
-
raise ArgumentError, "Missing required attribute #{key}" if self[key].nil?
|
14
|
-
end
|
20
|
+
guard_required!
|
15
21
|
freeze
|
22
|
+
self
|
16
23
|
end
|
17
24
|
|
18
|
-
private
|
19
|
-
|
20
25
|
# A hash of default values to set before yielding to the block
|
21
26
|
#
|
22
27
|
# @return [Hash<Symbol, Object>]
|
@@ -36,5 +41,14 @@ class Faulty
|
|
36
41
|
# @return [void]
|
37
42
|
def finalize
|
38
43
|
end
|
44
|
+
|
45
|
+
private
|
46
|
+
|
47
|
+
# Raise an error if required options are missing
|
48
|
+
def guard_required!
|
49
|
+
required.each do |key|
|
50
|
+
raise ArgumentError, "Missing required attribute #{key}" if self[key].nil?
|
51
|
+
end
|
52
|
+
end
|
39
53
|
end
|
40
54
|
end
|
data/lib/faulty/status.rb
CHANGED
@@ -26,14 +26,12 @@ class Faulty
|
|
26
26
|
) do
|
27
27
|
include ImmutableOptions
|
28
28
|
|
29
|
-
private
|
30
|
-
|
31
29
|
def finalize
|
32
30
|
raise ArgumentError, 'The circuit or notifier option must be given' unless notifier || circuit
|
33
31
|
|
34
32
|
self.circuit ||= Circuit.new(
|
35
33
|
Faulty::Storage::CircuitProxy.name,
|
36
|
-
notifier: notifier,
|
34
|
+
notifier: Events::FilterNotifier.new(notifier, exclude: %i[circuit_success]),
|
37
35
|
cache: Cache::Null.new
|
38
36
|
)
|
39
37
|
end
|
@@ -47,7 +45,20 @@ class Faulty
|
|
47
45
|
@options = Options.new(options, &block)
|
48
46
|
end
|
49
47
|
|
50
|
-
%i[
|
48
|
+
%i[
|
49
|
+
get_options
|
50
|
+
set_options
|
51
|
+
entry
|
52
|
+
open
|
53
|
+
reopen
|
54
|
+
close
|
55
|
+
lock
|
56
|
+
unlock
|
57
|
+
reset
|
58
|
+
status
|
59
|
+
history
|
60
|
+
list
|
61
|
+
].each do |method|
|
51
62
|
define_method(method) do |*args|
|
52
63
|
options.circuit.run { @storage.public_send(method, *args) }
|
53
64
|
end
|
@@ -30,8 +30,6 @@ class Faulty
|
|
30
30
|
) do
|
31
31
|
include ImmutableOptions
|
32
32
|
|
33
|
-
private
|
34
|
-
|
35
33
|
def required
|
36
34
|
%i[notifier]
|
37
35
|
end
|
@@ -49,6 +47,24 @@ class Faulty
|
|
49
47
|
@options = Options.new(options, &block)
|
50
48
|
end
|
51
49
|
|
50
|
+
# Get options from the first available storage backend
|
51
|
+
#
|
52
|
+
# @param (see Interface#get_options)
|
53
|
+
# @return (see Interface#get_options)
|
54
|
+
def get_options(circuit)
|
55
|
+
send_chain(:get_options, circuit) do |e|
|
56
|
+
options.notifier.notify(:storage_failure, circuit: circuit, action: :get_options, error: e)
|
57
|
+
end
|
58
|
+
end
|
59
|
+
|
60
|
+
# Try to set circuit options on all backends
|
61
|
+
#
|
62
|
+
# @param (see Interface#set_options)
|
63
|
+
# @return (see Interface#set_options)
|
64
|
+
def set_options(circuit, stored_options)
|
65
|
+
send_all(:set_options, circuit, stored_options)
|
66
|
+
end
|
67
|
+
|
52
68
|
# Create a circuit entry in the first available storage backend
|
53
69
|
#
|
54
70
|
# @param (see Interface#entry)
|
@@ -23,8 +23,6 @@ class Faulty
|
|
23
23
|
) do
|
24
24
|
include ImmutableOptions
|
25
25
|
|
26
|
-
private
|
27
|
-
|
28
26
|
def required
|
29
27
|
%i[notifier]
|
30
28
|
end
|
@@ -85,6 +83,30 @@ class Faulty
|
|
85
83
|
# @return (see Interface#list)
|
86
84
|
def_delegators :@storage, :lock, :unlock, :reset, :history, :list
|
87
85
|
|
86
|
+
# Get circuit options safely
|
87
|
+
#
|
88
|
+
# @see Interface#get_options
|
89
|
+
# @param (see Interface#get_options)
|
90
|
+
# @return (see Interface#get_options)
|
91
|
+
def get_options(circuit)
|
92
|
+
@storage.get_options(circuit)
|
93
|
+
rescue StandardError => e
|
94
|
+
options.notifier.notify(:storage_failure, circuit: circuit, action: :get_options, error: e)
|
95
|
+
nil
|
96
|
+
end
|
97
|
+
|
98
|
+
# Set circuit options safely
|
99
|
+
#
|
100
|
+
# @see Interface#get_options
|
101
|
+
# @param (see Interface#set_options)
|
102
|
+
# @return (see Interface#set_options)
|
103
|
+
def set_options(circuit, stored_options)
|
104
|
+
@storage.set_options(circuit, stored_options)
|
105
|
+
rescue StandardError => e
|
106
|
+
options.notifier.notify(:storage_failure, circuit: circuit, action: :set_options, error: e)
|
107
|
+
nil
|
108
|
+
end
|
109
|
+
|
88
110
|
# Add a history entry safely
|
89
111
|
#
|
90
112
|
# @see Interface#entry
|
@@ -6,6 +6,29 @@ class Faulty
|
|
6
6
|
#
|
7
7
|
# This is for documentation only and is not loaded
|
8
8
|
class Interface
|
9
|
+
# Get the options stored for circuit
|
10
|
+
#
|
11
|
+
# They should be returned exactly as given by {#set_options}
|
12
|
+
#
|
13
|
+
# @return [Hash] A hash of the options stored by {#set_options}. The keys
|
14
|
+
# must be symbols.
|
15
|
+
def get_options(circuit)
|
16
|
+
raise NotImplementedError
|
17
|
+
end
|
18
|
+
|
19
|
+
# Store the options for a circuit
|
20
|
+
#
|
21
|
+
# They should be returned exactly as given by {#set_options}
|
22
|
+
#
|
23
|
+
# @param circuit [Circuit] The circuit to set options for
|
24
|
+
# @param options [Hash<Symbol, Object>] A hash of symbol option names to
|
25
|
+
# circuit options. These option values are guranteed to be primive
|
26
|
+
# values.
|
27
|
+
# @return [void]
|
28
|
+
def set_options(circuit, stored_options)
|
29
|
+
raise NotImplementedError
|
30
|
+
end
|
31
|
+
|
9
32
|
# Add a circuit run entry to storage
|
10
33
|
#
|
11
34
|
# The backend may choose to store this in whatever manner it chooses as
|
@@ -99,7 +122,7 @@ class Faulty
|
|
99
122
|
# Reset the circuit to a fresh state
|
100
123
|
#
|
101
124
|
# Clears all circuit status including entries, state, locks,
|
102
|
-
# opened_at, and any other values that would affect Status.
|
125
|
+
# opened_at, options, and any other values that would affect Status.
|
103
126
|
#
|
104
127
|
# No concurrency gurantees are provided for resetting
|
105
128
|
#
|
@@ -33,8 +33,6 @@ class Faulty
|
|
33
33
|
Options = Struct.new(:max_sample_size) do
|
34
34
|
include ImmutableOptions
|
35
35
|
|
36
|
-
private
|
37
|
-
|
38
36
|
def defaults
|
39
37
|
{ max_sample_size: 100 }
|
40
38
|
end
|
@@ -43,7 +41,7 @@ class Faulty
|
|
43
41
|
# The internal object for storing a circuit
|
44
42
|
#
|
45
43
|
# @private
|
46
|
-
MemoryCircuit = Struct.new(:state, :runs, :opened_at, :lock) do
|
44
|
+
MemoryCircuit = Struct.new(:state, :runs, :opened_at, :lock, :options) do
|
47
45
|
def initialize
|
48
46
|
self.state = Concurrent::Atom.new(:closed)
|
49
47
|
self.runs = Concurrent::MVar.new([], dup_on_deref: true)
|
@@ -78,6 +76,24 @@ class Faulty
|
|
78
76
|
@options = Options.new(options, &block)
|
79
77
|
end
|
80
78
|
|
79
|
+
# Get the options stored for circuit
|
80
|
+
#
|
81
|
+
# @see Interface#get_options
|
82
|
+
# @param (see Interface#get_options)
|
83
|
+
# @return (see Interface#get_options)
|
84
|
+
def get_options(circuit)
|
85
|
+
fetch(circuit).options
|
86
|
+
end
|
87
|
+
|
88
|
+
# Store the options for a circuit
|
89
|
+
#
|
90
|
+
# @see Interface#set_options
|
91
|
+
# @param (see Interface#set_options)
|
92
|
+
# @return (see Interface#set_options)
|
93
|
+
def set_options(circuit, stored_options)
|
94
|
+
fetch(circuit).options = stored_options
|
95
|
+
end
|
96
|
+
|
81
97
|
# Add an entry to storage
|
82
98
|
#
|
83
99
|
# @see Interface#entry
|
@@ -0,0 +1,94 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
class Faulty
|
4
|
+
module Storage
|
5
|
+
# A no-op backend for disabling circuits
|
6
|
+
class Null
|
7
|
+
# Define a single global instance
|
8
|
+
@instance = new
|
9
|
+
|
10
|
+
def self.new
|
11
|
+
@instance
|
12
|
+
end
|
13
|
+
|
14
|
+
# @param (see Interface#get_options)
|
15
|
+
# @return (see Interface#get_options)
|
16
|
+
def get_options(_circuit)
|
17
|
+
{}
|
18
|
+
end
|
19
|
+
|
20
|
+
# @param (see Interface#set_options)
|
21
|
+
# @return (see Interface#set_options)
|
22
|
+
def set_options(_circuit, _stored_options)
|
23
|
+
end
|
24
|
+
|
25
|
+
# @param (see Interface#entry)
|
26
|
+
# @return (see Interface#entry)
|
27
|
+
def entry(_circuit, _time, _success)
|
28
|
+
[]
|
29
|
+
end
|
30
|
+
|
31
|
+
# @param (see Interface#open)
|
32
|
+
# @return (see Interface#open)
|
33
|
+
def open(_circuit, _opened_at)
|
34
|
+
true
|
35
|
+
end
|
36
|
+
|
37
|
+
# @param (see Interface#reopen)
|
38
|
+
# @return (see Interface#reopen)
|
39
|
+
def reopen(_circuit, _opened_at, _previous_opened_at)
|
40
|
+
true
|
41
|
+
end
|
42
|
+
|
43
|
+
# @param (see Interface#close)
|
44
|
+
# @return (see Interface#close)
|
45
|
+
def close(_circuit)
|
46
|
+
true
|
47
|
+
end
|
48
|
+
|
49
|
+
# @param (see Interface#lock)
|
50
|
+
# @return (see Interface#lock)
|
51
|
+
def lock(_circuit, _state)
|
52
|
+
end
|
53
|
+
|
54
|
+
# @param (see Interface#unlock)
|
55
|
+
# @return (see Interface#unlock)
|
56
|
+
def unlock(_circuit)
|
57
|
+
end
|
58
|
+
|
59
|
+
# @param (see Interface#reset)
|
60
|
+
# @return (see Interface#reset)
|
61
|
+
def reset(_circuit)
|
62
|
+
end
|
63
|
+
|
64
|
+
# @param (see Interface#status)
|
65
|
+
# @return (see Interface#status)
|
66
|
+
def status(circuit)
|
67
|
+
Faulty::Status.new(
|
68
|
+
options: circuit.options,
|
69
|
+
stub: true
|
70
|
+
)
|
71
|
+
end
|
72
|
+
|
73
|
+
# @param (see Interface#history)
|
74
|
+
# @return (see Interface#history)
|
75
|
+
def history(_circuit)
|
76
|
+
[]
|
77
|
+
end
|
78
|
+
|
79
|
+
# @param (see Interface#list)
|
80
|
+
# @return (see Interface#list)
|
81
|
+
def list
|
82
|
+
[]
|
83
|
+
end
|
84
|
+
|
85
|
+
# This backend is fault tolerant
|
86
|
+
#
|
87
|
+
# @param (see Interface#fault_tolerant?)
|
88
|
+
# @return (see Interface#fault_tolerant?)
|
89
|
+
def fault_tolerant?
|
90
|
+
true
|
91
|
+
end
|
92
|
+
end
|
93
|
+
end
|
94
|
+
end
|
data/lib/faulty/storage/redis.rb
CHANGED
@@ -57,8 +57,6 @@ class Faulty
|
|
57
57
|
) do
|
58
58
|
include ImmutableOptions
|
59
59
|
|
60
|
-
private
|
61
|
-
|
62
60
|
def defaults
|
63
61
|
{
|
64
62
|
key_prefix: 'faulty',
|
@@ -85,9 +83,37 @@ class Faulty
|
|
85
83
|
def initialize(**options, &block)
|
86
84
|
@options = Options.new(options, &block)
|
87
85
|
|
86
|
+
# Ensure JSON is available since we don't explicitly require it
|
87
|
+
JSON # rubocop:disable Lint/Void
|
88
|
+
|
88
89
|
check_client_options!
|
89
90
|
end
|
90
91
|
|
92
|
+
# Get the options stored for circuit
|
93
|
+
#
|
94
|
+
# @see Interface#get_options
|
95
|
+
# @param (see Interface#get_options)
|
96
|
+
# @return (see Interface#get_options)
|
97
|
+
def get_options(circuit)
|
98
|
+
json = redis { |r| r.get(options_key(circuit)) }
|
99
|
+
return if json.nil?
|
100
|
+
|
101
|
+
JSON.parse(json, symbolize_names: true)
|
102
|
+
end
|
103
|
+
|
104
|
+
# Store the options for a circuit
|
105
|
+
#
|
106
|
+
# These will be serialized as JSON
|
107
|
+
#
|
108
|
+
# @see Interface#set_options
|
109
|
+
# @param (see Interface#set_options)
|
110
|
+
# @return (see Interface#set_options)
|
111
|
+
def set_options(circuit, stored_options)
|
112
|
+
redis do |r|
|
113
|
+
r.set(options_key(circuit), JSON.dump(stored_options), ex: options.circuit_ttl)
|
114
|
+
end
|
115
|
+
end
|
116
|
+
|
91
117
|
# Add an entry to storage
|
92
118
|
#
|
93
119
|
# @see Interface#entry
|
@@ -173,7 +199,8 @@ class Faulty
|
|
173
199
|
r.del(
|
174
200
|
entries_key(circuit),
|
175
201
|
opened_at_key(circuit),
|
176
|
-
lock_key(circuit)
|
202
|
+
lock_key(circuit),
|
203
|
+
options_key(circuit)
|
177
204
|
)
|
178
205
|
r.set(state_key(circuit), 'closed', ex: options.circuit_ttl)
|
179
206
|
end
|
@@ -239,6 +266,11 @@ class Faulty
|
|
239
266
|
key('circuit', circuit.name, *parts)
|
240
267
|
end
|
241
268
|
|
269
|
+
# @return [String] The key for circuit options
|
270
|
+
def options_key(circuit)
|
271
|
+
ckey(circuit, 'options')
|
272
|
+
end
|
273
|
+
|
242
274
|
# @return [String] The key for circuit state
|
243
275
|
def state_key(circuit)
|
244
276
|
ckey(circuit, 'state')
|
data/lib/faulty/storage.rb
CHANGED
@@ -10,5 +10,6 @@ require 'faulty/storage/auto_wire'
|
|
10
10
|
require 'faulty/storage/circuit_proxy'
|
11
11
|
require 'faulty/storage/fallback_chain'
|
12
12
|
require 'faulty/storage/fault_tolerant_proxy'
|
13
|
+
require 'faulty/storage/null'
|
13
14
|
require 'faulty/storage/memory'
|
14
15
|
require 'faulty/storage/redis'
|
data/lib/faulty/version.rb
CHANGED
data/lib/faulty.rb
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
require 'securerandom'
|
4
4
|
require 'forwardable'
|
5
|
-
require 'concurrent
|
5
|
+
require 'concurrent'
|
6
6
|
|
7
7
|
require 'faulty/immutable_options'
|
8
8
|
require 'faulty/cache'
|
@@ -10,6 +10,7 @@ require 'faulty/circuit'
|
|
10
10
|
require 'faulty/error'
|
11
11
|
require 'faulty/events'
|
12
12
|
require 'faulty/patch'
|
13
|
+
require 'faulty/circuit_registry'
|
13
14
|
require 'faulty/result'
|
14
15
|
require 'faulty/status'
|
15
16
|
require 'faulty/storage'
|
@@ -128,6 +129,33 @@ class Faulty
|
|
128
129
|
def current_time
|
129
130
|
Time.now.to_i
|
130
131
|
end
|
132
|
+
|
133
|
+
# Disable Faulty circuits
|
134
|
+
#
|
135
|
+
# This allows circuits to run as if they were always closed. Does
|
136
|
+
# not disable caching.
|
137
|
+
#
|
138
|
+
# Intended for use in tests, or to disable Faulty entirely for an
|
139
|
+
# environment.
|
140
|
+
#
|
141
|
+
# @return [void]
|
142
|
+
def disable!
|
143
|
+
@disabled = true
|
144
|
+
end
|
145
|
+
|
146
|
+
# Re-enable Faulty if disabled with {#disable!}
|
147
|
+
#
|
148
|
+
# @return [void]
|
149
|
+
def enable!
|
150
|
+
@disabled = false
|
151
|
+
end
|
152
|
+
|
153
|
+
# Check whether Faulty was disabled with {#disable!}
|
154
|
+
#
|
155
|
+
# @return [Boolean] True if disabled
|
156
|
+
def disabled?
|
157
|
+
@disabled == true
|
158
|
+
end
|
131
159
|
end
|
132
160
|
|
133
161
|
attr_reader :options
|
@@ -199,8 +227,8 @@ class Faulty
|
|
199
227
|
# @param options [Hash] Attributes for {Options}
|
200
228
|
# @yield [Options] For setting options in a block
|
201
229
|
def initialize(**options, &block)
|
202
|
-
@circuits = Concurrent::Map.new
|
203
230
|
@options = Options.new(options, &block)
|
231
|
+
@registry = CircuitRegistry.new(circuit_options)
|
204
232
|
end
|
205
233
|
|
206
234
|
# Create or retrieve a circuit
|
@@ -216,10 +244,7 @@ class Faulty
|
|
216
244
|
# @return [Circuit] The new circuit or the existing circuit if it already exists
|
217
245
|
def circuit(name, **options, &block)
|
218
246
|
name = name.to_s
|
219
|
-
@
|
220
|
-
options = circuit_options.merge(options)
|
221
|
-
Circuit.new(name, **options, &block)
|
222
|
-
end
|
247
|
+
@registry.retrieve(name, options, &block)
|
223
248
|
end
|
224
249
|
|
225
250
|
# Get a list of all circuit names
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: faulty
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.8.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Justin Howard
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2021-
|
11
|
+
date: 2021-10-18 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: concurrent-ruby
|
@@ -38,6 +38,20 @@ dependencies:
|
|
38
38
|
- - "~>"
|
39
39
|
- !ruby/object:Gem::Version
|
40
40
|
version: '2.0'
|
41
|
+
- !ruby/object:Gem::Dependency
|
42
|
+
name: json
|
43
|
+
requirement: !ruby/object:Gem::Requirement
|
44
|
+
requirements:
|
45
|
+
- - ">="
|
46
|
+
- !ruby/object:Gem::Version
|
47
|
+
version: '0'
|
48
|
+
type: :development
|
49
|
+
prerelease: false
|
50
|
+
version_requirements: !ruby/object:Gem::Requirement
|
51
|
+
requirements:
|
52
|
+
- - ">="
|
53
|
+
- !ruby/object:Gem::Version
|
54
|
+
version: '0'
|
41
55
|
- !ruby/object:Gem::Dependency
|
42
56
|
name: redis
|
43
57
|
requirement: !ruby/object:Gem::Requirement
|
@@ -144,9 +158,11 @@ files:
|
|
144
158
|
- lib/faulty/cache/null.rb
|
145
159
|
- lib/faulty/cache/rails.rb
|
146
160
|
- lib/faulty/circuit.rb
|
161
|
+
- lib/faulty/circuit_registry.rb
|
147
162
|
- lib/faulty/error.rb
|
148
163
|
- lib/faulty/events.rb
|
149
164
|
- lib/faulty/events/callback_listener.rb
|
165
|
+
- lib/faulty/events/filter_notifier.rb
|
150
166
|
- lib/faulty/events/honeybadger_listener.rb
|
151
167
|
- lib/faulty/events/listener_interface.rb
|
152
168
|
- lib/faulty/events/log_listener.rb
|
@@ -165,6 +181,7 @@ files:
|
|
165
181
|
- lib/faulty/storage/fault_tolerant_proxy.rb
|
166
182
|
- lib/faulty/storage/interface.rb
|
167
183
|
- lib/faulty/storage/memory.rb
|
184
|
+
- lib/faulty/storage/null.rb
|
168
185
|
- lib/faulty/storage/redis.rb
|
169
186
|
- lib/faulty/version.rb
|
170
187
|
homepage: https://github.com/ParentSquare/faulty
|