stockpile_cache 1.1.0 → 1.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +3 -0
- data/Gemfile.lock +3 -3
- data/README.md +62 -2
- data/lib/stockpile.rb +33 -14
- data/lib/stockpile/cache.rb +8 -8
- data/lib/stockpile/cached_value_expirer.rb +2 -2
- data/lib/stockpile/cached_value_reader.rb +3 -3
- data/lib/stockpile/configuration.rb +12 -8
- data/lib/stockpile/constants.rb +1 -1
- data/lib/stockpile/{redis_connection.rb → default_redis_configuration.rb} +15 -10
- data/lib/stockpile/executor.rb +8 -6
- data/lib/stockpile/lock.rb +8 -7
- data/lib/stockpile/locked_execution_result.rb +4 -3
- data/lib/stockpile/redis_connections.rb +31 -0
- data/lib/stockpile/redis_connections_factory.rb +56 -0
- data/lib/stockpile/yaml_redis_configuration.rb +61 -0
- metadata +6 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e0cf5948f44622314a97af71ab6ce39aa3f59931092b7ac6d80f424e052be90d
|
4
|
+
data.tar.gz: bec173c3c11807350c3b4d4cb05f638fc4a90c71837882f7b64931db1dce8416
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: '08218ce4ab3946ae6b418dd0d72c354170415eff8b041d0e02bbc261ffca83fe4afd7718d9bb7907d2b80093afaeeaa8f83dc57e7b9a7aa4d91c1384ccf965b7'
|
7
|
+
data.tar.gz: fddd745d31dae1f5509cd3988e216d2fdfe5e520a971e58ef3ede3ed864c5b2e34191abef7b67bfab0115134fa4cdceec43fd404d8d8430c53df931b4f01bde1
|
data/CHANGELOG.md
CHANGED
data/Gemfile.lock
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
stockpile_cache (1.0
|
4
|
+
stockpile_cache (1.2.0)
|
5
5
|
connection_pool
|
6
6
|
oj
|
7
7
|
rake
|
@@ -14,12 +14,12 @@ GEM
|
|
14
14
|
connection_pool (2.2.2)
|
15
15
|
diff-lcs (1.3)
|
16
16
|
jaro_winkler (1.5.3)
|
17
|
-
oj (3.9.
|
17
|
+
oj (3.9.2)
|
18
18
|
parallel (1.17.0)
|
19
19
|
parser (2.6.4.1)
|
20
20
|
ast (~> 2.4.0)
|
21
21
|
rainbow (3.0.0)
|
22
|
-
rake (13.0.
|
22
|
+
rake (13.0.1)
|
23
23
|
redis (4.1.3)
|
24
24
|
rspec (3.8.0)
|
25
25
|
rspec-core (~> 3.8.0)
|
data/README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
# Stockpile [![Build Status][ci-image]][ci] [![Code Climate][codeclimate-image]][codeclimate] [![Gem Version][version-image]][version]
|
2
2
|
Stockpile is a simple cache written in Ruby backed by Redis. It has built in
|
3
3
|
[cache-stampede](https://en.wikipedia.org/wiki/Cache_stampede) (also known as
|
4
|
-
dog-piling) protection.
|
4
|
+
dog-piling) protection and support for multiple Redis servers.
|
5
5
|
|
6
6
|
Can be used with any Ruby or Ruby on Rails project. Can be used as a replacement for
|
7
7
|
existing Ruby on Rails cache.
|
@@ -71,6 +71,7 @@ Following settings are supported:
|
|
71
71
|
| `STOCKPILE_REDIS_URL` | `redis_url` | URL of your Redis server that will be used for caching. Defaults to `redis://localhost:6379/1`. |
|
72
72
|
| `STOCKPILE_REDIS_SENTINELS` | `sentinels` | (optional) Comma separated list of Sentinels IPs for Redis. Defaults to `nil`. Example value: `8.8.8.8:42,8.8.4.4:42`. |
|
73
73
|
| `STOCKPILE_SLUMBER` | `slumber` | Timeout (in seconds) for stampede protection lock. After timeout passed in code will be executed instead of reading a value from cache. Defaults to `2`. |
|
74
|
+
| `STOCKPILE_CONFIGURATION_FILE` | `configuration_file` | (optional) `.yml` configuration file to read connection information from. See [Multiple Database](#multiple-database). |
|
74
75
|
|
75
76
|
## Usage
|
76
77
|
To use simply wrap your code into `perform_cached` block:
|
@@ -81,14 +82,73 @@ Stockpile.perform_cached(key: 'meaning_of_life', ttl: 42) do
|
|
81
82
|
end
|
82
83
|
```
|
83
84
|
|
84
|
-
`perform` method accepts
|
85
|
+
`perform` method accepts 4 named arguments:
|
85
86
|
|
86
87
|
| Argument | Meaning |
|
87
88
|
| ------------- | ------------- |
|
88
89
|
| `key` | Pointer in cache by which a value will be either looked up or stored in cache once code provided in block is executed. |
|
89
90
|
| `ttl` | (optional) Time in seconds for which a cached value will be stored. Defaults to 300 seconds (5 minutes). |
|
91
|
+
| `db` | (optional) Name of the Redis database to cache value in. Defaults to `:default` |
|
90
92
|
| `&block` | Block of code to execute; it's return value will be stored in cache. |
|
91
93
|
|
94
|
+
### Multiple Database
|
95
|
+
Stockpile comes with a support for multiple databases. A word of caution: unless
|
96
|
+
you have very good reason to run multiple databases within single instance of
|
97
|
+
Redis server you probably should avoid doing so as you will not see any performance
|
98
|
+
improvements in doing so.
|
99
|
+
|
100
|
+
To allow multi-database support you have to do two things. First you have to set
|
101
|
+
`configuration_file` setting to point at `.yml` containing your configuration.
|
102
|
+
You can do so by either setting a `STOCKPILE_CONFIGURATION_FILE` environment
|
103
|
+
variable or by executing a configuration block during runtime (for Rails create
|
104
|
+
`config/initializers/stockpile.rb` with following content):
|
105
|
+
|
106
|
+
```
|
107
|
+
Stockpile.configure do |configuration|
|
108
|
+
configuration.configuration_file = <PATH/TO/FILE>
|
109
|
+
end
|
110
|
+
```
|
111
|
+
|
112
|
+
Second thing to do is to create a `.yml` configuration file. It has to have at
|
113
|
+
least one database definition. Providing `sentinels` is optional. Everything
|
114
|
+
else is mandatory:
|
115
|
+
|
116
|
+
```
|
117
|
+
---
|
118
|
+
master:
|
119
|
+
url: 'redis://redis-1-host:6379/1'
|
120
|
+
sentinels: '8.8.8.8:42,8.8.4.4:42'
|
121
|
+
pool_options:
|
122
|
+
size: 5
|
123
|
+
timeout: 5
|
124
|
+
|
125
|
+
commander:
|
126
|
+
url: 'redis://redis-2-host:6379/1'
|
127
|
+
pool_options:
|
128
|
+
size: 5
|
129
|
+
timeout: 5
|
130
|
+
```
|
131
|
+
|
132
|
+
To query different databases provide a corresponding `db:` param with
|
133
|
+
`perform_cached` method:
|
134
|
+
|
135
|
+
```
|
136
|
+
Stockpile.perform_cached(db: :master, key: 'meaning_of_life', ttl: 42) do
|
137
|
+
21 + 21
|
138
|
+
end
|
139
|
+
|
140
|
+
Stockpile.perform_cached(db: :commander, key: 'meaning_of_life', ttl: 21) do
|
141
|
+
21
|
142
|
+
end
|
143
|
+
```
|
144
|
+
|
145
|
+
If you do not provide a `db:` param then a `:default` database will be used; if
|
146
|
+
you do not define it in a configuration file your request will error out.
|
147
|
+
|
148
|
+
Using `configuration_file` setting will make Stockpile ignore all other
|
149
|
+
Redis connection related settings and it will read configuration from `.yml`
|
150
|
+
file instead.
|
151
|
+
|
92
152
|
## Caveats
|
93
153
|
There is no timeout or rescue set for code you will be running through the cache. If
|
94
154
|
you need to do either you have to handle it outside of Stockpile.
|
data/lib/stockpile.rb
CHANGED
@@ -18,10 +18,14 @@ require 'connection_pool'
|
|
18
18
|
require 'oj'
|
19
19
|
require 'redis'
|
20
20
|
require 'timeout'
|
21
|
+
require 'yaml'
|
21
22
|
|
22
23
|
require 'stockpile/constants'
|
23
24
|
require 'stockpile/configuration'
|
24
|
-
require 'stockpile/
|
25
|
+
require 'stockpile/redis_connections_factory'
|
26
|
+
require 'stockpile/default_redis_configuration'
|
27
|
+
require 'stockpile/yaml_redis_configuration'
|
28
|
+
require 'stockpile/redis_connections'
|
25
29
|
|
26
30
|
require 'stockpile/lock'
|
27
31
|
require 'stockpile/locked_execution_result'
|
@@ -36,16 +40,17 @@ require 'stockpile/executor'
|
|
36
40
|
# = Stockpile
|
37
41
|
#
|
38
42
|
# Simple cache with Redis as a backend and a built in cache-stampede
|
39
|
-
# protection. For more information on
|
40
|
-
# README.md file.
|
43
|
+
# protection and multiple Redis database support. For more information on
|
44
|
+
# general usage consider consulting README.md file.
|
41
45
|
#
|
42
46
|
# While interacting with the cache from within your application
|
43
47
|
# avoid re-using anything after :: notation as it is part of internal API
|
44
48
|
# and is subject to an un-announced breaking change.
|
45
49
|
#
|
46
|
-
# Stockpile provides
|
50
|
+
# Stockpile provides 6 methods as part of it's public API:
|
47
51
|
# * configuration
|
48
52
|
# * configure
|
53
|
+
# * expire_cached
|
49
54
|
# * perform_cached
|
50
55
|
# * redis
|
51
56
|
# * redis_connection_pool
|
@@ -59,7 +64,9 @@ module Stockpile
|
|
59
64
|
@configuration ||= Configuration.new
|
60
65
|
end
|
61
66
|
|
62
|
-
# API to configure cache dynamically during runtime.
|
67
|
+
# API to configure cache dynamically during runtime. Running dynamic
|
68
|
+
# configuration will rebuild connection pools releasing existing
|
69
|
+
# connections.
|
63
70
|
#
|
64
71
|
# @yield [configuration] Takes in a block of code of code that is setting
|
65
72
|
# or changing configuration values
|
@@ -70,17 +77,21 @@ module Stockpile
|
|
70
77
|
# @return [void]
|
71
78
|
def configure
|
72
79
|
yield(configuration)
|
80
|
+
@redis_connections = Stockpile::RedisConnectionsFactory.build_connections
|
81
|
+
|
73
82
|
nil
|
74
83
|
end
|
75
84
|
|
76
85
|
# Immediatelly expires a cached value for a given key.
|
77
86
|
#
|
78
87
|
# @params key [String] Key to expire
|
88
|
+
# @param db [Symbol] (optional) Which Redis database to expire data from.
|
89
|
+
# Defaults to `:default`
|
79
90
|
#
|
80
91
|
# @return [true, false] Returns true if value existed in cache and was
|
81
92
|
# succesfully expired. Returns false if value did not exist in cache.
|
82
|
-
def expire_cached(key:)
|
83
|
-
Stockpile::CachedValueExpirer.expire_cached(key: key)
|
93
|
+
def expire_cached(db: :default, key:)
|
94
|
+
Stockpile::CachedValueExpirer.expire_cached(db: db, key: key)
|
84
95
|
end
|
85
96
|
|
86
97
|
# Attempts to fetch a value from cache (for a given key). In case of miss
|
@@ -89,6 +100,8 @@ module Stockpile
|
|
89
100
|
#
|
90
101
|
# @param key [String] Key to use for a value lookup from cache or key
|
91
102
|
# to store value at once it is computed
|
103
|
+
# @param db [Symbol] (optional) Which Redis database to cache data in.
|
104
|
+
# Defaults to `:default`
|
92
105
|
# @param ttl [Integer] (optional) Time in seconds to expire cache after.
|
93
106
|
# Defaults to Stockpile::DEFAULT_TTL
|
94
107
|
#
|
@@ -98,8 +111,13 @@ module Stockpile
|
|
98
111
|
# Stockpile.perform_cached(key: 'meaning_of_life', ttl: 42) { 21 * 2 }
|
99
112
|
#
|
100
113
|
# @return Returns a result of block execution
|
101
|
-
def perform_cached(key:, ttl: Stockpile::DEFAULT_TTL, &block)
|
102
|
-
Stockpile::CachedValueReader.read_or_yield(
|
114
|
+
def perform_cached(db: :default, key:, ttl: Stockpile::DEFAULT_TTL, &block)
|
115
|
+
Stockpile::CachedValueReader.read_or_yield(
|
116
|
+
db: db,
|
117
|
+
key: key,
|
118
|
+
ttl: ttl,
|
119
|
+
&block
|
120
|
+
)
|
103
121
|
end
|
104
122
|
|
105
123
|
# API to communicate with Redis database backing cache up.
|
@@ -110,8 +128,8 @@ module Stockpile
|
|
110
128
|
# Store.redis { |r| r.set('meaning_of_life', 42) }
|
111
129
|
#
|
112
130
|
# @return Returns a result of interaction with Redis
|
113
|
-
def redis
|
114
|
-
|
131
|
+
def redis(db: :default)
|
132
|
+
redis_connections.with(db: db) do |connection|
|
115
133
|
yield connection
|
116
134
|
end
|
117
135
|
end
|
@@ -119,8 +137,9 @@ module Stockpile
|
|
119
137
|
# Accessor to connection pool. Defined on top level so it can be memoized
|
120
138
|
# on the topmost level
|
121
139
|
#
|
122
|
-
# @return [
|
123
|
-
|
124
|
-
|
140
|
+
# @return [Stockpile::RedisConnections] RedisConnections object holding all defined
|
141
|
+
# connection pools
|
142
|
+
def redis_connections
|
143
|
+
@redis_connections ||= Stockpile::RedisConnectionsFactory.build_connections
|
125
144
|
end
|
126
145
|
end
|
data/lib/stockpile/cache.rb
CHANGED
@@ -22,21 +22,21 @@ module Stockpile
|
|
22
22
|
module Cache
|
23
23
|
module_function
|
24
24
|
|
25
|
-
def get(key:)
|
26
|
-
value_from_cache = Stockpile.redis { |r| r.get(key) }
|
25
|
+
def get(db: :default, key:)
|
26
|
+
value_from_cache = Stockpile.redis(db: db) { |r| r.get(key) }
|
27
27
|
Oj.load(value_from_cache) if value_from_cache
|
28
28
|
end
|
29
29
|
|
30
|
-
def get_deferred(key:)
|
31
|
-
sleep(Stockpile::SLUMBER_COOLDOWN) until Stockpile.redis { |r| r.exists(key) }
|
32
|
-
value_from_cache = Stockpile.redis { |r| r.get(key) }
|
30
|
+
def get_deferred(db: :default, key:)
|
31
|
+
sleep(Stockpile::SLUMBER_COOLDOWN) until Stockpile.redis(db: db) { |r| r.exists(key) }
|
32
|
+
value_from_cache = Stockpile.redis(db: db) { |r| r.get(key) }
|
33
33
|
Oj.load(value_from_cache)
|
34
34
|
end
|
35
35
|
|
36
|
-
def set(key:, payload:, ttl:)
|
36
|
+
def set(db: :default, key:, payload:, ttl:)
|
37
37
|
payload = Oj.dump(payload)
|
38
|
-
Stockpile.redis { |r| r.set(key, payload) }
|
39
|
-
Stockpile.redis { |r| r.expire(key, ttl) }
|
38
|
+
Stockpile.redis(db: db) { |r| r.set(key, payload) }
|
39
|
+
Stockpile.redis(db: db) { |r| r.expire(key, ttl) }
|
40
40
|
end
|
41
41
|
end
|
42
42
|
end
|
@@ -22,11 +22,11 @@ module Stockpile
|
|
22
22
|
module CachedValueReader
|
23
23
|
module_function
|
24
24
|
|
25
|
-
def read_or_yield(key:, ttl:, &block)
|
26
|
-
if (result = Stockpile::Cache.get(key: key))
|
25
|
+
def read_or_yield(db: :default, key:, ttl:, &block)
|
26
|
+
if (result = Stockpile::Cache.get(db: db, key: key))
|
27
27
|
result
|
28
28
|
else
|
29
|
-
Stockpile::Executor.perform(key: key, ttl: ttl, &block)
|
29
|
+
Stockpile::Executor.perform(db: db, key: key, ttl: ttl, &block)
|
30
30
|
end
|
31
31
|
end
|
32
32
|
end
|
@@ -20,20 +20,25 @@ module Stockpile
|
|
20
20
|
# Holds configuration for cache with writeable attributes allowing
|
21
21
|
# dynamic change of configuration during runtime
|
22
22
|
class Configuration
|
23
|
-
attr_accessor :
|
24
|
-
:redis_url, :sentinels, :slumber
|
23
|
+
attr_accessor :configuration_file, :connection_pool, :connection_timeout,
|
24
|
+
:lock_expiration, :redis_url, :sentinels, :slumber
|
25
25
|
|
26
26
|
def initialize
|
27
|
+
@configuration_file = extract_configuration_file
|
27
28
|
@connection_pool = extract_connection_pool
|
28
29
|
@connection_timeout = extract_connection_timeout
|
29
30
|
@lock_expiration = extract_lock_expiration
|
30
31
|
@redis_url = extract_redis_url
|
31
|
-
@sentinels =
|
32
|
+
@sentinels = extract_sentinels
|
32
33
|
@slumber = extract_slumber
|
33
34
|
end
|
34
35
|
|
35
36
|
private
|
36
37
|
|
38
|
+
def extract_configuration_file
|
39
|
+
ENV.fetch('STOCKPILE_CONFIGURATION_FILE', nil)
|
40
|
+
end
|
41
|
+
|
37
42
|
def extract_connection_pool
|
38
43
|
ENV.fetch(
|
39
44
|
'STOCKPILE_CONNECTION_POOL',
|
@@ -69,11 +74,10 @@ module Stockpile
|
|
69
74
|
).to_i
|
70
75
|
end
|
71
76
|
|
72
|
-
def
|
73
|
-
|
74
|
-
|
75
|
-
|
76
|
-
end
|
77
|
+
def extract_sentinels
|
78
|
+
Stockpile::RedisConnectionsFactory.process_sentinels(
|
79
|
+
sentinels: ENV.fetch('STOCKPILE_REDIS_SENTINELS', '')
|
80
|
+
)
|
77
81
|
end
|
78
82
|
end
|
79
83
|
end
|
data/lib/stockpile/constants.rb
CHANGED
@@ -15,27 +15,32 @@
|
|
15
15
|
# limitations under the License.
|
16
16
|
|
17
17
|
module Stockpile
|
18
|
-
# == Stockpile::
|
18
|
+
# == Stockpile::DefaultRedisConfiguration
|
19
19
|
#
|
20
|
-
#
|
21
|
-
#
|
22
|
-
|
20
|
+
# Confiuration object for a single Redis database cache setup.
|
21
|
+
# Reads values out of environment, default values or uses
|
22
|
+
# configuration provided during runtime.
|
23
|
+
module DefaultRedisConfiguration
|
23
24
|
module_function
|
24
25
|
|
25
|
-
def
|
26
|
-
|
27
|
-
|
28
|
-
|
26
|
+
def configuration
|
27
|
+
[
|
28
|
+
{
|
29
|
+
db: :default,
|
30
|
+
pool_configuration: pool_configuration,
|
31
|
+
redis_configuration: redis_configuration
|
32
|
+
}
|
33
|
+
]
|
29
34
|
end
|
30
35
|
|
31
|
-
def
|
36
|
+
def redis_configuration
|
32
37
|
{
|
33
38
|
url: redis_url,
|
34
39
|
sentinels: sentinels
|
35
40
|
}.delete_if { |_k, v| v.nil? || v.empty? }
|
36
41
|
end
|
37
42
|
|
38
|
-
def
|
43
|
+
def pool_configuration
|
39
44
|
{
|
40
45
|
size: pool_size,
|
41
46
|
timeout: connection_timeout
|
data/lib/stockpile/executor.rb
CHANGED
@@ -22,13 +22,14 @@ module Stockpile
|
|
22
22
|
# value to appear in cache instead. Will timeout after given amount of time
|
23
23
|
# and will execute block if no value can be read from cache.
|
24
24
|
class Executor
|
25
|
-
attr_reader :key, :ttl
|
25
|
+
attr_reader :db, :key, :ttl
|
26
26
|
|
27
|
-
def self.perform(key:, ttl:, &block)
|
28
|
-
new(key, ttl).perform(&block)
|
27
|
+
def self.perform(db: :default, key:, ttl:, &block)
|
28
|
+
new(db, key, ttl).perform(&block)
|
29
29
|
end
|
30
30
|
|
31
|
-
def initialize(key, ttl)
|
31
|
+
def initialize(db, key, ttl)
|
32
|
+
@db = db
|
32
33
|
@key = key
|
33
34
|
@ttl = ttl
|
34
35
|
end
|
@@ -44,13 +45,14 @@ module Stockpile
|
|
44
45
|
private
|
45
46
|
|
46
47
|
def execution
|
47
|
-
@execution ||= Stockpile::Lock.perform_locked(lock_key: lock_key) do
|
48
|
+
@execution ||= Stockpile::Lock.perform_locked(db: db, lock_key: lock_key) do
|
48
49
|
yield
|
49
50
|
end
|
50
51
|
end
|
51
52
|
|
52
53
|
def cache_and_release_execution
|
53
54
|
Stockpile::Cache.set(
|
55
|
+
db: db,
|
54
56
|
key: key,
|
55
57
|
payload: execution.result,
|
56
58
|
ttl: ttl
|
@@ -66,7 +68,7 @@ module Stockpile
|
|
66
68
|
|
67
69
|
def wait_for_cache_or_yield
|
68
70
|
Timeout.timeout(Stockpile.configuration.slumber) do
|
69
|
-
Stockpile::Cache.get_deferred(key: key)
|
71
|
+
Stockpile::Cache.get_deferred(db: db, key: key)
|
70
72
|
end
|
71
73
|
rescue Timeout::Error
|
72
74
|
yield
|
data/lib/stockpile/lock.rb
CHANGED
@@ -23,13 +23,14 @@ module Stockpile
|
|
23
23
|
# Stockpile::LockedExcutionResult will hold Stockpile::FailedLockExecution
|
24
24
|
# as a result of execution
|
25
25
|
class Lock
|
26
|
-
attr_reader :lock_key
|
26
|
+
attr_reader :db, :lock_key
|
27
27
|
|
28
|
-
def self.perform_locked(lock_key:, &block)
|
29
|
-
new(lock_key).perform_locked(&block)
|
28
|
+
def self.perform_locked(db: :default, lock_key:, &block)
|
29
|
+
new(db, lock_key).perform_locked(&block)
|
30
30
|
end
|
31
31
|
|
32
|
-
def initialize(lock_key)
|
32
|
+
def initialize(db, lock_key)
|
33
|
+
@db = db
|
33
34
|
@lock_key = lock_key
|
34
35
|
end
|
35
36
|
|
@@ -44,7 +45,7 @@ module Stockpile
|
|
44
45
|
private
|
45
46
|
|
46
47
|
def failed_execution
|
47
|
-
Stockpile::LockedExcutionResult.new(result: failed_lock, lock_key: lock_key)
|
48
|
+
Stockpile::LockedExcutionResult.new(db: db, result: failed_lock, lock_key: lock_key)
|
48
49
|
end
|
49
50
|
|
50
51
|
def failed_lock
|
@@ -52,11 +53,11 @@ module Stockpile
|
|
52
53
|
end
|
53
54
|
|
54
55
|
def lock
|
55
|
-
Stockpile.redis { |r| r.set(lock_key, 1, nx: true, ex: Stockpile.configuration.lock_expiration) }
|
56
|
+
Stockpile.redis(db: db) { |r| r.set(lock_key, 1, nx: true, ex: Stockpile.configuration.lock_expiration) }
|
56
57
|
end
|
57
58
|
|
58
59
|
def successful_execution
|
59
|
-
Stockpile::LockedExcutionResult.new(result: yield, lock_key: lock_key)
|
60
|
+
Stockpile::LockedExcutionResult.new(db: db, result: yield, lock_key: lock_key)
|
60
61
|
end
|
61
62
|
end
|
62
63
|
end
|
@@ -19,15 +19,16 @@ module Stockpile
|
|
19
19
|
#
|
20
20
|
# Wrapper containing result of locked execution
|
21
21
|
class LockedExcutionResult
|
22
|
-
attr_reader :lock_key, :result
|
22
|
+
attr_reader :db, :lock_key, :result
|
23
23
|
|
24
|
-
def initialize(lock_key:, result:)
|
24
|
+
def initialize(db: :default, lock_key:, result:)
|
25
|
+
@db = db
|
25
26
|
@lock_key = lock_key
|
26
27
|
@result = result
|
27
28
|
end
|
28
29
|
|
29
30
|
def release_lock
|
30
|
-
Stockpile.redis { |r| r.expire(lock_key, 0) }
|
31
|
+
Stockpile.redis(db: db) { |r| r.expire(lock_key, 0) }
|
31
32
|
end
|
32
33
|
|
33
34
|
def success?
|
@@ -0,0 +1,31 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 ConvertKit, LLC
|
4
|
+
#
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
module Stockpile
|
18
|
+
# == Stockpile::RedisConnections
|
19
|
+
#
|
20
|
+
# Wrapper around pools of Redis connections to allow multiple
|
21
|
+
# Redis database support
|
22
|
+
module RedisConnections
|
23
|
+
module_function
|
24
|
+
|
25
|
+
def with(db:)
|
26
|
+
instance_variable_get("@#{db}".to_sym).with do |connection|
|
27
|
+
yield connection
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
31
|
+
end
|
@@ -0,0 +1,56 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 ConvertKit, LLC
|
4
|
+
#
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
module Stockpile
|
18
|
+
# == Stockpile::RedisConnectionsFactory
|
19
|
+
#
|
20
|
+
# Builds out connection pools out of provided configuration. Configurations
|
21
|
+
# are built with `*RedisConfiguration` classes. Providing a `.yml` file will
|
22
|
+
# override everything else and use that to build a config.
|
23
|
+
module RedisConnectionsFactory
|
24
|
+
module_function
|
25
|
+
|
26
|
+
def build_connections
|
27
|
+
configuration.each do |database|
|
28
|
+
pool = ConnectionPool.new(database[:pool_configuration]) do
|
29
|
+
Redis.new(database[:redis_configuration])
|
30
|
+
end
|
31
|
+
|
32
|
+
RedisConnections.instance_variable_set(
|
33
|
+
"@#{database[:db]}".to_sym,
|
34
|
+
pool
|
35
|
+
)
|
36
|
+
end
|
37
|
+
|
38
|
+
RedisConnections
|
39
|
+
end
|
40
|
+
|
41
|
+
def configuration
|
42
|
+
if Stockpile.configuration.configuration_file
|
43
|
+
Stockpile::YamlRedisConfiguration.configuration
|
44
|
+
else
|
45
|
+
Stockpile::DefaultRedisConfiguration.configuration
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
def process_sentinels(sentinels:)
|
50
|
+
sentinels.split(',').map do |sentinel|
|
51
|
+
host, port = sentinel.split(':')
|
52
|
+
{ host: host, port: port.to_i }
|
53
|
+
end
|
54
|
+
end
|
55
|
+
end
|
56
|
+
end
|
@@ -0,0 +1,61 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
# Copyright 2019 ConvertKit, LLC
|
4
|
+
#
|
5
|
+
# Licensed under the Apache License, Version 2.0 (the "License");
|
6
|
+
# you may not use this file except in compliance with the License.
|
7
|
+
# You may obtain a copy of the License at
|
8
|
+
#
|
9
|
+
# http://www.apache.org/licenses/LICENSE-2.0
|
10
|
+
#
|
11
|
+
# Unless required by applicable law or agreed to in writing, software
|
12
|
+
# distributed under the License is distributed on an "AS IS" BASIS,
|
13
|
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14
|
+
# See the License for the specific language governing permissions and
|
15
|
+
# limitations under the License.
|
16
|
+
|
17
|
+
module Stockpile
|
18
|
+
# == Stockpile::YamlRedisConfiguration
|
19
|
+
#
|
20
|
+
# Confiuration object a multiple Redis database cache setup. Reads
|
21
|
+
# configuration out of provided `.yml` file.
|
22
|
+
module YamlRedisConfiguration
|
23
|
+
module_function
|
24
|
+
|
25
|
+
def configuration
|
26
|
+
parsed_configuration.map do |database, settings|
|
27
|
+
{
|
28
|
+
db: database,
|
29
|
+
pool_configuration: extract_pool(settings: settings),
|
30
|
+
redis_configuration: extract_redis(settings: settings)
|
31
|
+
}
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
def extract_redis(settings:)
|
36
|
+
sentinels = Stockpile::RedisConnectionsFactory.process_sentinels(
|
37
|
+
sentinels: settings['sentinels'] || ''
|
38
|
+
)
|
39
|
+
|
40
|
+
{
|
41
|
+
url: settings['url'],
|
42
|
+
sentinels: sentinels
|
43
|
+
}.delete_if { |_k, v| v.nil? || v.empty? }
|
44
|
+
end
|
45
|
+
|
46
|
+
def extract_pool(settings:)
|
47
|
+
{
|
48
|
+
size: settings.dig('pool_options', 'size'),
|
49
|
+
timeout: settings.dig('pool_options', 'timeout')
|
50
|
+
}
|
51
|
+
end
|
52
|
+
|
53
|
+
def parsed_configuration
|
54
|
+
YAML.safe_load(raw_configuration)
|
55
|
+
end
|
56
|
+
|
57
|
+
def raw_configuration
|
58
|
+
File.open(Stockpile.configuration.configuration_file).read
|
59
|
+
end
|
60
|
+
end
|
61
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: stockpile_cache
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.2.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- ConvertKit, LLC
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-11-28 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: connection_pool
|
@@ -93,11 +93,14 @@ files:
|
|
93
93
|
- lib/stockpile/cached_value_reader.rb
|
94
94
|
- lib/stockpile/configuration.rb
|
95
95
|
- lib/stockpile/constants.rb
|
96
|
+
- lib/stockpile/default_redis_configuration.rb
|
96
97
|
- lib/stockpile/executor.rb
|
97
98
|
- lib/stockpile/failed_lock_execution.rb
|
98
99
|
- lib/stockpile/lock.rb
|
99
100
|
- lib/stockpile/locked_execution_result.rb
|
100
|
-
- lib/stockpile/
|
101
|
+
- lib/stockpile/redis_connections.rb
|
102
|
+
- lib/stockpile/redis_connections_factory.rb
|
103
|
+
- lib/stockpile/yaml_redis_configuration.rb
|
101
104
|
- lib/stockpile_cache.rb
|
102
105
|
- stockpile-cache.gemspec
|
103
106
|
homepage:
|