solid_cache 0.7.0 → 1.0.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +96 -56
- data/Rakefile +11 -2
- data/app/models/solid_cache/entry/encryption.rb +15 -0
- data/app/models/solid_cache/entry.rb +45 -91
- data/db/migrate/20240820123641_create_solid_cache_entries.rb +29 -0
- data/lib/generators/solid_cache/install/install_generator.rb +2 -4
- data/lib/generators/solid_cache/install/templates/config/solid_cache.yml.tt +1 -2
- data/lib/solid_cache/configuration.rb +20 -2
- data/lib/solid_cache/engine.rb +10 -0
- data/lib/solid_cache/store/entries.rb +2 -2
- data/lib/solid_cache/version.rb +1 -1
- metadata +10 -12
- data/db/migrate/20230724121448_create_solid_cache_entries.rb +0 -11
- data/db/migrate/20240108155507_add_key_hash_and_byte_size_to_solid_cache_entries.rb +0 -8
- data/db/migrate/20240110111600_add_key_hash_and_byte_size_indexes_and_null_constraints_to_solid_cache_entries.rb +0 -11
- data/db/migrate/20240110111702_remove_key_index_from_solid_cache_entries.rb +0 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0ef3fc7a701fc5af744a290377eae445564a040ccb938e2ea64caf1bab5e90da
|
4
|
+
data.tar.gz: dffd9ee1ab8ea5f6f96620dfcaba35092bd7d5fd9ff5bed210271666e2b839fe
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 22441a9d3132510beff5347788844f6bb05a7da369220ccdc2228587aabb8c2cd51dfb81eecff7ffc0febc0d8b1dea2f1ed852e6393e2a7bfa0af01418913d2a
|
7
|
+
data.tar.gz: 83b296209289d1399ac3b2b3857b6b2740d7d1da6e51f43ef9873e77179681ee65e05f6ca50afd9c10f0605279df637fa8ef8ba7dedd0f4b6a19349aa086c553
|
data/README.md
CHANGED
@@ -6,13 +6,7 @@ Solid Cache is a database-backed Active Support cache store implementation.
|
|
6
6
|
|
7
7
|
Using SQL databases backed by SSDs we can have caches that are much larger and cheaper than traditional memory-only Redis or Memcached backed caches.
|
8
8
|
|
9
|
-
##
|
10
|
-
|
11
|
-
To set Solid Cache as your Rails cache, you should add this to your environment config:
|
12
|
-
|
13
|
-
```ruby
|
14
|
-
config.cache_store = :solid_cache_store
|
15
|
-
```
|
9
|
+
## Introduction
|
16
10
|
|
17
11
|
Solid Cache is a FIFO (first in, first out) cache. While this is not as efficient as an LRU cache, it is mitigated by the longer cache lifespan.
|
18
12
|
|
@@ -21,7 +15,7 @@ A FIFO cache is much easier to manage:
|
|
21
15
|
2. We can estimate and control the cache size by comparing the maximum and minimum IDs.
|
22
16
|
3. By deleting from one end of the table and adding at the other end we can avoid fragmentation (on MySQL at least).
|
23
17
|
|
24
|
-
|
18
|
+
## Installation
|
25
19
|
Add this line to your application's `Gemfile`:
|
26
20
|
|
27
21
|
```ruby
|
@@ -38,13 +32,69 @@ Or install it yourself as:
|
|
38
32
|
$ gem install solid_cache
|
39
33
|
```
|
40
34
|
|
41
|
-
|
35
|
+
#### Cache database configuration
|
36
|
+
|
37
|
+
The default installation of Solid Cache expects a database named `cache` in `database.yml`. It should
|
38
|
+
have it's own connection pool to avoid mixing cache queries in other transactions.
|
39
|
+
|
40
|
+
You can use the primary database for your cache like this:
|
41
|
+
|
42
|
+
```yaml
|
43
|
+
# config/database.yml
|
44
|
+
production:
|
45
|
+
primary: &production_primary
|
46
|
+
...
|
47
|
+
cache:
|
48
|
+
<<: *production_primary
|
49
|
+
```
|
50
|
+
|
51
|
+
Or a separate database like this:
|
52
|
+
|
53
|
+
```yaml
|
54
|
+
production:
|
55
|
+
primary:
|
56
|
+
...
|
57
|
+
cache:
|
58
|
+
database: cache_development
|
59
|
+
host: 127.0.0.1
|
60
|
+
migrations_paths: "db/cache/migrate"
|
61
|
+
```
|
62
|
+
|
63
|
+
#### Install Solid Cache
|
64
|
+
|
65
|
+
Now, you need to install the necessary migrations and configure the cache store. You can do both at once using the provided generator:
|
66
|
+
|
67
|
+
```bash
|
68
|
+
# If using the primary database
|
69
|
+
$ bin/rails generate solid_cache:install
|
70
|
+
|
71
|
+
# Or if using a dedicated database
|
72
|
+
$ DATABASE=cache bin/rails generate solid_cache:install
|
73
|
+
```
|
74
|
+
|
75
|
+
This will set solid_cache as the cache store in production, and will copy the optional configuration file and the required migration over to your app.
|
76
|
+
|
77
|
+
Alternatively, you can add only the migration to your app:
|
42
78
|
|
43
79
|
```bash
|
44
|
-
|
80
|
+
# If using the primary database
|
81
|
+
$ bin/rails generate solid_cache:install:migrations
|
82
|
+
|
83
|
+
# Or if using a dedicated database
|
84
|
+
$ DATABASE=cache bin/rails generate solid_cache:install:migrations
|
85
|
+
```
|
86
|
+
|
87
|
+
And set Solid Cache as your application's cache store backend manually, in your environment config:
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
# config/environments/production.rb
|
91
|
+
config.cache_store = :solid_cache_store
|
45
92
|
```
|
46
93
|
|
47
|
-
|
94
|
+
#### Run migrations
|
95
|
+
|
96
|
+
Finally, you need to run the migrations:
|
97
|
+
|
48
98
|
```bash
|
49
99
|
$ bin/rails db:migrate
|
50
100
|
```
|
@@ -104,7 +154,9 @@ There are three options that can be set on the engine:
|
|
104
154
|
|
105
155
|
- `executor` - the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap asynchronous operations, defaults to the app executor
|
106
156
|
- `connects_to` - a custom connects to value for the abstract `SolidCache::Record` active record model. Required for sharding and/or using a separate cache database to the main app. This will overwrite any value set in `config/solid_cache.yml`
|
107
|
-
- `size_estimate_samples` - if `max_size` is set on the cache, the number of the samples used to
|
157
|
+
- `size_estimate_samples` - if `max_size` is set on the cache, the number of the samples used to estimate the size.
|
158
|
+
- `encrypted` - whether cache values should be encrypted (see [Enabling encryption](#enabling-encryption))
|
159
|
+
- `encryption_context_properties` - custom encryption context properties
|
108
160
|
|
109
161
|
These can be set in your Rails configuration:
|
110
162
|
|
@@ -150,46 +202,6 @@ Only triggering expiry when we write means that if the cache is idle, the backgr
|
|
150
202
|
|
151
203
|
If you want the cache expiry to be run in a background job instead of a thread, you can set `expiry_method` to `:job`. This will enqueue a `SolidCache::ExpiryJob`.
|
152
204
|
|
153
|
-
### Using a dedicated cache database
|
154
|
-
|
155
|
-
Add database configuration to database.yml, e.g.:
|
156
|
-
|
157
|
-
```
|
158
|
-
development:
|
159
|
-
cache:
|
160
|
-
database: cache_development
|
161
|
-
host: 127.0.0.1
|
162
|
-
migrations_paths: "db/cache/migrate"
|
163
|
-
```
|
164
|
-
|
165
|
-
Create database:
|
166
|
-
```
|
167
|
-
$ bin/rails db:create
|
168
|
-
```
|
169
|
-
|
170
|
-
Install migrations:
|
171
|
-
```
|
172
|
-
$ bin/rails solid_cache:install:migrations
|
173
|
-
```
|
174
|
-
|
175
|
-
Move migrations to custom migrations folder:
|
176
|
-
```
|
177
|
-
$ mkdir -p db/cache/migrate
|
178
|
-
$ mv db/migrate/*.solid_cache.rb db/cache/migrate
|
179
|
-
```
|
180
|
-
|
181
|
-
Set the engine configuration to point to the new database:
|
182
|
-
```yaml
|
183
|
-
# config/solid_cache.yml
|
184
|
-
production:
|
185
|
-
database: cache
|
186
|
-
```
|
187
|
-
|
188
|
-
Run migrations:
|
189
|
-
```
|
190
|
-
$ bin/rails db:migrate
|
191
|
-
```
|
192
|
-
|
193
205
|
### Sharding the cache
|
194
206
|
|
195
207
|
Solid Cache uses the [Maglev](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44824.pdf) consistent hashing scheme to shard the cache across multiple databases.
|
@@ -223,14 +235,42 @@ production:
|
|
223
235
|
|
224
236
|
### Enabling encryption
|
225
237
|
|
226
|
-
|
238
|
+
To encrypt the cache values, you can add set the encrypt property.
|
227
239
|
|
240
|
+
```yaml
|
241
|
+
# config/solid_cache.yml
|
242
|
+
production:
|
243
|
+
encrypt: true
|
244
|
+
```
|
245
|
+
or
|
228
246
|
```ruby
|
229
|
-
|
230
|
-
|
231
|
-
end
|
247
|
+
# application.rb
|
248
|
+
config.solid_cache.encrypt = true
|
232
249
|
```
|
233
250
|
|
251
|
+
You will need to set up your application to (use Active Record Encryption)[https://guides.rubyonrails.org/active_record_encryption.html].
|
252
|
+
|
253
|
+
Solid Cache by default uses a custom encryptor and message serializer that are optimised for it.
|
254
|
+
|
255
|
+
Firstly it disabled compression with the encryptor `ActiveRecord::Encryption::Encryptor.new(compress: false)` - the cache already compresses the data.
|
256
|
+
Secondly it uses `ActiveRecord::Encryption::MessagePackMessageSerializer.new` as the serializer. This serializer can only be used for binary columns,
|
257
|
+
but can store about 40% more data than the standard serializer.
|
258
|
+
|
259
|
+
You can choose your own context properties instead if you prefer:
|
260
|
+
|
261
|
+
```ruby
|
262
|
+
# application.rb
|
263
|
+
config.solid_cache.encryption_context_properties = {
|
264
|
+
encryptor: ActiveRecord::Encryption::Encryptor.new,
|
265
|
+
message_serializer: ActiveRecord::Encryption::MessageSerializer.new
|
266
|
+
}
|
267
|
+
```
|
268
|
+
|
269
|
+
**Note**
|
270
|
+
|
271
|
+
Encryption currently does not work for PostgreSQL, as Rails does not yet support encrypting binary columns for it.
|
272
|
+
See https://github.com/rails/rails/pull/52650.
|
273
|
+
|
234
274
|
### Index size limits
|
235
275
|
The Solid Cache migrations try to create an index with 1024 byte entries. If that is too big for your database, you should:
|
236
276
|
|
data/Rakefile
CHANGED
@@ -15,7 +15,9 @@ def run_without_aborting(*tasks)
|
|
15
15
|
|
16
16
|
tasks.each do |task|
|
17
17
|
Rake::Task[task].invoke
|
18
|
-
rescue Exception
|
18
|
+
rescue Exception => e
|
19
|
+
puts e.message
|
20
|
+
puts e.backtrace
|
19
21
|
errors << task
|
20
22
|
end
|
21
23
|
|
@@ -23,7 +25,7 @@ def run_without_aborting(*tasks)
|
|
23
25
|
end
|
24
26
|
|
25
27
|
def configs
|
26
|
-
[ :default, :connects_to, :database, :no_database, :shards, :unprepared_statements ]
|
28
|
+
[ :default, :connects_to, :database, :encrypted, :encrypted_custom, :no_database, :shards, :unprepared_statements ]
|
27
29
|
end
|
28
30
|
|
29
31
|
task :test do
|
@@ -34,6 +36,11 @@ end
|
|
34
36
|
configs.each do |config|
|
35
37
|
namespace :test do
|
36
38
|
task config do
|
39
|
+
if config.to_s.start_with?("encrypted") && ENV["TARGET_DB"] == "postgres"
|
40
|
+
puts "Skipping encrypted tests on PostgreSQL as binary encrypted columns are not supported by Rails yet"
|
41
|
+
next
|
42
|
+
end
|
43
|
+
|
37
44
|
if config == :default
|
38
45
|
sh("bin/rails test")
|
39
46
|
else
|
@@ -42,3 +49,5 @@ configs.each do |config|
|
|
42
49
|
end
|
43
50
|
end
|
44
51
|
end
|
52
|
+
|
53
|
+
task default: [:test]
|
@@ -0,0 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module SolidCache
|
4
|
+
class Entry
|
5
|
+
module Encryption
|
6
|
+
extend ActiveSupport::Concern
|
7
|
+
|
8
|
+
included do
|
9
|
+
if SolidCache.configuration.encrypt?
|
10
|
+
encrypts :value, **SolidCache.configuration.encryption_context_properties
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -2,41 +2,45 @@
|
|
2
2
|
|
3
3
|
module SolidCache
|
4
4
|
class Entry < Record
|
5
|
-
include Expiration, Size
|
5
|
+
include Encryption, Expiration, Size
|
6
6
|
|
7
7
|
# The estimated cost of an extra row in bytes, including fixed size columns, overhead, indexes and free space
|
8
8
|
# Based on experimentation on SQLite, MySQL and Postgresql.
|
9
9
|
# A bit high for SQLite (more like 90 bytes), but about right for MySQL/Postgresql.
|
10
10
|
ESTIMATED_ROW_OVERHEAD = 140
|
11
|
+
|
12
|
+
# Assuming MessagePack serialization
|
13
|
+
ESTIMATED_ENCRYPTION_OVERHEAD = 170
|
14
|
+
|
11
15
|
KEY_HASH_ID_RANGE = -(2**63)..(2**63 - 1)
|
12
16
|
|
13
17
|
class << self
|
14
18
|
def write(key, value)
|
15
|
-
|
19
|
+
write_multi([ { key: key, value: value } ])
|
16
20
|
end
|
17
21
|
|
18
22
|
def write_multi(payloads)
|
19
|
-
|
23
|
+
without_query_cache do
|
24
|
+
upsert_all \
|
25
|
+
add_key_hash_and_byte_size(payloads),
|
26
|
+
unique_by: upsert_unique_by, on_duplicate: :update, update_only: [ :key, :value, :byte_size ]
|
27
|
+
end
|
20
28
|
end
|
21
29
|
|
22
30
|
def read(key)
|
23
|
-
|
24
|
-
result[1] if result&.first == key
|
31
|
+
read_multi([key])[key]
|
25
32
|
end
|
26
33
|
|
27
34
|
def read_multi(keys)
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
end
|
32
|
-
|
33
|
-
def delete_by_key(key)
|
34
|
-
delete_no_query_cache(:key_hash, key_hash_for(key)) > 0
|
35
|
+
without_query_cache do
|
36
|
+
find_by_sql([select_sql(keys), *key_hashes_for(keys)]).pluck(:key, :value).to_h
|
37
|
+
end
|
35
38
|
end
|
36
39
|
|
37
|
-
def
|
38
|
-
|
39
|
-
|
40
|
+
def delete_by_key(*keys)
|
41
|
+
without_query_cache do
|
42
|
+
where(key_hash: key_hashes_for(keys)).delete_all
|
43
|
+
end
|
40
44
|
end
|
41
45
|
|
42
46
|
def clear_truncate
|
@@ -44,12 +48,14 @@ module SolidCache
|
|
44
48
|
end
|
45
49
|
|
46
50
|
def clear_delete
|
47
|
-
|
51
|
+
without_query_cache do
|
52
|
+
in_batches.delete_all
|
53
|
+
end
|
48
54
|
end
|
49
55
|
|
50
56
|
def lock_and_write(key, &block)
|
51
57
|
transaction do
|
52
|
-
|
58
|
+
without_query_cache do
|
53
59
|
result = lock.where(key_hash: key_hash_for(key)).pick(:key, :value)
|
54
60
|
new_value = block.call(result&.first == key ? result[1] : nil)
|
55
61
|
write(key, new_value) if new_value
|
@@ -59,33 +65,12 @@ module SolidCache
|
|
59
65
|
end
|
60
66
|
|
61
67
|
def id_range
|
62
|
-
|
68
|
+
without_query_cache do
|
63
69
|
pick(Arel.sql("max(id) - min(id) + 1")) || 0
|
64
70
|
end
|
65
71
|
end
|
66
72
|
|
67
73
|
private
|
68
|
-
def upsert_all_no_query_cache(payloads)
|
69
|
-
args = [ self.all,
|
70
|
-
connection_for_insert_all,
|
71
|
-
add_key_hash_and_byte_size(payloads) ].compact
|
72
|
-
options = { unique_by: upsert_unique_by,
|
73
|
-
on_duplicate: :update,
|
74
|
-
update_only: upsert_update_only }
|
75
|
-
insert_all = ActiveRecord::InsertAll.new(*args, **options)
|
76
|
-
sql = connection.build_insert_sql(ActiveRecord::InsertAll::Builder.new(insert_all))
|
77
|
-
|
78
|
-
message = +"#{self} "
|
79
|
-
message << "Bulk " if payloads.many?
|
80
|
-
message << "Upsert"
|
81
|
-
# exec_query_method does not clear the query cache, exec_insert_all does
|
82
|
-
connection.send exec_query_method, sql, message
|
83
|
-
end
|
84
|
-
|
85
|
-
def connection_for_insert_all
|
86
|
-
Rails.version >= "7.2" ? connection : nil
|
87
|
-
end
|
88
|
-
|
89
74
|
def add_key_hash_and_byte_size(payloads)
|
90
75
|
payloads.map do |payload|
|
91
76
|
payload.dup.tap do |payload|
|
@@ -95,73 +80,42 @@ module SolidCache
|
|
95
80
|
end
|
96
81
|
end
|
97
82
|
|
98
|
-
def exec_query_method
|
99
|
-
connection.respond_to?(:internal_exec_query) ? :internal_exec_query : :exec_query
|
100
|
-
end
|
101
|
-
|
102
83
|
def upsert_unique_by
|
103
84
|
connection.supports_insert_conflict_target? ? :key_hash : nil
|
104
85
|
end
|
105
86
|
|
106
|
-
def
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
87
|
+
def select_sql(keys)
|
88
|
+
@get_sql ||= {}
|
89
|
+
@get_sql[keys.count] ||= \
|
90
|
+
where(key_hash: [ "1111", "2222" ])
|
91
|
+
.select(:key, :value)
|
92
|
+
.to_sql
|
93
|
+
.gsub("1111, 2222", (["?"] * keys.count).join(", "))
|
112
94
|
end
|
113
95
|
|
114
|
-
def
|
115
|
-
|
116
|
-
|
96
|
+
def key_hash_for(key)
|
97
|
+
# Need to unpack this as a signed integer - Postgresql and SQLite don't support unsigned integers
|
98
|
+
Digest::SHA256.digest(key.to_s).unpack("q>").first
|
117
99
|
end
|
118
100
|
|
119
|
-
def
|
120
|
-
|
121
|
-
Arel::Collectors::SQLString.new,
|
122
|
-
Arel::Collectors::Bind.new,
|
123
|
-
)
|
124
|
-
|
125
|
-
connection.visitor.compile(relation.arel.ast, collector)[0]
|
101
|
+
def key_hashes_for(keys)
|
102
|
+
keys.map { |key| key_hash_for(key) }
|
126
103
|
end
|
127
104
|
|
128
|
-
def
|
129
|
-
|
130
|
-
if connection.prepared_statements?
|
131
|
-
result = connection.select_all(sanitize_sql(query), "#{name} Load", Array(values), preparable: true)
|
132
|
-
else
|
133
|
-
result = connection.select_all(sanitize_sql([ query, *values ]), "#{name} Load", Array(values), preparable: false)
|
134
|
-
end
|
135
|
-
|
136
|
-
result.cast_values(SolidCache::Entry.attribute_types)
|
137
|
-
end
|
105
|
+
def byte_size_for(payload)
|
106
|
+
payload[:key].to_s.bytesize + payload[:value].to_s.bytesize + estimated_row_overhead
|
138
107
|
end
|
139
108
|
|
140
|
-
def
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
145
|
-
# exec_delete does not clear the query cache
|
146
|
-
if connection.prepared_statements?
|
147
|
-
connection.exec_delete(sql, "#{name} Delete All", Array(values))
|
148
|
-
else
|
149
|
-
connection.exec_delete(sql, "#{name} Delete All")
|
150
|
-
end
|
109
|
+
def estimated_row_overhead
|
110
|
+
if SolidCache.configuration.encrypt?
|
111
|
+
ESTIMATED_ROW_OVERHEAD + ESTIMATED_ENCRYPTION_OVERHEAD
|
112
|
+
else
|
113
|
+
ESTIMATED_ROW_OVERHEAD
|
151
114
|
end
|
152
115
|
end
|
153
116
|
|
154
|
-
def
|
155
|
-
|
156
|
-
end
|
157
|
-
|
158
|
-
def key_hash_for(key)
|
159
|
-
# Need to unpack this as a signed integer - Postgresql and SQLite don't support unsigned integers
|
160
|
-
Digest::SHA256.digest(key.to_s).unpack("q>").first
|
161
|
-
end
|
162
|
-
|
163
|
-
def byte_size_for(payload)
|
164
|
-
payload[:key].to_s.bytesize + payload[:value].to_s.bytesize + ESTIMATED_ROW_OVERHEAD
|
117
|
+
def without_query_cache(&block)
|
118
|
+
uncached(dirties: false, &block)
|
165
119
|
end
|
166
120
|
end
|
167
121
|
end
|
@@ -0,0 +1,29 @@
|
|
1
|
+
class CreateSolidCacheEntries < ActiveRecord::Migration[7.2]
|
2
|
+
def up
|
3
|
+
create_table :solid_cache_entries, if_not_exists: true do |t|
|
4
|
+
t.binary :key, null: false, limit: 1024
|
5
|
+
t.binary :value, null: false, limit: 512.megabytes
|
6
|
+
t.datetime :created_at, null: false
|
7
|
+
t.integer :key_hash, null: false, limit: 8
|
8
|
+
t.integer :byte_size, null: false, limit: 4
|
9
|
+
|
10
|
+
t.index :key_hash, unique: true
|
11
|
+
t.index [:key_hash, :byte_size]
|
12
|
+
t.index :byte_size
|
13
|
+
end
|
14
|
+
|
15
|
+
raise "column \"key_hash\" does not exist" unless column_exists? :solid_cache_entries, :key_hash
|
16
|
+
rescue => e
|
17
|
+
if e.message =~ /(column "key_hash" does not exist|no such column: key_hash)/
|
18
|
+
raise \
|
19
|
+
"Could not find key_hash column on solid_cache_entries, if upgrading from v0.3 or earlier, have you followed " \
|
20
|
+
"the steps in https://github.com/rails/solid_cache/blob/main/upgrading_to_version_0.4.x.md?"
|
21
|
+
else
|
22
|
+
raise
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def down
|
27
|
+
drop_table :solid_cache_entries
|
28
|
+
end
|
29
|
+
end
|
@@ -7,10 +7,8 @@ class SolidCache::InstallGenerator < Rails::Generators::Base
|
|
7
7
|
desc: "Skip migrations"
|
8
8
|
|
9
9
|
def add_rails_cache
|
10
|
-
|
11
|
-
|
12
|
-
gsub_file env_config, /(# )?config\.cache_store = (:(?!null_store).*)/, "config.cache_store = :solid_cache_store"
|
13
|
-
end
|
10
|
+
if (env_config = Pathname(destination_root).join("config/environments/production.rb")).exist?
|
11
|
+
gsub_file env_config, /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
|
14
12
|
end
|
15
13
|
end
|
16
14
|
|
@@ -2,12 +2,15 @@
|
|
2
2
|
|
3
3
|
module SolidCache
|
4
4
|
class Configuration
|
5
|
-
attr_reader :store_options, :connects_to, :executor, :size_estimate_samples
|
5
|
+
attr_reader :store_options, :connects_to, :executor, :size_estimate_samples, :encrypt, :encryption_context_properties
|
6
6
|
|
7
|
-
def initialize(store_options: {}, database: nil, databases: nil, connects_to: nil, executor: nil, size_estimate_samples: 10_000)
|
7
|
+
def initialize(store_options: {}, database: nil, databases: nil, connects_to: nil, executor: nil, encrypt: false, encryption_context_properties: nil, size_estimate_samples: 10_000)
|
8
8
|
@store_options = store_options
|
9
9
|
@size_estimate_samples = size_estimate_samples
|
10
10
|
@executor = executor
|
11
|
+
@encrypt = encrypt
|
12
|
+
@encryption_context_properties = encryption_context_properties
|
13
|
+
@encryption_context_properties ||= default_encryption_context_properties if encrypt?
|
11
14
|
set_connects_to(database: database, databases: databases, connects_to: connects_to)
|
12
15
|
end
|
13
16
|
|
@@ -19,6 +22,10 @@ module SolidCache
|
|
19
22
|
sharded? ? connects_to[:shards].keys : []
|
20
23
|
end
|
21
24
|
|
25
|
+
def encrypt?
|
26
|
+
encrypt.present?
|
27
|
+
end
|
28
|
+
|
22
29
|
private
|
23
30
|
def set_connects_to(database:, databases:, connects_to:)
|
24
31
|
if [database, databases, connects_to].compact.size > 1
|
@@ -37,5 +44,16 @@ module SolidCache
|
|
37
44
|
nil
|
38
45
|
end
|
39
46
|
end
|
47
|
+
|
48
|
+
def default_encryption_context_properties
|
49
|
+
require "active_record/encryption/message_pack_message_serializer"
|
50
|
+
|
51
|
+
{
|
52
|
+
# No need to compress, the cache does that already
|
53
|
+
encryptor: ActiveRecord::Encryption::Encryptor.new(compress: false),
|
54
|
+
# Binary column only serializer that is 40% more efficient than the default MessageSerializer
|
55
|
+
message_serializer: ActiveRecord::Encryption::MessagePackMessageSerializer.new
|
56
|
+
}
|
57
|
+
end
|
40
58
|
end
|
41
59
|
end
|
data/lib/solid_cache/engine.rb
CHANGED
@@ -18,6 +18,8 @@ module SolidCache
|
|
18
18
|
|
19
19
|
options[:connects_to] = config.solid_cache.connects_to if config.solid_cache.connects_to
|
20
20
|
options[:size_estimate_samples] = config.solid_cache.size_estimate_samples if config.solid_cache.size_estimate_samples
|
21
|
+
options[:encrypt] = config.solid_cache.encrypt if config.solid_cache.encrypt
|
22
|
+
options[:encryption_context_properties] = config.solid_cache.encryption_context_properties if config.solid_cache.encryption_context_properties
|
21
23
|
|
22
24
|
SolidCache.configuration = SolidCache::Configuration.new(**options)
|
23
25
|
|
@@ -33,5 +35,13 @@ module SolidCache
|
|
33
35
|
config.after_initialize do
|
34
36
|
Rails.cache.setup! if Rails.cache.is_a?(Store)
|
35
37
|
end
|
38
|
+
|
39
|
+
config.after_initialize do
|
40
|
+
if SolidCache.configuration.encrypt? && SolidCache::Record.connection.adapter_name == "PostgreSQL"
|
41
|
+
raise \
|
42
|
+
"Cannot enable encryption for Solid Cache: Active Record Encryption does not currently support " \
|
43
|
+
"encrypting binary columns on PostgreSQL"
|
44
|
+
end
|
45
|
+
end
|
36
46
|
end
|
37
47
|
end
|
@@ -65,13 +65,13 @@ module SolidCache
|
|
65
65
|
|
66
66
|
def entry_delete(key)
|
67
67
|
writing_key(key, failsafe: :delete_entry, failsafe_returning: false) do
|
68
|
-
Entry.delete_by_key(key)
|
68
|
+
Entry.delete_by_key(key) > 0
|
69
69
|
end
|
70
70
|
end
|
71
71
|
|
72
72
|
def entry_delete_multi(entries)
|
73
73
|
writing_keys(entries, failsafe: :delete_multi_entries, failsafe_returning: 0) do
|
74
|
-
Entry.
|
74
|
+
Entry.delete_by_key(*entries)
|
75
75
|
end
|
76
76
|
end
|
77
77
|
end
|
data/lib/solid_cache/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: solid_cache
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Donal McBreen
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-08-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -16,42 +16,42 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '7'
|
19
|
+
version: '7.2'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '7'
|
26
|
+
version: '7.2'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: activejob
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '7'
|
33
|
+
version: '7.2'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version: '7'
|
40
|
+
version: '7.2'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: railties
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
45
|
- - ">="
|
46
46
|
- !ruby/object:Gem::Version
|
47
|
-
version: '7'
|
47
|
+
version: '7.2'
|
48
48
|
type: :runtime
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
52
|
- - ">="
|
53
53
|
- !ruby/object:Gem::Version
|
54
|
-
version: '7'
|
54
|
+
version: '7.2'
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
56
|
name: debug
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
@@ -106,15 +106,13 @@ files:
|
|
106
106
|
- Rakefile
|
107
107
|
- app/jobs/solid_cache/expiry_job.rb
|
108
108
|
- app/models/solid_cache/entry.rb
|
109
|
+
- app/models/solid_cache/entry/encryption.rb
|
109
110
|
- app/models/solid_cache/entry/expiration.rb
|
110
111
|
- app/models/solid_cache/entry/size.rb
|
111
112
|
- app/models/solid_cache/entry/size/estimate.rb
|
112
113
|
- app/models/solid_cache/entry/size/moving_average_estimate.rb
|
113
114
|
- app/models/solid_cache/record.rb
|
114
|
-
- db/migrate/
|
115
|
-
- db/migrate/20240108155507_add_key_hash_and_byte_size_to_solid_cache_entries.rb
|
116
|
-
- db/migrate/20240110111600_add_key_hash_and_byte_size_indexes_and_null_constraints_to_solid_cache_entries.rb
|
117
|
-
- db/migrate/20240110111702_remove_key_index_from_solid_cache_entries.rb
|
115
|
+
- db/migrate/20240820123641_create_solid_cache_entries.rb
|
118
116
|
- lib/active_support/cache/solid_cache_store.rb
|
119
117
|
- lib/generators/solid_cache/install/USAGE
|
120
118
|
- lib/generators/solid_cache/install/install_generator.rb
|
@@ -1,11 +0,0 @@
|
|
1
|
-
class CreateSolidCacheEntries < ActiveRecord::Migration[7.0]
|
2
|
-
def change
|
3
|
-
create_table :solid_cache_entries do |t|
|
4
|
-
t.binary :key, null: false, limit: 1024
|
5
|
-
t.binary :value, null: false, limit: 512.megabytes
|
6
|
-
t.datetime :created_at, null: false
|
7
|
-
|
8
|
-
t.index :key, unique: true
|
9
|
-
end
|
10
|
-
end
|
11
|
-
end
|
@@ -1,11 +0,0 @@
|
|
1
|
-
class AddKeyHashAndByteSizeIndexesAndNullConstraintsToSolidCacheEntries < ActiveRecord::Migration[7.0]
|
2
|
-
def change
|
3
|
-
change_table :solid_cache_entries, bulk: true do |t|
|
4
|
-
t.change_null :key_hash, false
|
5
|
-
t.change_null :byte_size, false
|
6
|
-
t.index :key_hash, unique: true
|
7
|
-
t.index [:key_hash, :byte_size]
|
8
|
-
t.index :byte_size
|
9
|
-
end
|
10
|
-
end
|
11
|
-
end
|