solid_cache 0.7.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +96 -56
- data/Rakefile +11 -2
- data/app/models/solid_cache/entry/encryption.rb +15 -0
- data/app/models/solid_cache/entry.rb +45 -91
- data/db/migrate/20240820123641_create_solid_cache_entries.rb +29 -0
- data/lib/generators/solid_cache/install/install_generator.rb +2 -4
- data/lib/generators/solid_cache/install/templates/config/solid_cache.yml.tt +1 -2
- data/lib/solid_cache/configuration.rb +20 -2
- data/lib/solid_cache/engine.rb +10 -0
- data/lib/solid_cache/store/entries.rb +2 -2
- data/lib/solid_cache/version.rb +1 -1
- metadata +10 -12
- data/db/migrate/20230724121448_create_solid_cache_entries.rb +0 -11
- data/db/migrate/20240108155507_add_key_hash_and_byte_size_to_solid_cache_entries.rb +0 -8
- data/db/migrate/20240110111600_add_key_hash_and_byte_size_indexes_and_null_constraints_to_solid_cache_entries.rb +0 -11
- data/db/migrate/20240110111702_remove_key_index_from_solid_cache_entries.rb +0 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 0ef3fc7a701fc5af744a290377eae445564a040ccb938e2ea64caf1bab5e90da
|
4
|
+
data.tar.gz: dffd9ee1ab8ea5f6f96620dfcaba35092bd7d5fd9ff5bed210271666e2b839fe
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 22441a9d3132510beff5347788844f6bb05a7da369220ccdc2228587aabb8c2cd51dfb81eecff7ffc0febc0d8b1dea2f1ed852e6393e2a7bfa0af01418913d2a
|
7
|
+
data.tar.gz: 83b296209289d1399ac3b2b3857b6b2740d7d1da6e51f43ef9873e77179681ee65e05f6ca50afd9c10f0605279df637fa8ef8ba7dedd0f4b6a19349aa086c553
|
data/README.md
CHANGED
@@ -6,13 +6,7 @@ Solid Cache is a database-backed Active Support cache store implementation.
|
|
6
6
|
|
7
7
|
Using SQL databases backed by SSDs we can have caches that are much larger and cheaper than traditional memory-only Redis or Memcached backed caches.
|
8
8
|
|
9
|
-
##
|
10
|
-
|
11
|
-
To set Solid Cache as your Rails cache, you should add this to your environment config:
|
12
|
-
|
13
|
-
```ruby
|
14
|
-
config.cache_store = :solid_cache_store
|
15
|
-
```
|
9
|
+
## Introduction
|
16
10
|
|
17
11
|
Solid Cache is a FIFO (first in, first out) cache. While this is not as efficient as an LRU cache, it is mitigated by the longer cache lifespan.
|
18
12
|
|
@@ -21,7 +15,7 @@ A FIFO cache is much easier to manage:
|
|
21
15
|
2. We can estimate and control the cache size by comparing the maximum and minimum IDs.
|
22
16
|
3. By deleting from one end of the table and adding at the other end we can avoid fragmentation (on MySQL at least).
|
23
17
|
|
24
|
-
|
18
|
+
## Installation
|
25
19
|
Add this line to your application's `Gemfile`:
|
26
20
|
|
27
21
|
```ruby
|
@@ -38,13 +32,69 @@ Or install it yourself as:
|
|
38
32
|
$ gem install solid_cache
|
39
33
|
```
|
40
34
|
|
41
|
-
|
35
|
+
#### Cache database configuration
|
36
|
+
|
37
|
+
The default installation of Solid Cache expects a database named `cache` in `database.yml`. It should
|
38
|
+
have it's own connection pool to avoid mixing cache queries in other transactions.
|
39
|
+
|
40
|
+
You can use the primary database for your cache like this:
|
41
|
+
|
42
|
+
```yaml
|
43
|
+
# config/database.yml
|
44
|
+
production:
|
45
|
+
primary: &production_primary
|
46
|
+
...
|
47
|
+
cache:
|
48
|
+
<<: *production_primary
|
49
|
+
```
|
50
|
+
|
51
|
+
Or a separate database like this:
|
52
|
+
|
53
|
+
```yaml
|
54
|
+
production:
|
55
|
+
primary:
|
56
|
+
...
|
57
|
+
cache:
|
58
|
+
database: cache_development
|
59
|
+
host: 127.0.0.1
|
60
|
+
migrations_paths: "db/cache/migrate"
|
61
|
+
```
|
62
|
+
|
63
|
+
#### Install Solid Cache
|
64
|
+
|
65
|
+
Now, you need to install the necessary migrations and configure the cache store. You can do both at once using the provided generator:
|
66
|
+
|
67
|
+
```bash
|
68
|
+
# If using the primary database
|
69
|
+
$ bin/rails generate solid_cache:install
|
70
|
+
|
71
|
+
# Or if using a dedicated database
|
72
|
+
$ DATABASE=cache bin/rails generate solid_cache:install
|
73
|
+
```
|
74
|
+
|
75
|
+
This will set solid_cache as the cache store in production, and will copy the optional configuration file and the required migration over to your app.
|
76
|
+
|
77
|
+
Alternatively, you can add only the migration to your app:
|
42
78
|
|
43
79
|
```bash
|
44
|
-
|
80
|
+
# If using the primary database
|
81
|
+
$ bin/rails generate solid_cache:install:migrations
|
82
|
+
|
83
|
+
# Or if using a dedicated database
|
84
|
+
$ DATABASE=cache bin/rails generate solid_cache:install:migrations
|
85
|
+
```
|
86
|
+
|
87
|
+
And set Solid Cache as your application's cache store backend manually, in your environment config:
|
88
|
+
|
89
|
+
```ruby
|
90
|
+
# config/environments/production.rb
|
91
|
+
config.cache_store = :solid_cache_store
|
45
92
|
```
|
46
93
|
|
47
|
-
|
94
|
+
#### Run migrations
|
95
|
+
|
96
|
+
Finally, you need to run the migrations:
|
97
|
+
|
48
98
|
```bash
|
49
99
|
$ bin/rails db:migrate
|
50
100
|
```
|
@@ -104,7 +154,9 @@ There are three options that can be set on the engine:
|
|
104
154
|
|
105
155
|
- `executor` - the [Rails executor](https://guides.rubyonrails.org/threading_and_code_execution.html#executor) used to wrap asynchronous operations, defaults to the app executor
|
106
156
|
- `connects_to` - a custom connects to value for the abstract `SolidCache::Record` active record model. Required for sharding and/or using a separate cache database to the main app. This will overwrite any value set in `config/solid_cache.yml`
|
107
|
-
- `size_estimate_samples` - if `max_size` is set on the cache, the number of the samples used to
|
157
|
+
- `size_estimate_samples` - if `max_size` is set on the cache, the number of the samples used to estimate the size.
|
158
|
+
- `encrypted` - whether cache values should be encrypted (see [Enabling encryption](#enabling-encryption))
|
159
|
+
- `encryption_context_properties` - custom encryption context properties
|
108
160
|
|
109
161
|
These can be set in your Rails configuration:
|
110
162
|
|
@@ -150,46 +202,6 @@ Only triggering expiry when we write means that if the cache is idle, the backgr
|
|
150
202
|
|
151
203
|
If you want the cache expiry to be run in a background job instead of a thread, you can set `expiry_method` to `:job`. This will enqueue a `SolidCache::ExpiryJob`.
|
152
204
|
|
153
|
-
### Using a dedicated cache database
|
154
|
-
|
155
|
-
Add database configuration to database.yml, e.g.:
|
156
|
-
|
157
|
-
```
|
158
|
-
development:
|
159
|
-
cache:
|
160
|
-
database: cache_development
|
161
|
-
host: 127.0.0.1
|
162
|
-
migrations_paths: "db/cache/migrate"
|
163
|
-
```
|
164
|
-
|
165
|
-
Create database:
|
166
|
-
```
|
167
|
-
$ bin/rails db:create
|
168
|
-
```
|
169
|
-
|
170
|
-
Install migrations:
|
171
|
-
```
|
172
|
-
$ bin/rails solid_cache:install:migrations
|
173
|
-
```
|
174
|
-
|
175
|
-
Move migrations to custom migrations folder:
|
176
|
-
```
|
177
|
-
$ mkdir -p db/cache/migrate
|
178
|
-
$ mv db/migrate/*.solid_cache.rb db/cache/migrate
|
179
|
-
```
|
180
|
-
|
181
|
-
Set the engine configuration to point to the new database:
|
182
|
-
```yaml
|
183
|
-
# config/solid_cache.yml
|
184
|
-
production:
|
185
|
-
database: cache
|
186
|
-
```
|
187
|
-
|
188
|
-
Run migrations:
|
189
|
-
```
|
190
|
-
$ bin/rails db:migrate
|
191
|
-
```
|
192
|
-
|
193
205
|
### Sharding the cache
|
194
206
|
|
195
207
|
Solid Cache uses the [Maglev](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44824.pdf) consistent hashing scheme to shard the cache across multiple databases.
|
@@ -223,14 +235,42 @@ production:
|
|
223
235
|
|
224
236
|
### Enabling encryption
|
225
237
|
|
226
|
-
|
238
|
+
To encrypt the cache values, you can add set the encrypt property.
|
227
239
|
|
240
|
+
```yaml
|
241
|
+
# config/solid_cache.yml
|
242
|
+
production:
|
243
|
+
encrypt: true
|
244
|
+
```
|
245
|
+
or
|
228
246
|
```ruby
|
229
|
-
|
230
|
-
|
231
|
-
end
|
247
|
+
# application.rb
|
248
|
+
config.solid_cache.encrypt = true
|
232
249
|
```
|
233
250
|
|
251
|
+
You will need to set up your application to (use Active Record Encryption)[https://guides.rubyonrails.org/active_record_encryption.html].
|
252
|
+
|
253
|
+
Solid Cache by default uses a custom encryptor and message serializer that are optimised for it.
|
254
|
+
|
255
|
+
Firstly it disabled compression with the encryptor `ActiveRecord::Encryption::Encryptor.new(compress: false)` - the cache already compresses the data.
|
256
|
+
Secondly it uses `ActiveRecord::Encryption::MessagePackMessageSerializer.new` as the serializer. This serializer can only be used for binary columns,
|
257
|
+
but can store about 40% more data than the standard serializer.
|
258
|
+
|
259
|
+
You can choose your own context properties instead if you prefer:
|
260
|
+
|
261
|
+
```ruby
|
262
|
+
# application.rb
|
263
|
+
config.solid_cache.encryption_context_properties = {
|
264
|
+
encryptor: ActiveRecord::Encryption::Encryptor.new,
|
265
|
+
message_serializer: ActiveRecord::Encryption::MessageSerializer.new
|
266
|
+
}
|
267
|
+
```
|
268
|
+
|
269
|
+
**Note**
|
270
|
+
|
271
|
+
Encryption currently does not work for PostgreSQL, as Rails does not yet support encrypting binary columns for it.
|
272
|
+
See https://github.com/rails/rails/pull/52650.
|
273
|
+
|
234
274
|
### Index size limits
|
235
275
|
The Solid Cache migrations try to create an index with 1024 byte entries. If that is too big for your database, you should:
|
236
276
|
|
data/Rakefile
CHANGED
@@ -15,7 +15,9 @@ def run_without_aborting(*tasks)
|
|
15
15
|
|
16
16
|
tasks.each do |task|
|
17
17
|
Rake::Task[task].invoke
|
18
|
-
rescue Exception
|
18
|
+
rescue Exception => e
|
19
|
+
puts e.message
|
20
|
+
puts e.backtrace
|
19
21
|
errors << task
|
20
22
|
end
|
21
23
|
|
@@ -23,7 +25,7 @@ def run_without_aborting(*tasks)
|
|
23
25
|
end
|
24
26
|
|
25
27
|
def configs
|
26
|
-
[ :default, :connects_to, :database, :no_database, :shards, :unprepared_statements ]
|
28
|
+
[ :default, :connects_to, :database, :encrypted, :encrypted_custom, :no_database, :shards, :unprepared_statements ]
|
27
29
|
end
|
28
30
|
|
29
31
|
task :test do
|
@@ -34,6 +36,11 @@ end
|
|
34
36
|
configs.each do |config|
|
35
37
|
namespace :test do
|
36
38
|
task config do
|
39
|
+
if config.to_s.start_with?("encrypted") && ENV["TARGET_DB"] == "postgres"
|
40
|
+
puts "Skipping encrypted tests on PostgreSQL as binary encrypted columns are not supported by Rails yet"
|
41
|
+
next
|
42
|
+
end
|
43
|
+
|
37
44
|
if config == :default
|
38
45
|
sh("bin/rails test")
|
39
46
|
else
|
@@ -42,3 +49,5 @@ configs.each do |config|
|
|
42
49
|
end
|
43
50
|
end
|
44
51
|
end
|
52
|
+
|
53
|
+
task default: [:test]
|
@@ -0,0 +1,15 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module SolidCache
|
4
|
+
class Entry
|
5
|
+
module Encryption
|
6
|
+
extend ActiveSupport::Concern
|
7
|
+
|
8
|
+
included do
|
9
|
+
if SolidCache.configuration.encrypt?
|
10
|
+
encrypts :value, **SolidCache.configuration.encryption_context_properties
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
@@ -2,41 +2,45 @@
|
|
2
2
|
|
3
3
|
module SolidCache
|
4
4
|
class Entry < Record
|
5
|
-
include Expiration, Size
|
5
|
+
include Encryption, Expiration, Size
|
6
6
|
|
7
7
|
# The estimated cost of an extra row in bytes, including fixed size columns, overhead, indexes and free space
|
8
8
|
# Based on experimentation on SQLite, MySQL and Postgresql.
|
9
9
|
# A bit high for SQLite (more like 90 bytes), but about right for MySQL/Postgresql.
|
10
10
|
ESTIMATED_ROW_OVERHEAD = 140
|
11
|
+
|
12
|
+
# Assuming MessagePack serialization
|
13
|
+
ESTIMATED_ENCRYPTION_OVERHEAD = 170
|
14
|
+
|
11
15
|
KEY_HASH_ID_RANGE = -(2**63)..(2**63 - 1)
|
12
16
|
|
13
17
|
class << self
|
14
18
|
def write(key, value)
|
15
|
-
|
19
|
+
write_multi([ { key: key, value: value } ])
|
16
20
|
end
|
17
21
|
|
18
22
|
def write_multi(payloads)
|
19
|
-
|
23
|
+
without_query_cache do
|
24
|
+
upsert_all \
|
25
|
+
add_key_hash_and_byte_size(payloads),
|
26
|
+
unique_by: upsert_unique_by, on_duplicate: :update, update_only: [ :key, :value, :byte_size ]
|
27
|
+
end
|
20
28
|
end
|
21
29
|
|
22
30
|
def read(key)
|
23
|
-
|
24
|
-
result[1] if result&.first == key
|
31
|
+
read_multi([key])[key]
|
25
32
|
end
|
26
33
|
|
27
34
|
def read_multi(keys)
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
end
|
32
|
-
|
33
|
-
def delete_by_key(key)
|
34
|
-
delete_no_query_cache(:key_hash, key_hash_for(key)) > 0
|
35
|
+
without_query_cache do
|
36
|
+
find_by_sql([select_sql(keys), *key_hashes_for(keys)]).pluck(:key, :value).to_h
|
37
|
+
end
|
35
38
|
end
|
36
39
|
|
37
|
-
def
|
38
|
-
|
39
|
-
|
40
|
+
def delete_by_key(*keys)
|
41
|
+
without_query_cache do
|
42
|
+
where(key_hash: key_hashes_for(keys)).delete_all
|
43
|
+
end
|
40
44
|
end
|
41
45
|
|
42
46
|
def clear_truncate
|
@@ -44,12 +48,14 @@ module SolidCache
|
|
44
48
|
end
|
45
49
|
|
46
50
|
def clear_delete
|
47
|
-
|
51
|
+
without_query_cache do
|
52
|
+
in_batches.delete_all
|
53
|
+
end
|
48
54
|
end
|
49
55
|
|
50
56
|
def lock_and_write(key, &block)
|
51
57
|
transaction do
|
52
|
-
|
58
|
+
without_query_cache do
|
53
59
|
result = lock.where(key_hash: key_hash_for(key)).pick(:key, :value)
|
54
60
|
new_value = block.call(result&.first == key ? result[1] : nil)
|
55
61
|
write(key, new_value) if new_value
|
@@ -59,33 +65,12 @@ module SolidCache
|
|
59
65
|
end
|
60
66
|
|
61
67
|
def id_range
|
62
|
-
|
68
|
+
without_query_cache do
|
63
69
|
pick(Arel.sql("max(id) - min(id) + 1")) || 0
|
64
70
|
end
|
65
71
|
end
|
66
72
|
|
67
73
|
private
|
68
|
-
def upsert_all_no_query_cache(payloads)
|
69
|
-
args = [ self.all,
|
70
|
-
connection_for_insert_all,
|
71
|
-
add_key_hash_and_byte_size(payloads) ].compact
|
72
|
-
options = { unique_by: upsert_unique_by,
|
73
|
-
on_duplicate: :update,
|
74
|
-
update_only: upsert_update_only }
|
75
|
-
insert_all = ActiveRecord::InsertAll.new(*args, **options)
|
76
|
-
sql = connection.build_insert_sql(ActiveRecord::InsertAll::Builder.new(insert_all))
|
77
|
-
|
78
|
-
message = +"#{self} "
|
79
|
-
message << "Bulk " if payloads.many?
|
80
|
-
message << "Upsert"
|
81
|
-
# exec_query_method does not clear the query cache, exec_insert_all does
|
82
|
-
connection.send exec_query_method, sql, message
|
83
|
-
end
|
84
|
-
|
85
|
-
def connection_for_insert_all
|
86
|
-
Rails.version >= "7.2" ? connection : nil
|
87
|
-
end
|
88
|
-
|
89
74
|
def add_key_hash_and_byte_size(payloads)
|
90
75
|
payloads.map do |payload|
|
91
76
|
payload.dup.tap do |payload|
|
@@ -95,73 +80,42 @@ module SolidCache
|
|
95
80
|
end
|
96
81
|
end
|
97
82
|
|
98
|
-
def exec_query_method
|
99
|
-
connection.respond_to?(:internal_exec_query) ? :internal_exec_query : :exec_query
|
100
|
-
end
|
101
|
-
|
102
83
|
def upsert_unique_by
|
103
84
|
connection.supports_insert_conflict_target? ? :key_hash : nil
|
104
85
|
end
|
105
86
|
|
106
|
-
def
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
87
|
+
def select_sql(keys)
|
88
|
+
@get_sql ||= {}
|
89
|
+
@get_sql[keys.count] ||= \
|
90
|
+
where(key_hash: [ "1111", "2222" ])
|
91
|
+
.select(:key, :value)
|
92
|
+
.to_sql
|
93
|
+
.gsub("1111, 2222", (["?"] * keys.count).join(", "))
|
112
94
|
end
|
113
95
|
|
114
|
-
def
|
115
|
-
|
116
|
-
|
96
|
+
def key_hash_for(key)
|
97
|
+
# Need to unpack this as a signed integer - Postgresql and SQLite don't support unsigned integers
|
98
|
+
Digest::SHA256.digest(key.to_s).unpack("q>").first
|
117
99
|
end
|
118
100
|
|
119
|
-
def
|
120
|
-
|
121
|
-
Arel::Collectors::SQLString.new,
|
122
|
-
Arel::Collectors::Bind.new,
|
123
|
-
)
|
124
|
-
|
125
|
-
connection.visitor.compile(relation.arel.ast, collector)[0]
|
101
|
+
def key_hashes_for(keys)
|
102
|
+
keys.map { |key| key_hash_for(key) }
|
126
103
|
end
|
127
104
|
|
128
|
-
def
|
129
|
-
|
130
|
-
if connection.prepared_statements?
|
131
|
-
result = connection.select_all(sanitize_sql(query), "#{name} Load", Array(values), preparable: true)
|
132
|
-
else
|
133
|
-
result = connection.select_all(sanitize_sql([ query, *values ]), "#{name} Load", Array(values), preparable: false)
|
134
|
-
end
|
135
|
-
|
136
|
-
result.cast_values(SolidCache::Entry.attribute_types)
|
137
|
-
end
|
105
|
+
def byte_size_for(payload)
|
106
|
+
payload[:key].to_s.bytesize + payload[:value].to_s.bytesize + estimated_row_overhead
|
138
107
|
end
|
139
108
|
|
140
|
-
def
|
141
|
-
|
142
|
-
|
143
|
-
|
144
|
-
|
145
|
-
# exec_delete does not clear the query cache
|
146
|
-
if connection.prepared_statements?
|
147
|
-
connection.exec_delete(sql, "#{name} Delete All", Array(values))
|
148
|
-
else
|
149
|
-
connection.exec_delete(sql, "#{name} Delete All")
|
150
|
-
end
|
109
|
+
def estimated_row_overhead
|
110
|
+
if SolidCache.configuration.encrypt?
|
111
|
+
ESTIMATED_ROW_OVERHEAD + ESTIMATED_ENCRYPTION_OVERHEAD
|
112
|
+
else
|
113
|
+
ESTIMATED_ROW_OVERHEAD
|
151
114
|
end
|
152
115
|
end
|
153
116
|
|
154
|
-
def
|
155
|
-
|
156
|
-
end
|
157
|
-
|
158
|
-
def key_hash_for(key)
|
159
|
-
# Need to unpack this as a signed integer - Postgresql and SQLite don't support unsigned integers
|
160
|
-
Digest::SHA256.digest(key.to_s).unpack("q>").first
|
161
|
-
end
|
162
|
-
|
163
|
-
def byte_size_for(payload)
|
164
|
-
payload[:key].to_s.bytesize + payload[:value].to_s.bytesize + ESTIMATED_ROW_OVERHEAD
|
117
|
+
def without_query_cache(&block)
|
118
|
+
uncached(dirties: false, &block)
|
165
119
|
end
|
166
120
|
end
|
167
121
|
end
|
@@ -0,0 +1,29 @@
|
|
1
|
+
class CreateSolidCacheEntries < ActiveRecord::Migration[7.2]
|
2
|
+
def up
|
3
|
+
create_table :solid_cache_entries, if_not_exists: true do |t|
|
4
|
+
t.binary :key, null: false, limit: 1024
|
5
|
+
t.binary :value, null: false, limit: 512.megabytes
|
6
|
+
t.datetime :created_at, null: false
|
7
|
+
t.integer :key_hash, null: false, limit: 8
|
8
|
+
t.integer :byte_size, null: false, limit: 4
|
9
|
+
|
10
|
+
t.index :key_hash, unique: true
|
11
|
+
t.index [:key_hash, :byte_size]
|
12
|
+
t.index :byte_size
|
13
|
+
end
|
14
|
+
|
15
|
+
raise "column \"key_hash\" does not exist" unless column_exists? :solid_cache_entries, :key_hash
|
16
|
+
rescue => e
|
17
|
+
if e.message =~ /(column "key_hash" does not exist|no such column: key_hash)/
|
18
|
+
raise \
|
19
|
+
"Could not find key_hash column on solid_cache_entries, if upgrading from v0.3 or earlier, have you followed " \
|
20
|
+
"the steps in https://github.com/rails/solid_cache/blob/main/upgrading_to_version_0.4.x.md?"
|
21
|
+
else
|
22
|
+
raise
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
def down
|
27
|
+
drop_table :solid_cache_entries
|
28
|
+
end
|
29
|
+
end
|
@@ -7,10 +7,8 @@ class SolidCache::InstallGenerator < Rails::Generators::Base
|
|
7
7
|
desc: "Skip migrations"
|
8
8
|
|
9
9
|
def add_rails_cache
|
10
|
-
|
11
|
-
|
12
|
-
gsub_file env_config, /(# )?config\.cache_store = (:(?!null_store).*)/, "config.cache_store = :solid_cache_store"
|
13
|
-
end
|
10
|
+
if (env_config = Pathname(destination_root).join("config/environments/production.rb")).exist?
|
11
|
+
gsub_file env_config, /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
|
14
12
|
end
|
15
13
|
end
|
16
14
|
|
@@ -2,12 +2,15 @@
|
|
2
2
|
|
3
3
|
module SolidCache
|
4
4
|
class Configuration
|
5
|
-
attr_reader :store_options, :connects_to, :executor, :size_estimate_samples
|
5
|
+
attr_reader :store_options, :connects_to, :executor, :size_estimate_samples, :encrypt, :encryption_context_properties
|
6
6
|
|
7
|
-
def initialize(store_options: {}, database: nil, databases: nil, connects_to: nil, executor: nil, size_estimate_samples: 10_000)
|
7
|
+
def initialize(store_options: {}, database: nil, databases: nil, connects_to: nil, executor: nil, encrypt: false, encryption_context_properties: nil, size_estimate_samples: 10_000)
|
8
8
|
@store_options = store_options
|
9
9
|
@size_estimate_samples = size_estimate_samples
|
10
10
|
@executor = executor
|
11
|
+
@encrypt = encrypt
|
12
|
+
@encryption_context_properties = encryption_context_properties
|
13
|
+
@encryption_context_properties ||= default_encryption_context_properties if encrypt?
|
11
14
|
set_connects_to(database: database, databases: databases, connects_to: connects_to)
|
12
15
|
end
|
13
16
|
|
@@ -19,6 +22,10 @@ module SolidCache
|
|
19
22
|
sharded? ? connects_to[:shards].keys : []
|
20
23
|
end
|
21
24
|
|
25
|
+
def encrypt?
|
26
|
+
encrypt.present?
|
27
|
+
end
|
28
|
+
|
22
29
|
private
|
23
30
|
def set_connects_to(database:, databases:, connects_to:)
|
24
31
|
if [database, databases, connects_to].compact.size > 1
|
@@ -37,5 +44,16 @@ module SolidCache
|
|
37
44
|
nil
|
38
45
|
end
|
39
46
|
end
|
47
|
+
|
48
|
+
def default_encryption_context_properties
|
49
|
+
require "active_record/encryption/message_pack_message_serializer"
|
50
|
+
|
51
|
+
{
|
52
|
+
# No need to compress, the cache does that already
|
53
|
+
encryptor: ActiveRecord::Encryption::Encryptor.new(compress: false),
|
54
|
+
# Binary column only serializer that is 40% more efficient than the default MessageSerializer
|
55
|
+
message_serializer: ActiveRecord::Encryption::MessagePackMessageSerializer.new
|
56
|
+
}
|
57
|
+
end
|
40
58
|
end
|
41
59
|
end
|
data/lib/solid_cache/engine.rb
CHANGED
@@ -18,6 +18,8 @@ module SolidCache
|
|
18
18
|
|
19
19
|
options[:connects_to] = config.solid_cache.connects_to if config.solid_cache.connects_to
|
20
20
|
options[:size_estimate_samples] = config.solid_cache.size_estimate_samples if config.solid_cache.size_estimate_samples
|
21
|
+
options[:encrypt] = config.solid_cache.encrypt if config.solid_cache.encrypt
|
22
|
+
options[:encryption_context_properties] = config.solid_cache.encryption_context_properties if config.solid_cache.encryption_context_properties
|
21
23
|
|
22
24
|
SolidCache.configuration = SolidCache::Configuration.new(**options)
|
23
25
|
|
@@ -33,5 +35,13 @@ module SolidCache
|
|
33
35
|
config.after_initialize do
|
34
36
|
Rails.cache.setup! if Rails.cache.is_a?(Store)
|
35
37
|
end
|
38
|
+
|
39
|
+
config.after_initialize do
|
40
|
+
if SolidCache.configuration.encrypt? && SolidCache::Record.connection.adapter_name == "PostgreSQL"
|
41
|
+
raise \
|
42
|
+
"Cannot enable encryption for Solid Cache: Active Record Encryption does not currently support " \
|
43
|
+
"encrypting binary columns on PostgreSQL"
|
44
|
+
end
|
45
|
+
end
|
36
46
|
end
|
37
47
|
end
|
@@ -65,13 +65,13 @@ module SolidCache
|
|
65
65
|
|
66
66
|
def entry_delete(key)
|
67
67
|
writing_key(key, failsafe: :delete_entry, failsafe_returning: false) do
|
68
|
-
Entry.delete_by_key(key)
|
68
|
+
Entry.delete_by_key(key) > 0
|
69
69
|
end
|
70
70
|
end
|
71
71
|
|
72
72
|
def entry_delete_multi(entries)
|
73
73
|
writing_keys(entries, failsafe: :delete_multi_entries, failsafe_returning: 0) do
|
74
|
-
Entry.
|
74
|
+
Entry.delete_by_key(*entries)
|
75
75
|
end
|
76
76
|
end
|
77
77
|
end
|
data/lib/solid_cache/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: solid_cache
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Donal McBreen
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-08-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activerecord
|
@@ -16,42 +16,42 @@ dependencies:
|
|
16
16
|
requirements:
|
17
17
|
- - ">="
|
18
18
|
- !ruby/object:Gem::Version
|
19
|
-
version: '7'
|
19
|
+
version: '7.2'
|
20
20
|
type: :runtime
|
21
21
|
prerelease: false
|
22
22
|
version_requirements: !ruby/object:Gem::Requirement
|
23
23
|
requirements:
|
24
24
|
- - ">="
|
25
25
|
- !ruby/object:Gem::Version
|
26
|
-
version: '7'
|
26
|
+
version: '7.2'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
28
|
name: activejob
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version: '7'
|
33
|
+
version: '7.2'
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version: '7'
|
40
|
+
version: '7.2'
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: railties
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
45
|
- - ">="
|
46
46
|
- !ruby/object:Gem::Version
|
47
|
-
version: '7'
|
47
|
+
version: '7.2'
|
48
48
|
type: :runtime
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
52
|
- - ">="
|
53
53
|
- !ruby/object:Gem::Version
|
54
|
-
version: '7'
|
54
|
+
version: '7.2'
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
56
|
name: debug
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
@@ -106,15 +106,13 @@ files:
|
|
106
106
|
- Rakefile
|
107
107
|
- app/jobs/solid_cache/expiry_job.rb
|
108
108
|
- app/models/solid_cache/entry.rb
|
109
|
+
- app/models/solid_cache/entry/encryption.rb
|
109
110
|
- app/models/solid_cache/entry/expiration.rb
|
110
111
|
- app/models/solid_cache/entry/size.rb
|
111
112
|
- app/models/solid_cache/entry/size/estimate.rb
|
112
113
|
- app/models/solid_cache/entry/size/moving_average_estimate.rb
|
113
114
|
- app/models/solid_cache/record.rb
|
114
|
-
- db/migrate/
|
115
|
-
- db/migrate/20240108155507_add_key_hash_and_byte_size_to_solid_cache_entries.rb
|
116
|
-
- db/migrate/20240110111600_add_key_hash_and_byte_size_indexes_and_null_constraints_to_solid_cache_entries.rb
|
117
|
-
- db/migrate/20240110111702_remove_key_index_from_solid_cache_entries.rb
|
115
|
+
- db/migrate/20240820123641_create_solid_cache_entries.rb
|
118
116
|
- lib/active_support/cache/solid_cache_store.rb
|
119
117
|
- lib/generators/solid_cache/install/USAGE
|
120
118
|
- lib/generators/solid_cache/install/install_generator.rb
|
@@ -1,11 +0,0 @@
|
|
1
|
-
class CreateSolidCacheEntries < ActiveRecord::Migration[7.0]
|
2
|
-
def change
|
3
|
-
create_table :solid_cache_entries do |t|
|
4
|
-
t.binary :key, null: false, limit: 1024
|
5
|
-
t.binary :value, null: false, limit: 512.megabytes
|
6
|
-
t.datetime :created_at, null: false
|
7
|
-
|
8
|
-
t.index :key, unique: true
|
9
|
-
end
|
10
|
-
end
|
11
|
-
end
|
@@ -1,11 +0,0 @@
|
|
1
|
-
class AddKeyHashAndByteSizeIndexesAndNullConstraintsToSolidCacheEntries < ActiveRecord::Migration[7.0]
|
2
|
-
def change
|
3
|
-
change_table :solid_cache_entries, bulk: true do |t|
|
4
|
-
t.change_null :key_hash, false
|
5
|
-
t.change_null :byte_size, false
|
6
|
-
t.index :key_hash, unique: true
|
7
|
-
t.index [:key_hash, :byte_size]
|
8
|
-
t.index :byte_size
|
9
|
-
end
|
10
|
-
end
|
11
|
-
end
|