blind_index 1.0.2 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +25 -0
- data/LICENSE.txt +1 -1
- data/README.md +110 -47
- data/lib/blind_index.rb +15 -18
- data/lib/blind_index/backfill.rb +113 -0
- data/lib/blind_index/extensions.rb +7 -32
- data/lib/blind_index/key_generator.rb +1 -1
- data/lib/blind_index/model.rb +20 -15
- data/lib/blind_index/mongoid.rb +13 -4
- data/lib/blind_index/version.rb +1 -1
- metadata +6 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 030c05f4653c350dbbfc5dd2f39727e44819c286865b27a18ac10c6e94fca449
|
4
|
+
data.tar.gz: f92d3e7d76a1b9ae65fe55bc1cf39e4a133dd633f2c620f0fa75dd0f35df8a55
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aa5d431804b50709280499845fd4609b0d576328c01a9be87f5d6cffeeb0a37d9fa10eba6a0f5fa33ccae83056bb939e25b85373839ba3d232f1488c88ce6d36
|
7
|
+
data.tar.gz: ddec86434f1b6908763d454a7fa67507bbb0b98bf30c0038b4331dae26d314c58c4a9312b3db1ffff087aa8a3162e8e30cc4082c60f8aadccb688e78d80c4934
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,28 @@
|
|
1
|
+
## 2.1.1 (2020-08-14)
|
2
|
+
|
3
|
+
- Fixed `version` option
|
4
|
+
|
5
|
+
## 2.1.0 (2020-07-06)
|
6
|
+
|
7
|
+
- Improved performance of uniqueness validations
|
8
|
+
- Fixed deprecation warnings in Ruby 2.7 with Mongoid
|
9
|
+
|
10
|
+
## 2.0.2 (2020-06-01)
|
11
|
+
|
12
|
+
- Improved error message for bad key length
|
13
|
+
- Fixed `backfill` method with relations for Mongoid
|
14
|
+
|
15
|
+
## 2.0.1 (2020-02-14)
|
16
|
+
|
17
|
+
- Added `BlindIndex.backfill` method
|
18
|
+
|
19
|
+
## 2.0.0 (2020-02-10)
|
20
|
+
|
21
|
+
- Blind indexes are updated immediately instead of in a `before_validation` callback
|
22
|
+
- Better Lockbox integration - no need to generate a separate key
|
23
|
+
- The `argon2` gem has been replaced with `argon2-kdf` for less dependencies and Windows support
|
24
|
+
- Removed deprecated `compute_email_bidx`
|
25
|
+
|
1
26
|
## 1.0.2 (2019-12-26)
|
2
27
|
|
3
28
|
- Fixed `OpenSSL::KDF` error on some platforms
|
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -10,7 +10,7 @@ Learn more about [securing sensitive data in Rails](https://ankane.org/sensitive
|
|
10
10
|
|
11
11
|
## How It Works
|
12
12
|
|
13
|
-
We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. `LIKE` queries are not possible, but you can index expressions.
|
13
|
+
We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. Efficient `LIKE` queries are [not possible](#like-ilike-and-full-text-searching), but you can index expressions.
|
14
14
|
|
15
15
|
## Leakage
|
16
16
|
|
@@ -26,38 +26,14 @@ Add this line to your application’s Gemfile:
|
|
26
26
|
gem 'blind_index'
|
27
27
|
```
|
28
28
|
|
29
|
-
|
29
|
+
## Prep
|
30
30
|
|
31
|
-
|
32
|
-
gem 'argon2', git: 'https://github.com/technion/ruby-argon2.git', submodules: true
|
33
|
-
```
|
31
|
+
Your model should already be set up with Lockbox or attr_encrypted. The examples are for a `User` model with `encrypts :email` or `attr_encrypted :email`. See the full examples for [Lockbox](https://ankane.org/securing-user-emails-lockbox) and [attr_encrypted](https://ankane.org/securing-user-emails-in-rails) if needed.
|
34
32
|
|
35
|
-
|
33
|
+
Also, if you use attr_encrypted, [generate a key](#key-generation).
|
36
34
|
|
37
35
|
## Getting Started
|
38
36
|
|
39
|
-
> Note: Your model should already be set up with Lockbox or attr_encrypted. The examples are for a `User` model with `encrypts :email` or `attr_encrypted :email`. See the full examples for [Lockbox](https://ankane.org/securing-user-emails-lockbox) and [attr_encrypted](https://ankane.org/securing-user-emails-in-rails) if needed.
|
40
|
-
|
41
|
-
First, generate a key
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
BlindIndex.generate_key
|
45
|
-
```
|
46
|
-
|
47
|
-
Store the key with your other secrets. This is typically Rails credentials or an environment variable ([dotenv](https://github.com/bkeepers/dotenv) is great for this). Be sure to use different keys in development and production. Keys don’t need to be hex-encoded, but it’s often easier to store them this way.
|
48
|
-
|
49
|
-
Set the following environment variable with your key (you can use this one in development)
|
50
|
-
|
51
|
-
```sh
|
52
|
-
BLIND_INDEX_MASTER_KEY=ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
|
53
|
-
```
|
54
|
-
|
55
|
-
or create `config/initializers/blind_index.rb` with something like
|
56
|
-
|
57
|
-
```ruby
|
58
|
-
BlindIndex.master_key = Rails.application.credentials.blind_index_master_key
|
59
|
-
```
|
60
|
-
|
61
37
|
Create a migration to add a column for the blind index
|
62
38
|
|
63
39
|
```ruby
|
@@ -84,10 +60,7 @@ end
|
|
84
60
|
Backfill existing records
|
85
61
|
|
86
62
|
```ruby
|
87
|
-
|
88
|
-
user.compute_email_bidx
|
89
|
-
user.save(validate: false)
|
90
|
-
end
|
63
|
+
BlindIndex.backfill(User)
|
91
64
|
```
|
92
65
|
|
93
66
|
And query away
|
@@ -96,9 +69,19 @@ And query away
|
|
96
69
|
User.where(email: "test@example.org")
|
97
70
|
```
|
98
71
|
|
72
|
+
## Expressions
|
73
|
+
|
74
|
+
You can apply expressions to attributes before indexing and searching. This gives you the the ability to perform case-insensitive searches and more.
|
75
|
+
|
76
|
+
```ruby
|
77
|
+
class User < ApplicationRecord
|
78
|
+
blind_index :email, expression: ->(v) { v.downcase }
|
79
|
+
end
|
80
|
+
```
|
81
|
+
|
99
82
|
## Validations
|
100
83
|
|
101
|
-
|
84
|
+
You can use blind indexes for uniqueness validations.
|
102
85
|
|
103
86
|
```ruby
|
104
87
|
class User < ApplicationRecord
|
@@ -106,15 +89,27 @@ class User < ApplicationRecord
|
|
106
89
|
end
|
107
90
|
```
|
108
91
|
|
109
|
-
We
|
92
|
+
We recommend adding a unique index to the blind index column through a database migration.
|
110
93
|
|
111
|
-
|
94
|
+
```ruby
|
95
|
+
add_index :users, :email_bidx, unique: true
|
96
|
+
```
|
112
97
|
|
113
|
-
|
98
|
+
For `allow_blank: true`, use:
|
99
|
+
|
100
|
+
```ruby
|
101
|
+
class User < ApplicationRecord
|
102
|
+
blind_index :email, expression: ->(v) { v.presence }
|
103
|
+
validates :email, uniqueness: {allow_blank: true}
|
104
|
+
end
|
105
|
+
```
|
106
|
+
|
107
|
+
For `case_sensitive: false`, use:
|
114
108
|
|
115
109
|
```ruby
|
116
110
|
class User < ApplicationRecord
|
117
111
|
blind_index :email, expression: ->(v) { v.downcase }
|
112
|
+
validates :email, uniqueness: true # for best performance, leave out {case_sensitive: false}
|
118
113
|
end
|
119
114
|
```
|
120
115
|
|
@@ -139,10 +134,7 @@ end
|
|
139
134
|
Backfill existing records
|
140
135
|
|
141
136
|
```ruby
|
142
|
-
|
143
|
-
user.compute_email_ci_bidx
|
144
|
-
user.save(validate: false)
|
145
|
-
end
|
137
|
+
BlindIndex.backfill(User, columns: [:email_ci_bidx])
|
146
138
|
```
|
147
139
|
|
148
140
|
And query away
|
@@ -169,18 +161,34 @@ You can also use virtual attributes to index data from multiple columns:
|
|
169
161
|
```ruby
|
170
162
|
class User < ApplicationRecord
|
171
163
|
attribute :initials, :string
|
164
|
+
blind_index :initials
|
172
165
|
|
173
|
-
# must come before the blind_index method so it runs first
|
174
166
|
before_validation :set_initials, if: -> { changes.key?(:first_name) || changes.key?(:last_name) }
|
175
167
|
|
176
|
-
blind_index :initials
|
177
|
-
|
178
168
|
def set_initials
|
179
169
|
self.initials = "#{first_name[0]}#{last_name[0]}"
|
180
170
|
end
|
181
171
|
end
|
182
172
|
```
|
183
173
|
|
174
|
+
## Migrating Data
|
175
|
+
|
176
|
+
If you’re encrypting a column and adding a blind index at the same time, use the `migrating` option.
|
177
|
+
|
178
|
+
```ruby
|
179
|
+
class User < ApplicationRecord
|
180
|
+
blind_index :email, migrating: true
|
181
|
+
end
|
182
|
+
```
|
183
|
+
|
184
|
+
This allows you to backfill records while still querying the unencrypted field.
|
185
|
+
|
186
|
+
```ruby
|
187
|
+
BlindIndex.backfill(User)
|
188
|
+
```
|
189
|
+
|
190
|
+
Once that completes, you can remove the `migrating` option.
|
191
|
+
|
184
192
|
## Key Rotation
|
185
193
|
|
186
194
|
To rotate keys without downtime, add a new column:
|
@@ -201,10 +209,7 @@ end
|
|
201
209
|
This will keep the new column synced going forward. Next, backfill the data:
|
202
210
|
|
203
211
|
```ruby
|
204
|
-
|
205
|
-
user.compute_rotated_email_bidx
|
206
|
-
user.save(validate: false)
|
207
|
-
end
|
212
|
+
BlindIndex.backfill(User, columns: [:email_bidx_v2])
|
208
213
|
```
|
209
214
|
|
210
215
|
Then update your model
|
@@ -260,6 +265,54 @@ class User
|
|
260
265
|
end
|
261
266
|
```
|
262
267
|
|
268
|
+
## Key Generation
|
269
|
+
|
270
|
+
This is optional for Lockbox, as its master key is used by default.
|
271
|
+
|
272
|
+
Generate a key with:
|
273
|
+
|
274
|
+
```ruby
|
275
|
+
BlindIndex.generate_key
|
276
|
+
```
|
277
|
+
|
278
|
+
Store the key with your other secrets. This is typically Rails credentials or an environment variable ([dotenv](https://github.com/bkeepers/dotenv) is great for this). Be sure to use different keys in development and production. Keys don’t need to be hex-encoded, but it’s often easier to store them this way.
|
279
|
+
|
280
|
+
Set the following environment variable with your key (you can use this one in development)
|
281
|
+
|
282
|
+
```sh
|
283
|
+
BLIND_INDEX_MASTER_KEY=ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
|
284
|
+
```
|
285
|
+
|
286
|
+
or create `config/initializers/blind_index.rb` with something like
|
287
|
+
|
288
|
+
```ruby
|
289
|
+
BlindIndex.master_key = Rails.application.credentials.blind_index_master_key
|
290
|
+
```
|
291
|
+
|
292
|
+
## LIKE, ILIKE, and Full-Text Searching
|
293
|
+
|
294
|
+
Unfortunately, blind indexes can’t be used for `LIKE`, `ILIKE`, or full-text searching. Instead, records must be loaded, decrypted, and searched in memory.
|
295
|
+
|
296
|
+
For `LIKE`, use:
|
297
|
+
|
298
|
+
```ruby
|
299
|
+
User.find { |u| u.email.include?("value") }
|
300
|
+
```
|
301
|
+
|
302
|
+
For `ILIKE`, use:
|
303
|
+
|
304
|
+
```ruby
|
305
|
+
User.find { |u| u.email =~ /value/i }
|
306
|
+
```
|
307
|
+
|
308
|
+
For full-text or fuzzy searching, use a gem like [FuzzyMatch](https://github.com/seamusabshere/fuzzy_match):
|
309
|
+
|
310
|
+
```ruby
|
311
|
+
FuzzyMatch.new(User.all, read: :email).find("value")
|
312
|
+
```
|
313
|
+
|
314
|
+
If the number of records is large, try to find a way to narrow it down. An [expression index](#expressions) is one way to do this, but leaks which records have the same value of the expression, so use it carefully.
|
315
|
+
|
263
316
|
## Reference
|
264
317
|
|
265
318
|
Set default options in an initializer with:
|
@@ -301,6 +354,14 @@ One alternative to blind indexing is to use a deterministic encryption scheme, l
|
|
301
354
|
|
302
355
|
## Upgrading
|
303
356
|
|
357
|
+
### 2.0.0
|
358
|
+
|
359
|
+
2.0.0 brings a number of improvements.
|
360
|
+
|
361
|
+
- Blind indexes are updated immediately instead of in a `before_validation` callback
|
362
|
+
- Better Lockbox integration - no need to generate a separate key
|
363
|
+
- There’s a new gem for Argon2 that has no dependencies and (officially) supports Windows
|
364
|
+
|
304
365
|
### 1.0.0
|
305
366
|
|
306
367
|
1.0.0 brings a number of improvements. Here are a few to be aware of:
|
@@ -421,3 +482,5 @@ cd blind_index
|
|
421
482
|
bundle install
|
422
483
|
bundle exec rake test
|
423
484
|
```
|
485
|
+
|
486
|
+
For security issues, send an email to the address on [this page](https://github.com/ankane).
|
data/lib/blind_index.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
1
|
# dependencies
|
2
2
|
require "active_support"
|
3
3
|
require "openssl"
|
4
|
-
require "argon2"
|
4
|
+
require "argon2/kdf"
|
5
5
|
|
6
6
|
# modules
|
7
|
+
require "blind_index/backfill"
|
7
8
|
require "blind_index/key_generator"
|
8
9
|
require "blind_index/model"
|
9
10
|
require "blind_index/version"
|
@@ -18,7 +19,7 @@ module BlindIndex
|
|
18
19
|
self.default_options = {}
|
19
20
|
|
20
21
|
def self.master_key
|
21
|
-
@master_key ||= ENV["BLIND_INDEX_MASTER_KEY"]
|
22
|
+
@master_key ||= ENV["BLIND_INDEX_MASTER_KEY"] || (defined?(Lockbox.master_key) && Lockbox.master_key)
|
22
23
|
end
|
23
24
|
|
24
25
|
def self.generate_bidx(value, key:, **options)
|
@@ -64,7 +65,7 @@ module BlindIndex
|
|
64
65
|
# use same bounds as rbnacl
|
65
66
|
raise BlindIndex::Error, "m must be between 3 and 22" if m < 3 || m > 22
|
66
67
|
|
67
|
-
|
68
|
+
Argon2::KDF.argon2id(value, salt: key, t: t, m: m, p: 1, length: size)
|
68
69
|
when :pbkdf2_sha256
|
69
70
|
iterations = cost_options[:iterations] || options[:iterations] || (options[:slow] ? 100000 : 10000)
|
70
71
|
OpenSSL::PKCS5.pbkdf2_hmac(value, key, iterations, size, "sha256")
|
@@ -78,7 +79,7 @@ module BlindIndex
|
|
78
79
|
# use same bounds as rbnacl
|
79
80
|
raise BlindIndex::Error, "m must be between 3 and 22" if m < 3 || m > 22
|
80
81
|
|
81
|
-
|
82
|
+
Argon2::KDF.argon2i(value, salt: key, t: t, m: m, p: 1, length: size)
|
82
83
|
when :scrypt
|
83
84
|
n = cost_options[:n] || 4096
|
84
85
|
r = cost_options[:r] || 8
|
@@ -116,29 +117,28 @@ module BlindIndex
|
|
116
117
|
key
|
117
118
|
end
|
118
119
|
|
119
|
-
def self.decode_key(key)
|
120
|
+
def self.decode_key(key, name: "Key")
|
120
121
|
# decode hex key
|
121
122
|
if key.encoding != Encoding::BINARY && key =~ /\A[0-9a-f]{64}\z/i
|
122
123
|
key = [key].pack("H*")
|
123
124
|
end
|
124
125
|
|
125
|
-
raise BlindIndex::Error, "
|
126
|
-
raise BlindIndex::Error, "
|
126
|
+
raise BlindIndex::Error, "#{name} must be 32 bytes (64 hex digits)" if key.bytesize != 32
|
127
|
+
raise BlindIndex::Error, "#{name} must use binary encoding" if key.encoding != Encoding::BINARY
|
127
128
|
|
128
129
|
key
|
129
130
|
end
|
131
|
+
|
132
|
+
def self.backfill(relation, columns: nil, batch_size: 1000)
|
133
|
+
Backfill.new(relation, columns: columns, batch_size: batch_size).perform
|
134
|
+
end
|
130
135
|
end
|
131
136
|
|
132
137
|
ActiveSupport.on_load(:active_record) do
|
133
138
|
require "blind_index/extensions"
|
134
139
|
extend BlindIndex::Model
|
135
140
|
|
136
|
-
|
137
|
-
ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
|
138
|
-
else
|
139
|
-
ActiveRecord::PredicateBuilder.singleton_class.prepend(BlindIndex::Extensions::PredicateBuilder)
|
140
|
-
end
|
141
|
-
|
141
|
+
ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
|
142
142
|
ActiveRecord::DynamicMatchers::Method.prepend(BlindIndex::Extensions::DynamicMatchers)
|
143
143
|
|
144
144
|
unless ActiveRecord::VERSION::STRING.start_with?("5.1.")
|
@@ -146,12 +146,9 @@ ActiveSupport.on_load(:active_record) do
|
|
146
146
|
end
|
147
147
|
end
|
148
148
|
|
149
|
-
|
150
|
-
# TODO find better ActiveModel hook
|
151
|
-
require "active_model/callbacks"
|
152
|
-
ActiveModel::Callbacks.include(BlindIndex::Model)
|
153
|
-
|
149
|
+
ActiveSupport.on_load(:mongoid) do
|
154
150
|
require "blind_index/mongoid"
|
151
|
+
Mongoid::Document::ClassMethods.include(BlindIndex::Model)
|
155
152
|
Mongoid::Criteria.prepend(BlindIndex::Mongoid::Criteria)
|
156
153
|
Mongoid::Validatable::UniquenessValidator.prepend(BlindIndex::Mongoid::UniquenessValidator)
|
157
154
|
end
|
@@ -0,0 +1,113 @@
|
|
1
|
+
module BlindIndex
|
2
|
+
class Backfill
|
3
|
+
attr_reader :blind_indexes
|
4
|
+
|
5
|
+
def initialize(relation, batch_size:, columns:)
|
6
|
+
@relation = relation
|
7
|
+
@transaction = @relation.respond_to?(:transaction)
|
8
|
+
@batch_size = batch_size
|
9
|
+
@blind_indexes = @relation.blind_indexes
|
10
|
+
filter_columns!(columns) if columns
|
11
|
+
end
|
12
|
+
|
13
|
+
def perform
|
14
|
+
each_batch do |records|
|
15
|
+
backfill_records(records)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
# modify in-place
|
22
|
+
def filter_columns!(columns)
|
23
|
+
columns = Array(columns).map(&:to_s)
|
24
|
+
blind_indexes.select! { |_, v| columns.include?(v[:bidx_attribute]) }
|
25
|
+
bad_columns = columns - blind_indexes.map { |_, v| v[:bidx_attribute] }
|
26
|
+
raise ArgumentError, "Bad column: #{bad_columns.first}" if bad_columns.any?
|
27
|
+
end
|
28
|
+
|
29
|
+
def build_relation
|
30
|
+
# build relation
|
31
|
+
relation = @relation
|
32
|
+
|
33
|
+
if defined?(ActiveRecord::Base) && relation.is_a?(ActiveRecord::Base)
|
34
|
+
relation = relation.unscoped
|
35
|
+
end
|
36
|
+
|
37
|
+
# convert from possible class to ActiveRecord::Relation or Mongoid::Criteria
|
38
|
+
relation = relation.all
|
39
|
+
|
40
|
+
attributes = blind_indexes.map { |_, v| v[:bidx_attribute] }
|
41
|
+
|
42
|
+
if defined?(ActiveRecord::Relation) && relation.is_a?(ActiveRecord::Relation)
|
43
|
+
base_relation = relation.unscoped
|
44
|
+
or_relation = relation.unscoped
|
45
|
+
|
46
|
+
attributes.each_with_index do |attribute, i|
|
47
|
+
or_relation =
|
48
|
+
if i == 0
|
49
|
+
base_relation.where(attribute => nil)
|
50
|
+
else
|
51
|
+
or_relation.or(base_relation.where(attribute => nil))
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
relation.merge(or_relation)
|
56
|
+
else
|
57
|
+
relation.merge(relation.unscoped.or(attributes.map { |a| {a => nil} }))
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
def each_batch
|
62
|
+
relation = build_relation
|
63
|
+
|
64
|
+
if relation.respond_to?(:find_in_batches)
|
65
|
+
relation.find_in_batches(batch_size: @batch_size) do |records|
|
66
|
+
yield records
|
67
|
+
end
|
68
|
+
else
|
69
|
+
# https://github.com/karmi/tire/blob/master/lib/tire/model/import.rb
|
70
|
+
# use cursor for Mongoid
|
71
|
+
records = []
|
72
|
+
relation.all.each do |record|
|
73
|
+
records << record
|
74
|
+
if records.length == @batch_size
|
75
|
+
yield records
|
76
|
+
records = []
|
77
|
+
end
|
78
|
+
end
|
79
|
+
yield records if records.any?
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
def backfill_records(records)
|
84
|
+
# do expensive blind index computation outside of transaction
|
85
|
+
records.each do |record|
|
86
|
+
blind_indexes.each do |k, v|
|
87
|
+
record.send("compute_#{k}_bidx") if !record.send(v[:bidx_attribute])
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
# don't need to save records that went from nil => nil
|
92
|
+
records.select! { |r| r.changed? }
|
93
|
+
|
94
|
+
if records.any?
|
95
|
+
with_transaction do
|
96
|
+
records.each do |record|
|
97
|
+
record.save!(validate: false)
|
98
|
+
end
|
99
|
+
end
|
100
|
+
end
|
101
|
+
end
|
102
|
+
|
103
|
+
def with_transaction
|
104
|
+
if @transaction
|
105
|
+
@relation.transaction do
|
106
|
+
yield
|
107
|
+
end
|
108
|
+
else
|
109
|
+
yield
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
113
|
+
end
|
@@ -1,6 +1,5 @@
|
|
1
1
|
module BlindIndex
|
2
2
|
module Extensions
|
3
|
-
# ActiveRecord 5.0+
|
4
3
|
module TableMetadata
|
5
4
|
def resolve_column_aliases(hash)
|
6
5
|
new_hash = super
|
@@ -29,42 +28,19 @@ module BlindIndex
|
|
29
28
|
end
|
30
29
|
end
|
31
30
|
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
hash.each do |key, _|
|
38
|
-
if key.respond_to?(:to_sym) && (bi = klass.blind_indexes[key.to_sym]) && !new_hash[key].is_a?(ActiveRecord::StatementCache::Substitute)
|
39
|
-
value = new_hash.delete(key)
|
40
|
-
new_hash[bi[:bidx_attribute]] =
|
41
|
-
if value.is_a?(Array)
|
42
|
-
value.map { |v| BlindIndex.generate_bidx(v, bi) }
|
43
|
-
else
|
44
|
-
BlindIndex.generate_bidx(value, bi)
|
45
|
-
end
|
46
|
-
end
|
47
|
-
end
|
48
|
-
end
|
49
|
-
new_hash
|
50
|
-
end
|
51
|
-
|
52
|
-
@@blind_index_cache = {}
|
53
|
-
|
54
|
-
# memoize for performance
|
55
|
-
def has_blind_indexes?(klass)
|
56
|
-
if @@blind_index_cache[klass].nil?
|
57
|
-
@@blind_index_cache[klass] = klass.respond_to?(:blind_indexes)
|
31
|
+
module UniquenessValidator
|
32
|
+
def validate_each(record, attribute, value)
|
33
|
+
klass = record.class
|
34
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
35
|
+
value = record.read_attribute_for_validation(bi[:bidx_attribute])
|
58
36
|
end
|
59
|
-
|
37
|
+
super(record, attribute, value)
|
60
38
|
end
|
61
|
-
end
|
62
39
|
|
63
|
-
|
40
|
+
# change attribute name here instead of validate_each for better error message
|
64
41
|
if ActiveRecord::VERSION::STRING >= "5.2"
|
65
42
|
def build_relation(klass, attribute, value)
|
66
43
|
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
67
|
-
value = BlindIndex.generate_bidx(value, **bi)
|
68
44
|
attribute = bi[:bidx_attribute]
|
69
45
|
end
|
70
46
|
super(klass, attribute, value)
|
@@ -72,7 +48,6 @@ module BlindIndex
|
|
72
48
|
else
|
73
49
|
def build_relation(klass, table, attribute, value)
|
74
50
|
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
75
|
-
value = BlindIndex.generate_bidx(value, **bi)
|
76
51
|
attribute = bi[:bidx_attribute]
|
77
52
|
end
|
78
53
|
super(klass, table, attribute, value)
|
@@ -11,7 +11,7 @@ module BlindIndex
|
|
11
11
|
raise ArgumentError, "Missing field for key generation" if bidx_attribute.to_s.empty?
|
12
12
|
|
13
13
|
c = "\x7E"*32
|
14
|
-
root_key = hkdf(BlindIndex.decode_key(@master_key), salt: table.to_s, info: "#{c}#{bidx_attribute}", length: 32, hash: "sha384")
|
14
|
+
root_key = hkdf(BlindIndex.decode_key(@master_key, name: "Master key"), salt: table.to_s, info: "#{c}#{bidx_attribute}", length: 32, hash: "sha384")
|
15
15
|
hash_hmac("sha256", pack([table, bidx_attribute, bidx_attribute]), root_key)
|
16
16
|
end
|
17
17
|
|
data/lib/blind_index/model.rb
CHANGED
@@ -10,7 +10,7 @@ module BlindIndex
|
|
10
10
|
# check here so we validate rotate options as well
|
11
11
|
unknown_keywords = options.keys - [:algorithm, :attribute, :bidx_attribute,
|
12
12
|
:callback, :cost, :encode, :expression, :insecure_key, :iterations, :key,
|
13
|
-
:legacy, :master_key, :size, :slow]
|
13
|
+
:legacy, :master_key, :size, :slow, :version]
|
14
14
|
raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
|
15
15
|
|
16
16
|
attribute = options[:attribute] || name
|
@@ -64,25 +64,30 @@ module BlindIndex
|
|
64
64
|
BlindIndex.generate_bidx(value, **blind_indexes[name])
|
65
65
|
end
|
66
66
|
|
67
|
-
define_singleton_method method_name do |value|
|
68
|
-
ActiveSupport::Deprecation.warn("Use #{class_method_name} instead")
|
69
|
-
send(class_method_name, value)
|
70
|
-
end
|
71
|
-
|
72
67
|
define_method method_name do
|
73
|
-
|
68
|
+
send("#{bidx_attribute}=", self.class.send(class_method_name, send(attribute)))
|
74
69
|
end
|
75
70
|
|
76
71
|
if callback
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
72
|
+
activerecord = defined?(ActiveRecord) && self < ActiveRecord::Base
|
73
|
+
|
74
|
+
# TODO reuse module
|
75
|
+
m = Module.new do
|
76
|
+
define_method "#{attribute}=" do |value|
|
77
|
+
result = super(value)
|
78
|
+
send(method_name)
|
79
|
+
result
|
80
|
+
end
|
81
|
+
|
82
|
+
unless activerecord
|
83
|
+
define_method "reset_#{attribute}!" do
|
84
|
+
result = super()
|
85
|
+
send(method_name)
|
86
|
+
result
|
87
|
+
end
|
88
|
+
end
|
85
89
|
end
|
90
|
+
prepend m
|
86
91
|
end
|
87
92
|
|
88
93
|
# use include so user can override
|
data/lib/blind_index/mongoid.rb
CHANGED
@@ -26,9 +26,9 @@ module BlindIndex
|
|
26
26
|
|
27
27
|
criterion[bidx_key] =
|
28
28
|
if value.is_a?(Array)
|
29
|
-
value.map { |v| BlindIndex.generate_bidx(v, bi) }
|
29
|
+
value.map { |v| BlindIndex.generate_bidx(v, **bi) }
|
30
30
|
else
|
31
|
-
BlindIndex.generate_bidx(value, bi)
|
31
|
+
BlindIndex.generate_bidx(value, **bi)
|
32
32
|
end
|
33
33
|
end
|
34
34
|
end
|
@@ -39,9 +39,18 @@ module BlindIndex
|
|
39
39
|
end
|
40
40
|
|
41
41
|
module UniquenessValidator
|
42
|
+
def validate_each(record, attribute, value)
|
43
|
+
klass = record.class
|
44
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
45
|
+
value = record.read_attribute_for_validation(bi[:bidx_attribute])
|
46
|
+
end
|
47
|
+
super(record, attribute, value)
|
48
|
+
end
|
49
|
+
|
50
|
+
# change attribute name here instead of validate_each for better error message
|
42
51
|
def create_criteria(base, document, attribute, value)
|
43
|
-
|
44
|
-
|
52
|
+
klass = document.class
|
53
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
45
54
|
attribute = bi[:bidx_attribute]
|
46
55
|
end
|
47
56
|
super(base, document, attribute, value)
|
data/lib/blind_index/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: blind_index
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 2.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-08-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -25,19 +25,19 @@ dependencies:
|
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '5'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
|
-
name: argon2
|
28
|
+
name: argon2-kdf
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
33
|
+
version: 0.1.1
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version:
|
40
|
+
version: 0.1.1
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: bundler
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -174,6 +174,7 @@ files:
|
|
174
174
|
- LICENSE.txt
|
175
175
|
- README.md
|
176
176
|
- lib/blind_index.rb
|
177
|
+
- lib/blind_index/backfill.rb
|
177
178
|
- lib/blind_index/extensions.rb
|
178
179
|
- lib/blind_index/key_generator.rb
|
179
180
|
- lib/blind_index/model.rb
|