blind_index 1.0.2 → 2.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +25 -0
- data/LICENSE.txt +1 -1
- data/README.md +110 -47
- data/lib/blind_index.rb +15 -18
- data/lib/blind_index/backfill.rb +113 -0
- data/lib/blind_index/extensions.rb +7 -32
- data/lib/blind_index/key_generator.rb +1 -1
- data/lib/blind_index/model.rb +20 -15
- data/lib/blind_index/mongoid.rb +13 -4
- data/lib/blind_index/version.rb +1 -1
- metadata +6 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 030c05f4653c350dbbfc5dd2f39727e44819c286865b27a18ac10c6e94fca449
|
4
|
+
data.tar.gz: f92d3e7d76a1b9ae65fe55bc1cf39e4a133dd633f2c620f0fa75dd0f35df8a55
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: aa5d431804b50709280499845fd4609b0d576328c01a9be87f5d6cffeeb0a37d9fa10eba6a0f5fa33ccae83056bb939e25b85373839ba3d232f1488c88ce6d36
|
7
|
+
data.tar.gz: ddec86434f1b6908763d454a7fa67507bbb0b98bf30c0038b4331dae26d314c58c4a9312b3db1ffff087aa8a3162e8e30cc4082c60f8aadccb688e78d80c4934
|
data/CHANGELOG.md
CHANGED
@@ -1,3 +1,28 @@
|
|
1
|
+
## 2.1.1 (2020-08-14)
|
2
|
+
|
3
|
+
- Fixed `version` option
|
4
|
+
|
5
|
+
## 2.1.0 (2020-07-06)
|
6
|
+
|
7
|
+
- Improved performance of uniqueness validations
|
8
|
+
- Fixed deprecation warnings in Ruby 2.7 with Mongoid
|
9
|
+
|
10
|
+
## 2.0.2 (2020-06-01)
|
11
|
+
|
12
|
+
- Improved error message for bad key length
|
13
|
+
- Fixed `backfill` method with relations for Mongoid
|
14
|
+
|
15
|
+
## 2.0.1 (2020-02-14)
|
16
|
+
|
17
|
+
- Added `BlindIndex.backfill` method
|
18
|
+
|
19
|
+
## 2.0.0 (2020-02-10)
|
20
|
+
|
21
|
+
- Blind indexes are updated immediately instead of in a `before_validation` callback
|
22
|
+
- Better Lockbox integration - no need to generate a separate key
|
23
|
+
- The `argon2` gem has been replaced with `argon2-kdf` for less dependencies and Windows support
|
24
|
+
- Removed deprecated `compute_email_bidx`
|
25
|
+
|
1
26
|
## 1.0.2 (2019-12-26)
|
2
27
|
|
3
28
|
- Fixed `OpenSSL::KDF` error on some platforms
|
data/LICENSE.txt
CHANGED
data/README.md
CHANGED
@@ -10,7 +10,7 @@ Learn more about [securing sensitive data in Rails](https://ankane.org/sensitive
|
|
10
10
|
|
11
11
|
## How It Works
|
12
12
|
|
13
|
-
We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. `LIKE` queries are not possible, but you can index expressions.
|
13
|
+
We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. Efficient `LIKE` queries are [not possible](#like-ilike-and-full-text-searching), but you can index expressions.
|
14
14
|
|
15
15
|
## Leakage
|
16
16
|
|
@@ -26,38 +26,14 @@ Add this line to your application’s Gemfile:
|
|
26
26
|
gem 'blind_index'
|
27
27
|
```
|
28
28
|
|
29
|
-
|
29
|
+
## Prep
|
30
30
|
|
31
|
-
|
32
|
-
gem 'argon2', git: 'https://github.com/technion/ruby-argon2.git', submodules: true
|
33
|
-
```
|
31
|
+
Your model should already be set up with Lockbox or attr_encrypted. The examples are for a `User` model with `encrypts :email` or `attr_encrypted :email`. See the full examples for [Lockbox](https://ankane.org/securing-user-emails-lockbox) and [attr_encrypted](https://ankane.org/securing-user-emails-in-rails) if needed.
|
34
32
|
|
35
|
-
|
33
|
+
Also, if you use attr_encrypted, [generate a key](#key-generation).
|
36
34
|
|
37
35
|
## Getting Started
|
38
36
|
|
39
|
-
> Note: Your model should already be set up with Lockbox or attr_encrypted. The examples are for a `User` model with `encrypts :email` or `attr_encrypted :email`. See the full examples for [Lockbox](https://ankane.org/securing-user-emails-lockbox) and [attr_encrypted](https://ankane.org/securing-user-emails-in-rails) if needed.
|
40
|
-
|
41
|
-
First, generate a key
|
42
|
-
|
43
|
-
```ruby
|
44
|
-
BlindIndex.generate_key
|
45
|
-
```
|
46
|
-
|
47
|
-
Store the key with your other secrets. This is typically Rails credentials or an environment variable ([dotenv](https://github.com/bkeepers/dotenv) is great for this). Be sure to use different keys in development and production. Keys don’t need to be hex-encoded, but it’s often easier to store them this way.
|
48
|
-
|
49
|
-
Set the following environment variable with your key (you can use this one in development)
|
50
|
-
|
51
|
-
```sh
|
52
|
-
BLIND_INDEX_MASTER_KEY=ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
|
53
|
-
```
|
54
|
-
|
55
|
-
or create `config/initializers/blind_index.rb` with something like
|
56
|
-
|
57
|
-
```ruby
|
58
|
-
BlindIndex.master_key = Rails.application.credentials.blind_index_master_key
|
59
|
-
```
|
60
|
-
|
61
37
|
Create a migration to add a column for the blind index
|
62
38
|
|
63
39
|
```ruby
|
@@ -84,10 +60,7 @@ end
|
|
84
60
|
Backfill existing records
|
85
61
|
|
86
62
|
```ruby
|
87
|
-
|
88
|
-
user.compute_email_bidx
|
89
|
-
user.save(validate: false)
|
90
|
-
end
|
63
|
+
BlindIndex.backfill(User)
|
91
64
|
```
|
92
65
|
|
93
66
|
And query away
|
@@ -96,9 +69,19 @@ And query away
|
|
96
69
|
User.where(email: "test@example.org")
|
97
70
|
```
|
98
71
|
|
72
|
+
## Expressions
|
73
|
+
|
74
|
+
You can apply expressions to attributes before indexing and searching. This gives you the the ability to perform case-insensitive searches and more.
|
75
|
+
|
76
|
+
```ruby
|
77
|
+
class User < ApplicationRecord
|
78
|
+
blind_index :email, expression: ->(v) { v.downcase }
|
79
|
+
end
|
80
|
+
```
|
81
|
+
|
99
82
|
## Validations
|
100
83
|
|
101
|
-
|
84
|
+
You can use blind indexes for uniqueness validations.
|
102
85
|
|
103
86
|
```ruby
|
104
87
|
class User < ApplicationRecord
|
@@ -106,15 +89,27 @@ class User < ApplicationRecord
|
|
106
89
|
end
|
107
90
|
```
|
108
91
|
|
109
|
-
We
|
92
|
+
We recommend adding a unique index to the blind index column through a database migration.
|
110
93
|
|
111
|
-
|
94
|
+
```ruby
|
95
|
+
add_index :users, :email_bidx, unique: true
|
96
|
+
```
|
112
97
|
|
113
|
-
|
98
|
+
For `allow_blank: true`, use:
|
99
|
+
|
100
|
+
```ruby
|
101
|
+
class User < ApplicationRecord
|
102
|
+
blind_index :email, expression: ->(v) { v.presence }
|
103
|
+
validates :email, uniqueness: {allow_blank: true}
|
104
|
+
end
|
105
|
+
```
|
106
|
+
|
107
|
+
For `case_sensitive: false`, use:
|
114
108
|
|
115
109
|
```ruby
|
116
110
|
class User < ApplicationRecord
|
117
111
|
blind_index :email, expression: ->(v) { v.downcase }
|
112
|
+
validates :email, uniqueness: true # for best performance, leave out {case_sensitive: false}
|
118
113
|
end
|
119
114
|
```
|
120
115
|
|
@@ -139,10 +134,7 @@ end
|
|
139
134
|
Backfill existing records
|
140
135
|
|
141
136
|
```ruby
|
142
|
-
|
143
|
-
user.compute_email_ci_bidx
|
144
|
-
user.save(validate: false)
|
145
|
-
end
|
137
|
+
BlindIndex.backfill(User, columns: [:email_ci_bidx])
|
146
138
|
```
|
147
139
|
|
148
140
|
And query away
|
@@ -169,18 +161,34 @@ You can also use virtual attributes to index data from multiple columns:
|
|
169
161
|
```ruby
|
170
162
|
class User < ApplicationRecord
|
171
163
|
attribute :initials, :string
|
164
|
+
blind_index :initials
|
172
165
|
|
173
|
-
# must come before the blind_index method so it runs first
|
174
166
|
before_validation :set_initials, if: -> { changes.key?(:first_name) || changes.key?(:last_name) }
|
175
167
|
|
176
|
-
blind_index :initials
|
177
|
-
|
178
168
|
def set_initials
|
179
169
|
self.initials = "#{first_name[0]}#{last_name[0]}"
|
180
170
|
end
|
181
171
|
end
|
182
172
|
```
|
183
173
|
|
174
|
+
## Migrating Data
|
175
|
+
|
176
|
+
If you’re encrypting a column and adding a blind index at the same time, use the `migrating` option.
|
177
|
+
|
178
|
+
```ruby
|
179
|
+
class User < ApplicationRecord
|
180
|
+
blind_index :email, migrating: true
|
181
|
+
end
|
182
|
+
```
|
183
|
+
|
184
|
+
This allows you to backfill records while still querying the unencrypted field.
|
185
|
+
|
186
|
+
```ruby
|
187
|
+
BlindIndex.backfill(User)
|
188
|
+
```
|
189
|
+
|
190
|
+
Once that completes, you can remove the `migrating` option.
|
191
|
+
|
184
192
|
## Key Rotation
|
185
193
|
|
186
194
|
To rotate keys without downtime, add a new column:
|
@@ -201,10 +209,7 @@ end
|
|
201
209
|
This will keep the new column synced going forward. Next, backfill the data:
|
202
210
|
|
203
211
|
```ruby
|
204
|
-
|
205
|
-
user.compute_rotated_email_bidx
|
206
|
-
user.save(validate: false)
|
207
|
-
end
|
212
|
+
BlindIndex.backfill(User, columns: [:email_bidx_v2])
|
208
213
|
```
|
209
214
|
|
210
215
|
Then update your model
|
@@ -260,6 +265,54 @@ class User
|
|
260
265
|
end
|
261
266
|
```
|
262
267
|
|
268
|
+
## Key Generation
|
269
|
+
|
270
|
+
This is optional for Lockbox, as its master key is used by default.
|
271
|
+
|
272
|
+
Generate a key with:
|
273
|
+
|
274
|
+
```ruby
|
275
|
+
BlindIndex.generate_key
|
276
|
+
```
|
277
|
+
|
278
|
+
Store the key with your other secrets. This is typically Rails credentials or an environment variable ([dotenv](https://github.com/bkeepers/dotenv) is great for this). Be sure to use different keys in development and production. Keys don’t need to be hex-encoded, but it’s often easier to store them this way.
|
279
|
+
|
280
|
+
Set the following environment variable with your key (you can use this one in development)
|
281
|
+
|
282
|
+
```sh
|
283
|
+
BLIND_INDEX_MASTER_KEY=ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
|
284
|
+
```
|
285
|
+
|
286
|
+
or create `config/initializers/blind_index.rb` with something like
|
287
|
+
|
288
|
+
```ruby
|
289
|
+
BlindIndex.master_key = Rails.application.credentials.blind_index_master_key
|
290
|
+
```
|
291
|
+
|
292
|
+
## LIKE, ILIKE, and Full-Text Searching
|
293
|
+
|
294
|
+
Unfortunately, blind indexes can’t be used for `LIKE`, `ILIKE`, or full-text searching. Instead, records must be loaded, decrypted, and searched in memory.
|
295
|
+
|
296
|
+
For `LIKE`, use:
|
297
|
+
|
298
|
+
```ruby
|
299
|
+
User.find { |u| u.email.include?("value") }
|
300
|
+
```
|
301
|
+
|
302
|
+
For `ILIKE`, use:
|
303
|
+
|
304
|
+
```ruby
|
305
|
+
User.find { |u| u.email =~ /value/i }
|
306
|
+
```
|
307
|
+
|
308
|
+
For full-text or fuzzy searching, use a gem like [FuzzyMatch](https://github.com/seamusabshere/fuzzy_match):
|
309
|
+
|
310
|
+
```ruby
|
311
|
+
FuzzyMatch.new(User.all, read: :email).find("value")
|
312
|
+
```
|
313
|
+
|
314
|
+
If the number of records is large, try to find a way to narrow it down. An [expression index](#expressions) is one way to do this, but leaks which records have the same value of the expression, so use it carefully.
|
315
|
+
|
263
316
|
## Reference
|
264
317
|
|
265
318
|
Set default options in an initializer with:
|
@@ -301,6 +354,14 @@ One alternative to blind indexing is to use a deterministic encryption scheme, l
|
|
301
354
|
|
302
355
|
## Upgrading
|
303
356
|
|
357
|
+
### 2.0.0
|
358
|
+
|
359
|
+
2.0.0 brings a number of improvements.
|
360
|
+
|
361
|
+
- Blind indexes are updated immediately instead of in a `before_validation` callback
|
362
|
+
- Better Lockbox integration - no need to generate a separate key
|
363
|
+
- There’s a new gem for Argon2 that has no dependencies and (officially) supports Windows
|
364
|
+
|
304
365
|
### 1.0.0
|
305
366
|
|
306
367
|
1.0.0 brings a number of improvements. Here are a few to be aware of:
|
@@ -421,3 +482,5 @@ cd blind_index
|
|
421
482
|
bundle install
|
422
483
|
bundle exec rake test
|
423
484
|
```
|
485
|
+
|
486
|
+
For security issues, send an email to the address on [this page](https://github.com/ankane).
|
data/lib/blind_index.rb
CHANGED
@@ -1,9 +1,10 @@
|
|
1
1
|
# dependencies
|
2
2
|
require "active_support"
|
3
3
|
require "openssl"
|
4
|
-
require "argon2"
|
4
|
+
require "argon2/kdf"
|
5
5
|
|
6
6
|
# modules
|
7
|
+
require "blind_index/backfill"
|
7
8
|
require "blind_index/key_generator"
|
8
9
|
require "blind_index/model"
|
9
10
|
require "blind_index/version"
|
@@ -18,7 +19,7 @@ module BlindIndex
|
|
18
19
|
self.default_options = {}
|
19
20
|
|
20
21
|
def self.master_key
|
21
|
-
@master_key ||= ENV["BLIND_INDEX_MASTER_KEY"]
|
22
|
+
@master_key ||= ENV["BLIND_INDEX_MASTER_KEY"] || (defined?(Lockbox.master_key) && Lockbox.master_key)
|
22
23
|
end
|
23
24
|
|
24
25
|
def self.generate_bidx(value, key:, **options)
|
@@ -64,7 +65,7 @@ module BlindIndex
|
|
64
65
|
# use same bounds as rbnacl
|
65
66
|
raise BlindIndex::Error, "m must be between 3 and 22" if m < 3 || m > 22
|
66
67
|
|
67
|
-
|
68
|
+
Argon2::KDF.argon2id(value, salt: key, t: t, m: m, p: 1, length: size)
|
68
69
|
when :pbkdf2_sha256
|
69
70
|
iterations = cost_options[:iterations] || options[:iterations] || (options[:slow] ? 100000 : 10000)
|
70
71
|
OpenSSL::PKCS5.pbkdf2_hmac(value, key, iterations, size, "sha256")
|
@@ -78,7 +79,7 @@ module BlindIndex
|
|
78
79
|
# use same bounds as rbnacl
|
79
80
|
raise BlindIndex::Error, "m must be between 3 and 22" if m < 3 || m > 22
|
80
81
|
|
81
|
-
|
82
|
+
Argon2::KDF.argon2i(value, salt: key, t: t, m: m, p: 1, length: size)
|
82
83
|
when :scrypt
|
83
84
|
n = cost_options[:n] || 4096
|
84
85
|
r = cost_options[:r] || 8
|
@@ -116,29 +117,28 @@ module BlindIndex
|
|
116
117
|
key
|
117
118
|
end
|
118
119
|
|
119
|
-
def self.decode_key(key)
|
120
|
+
def self.decode_key(key, name: "Key")
|
120
121
|
# decode hex key
|
121
122
|
if key.encoding != Encoding::BINARY && key =~ /\A[0-9a-f]{64}\z/i
|
122
123
|
key = [key].pack("H*")
|
123
124
|
end
|
124
125
|
|
125
|
-
raise BlindIndex::Error, "
|
126
|
-
raise BlindIndex::Error, "
|
126
|
+
raise BlindIndex::Error, "#{name} must be 32 bytes (64 hex digits)" if key.bytesize != 32
|
127
|
+
raise BlindIndex::Error, "#{name} must use binary encoding" if key.encoding != Encoding::BINARY
|
127
128
|
|
128
129
|
key
|
129
130
|
end
|
131
|
+
|
132
|
+
def self.backfill(relation, columns: nil, batch_size: 1000)
|
133
|
+
Backfill.new(relation, columns: columns, batch_size: batch_size).perform
|
134
|
+
end
|
130
135
|
end
|
131
136
|
|
132
137
|
ActiveSupport.on_load(:active_record) do
|
133
138
|
require "blind_index/extensions"
|
134
139
|
extend BlindIndex::Model
|
135
140
|
|
136
|
-
|
137
|
-
ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
|
138
|
-
else
|
139
|
-
ActiveRecord::PredicateBuilder.singleton_class.prepend(BlindIndex::Extensions::PredicateBuilder)
|
140
|
-
end
|
141
|
-
|
141
|
+
ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
|
142
142
|
ActiveRecord::DynamicMatchers::Method.prepend(BlindIndex::Extensions::DynamicMatchers)
|
143
143
|
|
144
144
|
unless ActiveRecord::VERSION::STRING.start_with?("5.1.")
|
@@ -146,12 +146,9 @@ ActiveSupport.on_load(:active_record) do
|
|
146
146
|
end
|
147
147
|
end
|
148
148
|
|
149
|
-
|
150
|
-
# TODO find better ActiveModel hook
|
151
|
-
require "active_model/callbacks"
|
152
|
-
ActiveModel::Callbacks.include(BlindIndex::Model)
|
153
|
-
|
149
|
+
ActiveSupport.on_load(:mongoid) do
|
154
150
|
require "blind_index/mongoid"
|
151
|
+
Mongoid::Document::ClassMethods.include(BlindIndex::Model)
|
155
152
|
Mongoid::Criteria.prepend(BlindIndex::Mongoid::Criteria)
|
156
153
|
Mongoid::Validatable::UniquenessValidator.prepend(BlindIndex::Mongoid::UniquenessValidator)
|
157
154
|
end
|
@@ -0,0 +1,113 @@
|
|
1
|
+
module BlindIndex
|
2
|
+
class Backfill
|
3
|
+
attr_reader :blind_indexes
|
4
|
+
|
5
|
+
def initialize(relation, batch_size:, columns:)
|
6
|
+
@relation = relation
|
7
|
+
@transaction = @relation.respond_to?(:transaction)
|
8
|
+
@batch_size = batch_size
|
9
|
+
@blind_indexes = @relation.blind_indexes
|
10
|
+
filter_columns!(columns) if columns
|
11
|
+
end
|
12
|
+
|
13
|
+
def perform
|
14
|
+
each_batch do |records|
|
15
|
+
backfill_records(records)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
|
19
|
+
private
|
20
|
+
|
21
|
+
# modify in-place
|
22
|
+
def filter_columns!(columns)
|
23
|
+
columns = Array(columns).map(&:to_s)
|
24
|
+
blind_indexes.select! { |_, v| columns.include?(v[:bidx_attribute]) }
|
25
|
+
bad_columns = columns - blind_indexes.map { |_, v| v[:bidx_attribute] }
|
26
|
+
raise ArgumentError, "Bad column: #{bad_columns.first}" if bad_columns.any?
|
27
|
+
end
|
28
|
+
|
29
|
+
def build_relation
|
30
|
+
# build relation
|
31
|
+
relation = @relation
|
32
|
+
|
33
|
+
if defined?(ActiveRecord::Base) && relation.is_a?(ActiveRecord::Base)
|
34
|
+
relation = relation.unscoped
|
35
|
+
end
|
36
|
+
|
37
|
+
# convert from possible class to ActiveRecord::Relation or Mongoid::Criteria
|
38
|
+
relation = relation.all
|
39
|
+
|
40
|
+
attributes = blind_indexes.map { |_, v| v[:bidx_attribute] }
|
41
|
+
|
42
|
+
if defined?(ActiveRecord::Relation) && relation.is_a?(ActiveRecord::Relation)
|
43
|
+
base_relation = relation.unscoped
|
44
|
+
or_relation = relation.unscoped
|
45
|
+
|
46
|
+
attributes.each_with_index do |attribute, i|
|
47
|
+
or_relation =
|
48
|
+
if i == 0
|
49
|
+
base_relation.where(attribute => nil)
|
50
|
+
else
|
51
|
+
or_relation.or(base_relation.where(attribute => nil))
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
relation.merge(or_relation)
|
56
|
+
else
|
57
|
+
relation.merge(relation.unscoped.or(attributes.map { |a| {a => nil} }))
|
58
|
+
end
|
59
|
+
end
|
60
|
+
|
61
|
+
def each_batch
|
62
|
+
relation = build_relation
|
63
|
+
|
64
|
+
if relation.respond_to?(:find_in_batches)
|
65
|
+
relation.find_in_batches(batch_size: @batch_size) do |records|
|
66
|
+
yield records
|
67
|
+
end
|
68
|
+
else
|
69
|
+
# https://github.com/karmi/tire/blob/master/lib/tire/model/import.rb
|
70
|
+
# use cursor for Mongoid
|
71
|
+
records = []
|
72
|
+
relation.all.each do |record|
|
73
|
+
records << record
|
74
|
+
if records.length == @batch_size
|
75
|
+
yield records
|
76
|
+
records = []
|
77
|
+
end
|
78
|
+
end
|
79
|
+
yield records if records.any?
|
80
|
+
end
|
81
|
+
end
|
82
|
+
|
83
|
+
def backfill_records(records)
|
84
|
+
# do expensive blind index computation outside of transaction
|
85
|
+
records.each do |record|
|
86
|
+
blind_indexes.each do |k, v|
|
87
|
+
record.send("compute_#{k}_bidx") if !record.send(v[:bidx_attribute])
|
88
|
+
end
|
89
|
+
end
|
90
|
+
|
91
|
+
# don't need to save records that went from nil => nil
|
92
|
+
records.select! { |r| r.changed? }
|
93
|
+
|
94
|
+
if records.any?
|
95
|
+
with_transaction do
|
96
|
+
records.each do |record|
|
97
|
+
record.save!(validate: false)
|
98
|
+
end
|
99
|
+
end
|
100
|
+
end
|
101
|
+
end
|
102
|
+
|
103
|
+
def with_transaction
|
104
|
+
if @transaction
|
105
|
+
@relation.transaction do
|
106
|
+
yield
|
107
|
+
end
|
108
|
+
else
|
109
|
+
yield
|
110
|
+
end
|
111
|
+
end
|
112
|
+
end
|
113
|
+
end
|
@@ -1,6 +1,5 @@
|
|
1
1
|
module BlindIndex
|
2
2
|
module Extensions
|
3
|
-
# ActiveRecord 5.0+
|
4
3
|
module TableMetadata
|
5
4
|
def resolve_column_aliases(hash)
|
6
5
|
new_hash = super
|
@@ -29,42 +28,19 @@ module BlindIndex
|
|
29
28
|
end
|
30
29
|
end
|
31
30
|
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
hash.each do |key, _|
|
38
|
-
if key.respond_to?(:to_sym) && (bi = klass.blind_indexes[key.to_sym]) && !new_hash[key].is_a?(ActiveRecord::StatementCache::Substitute)
|
39
|
-
value = new_hash.delete(key)
|
40
|
-
new_hash[bi[:bidx_attribute]] =
|
41
|
-
if value.is_a?(Array)
|
42
|
-
value.map { |v| BlindIndex.generate_bidx(v, bi) }
|
43
|
-
else
|
44
|
-
BlindIndex.generate_bidx(value, bi)
|
45
|
-
end
|
46
|
-
end
|
47
|
-
end
|
48
|
-
end
|
49
|
-
new_hash
|
50
|
-
end
|
51
|
-
|
52
|
-
@@blind_index_cache = {}
|
53
|
-
|
54
|
-
# memoize for performance
|
55
|
-
def has_blind_indexes?(klass)
|
56
|
-
if @@blind_index_cache[klass].nil?
|
57
|
-
@@blind_index_cache[klass] = klass.respond_to?(:blind_indexes)
|
31
|
+
module UniquenessValidator
|
32
|
+
def validate_each(record, attribute, value)
|
33
|
+
klass = record.class
|
34
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
35
|
+
value = record.read_attribute_for_validation(bi[:bidx_attribute])
|
58
36
|
end
|
59
|
-
|
37
|
+
super(record, attribute, value)
|
60
38
|
end
|
61
|
-
end
|
62
39
|
|
63
|
-
|
40
|
+
# change attribute name here instead of validate_each for better error message
|
64
41
|
if ActiveRecord::VERSION::STRING >= "5.2"
|
65
42
|
def build_relation(klass, attribute, value)
|
66
43
|
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
67
|
-
value = BlindIndex.generate_bidx(value, **bi)
|
68
44
|
attribute = bi[:bidx_attribute]
|
69
45
|
end
|
70
46
|
super(klass, attribute, value)
|
@@ -72,7 +48,6 @@ module BlindIndex
|
|
72
48
|
else
|
73
49
|
def build_relation(klass, table, attribute, value)
|
74
50
|
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
75
|
-
value = BlindIndex.generate_bidx(value, **bi)
|
76
51
|
attribute = bi[:bidx_attribute]
|
77
52
|
end
|
78
53
|
super(klass, table, attribute, value)
|
@@ -11,7 +11,7 @@ module BlindIndex
|
|
11
11
|
raise ArgumentError, "Missing field for key generation" if bidx_attribute.to_s.empty?
|
12
12
|
|
13
13
|
c = "\x7E"*32
|
14
|
-
root_key = hkdf(BlindIndex.decode_key(@master_key), salt: table.to_s, info: "#{c}#{bidx_attribute}", length: 32, hash: "sha384")
|
14
|
+
root_key = hkdf(BlindIndex.decode_key(@master_key, name: "Master key"), salt: table.to_s, info: "#{c}#{bidx_attribute}", length: 32, hash: "sha384")
|
15
15
|
hash_hmac("sha256", pack([table, bidx_attribute, bidx_attribute]), root_key)
|
16
16
|
end
|
17
17
|
|
data/lib/blind_index/model.rb
CHANGED
@@ -10,7 +10,7 @@ module BlindIndex
|
|
10
10
|
# check here so we validate rotate options as well
|
11
11
|
unknown_keywords = options.keys - [:algorithm, :attribute, :bidx_attribute,
|
12
12
|
:callback, :cost, :encode, :expression, :insecure_key, :iterations, :key,
|
13
|
-
:legacy, :master_key, :size, :slow]
|
13
|
+
:legacy, :master_key, :size, :slow, :version]
|
14
14
|
raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
|
15
15
|
|
16
16
|
attribute = options[:attribute] || name
|
@@ -64,25 +64,30 @@ module BlindIndex
|
|
64
64
|
BlindIndex.generate_bidx(value, **blind_indexes[name])
|
65
65
|
end
|
66
66
|
|
67
|
-
define_singleton_method method_name do |value|
|
68
|
-
ActiveSupport::Deprecation.warn("Use #{class_method_name} instead")
|
69
|
-
send(class_method_name, value)
|
70
|
-
end
|
71
|
-
|
72
67
|
define_method method_name do
|
73
|
-
|
68
|
+
send("#{bidx_attribute}=", self.class.send(class_method_name, send(attribute)))
|
74
69
|
end
|
75
70
|
|
76
71
|
if callback
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
|
82
|
-
|
83
|
-
|
84
|
-
|
72
|
+
activerecord = defined?(ActiveRecord) && self < ActiveRecord::Base
|
73
|
+
|
74
|
+
# TODO reuse module
|
75
|
+
m = Module.new do
|
76
|
+
define_method "#{attribute}=" do |value|
|
77
|
+
result = super(value)
|
78
|
+
send(method_name)
|
79
|
+
result
|
80
|
+
end
|
81
|
+
|
82
|
+
unless activerecord
|
83
|
+
define_method "reset_#{attribute}!" do
|
84
|
+
result = super()
|
85
|
+
send(method_name)
|
86
|
+
result
|
87
|
+
end
|
88
|
+
end
|
85
89
|
end
|
90
|
+
prepend m
|
86
91
|
end
|
87
92
|
|
88
93
|
# use include so user can override
|
data/lib/blind_index/mongoid.rb
CHANGED
@@ -26,9 +26,9 @@ module BlindIndex
|
|
26
26
|
|
27
27
|
criterion[bidx_key] =
|
28
28
|
if value.is_a?(Array)
|
29
|
-
value.map { |v| BlindIndex.generate_bidx(v, bi) }
|
29
|
+
value.map { |v| BlindIndex.generate_bidx(v, **bi) }
|
30
30
|
else
|
31
|
-
BlindIndex.generate_bidx(value, bi)
|
31
|
+
BlindIndex.generate_bidx(value, **bi)
|
32
32
|
end
|
33
33
|
end
|
34
34
|
end
|
@@ -39,9 +39,18 @@ module BlindIndex
|
|
39
39
|
end
|
40
40
|
|
41
41
|
module UniquenessValidator
|
42
|
+
def validate_each(record, attribute, value)
|
43
|
+
klass = record.class
|
44
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
45
|
+
value = record.read_attribute_for_validation(bi[:bidx_attribute])
|
46
|
+
end
|
47
|
+
super(record, attribute, value)
|
48
|
+
end
|
49
|
+
|
50
|
+
# change attribute name here instead of validate_each for better error message
|
42
51
|
def create_criteria(base, document, attribute, value)
|
43
|
-
|
44
|
-
|
52
|
+
klass = document.class
|
53
|
+
if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
|
45
54
|
attribute = bi[:bidx_attribute]
|
46
55
|
end
|
47
56
|
super(base, document, attribute, value)
|
data/lib/blind_index/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: blind_index
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 2.1.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Andrew Kane
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2020-08-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -25,19 +25,19 @@ dependencies:
|
|
25
25
|
- !ruby/object:Gem::Version
|
26
26
|
version: '5'
|
27
27
|
- !ruby/object:Gem::Dependency
|
28
|
-
name: argon2
|
28
|
+
name: argon2-kdf
|
29
29
|
requirement: !ruby/object:Gem::Requirement
|
30
30
|
requirements:
|
31
31
|
- - ">="
|
32
32
|
- !ruby/object:Gem::Version
|
33
|
-
version:
|
33
|
+
version: 0.1.1
|
34
34
|
type: :runtime
|
35
35
|
prerelease: false
|
36
36
|
version_requirements: !ruby/object:Gem::Requirement
|
37
37
|
requirements:
|
38
38
|
- - ">="
|
39
39
|
- !ruby/object:Gem::Version
|
40
|
-
version:
|
40
|
+
version: 0.1.1
|
41
41
|
- !ruby/object:Gem::Dependency
|
42
42
|
name: bundler
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
@@ -174,6 +174,7 @@ files:
|
|
174
174
|
- LICENSE.txt
|
175
175
|
- README.md
|
176
176
|
- lib/blind_index.rb
|
177
|
+
- lib/blind_index/backfill.rb
|
177
178
|
- lib/blind_index/extensions.rb
|
178
179
|
- lib/blind_index/key_generator.rb
|
179
180
|
- lib/blind_index/model.rb
|