blind_index 2.3.1 → 2.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
 - data/CHANGELOG.md +4 -0
 - data/README.md +35 -102
 - data/lib/blind_index/model.rb +2 -2
 - data/lib/blind_index/version.rb +1 -1
 - data/lib/blind_index.rb +6 -6
 - metadata +3 -3
 
    
        checksums.yaml
    CHANGED
    
    | 
         @@ -1,7 +1,7 @@ 
     | 
|
| 
       1 
1 
     | 
    
         
             
            ---
         
     | 
| 
       2 
2 
     | 
    
         
             
            SHA256:
         
     | 
| 
       3 
     | 
    
         
            -
              metadata.gz:  
     | 
| 
       4 
     | 
    
         
            -
              data.tar.gz:  
     | 
| 
      
 3 
     | 
    
         
            +
              metadata.gz: 5a8f12336176bc78927ea8f645fa707e872db1a4523ee00ea5e43e5b6663db19
         
     | 
| 
      
 4 
     | 
    
         
            +
              data.tar.gz: fb0eab6db99944ab6e249f26e2f5f01846127732d23443944a11e2d3bcd05b59
         
     | 
| 
       5 
5 
     | 
    
         
             
            SHA512:
         
     | 
| 
       6 
     | 
    
         
            -
              metadata.gz:  
     | 
| 
       7 
     | 
    
         
            -
              data.tar.gz:  
     | 
| 
      
 6 
     | 
    
         
            +
              metadata.gz: 170b3ce8ee37f46f930c9bc7a16d828bf86c2653d215b05b1d34c93d0f60f1cb6f246f201095be33cff78e2f1c8de440987177e55462f1f5c662fd9b4616ffc7
         
     | 
| 
      
 7 
     | 
    
         
            +
              data.tar.gz: 35690b9711146af73d4dfc4d8b92d7f1562619d2a98212ce9c2d4f6edafc4a785ae2458c5455c12c2fb312d986b1fb0f4df34883987ec1a4840fa49436e3ff26
         
     | 
    
        data/CHANGELOG.md
    CHANGED
    
    
    
        data/README.md
    CHANGED
    
    | 
         @@ -224,17 +224,27 @@ Finally, drop the old column. 
     | 
|
| 
       224 
224 
     | 
    
         | 
| 
       225 
225 
     | 
    
         
             
            ## Key Separation
         
     | 
| 
       226 
226 
     | 
    
         | 
| 
       227 
     | 
    
         
            -
            The master key is used to generate unique keys for each blind index. This technique comes from [CipherSweet](https://ciphersweet.paragonie.com/internals/key-hierarchy). The table name and blind index column name are both used in this process. 
     | 
| 
      
 227 
     | 
    
         
            +
            The master key is used to generate unique keys for each blind index. This technique comes from [CipherSweet](https://ciphersweet.paragonie.com/internals/key-hierarchy). The table name and blind index column name are both used in this process.
         
     | 
| 
      
 228 
     | 
    
         
            +
             
     | 
| 
      
 229 
     | 
    
         
            +
            You can get an individual key with:
         
     | 
| 
       228 
230 
     | 
    
         | 
| 
       229 
231 
     | 
    
         
             
            ```ruby
         
     | 
| 
       230 
232 
     | 
    
         
             
            BlindIndex.index_key(table: "users", bidx_attribute: "email_bidx")
         
     | 
| 
       231 
233 
     | 
    
         
             
            ```
         
     | 
| 
       232 
234 
     | 
    
         | 
| 
       233 
     | 
    
         
            -
             
     | 
| 
      
 235 
     | 
    
         
            +
            To rename a table with blind indexes, use:
         
     | 
| 
       234 
236 
     | 
    
         | 
| 
       235 
237 
     | 
    
         
             
            ```ruby
         
     | 
| 
       236 
238 
     | 
    
         
             
            class User < ApplicationRecord
         
     | 
| 
       237 
     | 
    
         
            -
              blind_index :email,  
     | 
| 
      
 239 
     | 
    
         
            +
              blind_index :email, key_table: "original_table"
         
     | 
| 
      
 240 
     | 
    
         
            +
            end
         
     | 
| 
      
 241 
     | 
    
         
            +
            ```
         
     | 
| 
      
 242 
     | 
    
         
            +
             
     | 
| 
      
 243 
     | 
    
         
            +
            To rename a blind index column, use:
         
     | 
| 
      
 244 
     | 
    
         
            +
             
     | 
| 
      
 245 
     | 
    
         
            +
            ```ruby
         
     | 
| 
      
 246 
     | 
    
         
            +
            class User < ApplicationRecord
         
     | 
| 
      
 247 
     | 
    
         
            +
              blind_index :email, key_attribute: "original_column"
         
     | 
| 
       238 
248 
     | 
    
         
             
            end
         
     | 
| 
       239 
249 
     | 
    
         
             
            ```
         
     | 
| 
       240 
250 
     | 
    
         | 
| 
         @@ -346,6 +356,28 @@ class User < ApplicationRecord 
     | 
|
| 
       346 
356 
     | 
    
         
             
            end
         
     | 
| 
       347 
357 
     | 
    
         
             
            ```
         
     | 
| 
       348 
358 
     | 
    
         | 
| 
      
 359 
     | 
    
         
            +
            ## Compatibility
         
     | 
| 
      
 360 
     | 
    
         
            +
             
     | 
| 
      
 361 
     | 
    
         
            +
            You can generate blind indexes from other languages as well. For Python, you can use [argon2-cffi](https://github.com/hynek/argon2-cffi).
         
     | 
| 
      
 362 
     | 
    
         
            +
             
     | 
| 
      
 363 
     | 
    
         
            +
            ```python
         
     | 
| 
      
 364 
     | 
    
         
            +
            from argon2.low_level import Type, hash_secret_raw
         
     | 
| 
      
 365 
     | 
    
         
            +
            from base64 import b64encode
         
     | 
| 
      
 366 
     | 
    
         
            +
             
     | 
| 
      
 367 
     | 
    
         
            +
            key = '289737bab72fa97b1f4b081cef00d7b7d75034bcf3183c363feaf3e6441777bc'
         
     | 
| 
      
 368 
     | 
    
         
            +
            value = 'test@example.org'
         
     | 
| 
      
 369 
     | 
    
         
            +
             
     | 
| 
      
 370 
     | 
    
         
            +
            bidx = b64encode(hash_secret_raw(
         
     | 
| 
      
 371 
     | 
    
         
            +
                secret=value.encode(),
         
     | 
| 
      
 372 
     | 
    
         
            +
                salt=bytes.fromhex(key),
         
     | 
| 
      
 373 
     | 
    
         
            +
                time_cost=3,
         
     | 
| 
      
 374 
     | 
    
         
            +
                memory_cost=2**12,
         
     | 
| 
      
 375 
     | 
    
         
            +
                parallelism=1,
         
     | 
| 
      
 376 
     | 
    
         
            +
                hash_len=32,
         
     | 
| 
      
 377 
     | 
    
         
            +
                type=Type.ID
         
     | 
| 
      
 378 
     | 
    
         
            +
            ))
         
     | 
| 
      
 379 
     | 
    
         
            +
            ```
         
     | 
| 
      
 380 
     | 
    
         
            +
             
     | 
| 
       349 
381 
     | 
    
         
             
            ## Alternatives
         
     | 
| 
       350 
382 
     | 
    
         | 
| 
       351 
383 
     | 
    
         
             
            One alternative to blind indexing is to use a deterministic encryption scheme, like [AES-SIV](https://github.com/miscreant/miscreant). In this approach, the encrypted data will be the same for matches. We recommend blind indexing over deterministic encryption because:
         
     | 
| 
         @@ -363,105 +395,6 @@ One alternative to blind indexing is to use a deterministic encryption scheme, l 
     | 
|
| 
       363 
395 
     | 
    
         
             
            - Better Lockbox integration - no need to generate a separate key
         
     | 
| 
       364 
396 
     | 
    
         
             
            - There’s a new gem for Argon2 that has no dependencies and (officially) supports Windows
         
     | 
| 
       365 
397 
     | 
    
         | 
| 
       366 
     | 
    
         
            -
            ### 1.0.0
         
     | 
| 
       367 
     | 
    
         
            -
             
     | 
| 
       368 
     | 
    
         
            -
            1.0.0 brings a number of improvements. Here are a few to be aware of:
         
     | 
| 
       369 
     | 
    
         
            -
             
     | 
| 
       370 
     | 
    
         
            -
            - Argon2id is the default algorithm for stronger security
         
     | 
| 
       371 
     | 
    
         
            -
            - You can use a master key instead of individual keys for each column
         
     | 
| 
       372 
     | 
    
         
            -
            - Columns no longer have an `encrypted_` prefix
         
     | 
| 
       373 
     | 
    
         
            -
             
     | 
| 
       374 
     | 
    
         
            -
            For existing fields, add:
         
     | 
| 
       375 
     | 
    
         
            -
             
     | 
| 
       376 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       377 
     | 
    
         
            -
            class User < ApplicationRecord
         
     | 
| 
       378 
     | 
    
         
            -
              blind_index :email, legacy: true
         
     | 
| 
       379 
     | 
    
         
            -
            end
         
     | 
| 
       380 
     | 
    
         
            -
            ```
         
     | 
| 
       381 
     | 
    
         
            -
             
     | 
| 
       382 
     | 
    
         
            -
            #### Optional
         
     | 
| 
       383 
     | 
    
         
            -
             
     | 
| 
       384 
     | 
    
         
            -
            To rotate to new fields that use Argon2id and a master key, generate a master key:
         
     | 
| 
       385 
     | 
    
         
            -
             
     | 
| 
       386 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       387 
     | 
    
         
            -
            BlindIndex.generate_key
         
     | 
| 
       388 
     | 
    
         
            -
            ```
         
     | 
| 
       389 
     | 
    
         
            -
             
     | 
| 
       390 
     | 
    
         
            -
            And set `ENV["BLIND_INDEX_MASTER_KEY"]` or `BlindIndex.master_key`.
         
     | 
| 
       391 
     | 
    
         
            -
             
     | 
| 
       392 
     | 
    
         
            -
            Add a new column without the `encrypted_` prefix:
         
     | 
| 
       393 
     | 
    
         
            -
             
     | 
| 
       394 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       395 
     | 
    
         
            -
            add_column :users, :email_bidx, :string
         
     | 
| 
       396 
     | 
    
         
            -
            add_index :users, :email_bidx # unique: true if needed
         
     | 
| 
       397 
     | 
    
         
            -
            ```
         
     | 
| 
       398 
     | 
    
         
            -
             
     | 
| 
       399 
     | 
    
         
            -
            And add to your model
         
     | 
| 
       400 
     | 
    
         
            -
             
     | 
| 
       401 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       402 
     | 
    
         
            -
            class User < ApplicationRecord
         
     | 
| 
       403 
     | 
    
         
            -
              blind_index :email, key: ENV["USER_EMAIL_BLIND_INDEX_KEY"], legacy: true, rotate: {}
         
     | 
| 
       404 
     | 
    
         
            -
            end
         
     | 
| 
       405 
     | 
    
         
            -
            ```
         
     | 
| 
       406 
     | 
    
         
            -
             
     | 
| 
       407 
     | 
    
         
            -
            > For more sensitive fields, use `rotate: {slow: true}`
         
     | 
| 
       408 
     | 
    
         
            -
             
     | 
| 
       409 
     | 
    
         
            -
            This will keep the new column synced going forward. Next, backfill the data:
         
     | 
| 
       410 
     | 
    
         
            -
             
     | 
| 
       411 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       412 
     | 
    
         
            -
            User.unscoped.where(email_bidx: nil).find_each do |user|
         
     | 
| 
       413 
     | 
    
         
            -
              user.compute_rotated_email_bidx
         
     | 
| 
       414 
     | 
    
         
            -
              user.save(validate: false)
         
     | 
| 
       415 
     | 
    
         
            -
            end
         
     | 
| 
       416 
     | 
    
         
            -
            ```
         
     | 
| 
       417 
     | 
    
         
            -
             
     | 
| 
       418 
     | 
    
         
            -
            Then update your model
         
     | 
| 
       419 
     | 
    
         
            -
             
     | 
| 
       420 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       421 
     | 
    
         
            -
            class User < ApplicationRecord
         
     | 
| 
       422 
     | 
    
         
            -
              blind_index :email
         
     | 
| 
       423 
     | 
    
         
            -
            end
         
     | 
| 
       424 
     | 
    
         
            -
            ```
         
     | 
| 
       425 
     | 
    
         
            -
             
     | 
| 
       426 
     | 
    
         
            -
            > For more sensitive fields, add `slow: true`
         
     | 
| 
       427 
     | 
    
         
            -
             
     | 
| 
       428 
     | 
    
         
            -
            Finally, drop the old column.
         
     | 
| 
       429 
     | 
    
         
            -
             
     | 
| 
       430 
     | 
    
         
            -
            ### 0.3.0
         
     | 
| 
       431 
     | 
    
         
            -
             
     | 
| 
       432 
     | 
    
         
            -
            This version introduces a breaking change to enforce secure key generation. An error is thrown if your blind index key isn’t both binary and 32 bytes.
         
     | 
| 
       433 
     | 
    
         
            -
             
     | 
| 
       434 
     | 
    
         
            -
            We recommend rotating your key if it doesn’t meet this criteria. You can generate a new key in the Rails console with:
         
     | 
| 
       435 
     | 
    
         
            -
             
     | 
| 
       436 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       437 
     | 
    
         
            -
            SecureRandom.hex(32)
         
     | 
| 
       438 
     | 
    
         
            -
            ```
         
     | 
| 
       439 
     | 
    
         
            -
             
     | 
| 
       440 
     | 
    
         
            -
            Update your model to convert the hex key to binary.
         
     | 
| 
       441 
     | 
    
         
            -
             
     | 
| 
       442 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       443 
     | 
    
         
            -
            class User < ApplicationRecord
         
     | 
| 
       444 
     | 
    
         
            -
              blind_index :email, key: [ENV["USER_EMAIL_BLIND_INDEX_KEY"]].pack("H*")
         
     | 
| 
       445 
     | 
    
         
            -
            end
         
     | 
| 
       446 
     | 
    
         
            -
            ```
         
     | 
| 
       447 
     | 
    
         
            -
             
     | 
| 
       448 
     | 
    
         
            -
            And recompute the blind index.
         
     | 
| 
       449 
     | 
    
         
            -
             
     | 
| 
       450 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       451 
     | 
    
         
            -
            User.unscoped.find_each do |user|
         
     | 
| 
       452 
     | 
    
         
            -
              user.compute_email_bidx
         
     | 
| 
       453 
     | 
    
         
            -
              user.save(validate: false)
         
     | 
| 
       454 
     | 
    
         
            -
            end
         
     | 
| 
       455 
     | 
    
         
            -
            ```
         
     | 
| 
       456 
     | 
    
         
            -
             
     | 
| 
       457 
     | 
    
         
            -
            To continue without rotating, set:
         
     | 
| 
       458 
     | 
    
         
            -
             
     | 
| 
       459 
     | 
    
         
            -
            ```ruby
         
     | 
| 
       460 
     | 
    
         
            -
            class User < ApplicationRecord
         
     | 
| 
       461 
     | 
    
         
            -
              blind_index :email, insecure_key: true
         
     | 
| 
       462 
     | 
    
         
            -
            end
         
     | 
| 
       463 
     | 
    
         
            -
            ```
         
     | 
| 
       464 
     | 
    
         
            -
             
     | 
| 
       465 
398 
     | 
    
         
             
            ## History
         
     | 
| 
       466 
399 
     | 
    
         | 
| 
       467 
400 
     | 
    
         
             
            View the [changelog](https://github.com/ankane/blind_index/blob/master/CHANGELOG.md)
         
     | 
    
        data/lib/blind_index/model.rb
    CHANGED
    
    | 
         @@ -10,7 +10,7 @@ module BlindIndex 
     | 
|
| 
       10 
10 
     | 
    
         
             
                    # check here so we validate rotate options as well
         
     | 
| 
       11 
11 
     | 
    
         
             
                    unknown_keywords = options.keys - [:algorithm, :attribute, :bidx_attribute,
         
     | 
| 
       12 
12 
     | 
    
         
             
                      :callback, :cost, :encode, :expression, :insecure_key, :iterations, :key,
         
     | 
| 
       13 
     | 
    
         
            -
                      :legacy, :master_key, :size, :slow, :version]
         
     | 
| 
      
 13 
     | 
    
         
            +
                      :key_attribute, :key_table, :legacy, :master_key, :size, :slow, :version]
         
     | 
| 
       14 
14 
     | 
    
         
             
                    raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
         
     | 
| 
       15 
15 
     | 
    
         | 
| 
       16 
16 
     | 
    
         
             
                    attribute = options[:attribute] || name
         
     | 
| 
         @@ -33,7 +33,7 @@ module BlindIndex 
     | 
|
| 
       33 
33 
     | 
    
         
             
                    class_method_name = :"generate_#{name}_bidx"
         
     | 
| 
       34 
34 
     | 
    
         | 
| 
       35 
35 
     | 
    
         
             
                    key = options[:key]
         
     | 
| 
       36 
     | 
    
         
            -
                    key ||= -> { BlindIndex.index_key(table: try(:table_name) || collection_name.to_s, bidx_attribute: bidx_attribute, master_key: options[:master_key], encode: false) }
         
     | 
| 
      
 36 
     | 
    
         
            +
                    key ||= -> { BlindIndex.index_key(table: options[:key_table] || try(:table_name) || collection_name.to_s, bidx_attribute: options[:key_attribute] || bidx_attribute, master_key: options[:master_key], encode: false) }
         
     | 
| 
       37 
37 
     | 
    
         | 
| 
       38 
38 
     | 
    
         
             
                    class_eval do
         
     | 
| 
       39 
39 
     | 
    
         
             
                      activerecord = defined?(ActiveRecord) && self < ActiveRecord::Base
         
     | 
    
        data/lib/blind_index/version.rb
    CHANGED
    
    
    
        data/lib/blind_index.rb
    CHANGED
    
    | 
         @@ -6,10 +6,10 @@ require "argon2/kdf" 
     | 
|
| 
       6 
6 
     | 
    
         
             
            require "openssl"
         
     | 
| 
       7 
7 
     | 
    
         | 
| 
       8 
8 
     | 
    
         
             
            # modules
         
     | 
| 
       9 
     | 
    
         
            -
             
     | 
| 
       10 
     | 
    
         
            -
             
     | 
| 
       11 
     | 
    
         
            -
             
     | 
| 
       12 
     | 
    
         
            -
             
     | 
| 
      
 9 
     | 
    
         
            +
            require_relative "blind_index/backfill"
         
     | 
| 
      
 10 
     | 
    
         
            +
            require_relative "blind_index/key_generator"
         
     | 
| 
      
 11 
     | 
    
         
            +
            require_relative "blind_index/model"
         
     | 
| 
      
 12 
     | 
    
         
            +
            require_relative "blind_index/version"
         
     | 
| 
       13 
13 
     | 
    
         | 
| 
       14 
14 
     | 
    
         
             
            module BlindIndex
         
     | 
| 
       15 
15 
     | 
    
         
             
              class Error < StandardError; end
         
     | 
| 
         @@ -137,7 +137,7 @@ module BlindIndex 
     | 
|
| 
       137 
137 
     | 
    
         
             
            end
         
     | 
| 
       138 
138 
     | 
    
         | 
| 
       139 
139 
     | 
    
         
             
            ActiveSupport.on_load(:active_record) do
         
     | 
| 
       140 
     | 
    
         
            -
               
     | 
| 
      
 140 
     | 
    
         
            +
              require_relative "blind_index/extensions"
         
     | 
| 
       141 
141 
     | 
    
         
             
              extend BlindIndex::Model
         
     | 
| 
       142 
142 
     | 
    
         | 
| 
       143 
143 
     | 
    
         
             
              ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
         
     | 
| 
         @@ -147,7 +147,7 @@ ActiveSupport.on_load(:active_record) do 
     | 
|
| 
       147 
147 
     | 
    
         
             
            end
         
     | 
| 
       148 
148 
     | 
    
         | 
| 
       149 
149 
     | 
    
         
             
            ActiveSupport.on_load(:mongoid) do
         
     | 
| 
       150 
     | 
    
         
            -
               
     | 
| 
      
 150 
     | 
    
         
            +
              require_relative "blind_index/mongoid"
         
     | 
| 
       151 
151 
     | 
    
         
             
              Mongoid::Document::ClassMethods.include(BlindIndex::Model)
         
     | 
| 
       152 
152 
     | 
    
         
             
              Mongoid::Criteria.prepend(BlindIndex::Mongoid::Criteria)
         
     | 
| 
       153 
153 
     | 
    
         
             
              Mongoid::Validatable::UniquenessValidator.prepend(BlindIndex::Mongoid::UniquenessValidator)
         
     | 
    
        metadata
    CHANGED
    
    | 
         @@ -1,14 +1,14 @@ 
     | 
|
| 
       1 
1 
     | 
    
         
             
            --- !ruby/object:Gem::Specification
         
     | 
| 
       2 
2 
     | 
    
         
             
            name: blind_index
         
     | 
| 
       3 
3 
     | 
    
         
             
            version: !ruby/object:Gem::Version
         
     | 
| 
       4 
     | 
    
         
            -
              version: 2.3. 
     | 
| 
      
 4 
     | 
    
         
            +
              version: 2.3.2
         
     | 
| 
       5 
5 
     | 
    
         
             
            platform: ruby
         
     | 
| 
       6 
6 
     | 
    
         
             
            authors:
         
     | 
| 
       7 
7 
     | 
    
         
             
            - Andrew Kane
         
     | 
| 
       8 
8 
     | 
    
         
             
            autorequire:
         
     | 
| 
       9 
9 
     | 
    
         
             
            bindir: bin
         
     | 
| 
       10 
10 
     | 
    
         
             
            cert_chain: []
         
     | 
| 
       11 
     | 
    
         
            -
            date:  
     | 
| 
      
 11 
     | 
    
         
            +
            date: 2023-04-27 00:00:00.000000000 Z
         
     | 
| 
       12 
12 
     | 
    
         
             
            dependencies:
         
     | 
| 
       13 
13 
     | 
    
         
             
            - !ruby/object:Gem::Dependency
         
     | 
| 
       14 
14 
     | 
    
         
             
              name: activesupport
         
     | 
| 
         @@ -73,7 +73,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement 
     | 
|
| 
       73 
73 
     | 
    
         
             
                - !ruby/object:Gem::Version
         
     | 
| 
       74 
74 
     | 
    
         
             
                  version: '0'
         
     | 
| 
       75 
75 
     | 
    
         
             
            requirements: []
         
     | 
| 
       76 
     | 
    
         
            -
            rubygems_version: 3. 
     | 
| 
      
 76 
     | 
    
         
            +
            rubygems_version: 3.4.10
         
     | 
| 
       77 
77 
     | 
    
         
             
            signing_key:
         
     | 
| 
       78 
78 
     | 
    
         
             
            specification_version: 4
         
     | 
| 
       79 
79 
     | 
    
         
             
            summary: Securely search encrypted database fields
         
     |