blind_index 2.0.2 → 2.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 06112a25e062c04740b0a01173300d37c364c05ef3bef0de0fa214e3dfbca61e
-  data.tar.gz: 41c534b7fd3266559821bba50d122a08adbf3215212304dc7c911cb62e204596
+  metadata.gz: 89f2640c25eea46ce76f32e3f8c725744f356c1037bc719c7cf2e1ab2728a1ee
+  data.tar.gz: 984cf9b1d682d5662129e0ce9dee0047c3c57fbbc6d2922913bc59c0024334dc
 SHA512:
-  metadata.gz: 7f3369a1b5ba9b36be21e18c3a41ea9060c779f7cb2b92c2b2539144acd5fb01d99a797b123f6f7b1a97dd53b92c6ff45ba4c5517e43bf99f3e8363875ade522
-  data.tar.gz: 0e5a857dd66532f72635c5ec13f56c4e864f8197533a3d300a5630cda8e9dc2d62b3c3bf771c794df2af668a2249830f6d0ba4e06a396f303131465f17dbd6a2
+  metadata.gz: 99d6f55c8664ba560ac4f7171a402eb6a18347d7be1bfb70425b8f506ba829c808a1e18fc26fda5d8eb7bb88ec7cdcce989e0aadd7e94c22cbfefc915f01150d
+  data.tar.gz: b9df49cf70f1e62e2bc753c0dc065c5fb236b38962cc3bdd7d15d6a7068521aaacd051ac996a9fc05018c48a32b4ca84494f268abee9900b447d3fb107416063

data/CHANGELOG.md

@@ -1,3 +1,21 @@
+## 2.3.0 (2022-01-16)
+
+- Added blind indexes to `filter_attributes`
+- Dropped support for Ruby < 2.6 and Rails < 5.2
+
+## 2.2.0 (2020-09-07)
+
+- Added support for `where` with table in Active Record 5.2+
+
+## 2.1.1 (2020-08-14)
+
+- Fixed `version` option
+
+## 2.1.0 (2020-07-06)
+
+- Improved performance of uniqueness validations
+- Fixed deprecation warnings in Ruby 2.7 with Mongoid
+
 ## 2.0.2 (2020-06-01)
 
 - Improved error message for bad key length
@@ -7,7 +25,7 @@
 
 - Added `BlindIndex.backfill` method
 
-## 2.0.0 (2019-02-10)
+## 2.0.0 (2020-02-10)
 
 - Blind indexes are updated immediately instead of in a `before_validation` callback
 - Better Lockbox integration - no need to generate a separate key

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2017-2020 Andrew Kane
+Copyright (c) 2017-2021 Andrew Kane
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -6,11 +6,11 @@ Works with [Lockbox](https://github.com/ankane/lockbox) ([full example](https://
 
 Learn more about [securing sensitive data in Rails](https://ankane.org/sensitive-data-rails)
 
-[![Build Status](https://travis-ci.org/ankane/blind_index.svg?branch=master)](https://travis-ci.org/ankane/blind_index)
+[![Build Status](https://github.com/ankane/blind_index/workflows/build/badge.svg?branch=master)](https://github.com/ankane/blind_index/actions)
 
 ## How It Works
 
-We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. `LIKE` queries are not possible, but you can index expressions.
+We use [this approach](https://paragonie.com/blog/2017/05/building-searchable-encrypted-databases-with-php-and-sql) by Scott Arciszewski. To summarize, we compute a keyed hash of the sensitive data and store it in a column. To query, we apply the keyed hash function to the value we’re searching and then perform a database search. This results in performant queries for exact matches. Efficient `LIKE` queries are [not possible](#like-ilike-and-full-text-searching), but you can index expressions.
 
 ## Leakage
 
@@ -26,13 +26,13 @@ Add this line to your application’s Gemfile:
 gem 'blind_index'
 ```
 
-## Getting Started
+## Prep
 
 Your model should already be set up with Lockbox or attr_encrypted. The examples are for a `User` model with `encrypts :email` or `attr_encrypted :email`. See the full examples for [Lockbox](https://ankane.org/securing-user-emails-lockbox) and [attr_encrypted](https://ankane.org/securing-user-emails-in-rails) if needed.
 
 Also, if you use attr_encrypted, [generate a key](#key-generation).
 
----
+## Getting Started
 
 Create a migration to add a column for the blind index
 
@@ -69,9 +69,19 @@ And query away
 User.where(email: "test@example.org")
 ```
 
+## Expressions
+
+You can apply expressions to attributes before indexing and searching. This gives you the the ability to perform case-insensitive searches and more.
+
+```ruby
+class User < ApplicationRecord
+  blind_index :email, expression: ->(v) { v.downcase }
+end
+```
+
 ## Validations
 
-To prevent duplicates, use:
+You can use blind indexes for uniqueness validations.
 
 ```ruby
 class User < ApplicationRecord
@@ -79,15 +89,27 @@ class User < ApplicationRecord
 end
 ```
 
-We also recommend adding a unique index to the blind index column through a database migration.
+We recommend adding a unique index to the blind index column through a database migration.
 
-## Expressions
+```ruby
+add_index :users, :email_bidx, unique: true
+```
 
-You can apply expressions to attributes before indexing and searching. This gives you the the ability to perform case-insensitive searches and more.
+For `allow_blank: true`, use:
+
+```ruby
+class User < ApplicationRecord
+  blind_index :email, expression: ->(v) { v.presence }
+  validates :email, uniqueness: {allow_blank: true}
+end
+```
+
+For `case_sensitive: false`, use:
 
 ```ruby
 class User < ApplicationRecord
   blind_index :email, expression: ->(v) { v.downcase }
+  validates :email, uniqueness: true # for best performance, leave out {case_sensitive: false}
 end
 ```
 
@@ -240,6 +262,7 @@ For Mongoid, use:
 ```ruby
 class User
   field :email_bidx, type: String
+  index({email_bidx: 1})
 end
 ```
 
@@ -267,6 +290,30 @@ or create `config/initializers/blind_index.rb` with something like
 BlindIndex.master_key = Rails.application.credentials.blind_index_master_key
 ```
 
+## LIKE, ILIKE, and Full-Text Searching
+
+Unfortunately, blind indexes can’t be used for `LIKE`, `ILIKE`, or full-text searching. Instead, records must be loaded, decrypted, and searched in memory.
+
+For `LIKE`, use:
+
+```ruby
+User.select { |u| u.email.include?("value") }
+```
+
+For `ILIKE`, use:
+
+```ruby
+User.select { |u| u.email =~ /value/i }
+```
+
+For full-text or fuzzy searching, use a gem like [FuzzyMatch](https://github.com/seamusabshere/fuzzy_match):
+
+```ruby
+FuzzyMatch.new(User.all, read: :email).find("value")
+```
+
+If the number of records is large, try to find a way to narrow it down. An [expression index](#expressions) is one way to do this, but leaks which records have the same value of the expression, so use it carefully.
+
 ## Reference
 
 Set default options in an initializer with:
@@ -436,3 +483,5 @@ cd blind_index
 bundle install
 bundle exec rake test
 ```
+
+For security issues, send an email to the address on [this page](https://github.com/ankane).

data/lib/blind_index/extensions.rb

@@ -1,24 +1,6 @@
 module BlindIndex
   module Extensions
     module TableMetadata
-      def resolve_column_aliases(hash)
-        new_hash = super
-        if has_blind_indexes?
-          hash.each do |key, _|
-            if key.respond_to?(:to_sym) && (bi = klass.blind_indexes[key.to_sym]) && !new_hash[key].is_a?(ActiveRecord::StatementCache::Substitute)
-              value = new_hash.delete(key)
-              new_hash[bi[:bidx_attribute]] =
-                if value.is_a?(Array)
-                  value.map { |v| BlindIndex.generate_bidx(v, **bi) }
-                else
-                  BlindIndex.generate_bidx(value, **bi)
-                end
-            end
-          end
-        end
-        new_hash
-      end
-
       # memoize for performance
       def has_blind_indexes?
         unless defined?(@has_blind_indexes)
@@ -28,23 +10,38 @@ module BlindIndex
       end
     end
 
+    module PredicateBuilder
+      # https://github.com/rails/rails/commit/56f30962b84fc53b76001301fb830c1594fd377e
+      def build(attribute, value, *args)
+        if table.has_blind_indexes? && (bi = table.send(:klass).blind_indexes[attribute.name.to_sym]) && !value.is_a?(ActiveRecord::StatementCache::Substitute)
+          attribute = attribute.relation[bi[:bidx_attribute]]
+          value =
+            if value.is_a?(Array)
+              value.map { |v| BlindIndex.generate_bidx(v, **bi) }
+            else
+              BlindIndex.generate_bidx(value, **bi)
+            end
+        end
+
+        super(attribute, value, *args)
+      end
+    end
+
     module UniquenessValidator
-      if ActiveRecord::VERSION::STRING >= "5.2"
-        def build_relation(klass, attribute, value)
-          if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
-            value = BlindIndex.generate_bidx(value, **bi)
-            attribute = bi[:bidx_attribute]
-          end
-          super(klass, attribute, value)
+      def validate_each(record, attribute, value)
+        klass = record.class
+        if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
+          value = record.read_attribute_for_validation(bi[:bidx_attribute])
         end
-      else
-        def build_relation(klass, table, attribute, value)
-          if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
-            value = BlindIndex.generate_bidx(value, **bi)
-            attribute = bi[:bidx_attribute]
-          end
-          super(klass, table, attribute, value)
+        super(record, attribute, value)
+      end
+
+      # change attribute name here instead of validate_each for better error message
+      def build_relation(klass, attribute, value)
+        if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
+          attribute = bi[:bidx_attribute]
         end
+        super(klass, attribute, value)
       end
     end
 

data/lib/blind_index/model.rb

@@ -10,7 +10,7 @@ module BlindIndex
         # check here so we validate rotate options as well
         unknown_keywords = options.keys - [:algorithm, :attribute, :bidx_attribute,
           :callback, :cost, :encode, :expression, :insecure_key, :iterations, :key,
-          :legacy, :master_key, :size, :slow]
+          :legacy, :master_key, :size, :slow, :version]
         raise ArgumentError, "unknown keywords: #{unknown_keywords.join(", ")}" if unknown_keywords.any?
 
         attribute = options[:attribute] || name
@@ -36,6 +36,14 @@ module BlindIndex
         key ||= -> { BlindIndex.index_key(table: try(:table_name) || collection_name.to_s, bidx_attribute: bidx_attribute, master_key: options[:master_key], encode: false) }
 
         class_eval do
+          activerecord = defined?(ActiveRecord) && self < ActiveRecord::Base
+
+          if activerecord && ActiveRecord::VERSION::MAJOR >= 6
+            # blind index value isn't really sensitive
+            # but don't need to show it in the Rails console
+            self.filter_attributes += [/\A#{Regexp.escape(bidx_attribute)}\z/]
+          end
+
           @blind_indexes ||= {}
 
           unless respond_to?(:blind_indexes)
@@ -69,8 +77,6 @@ module BlindIndex
           end
 
           if callback
-            activerecord = defined?(ActiveRecord) && self < ActiveRecord::Base
-
             # TODO reuse module
             m = Module.new do
               define_method "#{attribute}=" do |value|

data/lib/blind_index/mongoid.rb

@@ -26,9 +26,9 @@ module BlindIndex
 
               criterion[bidx_key] =
                 if value.is_a?(Array)
-                  value.map { |v| BlindIndex.generate_bidx(v, bi) }
+                  value.map { |v| BlindIndex.generate_bidx(v, **bi) }
                 else
-                  BlindIndex.generate_bidx(value, bi)
+                  BlindIndex.generate_bidx(value, **bi)
                 end
             end
           end
@@ -39,9 +39,18 @@ module BlindIndex
     end
 
     module UniquenessValidator
+      def validate_each(record, attribute, value)
+        klass = record.class
+        if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
+          value = record.read_attribute_for_validation(bi[:bidx_attribute])
+        end
+        super(record, attribute, value)
+      end
+
+      # change attribute name here instead of validate_each for better error message
       def create_criteria(base, document, attribute, value)
-        if base.respond_to?(:blind_indexes) && (bi = base.blind_indexes[attribute])
-          value = BlindIndex.generate_bidx(value, bi)
+        klass = document.class
+        if klass.respond_to?(:blind_indexes) && (bi = klass.blind_indexes[attribute])
           attribute = bi[:bidx_attribute]
         end
         super(base, document, attribute, value)

data/lib/blind_index/version.rb

@@ -1,3 +1,3 @@
 module BlindIndex
-  VERSION = "2.0.2"
+  VERSION = "2.3.0"
 end

data/lib/blind_index.rb

@@ -1,8 +1,10 @@
 # dependencies
 require "active_support"
-require "openssl"
 require "argon2/kdf"
 
+# stdlib
+require "openssl"
+
 # modules
 require "blind_index/backfill"
 require "blind_index/key_generator"
@@ -140,18 +142,13 @@ ActiveSupport.on_load(:active_record) do
 
   ActiveRecord::TableMetadata.prepend(BlindIndex::Extensions::TableMetadata)
   ActiveRecord::DynamicMatchers::Method.prepend(BlindIndex::Extensions::DynamicMatchers)
-
-  unless ActiveRecord::VERSION::STRING.start_with?("5.1.")
-    ActiveRecord::Validations::UniquenessValidator.prepend(BlindIndex::Extensions::UniquenessValidator)
-  end
+  ActiveRecord::Validations::UniquenessValidator.prepend(BlindIndex::Extensions::UniquenessValidator)
+  ActiveRecord::PredicateBuilder.prepend(BlindIndex::Extensions::PredicateBuilder)
 end
 
-if defined?(Mongoid)
-  # TODO find better ActiveModel hook
-  require "active_model/callbacks"
-  ActiveModel::Callbacks.include(BlindIndex::Model)
-
+ActiveSupport.on_load(:mongoid) do
   require "blind_index/mongoid"
+  Mongoid::Document::ClassMethods.include(BlindIndex::Model)
   Mongoid::Criteria.prepend(BlindIndex::Mongoid::Criteria)
   Mongoid::Validatable::UniquenessValidator.prepend(BlindIndex::Mongoid::UniquenessValidator)
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: blind_index
 version: !ruby/object:Gem::Version
-  version: 2.0.2
+  version: 2.3.0
 platform: ruby
 authors:
 - Andrew Kane
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-06-01 00:00:00.000000000 Z
+date: 2022-01-16 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -16,14 +16,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '5'
+        version: '5.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '5'
+        version: '5.2'
 - !ruby/object:Gem::Dependency
   name: argon2-kdf
   requirement: !ruby/object:Gem::Requirement
@@ -38,134 +38,8 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 0.1.1
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: rake
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: minitest
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: attr_encrypted
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: activerecord
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: sqlite3
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: scrypt
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: benchmark-ips
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0'
-- !ruby/object:Gem::Dependency
-  name: lockbox
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0.2'
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '0.2'
-description: 
-email: andrew@chartkick.com
+description:
+email: andrew@ankane.org
 executables: []
 extensions: []
 extra_rdoc_files: []
@@ -184,7 +58,7 @@ homepage: https://github.com/ankane/blind_index
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -192,15 +66,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.4'
+      version: '2.6'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.2
-signing_key: 
+rubygems_version: 3.3.3
+signing_key:
 specification_version: 4
 summary: Securely search encrypted database fields
 test_files: []