has_unpublished_password 0.2.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f3b72f37de09c2757a24ed07a9324a635caeb962a3cc8c947593ffe082bda976
-  data.tar.gz: a7717f80f508d92fe81359111d1e1ac79e4b73faf19dbcd64ad806d2a82025f4
+  metadata.gz: ccd1bd39bcceaea24cb767e8378b0a2a95590a8f3639585cb571072ec3d7d40b
+  data.tar.gz: 71170d7d08eeb3452a39970216f2188ee5e68ae883db8b3a6dee80387161c635
 SHA512:
-  metadata.gz: 337475d798ece49c9cb9863a42c590dc300905d885d42622d037ce6d6376ccc4c41bc7724184f0ccd2494aab10efc675f051991548e6da16e5df5ef902b35dba
-  data.tar.gz: 527ca7b0b75f9cc5ac6d9e496c1eb8caa374efbfd81484808dd8d083bb738c6d4090b2e3d4510be217214ed87f18c70b0707e1389d4717031d6d2c05b72df245
+  metadata.gz: b615eb2ad57cb7678ba7ae60e4d1d7f6dfd1df8de58e01e4387bf34293d91721d4384029489098b065a9a61f6b9c3ea717a094dd550bbe37e9f37d9e995e70ea
+  data.tar.gz: 931357de56d6cbbc8b3e0e8eb36ad69790d5d1db108ce0d4dbd63ceee13dea757ad1b929ad5992242800661208957e8cb4a40de5f423ebab9a694e007a474f74

data/README.md

@@ -2,23 +2,24 @@
 
 ## What is it?
 
-This is a gem which performs offline checks against the [HIBP](https://haveibeenpwned.com/) master list.
+This is a gem which performs offline checks against the [HIBP](https://haveibeenpwned.com/) master list (well, the top 11,000,000 passwords in it).
 
 It can be used to ensure that your users are not using credentials which have previously been leaked.
 
-The checks are performed using a pre-built cuckoo filter.
+The checks are performed using a pre-built bloom filter.
 
-## Status
+### Why not the full list?
+There's a tradeoff to be made between the false positive rate, the number of passwords checked, and the amount of disk/network bandwidth used.
+
+The full list is ~11gb compressed, and the smallest bloom filter that'll get an acceptable false positive rate on the full list is ~1gb. This gem is 32mb.
 
-I just threw this together and haven't used it in production yet.
+## Status
 
-I've pre-built filters from three datasets (found in the `hibp-cuckoo-filter` gem):
+In use in production on a fairly large site (https://radiopaedia.org).
 
- * The 'top 560k passwords' filter is 992kb, with a 1.7% false positive rate. Uses 1mb ram once loaded.
- * The 'top 5600k passwords' filter is 9.6mb, with a 2.1% false positive rate. Uses 8mb ram once loaded.
- * The 'top 56m passwords' filter is 80mb, with a 2.6% false positive rate. Uses 67mb ram once loaded.
+The released version of this gem includes ~30mb of bloom filter containing the top 11,200,000 most-leaked passwords according to HIBP.
 
-On my recent macbook pro, checking a single password against the largest filter takes about 3.2 microseconds.
+Checking set membership is *fast*, and the false positive rate is about 0.001%.
 
 ## Installation
 
@@ -26,7 +27,6 @@ Add this line to your application's Gemfile:
 
 ```ruby
 gem 'has_unpublished_password'
-gem 'hibp-cuckoo-filter' # optional data files, if you don't want to build your own.
 ```
 
 And then execute:
@@ -39,53 +39,20 @@ Or install it yourself as:
 
 ## Usage
 
-### Configuration
-
-Add an initializer (eg `config/initializers/has_unpublished_password.rb`):
-
-    HasUnpublishedPassword.configure do |config|
-      # Valid values: :small, :medium, :large
-      config.filter = :large
-    end
-
-
 ### Validation
 
 `validates :password, never_leaked_to_hibp: true`
 
-### Low level usage
-```
-filter = HasUnpublishedPassword.import('serialized.json.gz')
-filter.has? Digest::SHA1.hexdigest('password') # true
-```
-
 ## Development
 
-### Native component
-
-The [native component](https://github.com/DanielHeath/rust-cuckoofilter) is written in rust.
-
-To update it to a new version, cross compile it in release mode, then:
-```
-cd ../rust-cuckoofilter/cabi
-
-cargo build --target=x86_64-unknown-linux-gnu --release
-cp target/release/libcuckoofilter_cabi.d ../../has_unpublished_password/nativeext/x86_64/
-
-cargo build --target=x86_64-apple-darwin --release
-cp target/release/libcuckoofilter_cabi.dylib ../../has_unpublished_password/nativeext/x86_64/
-```
-
 ### Building the filter
 
 First, download the master list from HIBP (I used the 'ordered by frequency' list) and decompress it.
 
-Then, run `data/prepare.sh <path-to-master-list-file>`.
+Then, run `bundle exec data/prepare-and-validate.rb <path-to-master-list-file>`.
 
 This takes quite awhile; it'll print how many lines it's completed periodically.
 
-This process writes its result to `serialized.json.gz` when complete.
-
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/danielheath/has_unpublished_password. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [Contributor Covenant](http://contributor-covenant.org) code of conduct.
@@ -98,16 +65,3 @@ The gem is available as open source under the terms of the [MIT License](https:/
 
 Everyone interacting in the HasUnpublishedPassword project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/has_unpublished_password/blob/master/CODE_OF_CONDUCT.md).
 
-
-
-
-
-
-
-
-
-
-
-24 bits is 2mb. 1/3rd of values will be filled by 5 million rows.
-
-take 6 24-bit sequences (for 12mb). That yields a 0.13% collision chance. That's promising.

data/lib/has_unpublished_password/never_leaked_to_hibp_validator.rb

@@ -2,6 +2,7 @@
 
 class NeverLeakedToHibpValidator < ActiveModel::EachValidator
   def validate_each(record, attribute, value)
+    return unless value
     if HasUnpublishedPassword.has_been_published? value
       record.errors[attribute] << (options[:message] || "That's one of the first passwords a hacker would try.")
     end

data/lib/has_unpublished_password/version.rb

@@ -1,3 +1,3 @@
 module HasUnpublishedPassword
-  VERSION = "0.2.0"
+  VERSION = "0.2.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: has_unpublished_password
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Daniel Heath
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2019-06-03 00:00:00.000000000 Z
+date: 2019-06-04 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler