dhash-vips 0.1.0.3 → 0.1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 2b90b4abcd617f0e285dd3399338b4114d921624
-  data.tar.gz: d47a8e51a8e7a2bf28b60d0259dc350f8b5e4fa0
+  metadata.gz: e462c5c41c5f2c5916223c083f22d16ae37a062b
+  data.tar.gz: 3261d15852334f30b59bd53cbfdd03e4a6db8758
 SHA512:
-  metadata.gz: 3b8b2ce65f3bdfd90d03e3a069e30c61be4a9bf1fc40f9d4beace8216165383d7f54572e996e101ab9fd6f056b260676a9ed465b8860bd1a6e17a4228b859909
-  data.tar.gz: 2ea40198fa4c6ed05183d610b2ca558a6ad67a386e69c127dbc0afac5723686025bd4644cc7297c300d9173b212788cc1ab4f242ad849ed7ee50dbc462dd5f08
+  metadata.gz: 80c90dd232d5c9ea5ccfd65d715c48bafb56ca94aac70e45ced117ad27e748ae81030010f4eb62f963b86f0854d1a9c6d3f4174de730cb8f763fd7dcfcf60f89
+  data.tar.gz: bbe1bee6c35949f68fe886512668d3ea5215daa2207f49fe3f2db51b5ea3c1829d2fb7c69a4d3b05a05afab6df7d243ba37d10d6de39aae1088b404189d43326

data/README.md

@@ -93,10 +93,6 @@ end
 
 * The above `15` and `25` constants are found empirically and just work enough well for 8-byte hashes. To find these thresholds we can run a rake task with hardcoded test cases (pairs of photos from the same photosession are not the same but are considered to be enough 'similar' for the purpose of this benchmark):
 
-      $ vips -v
-      vips-8.9.2-Tue Apr 21 09:26:11 UTC 2020
-      $ identify -version | head -1
-      Version: ImageMagick 6.9.11-24 Q16 x86_64 2020-07-18 https://imagemagick.org
       $ rake compare_quality
 
                             Dhash  Phamilie  DHashVips::DHash  DHashVips::IDHash  DHashVips::IDHash(4)
@@ -111,45 +107,9 @@ end
 
 * Methods were renamed from `#calculate` to `#fingerprint` and from `#hamming` to `#distance`.  
 * The `DHash#calculate` accepts `hash_size` optional parameter that is 8 by default. The `IDHash#fingerprint`'s optional parameter is called `power` and works in a bit different way: 3 means 8 and 4 means 16 -- other sizes are not supported because they don't seem to be useful (higher fingerprint resolution makes it vulnerable to image shifts and croppings, also `#distance` becomes much slower). Because IDHash's fingerprint is more complex than DHash's one it's not that straight forward to compare them so under the hood the `#distance` method have to check the size of fingerprint. If you are sure that fingerprints were made with power=3 then to skip the check you may use the `#distance3` method directly.  
-* The `#distance3` method will use Ruby C extension that is around 15 times faster than pure Ruby implementation -- native extension is currently hardcoded to be compiled only if it's macOS and rbenv Ruby 2.3.8 installed with `-k` flag but if you know how to make the gem gracefully fallback to native Ruby if `make` fails let me know or make a pull request. So the full benchmark:
+* The `#distance3` method will try to compile and use the Ruby C extension that is around 15 times faster than pure Ruby implementation -- native extension currently works on macOS rbenv Ruby from 2.3.8 to 2.4.9 installed with rbenv `-k` flag. So the full benchmark:
 
-  * Ruby 2.0.0
-
-        $ bundle exec rake compare_speed
-
-        load the image and calculate the fingerprint:
-                                  user     system      total        real
-        Dhash                12.400000   0.820000  13.220000 ( 13.329952)
-        DHashVips::DHash      1.330000   0.230000   1.560000 (  1.509826)
-        DHashVips::IDHash     1.060000   0.090000   1.150000 (  1.100332)
-        DHashVips::IDHash 4   1.030000   0.080000   1.110000 (  1.089148)
-
-        measure the distance (32*32*1000 times):
-                                            user     system      total        real
-        Dhash hamming                   3.140000   0.020000   3.160000 (  3.179392)
-        DHashVips::DHash hamming        3.040000   0.020000   3.060000 (  3.095190)
-        DHashVips::IDHash distance      8.170000   0.040000   8.210000 (  8.279950)
-        DHashVips::IDHash distance3     6.720000   0.030000   6.750000 (  6.790900)
-        DHashVips::IDHash distance 4   24.430000   0.130000  24.560000 ( 24.652625)
-
-  * Ruby 2.3.3 seems to have some bit arithmetics improvement compared to 2.0:
-
-        load the image and calculate the fingerprint:
-                                  user     system      total        real
-        Dhash                13.110000   0.950000  14.060000 ( 14.537057)
-        DHashVips::DHash      1.480000   0.310000   1.790000 (  1.808787)
-        DHashVips::IDHash     1.080000   0.100000   1.180000 (  1.156446)
-        DHashVips::IDHash 4   1.030000   0.090000   1.120000 (  1.076117)
-
-        measure the distance (32*32*1000 times):
-                                            user     system      total        real
-        Dhash hamming                   1.770000   0.010000   1.780000 (  1.815612)
-        DHashVips::DHash hamming        1.810000   0.010000   1.820000 (  1.875666)
-        DHashVips::IDHash distance      4.250000   0.020000   4.270000 (  4.350071)
-        DHashVips::IDHash distance3     3.430000   0.020000   3.450000 (  3.499031)
-        DHashVips::IDHash distance 4    8.210000   0.110000   8.320000 (  8.510735)
-
-  * Ruby 2.3.8p459 (2.4.6, 2.5.5 and 2.6.3 are all similar) with newer CPU (`sysctl -n machdep.cpu.brand_string #=> Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz`):
+  * Ruby 2.3.8p459:
 
         load the image and calculate the fingerprint:
                                   user     system      total        real
@@ -169,6 +129,19 @@ end
         DHashVips::IDHash distance3_c      0.210000   0.000000   0.210000 (  0.212864)
         DHashVips::IDHash distance 4       8.300000   0.120000   8.420000 (  8.499735)
 
+* There is now a benchmark that runs both speed and quality tests summing results to a single table where lower numbers are better:
+
+    ruby 2.3.8p459 (2018-10-18 revision 65136) [x86_64-darwin18]
+    vips-8.9.2-Tue Apr 21 09:26:11 UTC 2020
+    Version: ImageMagick 6.9.11-24 Q16 x86_64 2020-07-18
+    Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz
+    
+                Fingerprint  Compare  1/FMI^2
+      Phamilie        3.943    0.630    4.000
+         Dhash        4.969    1.097    1.375
+         DHash        0.434    1.089    1.556
+        IDHash        0.396    0.126    1.250
+
 * Also note that to make `#distance` able to assume the fingerprint resolution from the size of Integer that represents it, the change in its structure was needed (left half of bits was swapped with right one), so fingerprints between versions 0.0.4 and 0.0.5 became incompatible, but you probably can convert them manually. Otherwise if we put the version or structure information inside fingerprint it would became slow to (de)serialize and store.
 
 ## Development notes

data/Rakefile

@@ -1,7 +1,11 @@
-STDOUT.sync = true
-require "pp"
+begin
+  # for `rake release`
+  require "bundler/gem_tasks"
+rescue LoadError
+  puts "consider to `gem install bundler` to be able to `rake release`"
+end
 
-require "bundler/gem_tasks" # to push to rubygems
+require "pp"
 
 visualize_hash = lambda do |hash|
   puts hash.to_s(2).rjust(64, ?0).gsub(/(?<=.)/, '\0 ').scan(/.{16}/)
@@ -346,6 +350,10 @@ task :benchmark do
     fmi
   end
 
+  puts RUBY_DESCRIPTION
+  system "vips -v"
+  system "identify -version | /usr/bin/head -1"
+  system "sysctl -n machdep.cpu.brand_string"
   require "mll"
   puts MLL::grid.call %w{ \  Fingerprint Compare 1/FMI^2 }.zip(*[
     %w{ Phamilie Dhash DHash IDHash },

data/dhash-vips.gemspec

@@ -1,9 +1,9 @@
 Gem::Specification.new do |spec|
   spec.name          = "dhash-vips"
-  spec.version       = "0.1.0.3"
+  spec.version       = "0.1.1.0"
   spec.author        = "Victor Maslov"
   spec.email         = "nakilon@gmail.com"
-  spec.summary       = "dHash and IDHash powered by Vips"
+  spec.summary       = "dHash and IDHash perceptual image hashing/fingerprinting"
   spec.homepage      = "https://github.com/nakilon/dhash-vips"
   spec.license       = "MIT"
 
@@ -20,12 +20,13 @@ Gem::Specification.new do |spec|
 
   spec.add_development_dependency "rake"
   spec.add_development_dependency "minitest"
-  spec.add_development_dependency "get_process_mem"
 
   spec.add_development_dependency "rmagick", "~>2.16"
   spec.add_development_dependency "phamilie"
   spec.add_development_dependency "dhash"
 
+  spec.add_development_dependency "get_process_mem"
+
   spec.add_development_dependency "mll"
   spec.add_development_dependency "byebug"
 end

data/extconf.rb

@@ -1,13 +1,17 @@
 require "mkmf"
 
 File.write "Makefile", dummy_makefile(?.).join
-unless Gem::Platform.local.os == "darwin" && Gem::Version.new(RUBY_VERSION) == Gem::Version.new("2.3.8")
+unless Gem::Platform.local.os == "darwin" && ENV["RBENV_ROOT"] && ENV["RBENV_VERSION"]
 else
-  begin
-    # https://github.com/rbenv/rbenv/issues/1199
-    append_cppflags "-I#{Dir.glob("#{`rbenv root`.chomp}/sources/#{`rbenv version-name`.chomp}/*/").first}"
-  rescue
+  if Gem::Version.new(RUBY_VERSION) < Gem::Version.new("2.3.8") ||
+     Gem::Version.new(RUBY_VERSION) > Gem::Version.new("2.4.9")
   else
+    if Gem::Version.new(RUBY_VERSION) < Gem::Version.new("2.4")
+    else
+      append_cppflags "-DRUBY_EXPORT"
+    end
+    # https://github.com/rbenv/rbenv/issues/1199
+    append_cppflags "-I#{Dir.glob("#{ENV["RBENV_ROOT"]}/sources/#{ENV["RBENV_VERSION"]}/ruby-*/").first}"
     create_makefile "idhash"
     # Why this hack?
     # 1. Because I want to use Ruby and ./idhash.bundle for tests, not C.
@@ -21,10 +25,11 @@ else
 end
 
 # Cases to check:
-# 0. all is ok
-# `rm -rf idhash.o idhash.bundle pkg && bundle exec rake install`    # w/o ext  # ["/Users/nakilon/_/dhash-vips/lib/dhash-vips.rb", 32]
-# `rm -f idhash.o idhash.bundle Makefile && ruby extconf.rb && make` # with ext # ["/Users/nakilon/_/dhash-vips/lib/dhash-vips.rb", 40]
-# `bundle exec rake -rdhash-vips -e "p DHashVips::IDHash.method(:distance3).source_location"`
+# 0. everything is ok
+# `rm -rf idhash.o idhash.bundle idhash.so pkg && bundle exec rake install`
+# `bundle exec rake -rdhash-vips -e "p DHashVips::IDHash.method(:distance3).source_location"` # => # ["/Users/nakilon/_/dhash-vips/lib/dhash-vips.rb", 32]  # currently falsely says that gem install failed idk why
+# `rm -f idhash.o idhash.bundle idhash.so Makefile && ruby extconf.rb && make`
+# `bundle exec rake -rdhash-vips -e "p DHashVips::IDHash.method(:distance3).source_location"` # => # ["/Users/nakilon/_/dhash-vips/lib/dhash-vips.rb", 53]
 # 1. not macOS && rbenv
 # 2. fail during append_cppflags
 # 3. failed compilation

data/lib/dhash-vips-post-install-test.rb

@@ -20,8 +20,20 @@ ss = s.repeated_permutation(4).map do |s1, s2, s3, s4|
   end
 end
 fail unless :distance3 == DHashVips::IDHash.method(:distance3).original_name
+if Gem::Version.new(RUBY_VERSION) < Gem::Version.new("2.4")
+  check = lambda do |s1, s2|
+    s1.is_a?(Bignum) && s2.is_a?(Bignum)
+  end
+else
+  require "rbconfig/sizeof"
+  check = lambda do |s1, s2|
+    # https://github.com/ruby/ruby/commit/de2f7416d2deb4166d78638a41037cb550d64484#diff-16b196bc6bfe8fba63951420f843cfb4R10
+    _FIXNUM_MAX = (1 << (8 * RbConfig::SIZEOF["long"] - 2)) - 1
+    s1 > _FIXNUM_MAX && s2 > _FIXNUM_MAX
+  end
+end
 ss.product ss do |s1, s2|
-  next unless s1.is_a?(Bignum) && s2.is_a?(Bignum)
+  next unless check.call s1, s2
   unless f[s1, s2] == DHashVips::IDHash.distance3_c(s1, s2)
     p [s1, s2]
     p [s1.to_s(16).rjust(64,?0)].pack("H*").unpack("N*").map{ |_| _.to_s(2).rjust(32, ?0) }

data/lib/dhash-vips.rb

@@ -37,11 +37,25 @@ module DHashVips
     rescue LoadError
       alias_method :distance3, :distance3_ruby
     else
-      def distance3 a, b
-        if a.is_a?(Bignum) && b.is_a?(Bignum)
-          distance3_c a, b
-        else
-          distance3_ruby a, b
+      # we can't just do `defined? Bignum` because it's defined but deprecated (some internal CONST_DEPRECATED flag)
+      if Gem::Version.new(RUBY_VERSION) < Gem::Version.new("2.4")
+        def distance3 a, b
+          if a.is_a?(Bignum) && b.is_a?(Bignum)
+            distance3_c a, b
+          else
+            distance3_ruby a, b
+          end
+        end
+      else
+        # https://github.com/ruby/ruby/commit/de2f7416d2deb4166d78638a41037cb550d64484#diff-16b196bc6bfe8fba63951420f843cfb4R10
+        require "rbconfig/sizeof"
+        FIXNUM_MAX = (1 << (8 * RbConfig::SIZEOF["long"] - 2)) - 1
+        def distance3 a, b
+          if a > FIXNUM_MAX && b > FIXNUM_MAX
+            distance3_c a, b
+          else
+            distance3_ruby a, b
+          end
         end
       end
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dhash-vips
 version: !ruby/object:Gem::Version
-  version: 0.1.0.3
+  version: 0.1.1.0
 platform: ruby
 authors:
 - Victor Maslov
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2020-07-25 00:00:00.000000000 Z
+date: 2020-07-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ruby-vips
@@ -53,35 +53,35 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: get_process_mem
+  name: rmagick
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '2.16'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '2.16'
 - !ruby/object:Gem::Dependency
-  name: rmagick
+  name: phamilie
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '2.16'
+        version: '0'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: '2.16'
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: phamilie
+  name: dhash
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -95,7 +95,7 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: dhash
+  name: get_process_mem
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
@@ -174,9 +174,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.5.2.3
+rubygems_version: 2.6.14.4
 signing_key: 
 specification_version: 4
-summary: dHash and IDHash powered by Vips
+summary: dHash and IDHash perceptual image hashing/fingerprinting
 test_files:
 - test.rb