google_hash 0.2.1 → 0.3.0

data/README

@@ -2,7 +2,6 @@ Pleased to announce the initial release of the "google_hash" gem.
 
 Its goal. To boldly be faster than any hash hash before (cue star trek TNG theme).
 
-
 The goal is basically a better hash, either one that is faster or more space efficient than ruby's default.  To attempt this we wrap the google sparse and dense hashes [1].
 
 Speed results (populating 500000 integers):
@@ -13,50 +12,63 @@ Hash (Ruby default)
 0.359375 (populate)
 1.1875   (each)
 
-GoogleHashDense
+GoogleHashDenseIntToInt
 0.1875   (populate)
 0.078125 (each)
 
-GoogleHashSparse
+GoogleHashSparseIntToInt
 0.53125  (populate)
 0.078125 (each)
 
 Usage:
 
-a = GoogleHashDenseRubyToRuby.new # or GoogleHash.new
-b = GoogleHashDenseLongToRuby.new # :int => Ruby
-b = GoogleHashSparseLongToRuby.new # :int => Ruby
+a = GoogleHashDenseRubyToRuby.new
+b = GoogleHashDenseIntToRuby.new # :int => Ruby
+c = GoogleHashSparseIntToInt.new # :int => :int
+d = GoogleHashSparseIntToInt.new # :long => :long  (longs are 8 byte ints on 64-bit machines, so can store larger numbers)
 
 a[3] = 4
 b[4] = 'abc'
-b['abc'.hash] = 'some complex object' 
+b['abc'] = 'some complex object' 
+c[3] = 4 # all you can use are ints
 
 a.each{|k, v| ... }
 
 a.keys => Array
 a.values => Array
 
+For a complete list, see types.txt, but it's all the various combinations of Ruby, Int, and Long.
+
 To learn if sparse or dense is right for you, check their documentation: 
 
 http://google-sparsehash.googlecode.com/svn/trunk/doc/index.html
+Dense is faster, sparse uses less memory.
 
 Installation:
 
-gem install google_hash (if on doze, you'll need the devkit installed)
+gem install os # install a dependency
+gem install sane # install a dependency
+gem install google_hash (if on doze, you'll also first need the devkit installed)
 
-Both classes are currently more space efficient than a hash, because they store keys as "native" ints, so the keys no longer affect GC time, as well as only use 4 bytes instead of 20 (or 8 instead of 40, on 64 bit).  This should release stress on the GC, and both GoogleHashDense and GoogleHashSparse do this.  In terms of total memory usage GoogleHashDense uses more, and is more speedy, GoogleHashSparse uses less, and is much more memory efficient (2 bits per entry, or so I'm told).
+Note that if you use a type other than "Ruby" that you will be saving the values away as native values, so they will be using less space than a typical ruby object (which is 20 bytes), using 4 bytes instead of 20 (or instead of 40, in the case of 64 bit machines).  In addition to taking less memory, they are also stored separate from Ruby's heap, meaning that they should release stress on the GC, if you are spending much time in GC.
 
-If you have a desired use case let me know and I might well be able to code it up for you.
+If you have a desired use case that's not covered, let me know and I might well be able to code it up for you and add it.
 
 ex: currently it uses longs internally instead of ints--if you want ints or chars or what not, let me know.
 
-This is meant to be one more tool in the rubyists toolbelt when trying to optimize speed-wise, and plans to expand to more types, but at least with this release it has a #each method.
+if you want it to remember insertion order, I could do that, too, or native "store away" strings/bignums, whatever.
+
 
-Note that sparse maps can be saved to disk--ask if you want/need this functionality.
+This is meant to be one more tool in the rubyists toolbelt when trying to optimize speed-wise, and plans to expand to more types, but at least with this release it has a #each method.
 
 Enjoy.
+
 -r
 
 [1] http://code.google.com/p/google-sparsehash
 
-If you want to see the code/hack on it, run extconf.rb within the ext directory, to create the code it actually uses (from a template).
+If you want to see the code/hack on it, run extconf.rb within the ext directory, to create the code it actually uses (from a template).
+
+Related:
+
+judy http://groups.google.com/group/ruby-talk-google/browse_thread/thread/05ed587925526a7f/314375891d12b672?lnk=raot

data/VERSION

@@ -1 +1 @@
-0.2.1
+0.3.0

data/ext/extconf.rb

@@ -17,8 +17,8 @@ end
 $CFLAGS += " -I./local_installed/include "
 
 if RUBY_VERSION < '1.9'
- # appears to link using gcc on 1.8 [mingw at least]
- $LDFLAGS += " -lstdc++ "
+  # appears to link using gcc on 1.8 [mingw at least]
+  $LDFLAGS += " -lstdc++ "
 end
 
 # create our files...
@@ -35,56 +35,64 @@ else
 end
 
 ruby_key =  {:convert_keys_from_ruby => "", :convert_keys_to_ruby => "", :key_type => "VALUE", :unreachable_key => "current_instance"} # TODO NULL is false here?
-long_key = {:assert_key_type => 'T_FIXNUM', :convert_keys_from_ruby => "FIX2LONG", 
-  :convert_keys_to_ruby => "LONG2FIX", :key_type => "long", :unreachable_key => "1<<#{unreachable_int}"}  
-int_key = {:assert_key_type => 'T_FIXNUM', :convert_keys_from_ruby => "FIX2INT", 
-  :convert_keys_to_ruby => "INT2FIX", :key_type => "int", :unreachable_key => "1<<#{unreachable_int}"}
+long_key = {:assert_key_type => 'T_FIXNUM', :convert_keys_from_ruby => "FIX2LONG",
+:convert_keys_to_ruby => "LONG2FIX", :key_type => "long", :unreachable_key => "1<<#{unreachable_int}"}
+int_key = {:assert_key_type => 'T_FIXNUM', :convert_keys_from_ruby => "FIX2INT",
+:convert_keys_to_ruby => "INT2FIX", :key_type => "int", :unreachable_key => "1<<#{unreachable_int}"}
 
 
 ruby_value =  {:value_type => "VALUE"}
-long_value = {:assert_value_type => 'T_FIXNUM', :convert_values_from_ruby => "FIX2LONG", 
-  :convert_values_to_ruby => "LONG2FIX", :value_type => "long"}
+long_value = {:assert_value_type => 'T_FIXNUM', :convert_values_from_ruby => "FIX2LONG",
+:convert_values_to_ruby => "LONG2FIX", :value_type => "long"}
+int_value = {:assert_value_type => 'T_FIXNUM', :convert_values_from_ruby => "FIX2INT",
+:convert_values_to_ruby => "INT2FIX", :value_type => "int"}
 
 
-
-int_to_ruby = long_key.merge(ruby_value)
+long_to_ruby = long_key.merge(ruby_value)
 ruby_to_ruby = ruby_key.merge(ruby_value)
 
+long_to_long = long_key.merge(long_value)
+
 init_funcs = []
+require 'sane'
 
-for options in [int_to_ruby, ruby_to_ruby] do
- for type in ['sparse', 'dense'] do
-  raise unless options[:value_type] && options[:key_type]
-  
-  # create local variables so that the template can look cleaner
-  unreachable_key = options[:unreachable_key]
-  convert_keys_from_ruby = options[:convert_keys_from_ruby]
-  convert_keys_to_ruby = options[:convert_keys_to_ruby]
-  key_type = options[:key_type]
-  value_type = options[:value_type]
-  english_key_type = options[:key_type] == 'VALUE' ? 'ruby' : options[:key_type]
-  english_value_type = options[:value_type] == 'VALUE' ? 'ruby' : options[:value_type]
-  
-  assert_key_type = options[:assert_key_type]
-  convert_values_from_ruby = options[:convert_values_from_ruby]
-  convert_values_to_ruby = options[:convert_values_to_ruby]
-  
-  
-  if options[:key_type] == 'VALUE'
-    extra_hash_params =  ", hashrb, eqrb"  # use these methods for comparison
-    # ltodo is that the right hash -- is is type_t
+for key in [ruby_key, long_key, int_key] do
+  for value in [ruby_value, long_value, int_value] do
+    options = key.merge(value)
+    for type in ['sparse', 'dense'] do
+
+
+      # create local variables so that the template can look cleaner
+      unreachable_key = options[:unreachable_key]
+      convert_keys_from_ruby = options[:convert_keys_from_ruby]
+      convert_keys_to_ruby = options[:convert_keys_to_ruby]
+      key_type = options[:key_type]
+      value_type = options[:value_type]
+      english_key_type = options[:key_type] == 'VALUE' ? 'ruby' : options[:key_type]
+      english_value_type = options[:value_type] == 'VALUE' ? 'ruby' : options[:value_type]
+
+      assert_key_type = options[:assert_key_type]
+      convert_values_from_ruby = options[:convert_values_from_ruby]
+      convert_values_to_ruby = options[:convert_values_to_ruby]
+      assert_value_type = options[:assert_value_type]
+
+      if options[:key_type] == 'VALUE'
+        extra_hash_params =  ", hashrb, eqrb"
+      else
+        extra_hash_params = nil
+      end
+
+      template = ERB.new(File.read('template/google_hash.cpp.erb'))
+      descriptor = type + '_' + english_key_type + '_to_' + english_value_type;
+      File.write(descriptor + '.cpp', template.result(binding))
+      init_funcs << "init_" + descriptor
+    end
   end
-  
-  template = ERB.new(File.read('template/google_hash.cpp.erb'))  
-  descriptor = type + '_' + english_key_type + '_to_' + english_value_type;
-  File.write(descriptor + '.cpp', template.result(binding))
-  init_funcs << "init_" + descriptor
- end
 end
 
 # write our Init method
 
-template = ERB.new(File.read('template/main.cpp.erb'))  
+template = ERB.new(File.read('template/main.cpp.erb'))
 File.write 'main.cpp', template.result(binding)
 
 create_makefile('google_hash')

data/ext/template/google_hash.cpp.erb

@@ -15,6 +15,10 @@ extern "C" {
 
 // some helpers
 
+<% 
+ # not yet used...
+  if false   
+%>
 struct eqstr
 {
   bool operator()(const char* s1, const char* s2) const
@@ -23,12 +27,24 @@ struct eqstr
   }
 };
 
-static hash<const char*> H;
+struct eqint
+{
+  inline bool operator()(int s1, int s2) const
+  {
+    return s1 == s2;
+  }
+};
+
+<% end %>
 
-// hashing it is like hash<const char*>
 
 static ID id_eql, id_hash;
 
+<% if key_type == 'VALUE' %>
+static hash<const char*> H;
+
+// hashing it is like hash<const char*>
+
 struct eqrb
 {
   bool operator()(const VALUE s1, const VALUE s2) const
@@ -47,6 +63,10 @@ struct eqrb
   }
 };
 
+#ifndef RBIGNUM_DIGITS
+# define RBIGNUM_DIGITS(a) RBIGNUM(a)->digits
+#endif
+
 struct hashrb
 {
     size_t operator()(VALUE hash_me) const
@@ -68,14 +88,13 @@ struct hashrb
               case T_SYMBOL:
                // ltodo
                 return hash_me;
-            <% unless RUBY_VERSION < '1.9' %>
               case T_BIGNUM:
-	              return LONG2FIX(((long*)(RBIGNUM_DIGITS(hash_me)))[0]); // its first digit...I'm thinkin'
-            <% end %>
-	            case T_STRING:
+	            return LONG2FIX(((long*)(RBIGNUM_DIGITS(hash_me)))[0]); // its first digit...I'm thinkin'
+	             // not sure if this is faster or not...
+	             //case T_STRING:
 	             //return H(StringValueCStr(hash_me)); // populate/lookup 0.26 -> 0.23 [core is 0.16 somehow]
 	             // perhaps they cache?
-	             return H(RSTRING_PTR(hash_me)); // 0.23 -> -.22
+	             //return H(RSTRING_PTR(hash_me)); // 0.23 -> -.22
 	          }        
            
             VALUE hval = rb_funcall(hash_me, id_hash, 0);
@@ -97,16 +116,9 @@ struct hashrb
       }
 };
 
-
+<% end %>
 
 /* we end up not needing this...or at least not using it, I don't know if it would be faster than using the default or not
-struct eqint
-{
-  inline bool operator()(int s1, int s2) const
-  {
-    return s1 == s2;
-  }
-};
 */
 
 
@@ -168,9 +180,17 @@ rb_mri_hash_new(VALUE freshly_created) {
 static VALUE rb_ghash_set(VALUE cb, VALUE set_this, VALUE to_this) {
   <% if assert_key_type %>
     if(!(TYPE(set_this) == <%= assert_key_type %>)) {
-       rb_raise(rb_eTypeError, "not valid value #{assert_key_type}");
+       rb_raise(rb_eTypeError, "not valid key #{assert_key_type}");
     }
   <% end %>
+  
+  <% if assert_value_type %>
+    if(!(TYPE(to_this) == <%= assert_value_type %>)) {
+       rb_raise(rb_eTypeError, "not valid value #{assert_value_type}");
+    }
+  <% end %>
+
+  
   RCallback* cbs = GetCallbackStruct(cb);
   (*cbs->hash_map)[ <%= convert_keys_from_ruby %>(set_this)] = <%= convert_values_from_ruby %>(to_this);  
   return to_this; // ltodo test that it returns value...
@@ -180,7 +200,7 @@ static VALUE rb_ghash_get(VALUE cb, VALUE get_this) {
   // TODO optionally not assert [?]
   <% if assert_key_type %>
     if(!(TYPE(get_this) == <%= assert_key_type %>)) {
-       rb_raise(rb_eTypeError, "not valid value #{assert_key_type}");
+       rb_raise(rb_eTypeError, "not valid key #{assert_key_type}");
     }
   <% end %>
   RCallback* cbs = GetCallbackStruct(cb);

data/spec/spec.google_hash.rb

@@ -156,7 +156,7 @@ describe "google_hash" do
   
   it "should raise on errant values" do
     a = GoogleHashDenseLongToLong.new
-    proc { a[3] = 4}.should raise_error
+    proc { a[3] = 'abc'}.should raise_error
   end  
   
   it "should have an Enumerator for values, keys, on demand"

data/types.txt

@@ -0,0 +1,18 @@
+GoogleHashSparseRubyToRuby
+GoogleHashDenseRubyToRuby
+GoogleHashSparseRubyToLong
+GoogleHashDenseRubyToLong
+GoogleHashSparseRubyToInt
+GoogleHashDenseRubyToInt
+GoogleHashSparseLongToRuby
+GoogleHashDenseLongToRuby
+GoogleHashSparseLongToLong
+GoogleHashDenseLongToLong
+GoogleHashSparseLongToInt
+GoogleHashDenseLongToInt
+GoogleHashSparseIntToRuby
+GoogleHashDenseIntToRuby
+GoogleHashSparseIntToLong
+GoogleHashDenseIntToLong
+GoogleHashSparseIntToInt
+GoogleHashDenseIntToInt

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification 
 name: google_hash
 version: !ruby/object:Gem::Version 
-  version: 0.2.1
+  version: 0.3.0
 platform: ruby
 authors: 
 - rogerdpack
@@ -9,7 +9,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2009-12-21 00:00:00 -07:00
+date: 2009-12-26 00:00:00 -07:00
 default_executable: 
 dependencies: 
 - !ruby/object:Gem::Dependency 
@@ -119,6 +119,7 @@ files:
 - ext/template/main.cpp.erb
 - results.txt
 - spec/spec.google_hash.rb
+- types.txt
 has_rdoc: true
 homepage: http://github.com/rdp/ruby_google_hash
 licenses: []