surus 0.4.2 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c9704e8ae62f2951b6dafe408bd6136fc785f3a1
-  data.tar.gz: 38471bda203953316a99e095dac5e0dd93751f67
+  metadata.gz: 0a513dd97a43e25dbe91de60dee39323f42b410a
+  data.tar.gz: 092f7b1ea714785b14f59f68088f33101ae57244
 SHA512:
-  metadata.gz: 0d09100c14636b85460f7786ebf97393ffa730be521e7558c2d95075e0dab8093ab71acc11a575e111de5de5311fccf330136564c5fc83e8d0c8a2bb5fe08c87
-  data.tar.gz: 14e64a0de18c968df746f5a93eac0696ef4bbdee12a2f66f8fcb1be97d8c1c2fe4424399cf7defb90fbc4827d7395f7faf95cfbd9cb3083a7d5dfb32de454a14
+  metadata.gz: 111bcea66d6e02e09a8b7314e3768399dd55422ecb403305461bdd4d4f16fcc62f97181d41e6cbf7551fdd4772201bbcdc9d7f5b9a77c2f3ce0482efe0a64d3c
+  data.tar.gz: e4e759ac1c896166afae2e26033667b30f88f578ba1a9dc3c3caef6374252b3a19cc1491d397dd1dc11ea383e45d15246e779f97eacee2219124eed47ff52473

data/README.md

@@ -3,11 +3,11 @@ Surus
 
 # Description
 
-Surus accelerates ActiveRecord with PostgreSQL specific types and
-functionality. It enables indexed searching of serialized arrays and hashes.
+Surus adds PostgreSQL specific functionality to ActiveRecord. It adds
+helper methods for searching PostgreSQL arrays and hstores.
 It also can control PostgreSQL synchronous commit behavior. By relaxing
 PostgreSQL's durability guarantee, transaction commit rate can be increased by
-50% or more. It also directly generate JSON in PostgreSQL which can be
+50% or more. It can also directly generate JSON in PostgreSQL which can be
 substantially faster than converting ActiveRecord objects to JSON.
 
 # Installation
@@ -18,33 +18,13 @@ Or add to your Gemfile.
 
     gem 'surus'
 
-# Rails 4 Support
+## Rails 3
 
-This version of Surus is compatible with Rails 3.1.x and Rails 3.2.x. It is not
-compatible with Rails 4. Checkout the rails4 branch if you are using Rails 4.
+This version of Surus only works on Rails 4. Use the 0.4 line for Rails 3
 
-# Hstore
-
-Hashes can be serialized to an hstore column. hstore is a PostgreSQL key/value
-type that can be indexed for fast searching.
-
-    class User < ActiveRecord::Base
-      serialize :properties, Surus::Hstore::Serializer.new
-    end
-
-    User.create :properties => { :favorite_color => "green", :results_per_page => 20 }
-    User.create :properties => { :favorite_colors => ["green", "blue", "red"] }
-
-Even though the underlying hstore can only use strings for keys and values
-(and NULL for values) Surus can successfully maintain type for integers,
-floats, bigdecimals, dates, and any value that YAML can serialize. It does
-this by storing an extra key value pair (or two) to maintain type information.
+    gem 'surus', '~> 0.4.2'
 
-Because it falls back to YAML serialization for complex types, this means that
-nested data structures can be serialized to an hstore. In other words, any
-hash that can be serialized with the normal Rails YAML serialization can be
-serialized with Surus. But you can get the benefits of PostgreSQL indexing
-on the top level keys and values for free.
+# Hstore
 
 Hstores can be searched with helper scopes.
 
@@ -58,33 +38,18 @@ Hstore is a PostgreSQL extension. You can generate a migration to install it.
     rails g surus:hstore:install
     rake db:migrate
 
-You can then use the hstore type in migrations.
-
-    class AddPropertiesToUsers < ActiveRecord::Migration
-      def change
-        add_column :users, :properties, :hstore
-      end
-    end
+Even though the underlying hstore can only use strings for keys and values
+(and NULL for values) Surus can successfully maintain type for integers,
+floats, bigdecimals, dates, and any value that YAML can serialize. It does
+this by storing an extra key value pair (or two) to maintain type information.
 
-Read more in the [PostgreSQL hstore documentation](http://www.postgresql.org/docs/9.1/static/hstore.html).
+Because it falls back to YAML serialization for complex types, this means that
+nested data structures can be serialized to an hstore. In other words, any
+hash that can be serialized with the normal Rails YAML serialization can be
+serialized with Surus.
 
 # Array
 
-Ruby arrays can be serialized to PostgreSQL arrays. Surus includes support
-for text, integer, float, and decimal arrays.
-
-    class User < ActiveRecord::Base
-
-      serialize :favorite_integers, Surus::Array::IntegerSerializer.new
-      serialize :favorite_floats, Surus::Array::FloatSerializer.new
-      serialize :favorite_decimals, Surus::Array::DecimalSerializer.new
-    end
-
-    User.create :permissions => %w{ read_notes write_notes, manage_topics },
-      :favorite_integers => [1, 2, 3],
-      :favorite_floats => [1.3, 2.2, 3.1],
-      :favorite_decimals => [BigDecimal("3.14"), BigDecimal("4.23"]
-
 Arrays can be searched with helper scopes.
 
     User.array_has(:permissions, "admin")
@@ -131,40 +96,8 @@ the Rails `to_json` interface.
     User.all_json(columns: [:id, :name, :email], include: {posts: {columns: [:id, :subject]}})
     Post.all_json(include: [:forum, :post])
 
-You can use the json type in migrations.
-
-    class AddPropertiesToUsers < ActiveRecord::Migration
-      def change
-        add_column :users, :properties, :json
-      end
-    end
-
-
 # Benchmarks
 
-Using PostgreSQL's hstore enables searches to be done quickly in the database.
-
-    jack@moya:~/work/surus$ ruby -I lib -I bench bench/hstore_find.rb
-    Skipping EAV test. Use -e to enable (VERY SLOW!)
-    Skipping YAML test. Use -y to enable (VERY SLOW!)
-    Creating Surus test data... Done.
-
-    2000 records with 5 string key/value pairs
-    Finding all by inclusion of a key 200 times
-                   user     system      total        real
-    Surus      0.120000   0.030000   0.150000 (  0.356240)
-
-Arrays are also searchable.
-
-    jack@moya:~/work/surus$ ruby -I lib -I bench bench/array_find.rb
-    Skipping YAML test. Use -y to enable (VERY SLOW!)
-    Creating Surus test data... Done.
-
-    2000 records with 10 element arrays
-    Finding all where array includes value 200 times
-                   user     system      total        real
-    Surus      0.120000   0.040000   0.160000 (  0.531735)
-
 JSON generation is with all_json and find_json is substantially faster than to_json.
 
     jack@hk-47~/dev/surus$ ruby -I lib -I bench bench/json_generation.rb

data/bench/benchmark_helper.rb

@@ -40,14 +40,6 @@ class EavDetailRecord < ActiveRecord::Base
   belongs_to :eav_master_record
 end
 
-class YamlArrayRecord < ActiveRecord::Base
-  serialize :names
-end
-
-class SurusTextArrayRecord < ActiveRecord::Base
-  serialize :names, Surus::Array::TextSerializer.new
-end
-
 class WideRecord < ActiveRecord::Base
 end
 
@@ -77,7 +69,6 @@ def clean_database
   SurusKeyValueRecord.delete_all
   EavDetailRecord.delete_all
   EavMasterRecord.delete_all
-  YamlArrayRecord.delete_all
   WideRecord.delete_all
   NarrowRecord.delete_all
   Post.destroy_all # destroy instead of delete so it removes join records in posts_tags

data/bench/database_structure.sql

@@ -38,24 +38,6 @@ CREATE UNIQUE INDEX ON eav_detail_records(eav_master_record_id, "key");
 CREATE INDEX ON eav_detail_records ("value");
 
 
-DROP TABLE IF EXISTS yaml_array_records;
-
-CREATE TABLE yaml_array_records(
-  id serial PRIMARY KEY,
-  names text
-);
-
-
-DROP TABLE IF EXISTS surus_text_array_records;
-
-CREATE TABLE surus_text_array_records(
-  id serial PRIMARY KEY,
-  names text[]
-);
-
-CREATE INDEX ON surus_text_array_records USING GIN (names);
-
-
 
 DROP TABLE IF EXISTS wide_records;
 

data/bench/hstore_find.rb

@@ -6,7 +6,7 @@ optparse = OptionParser.new do |opts|
   opts.on '-r NUM', '--records NUM', Integer, 'Number of records to create' do |n|
     options[:records] = n
   end
-  
+
   options[:pairs] = 5
   opts.on '-p NUM', '--pairs NUM', Integer, 'Number of key/value pairs' do |n|
     options[:pairs] = n
@@ -16,7 +16,7 @@ optparse = OptionParser.new do |opts|
   opts.on '-e', '--eav', 'Include EAV in benchmark (VERY SLOW!)' do
     options[:eav] = true
   end
-  
+
   options[:yaml] = false
   opts.on '-y', '--yaml', 'Include YAML in benchmark (VERY SLOW!)' do
     options[:yaml] = true
@@ -84,22 +84,22 @@ Benchmark.bm(8) do |x|
         EavMasterRecord
           .includes(:eav_detail_records)
           .where("EXISTS(SELECT 1 FROM eav_detail_records WHERE eav_master_records.id=eav_detail_records.eav_master_record_id AND key=?)", key_to_find)
-          .all
+          .to_a
       end
-    end  
+    end
   end
 
   x.report("Surus") do
     keys_to_find.each do |key_to_find|
-      SurusKeyValueRecord.hstore_has_key(:properties, key_to_find).all
+      SurusKeyValueRecord.hstore_has_key(:properties, key_to_find).to_a
     end
   end
 
   if yaml
     x.report("YAML") do
       keys_to_find.each do |key_to_find|
-        YamlKeyValueRecord.all.select { |r| r.properties.key?(key_to_find) }
+        YamlKeyValueRecord.to_a.select { |r| r.properties.key?(key_to_find) }
       end
-    end    
+    end
   end
 end

data/bench/json_generation.rb

@@ -55,7 +55,7 @@ Benchmark.bm(55) do |x|
 
   x.report("to_json:   50 records with 3 associations #{num_long_iterations} times") do
     num_long_iterations.times do
-      Post.includes(:author, :forum).all.to_json include: [:author, :forum, :tags]
+      Post.includes(:author, :forum).to_a.to_json include: [:author, :forum, :tags]
     end
   end
 end

data/lib/surus.rb

@@ -2,15 +2,9 @@ require 'active_record'
 require 'surus/version'
 require 'surus/hstore/serializer'
 require 'surus/hstore/scope'
-require 'surus/hstore/connection_adapters'
-require 'surus/array/text_serializer'
-require 'surus/array/integer_serializer'
-require 'surus/array/float_serializer'
-require 'surus/array/decimal_serializer'
 require 'surus/array/scope'
 require 'surus/synchronous_commit/connection'
 require 'surus/synchronous_commit/model'
-require 'surus/json/connection_adapters'
 require 'surus/json/query'
 require 'surus/json/row_query'
 require 'surus/json/array_agg_query'

data/lib/surus/array/scope.rb

@@ -24,7 +24,7 @@ module Surus
       private
       def array_cast(column_name)
         column = columns_hash[column_name.to_s]
-        "::#{column.sql_type}"
+        "::#{column.sql_type}[]"
       end
     end
   end

data/lib/surus/hstore/serializer.rb

@@ -1,71 +1,47 @@
 module Surus
   module Hstore
     class Serializer
-      KEY_VALUE_REGEX = %r{
-        "
-        ((?:[^"\\]|\\.)*)
-        "
-        =>
-        (
-          "
-          (?:[^"\\]|\\.)*
-          "
-          |(NULL)
-        )
-      }x
-
-      def load(string)
-        return unless string
-        stringified_hash = string.scan(KEY_VALUE_REGEX).each_with_object({}) do |key_value, hash|
-          key, value = key_value
-          key = unescape(key)
-          value = if value == "NULL"
-            nil
-          else
-            unescape(value[1..-2])
-          end
-          
-          hash[key] = value
-        end
+      def load(stringified_hash)
+        return unless stringified_hash
 
         key_types = stringified_hash.delete "__key_types"
         key_types = YAML.load key_types if key_types
         value_types = stringified_hash.delete "__value_types"
         value_types = YAML.load value_types if value_types
-        
+
         return stringified_hash unless key_types || value_types
-        
+
         stringified_hash.each_with_object({}) do |key_value, hash|
           string_key, string_value = key_value
-          
-          key = if key_types && key_types.key?(string_key) 
+
+          key = if key_types && key_types.key?(string_key)
             typecast(string_key, key_types[string_key])
           else
             string_key
           end
-          
+
           value = if value_types && value_types.key?(string_key)
             typecast(string_value, value_types[string_key])
           else
             string_value
           end
-          
+
           hash[key] = value
-        end    
+        end
       end
-      
+
       def dump(hash)
         return unless hash
-        
+
         key_types = {}
         value_types = {}
-        
+
         stringified_hash = hash.each_with_object({}) do |key_value, stringified_hash|
           key_string, key_type = stringify(key_value[0])
           value_string, value_type = stringify(key_value[1])
-          
+
           stringified_hash[key_string] = value_string
-          
+
           key_types[key_string] = key_type unless key_type == "String"
           value_types[key_string] = value_type unless value_type == "String"
         end
@@ -76,34 +52,34 @@ module Surus
         # the mess for us.
         stringified_hash["__key_types"] = YAML.dump(key_types) if key_types.present?
         stringified_hash["__value_types"] = YAML.dump(value_types) if value_types.present?
-        
+
         stringified_hash.map do |key, value|
           "#{format_key(key)}=>#{format_value(value)}"
         end.join(", ")
       end
-      
+
       def format_key(key)
         %Q("#{escape(key)}")
       end
-      
+
       def format_value(value)
         value ? %Q("#{escape(value)}") : "NULL"
       end
-      
+
       # Escape a value for use as a key or value in an hstore
       def escape(value)
         value
           .gsub('\\', '\\\\\\')
           .gsub('"', '\\"')
       end
-      
+
       # Unescape a value from a key or value in an hstore
       def unescape(value)
         value
           .gsub('\\\\', '\\')
           .gsub('\\"', '"')
       end
-      
+
       # Returns an array of value as a string and value type
       def stringify(value)
         if value.kind_of?(String)
@@ -126,9 +102,9 @@ module Surus
           [nil, "String"] # we don't actually stringify nil because format_value special cases nil
         else
           [YAML.dump(value), "YAML"]
-        end  
+        end
       end
-      
+
       def typecast(value, type)
         case type
         when "Symbol"
@@ -152,5 +128,5 @@ module Surus
         end
       end
     end
-  end  
+  end
 end

data/lib/surus/json/belongs_to_scope_builder.rb

@@ -2,11 +2,11 @@ module Surus
   module JSON
     class BelongsToScopeBuilder < AssociationScopeBuilder
       def scope
-        association_scope = association
+        s = association
           .klass
           .where("#{quote_column_name association.active_record_primary_key}=#{quote_column_name association.foreign_key}")
-        association_scope = association_scope.where(conditions) if conditions
-        association_scope
+        s = s.instance_eval(&association.scope) if association.scope
+        s
       end
     end
   end

data/lib/surus/json/has_and_belongs_to_many_scope_builder.rb

@@ -2,17 +2,16 @@ module Surus
   module JSON
     class HasAndBelongsToManyScopeBuilder < AssociationScopeBuilder
       def scope
-        association_scope = association
+        s = association
           .klass
           .joins("JOIN #{join_table} ON #{join_table}.#{association_foreign_key}=#{association_table}.#{association_primary_key}")
           .where("#{outside_class.quoted_table_name}.#{association_primary_key}=#{join_table}.#{foreign_key}")
-        association_scope = association_scope.where(conditions) if conditions
-        association_scope = association_scope.order(order) if order
-        association_scope
+        s = s.instance_eval(&association.scope) if association.scope
+        s
       end
 
       def join_table
-        connection.quote_table_name association.options[:join_table]
+        connection.quote_table_name association.join_table
       end
 
       def association_foreign_key