pacer 1.0.2-java → 1.0.3-java

data/blog/2012-09-18-pacer-1.0.md

@@ -0,0 +1,63 @@
+What's new in Pacer 1.0
+===
+
+We're very excited about the latest version of Pacer. We've used Pacer
+0.9 for almost a year now and it's served us very well, but in that time
+we found a few pieces that we weren't happy with. We've addressed those
+issues and also made improvements in its architecture, consistency, and
+performance. So, here's what's new in Pacer 1.0:
+
+* The already comprehensive spec suite has been improved and also
+  runs almost 10x faster.
+
+* Graphs and elements are always wrapped. Previously, Pacer
+  patched the raw Java classes to add the methods it needs in addition
+  to allowing elements to be wrapped. By choosing to always use
+  wrapping, we can provide a cleaner, easier, and more consistent
+  interface.
+* It uses the current [Tinkerpop](http://tinkerpop.com/) 2.1 stack with the most current versions
+  of all of its supported databases including [Neo4j](http://neo4j.org),
+  [Dex](http://www.sparsity-technologies.com/dex.php),
+  [OrientDB](http://www.orientdb.org/index.htm), and
+  [TinkerGraph](https://github.com/tinkerpop/blueprints/wiki/TinkerGraph). This is great news because
+  the Tinkerpop stack and the graphs that it supports are constantly
+  improving.
+* The pacer-[graphdb] gems are no longer required but are all updated as a
+  convenient way to get the dependent jars without the pain of using maven.
+* Pacer can fully support all of [Blueprints](https://github.com/tinkerpop/blueprints/wiki)'
+  WrapperGraph implementations. If you need the functionality behind one of the wrappers or want to
+  create your own, simply initialize a new PacerGraph with the wrapped
+  graph.
+* Pacer is now able to fully support all [Gremlin](https://github.com/tinkerpop/gremlin/wiki) pipes including those
+  that require PipeFunctions, making all features in Gremlin available
+  to Pacer in addition to Pacer's own unique features.
+* Many internal details of the implementation have been improved to make
+  Pacer's source much easier to understand and extend. This provides a
+  solid base on which to develop sophisticated projects.
+
+
+Breaking changes and upgrade notes
+---
+
+Breaking changes have been kept to a minimum but there were a number of
+places that we weren't happy with the previous implementation and have
+addressed them.
+
+* Pacer now requires [JRuby](http://jruby.org/) 1.7. This latest version of JRuby handles Java
+  exceptions significantly more cleanly which eliminated a problem we
+  previously had to work around in numerous places. In addition it is
+  now in 1.9 mode by default, starts up faster, and conforms to the more
+  recent Ruby 1.9.3 which brings valuable improvements to the standard
+  Ruby libraries.
+
+* To find an index, the `#index_name` method has been renamed to `#index`.
+* All `#element_type` methods now return simply `:vertex`, `:edge`, `:mixed`
+  or `:object`.
+* The `#extensions` method sometimes returned a Set and sometimes an Array. Now it
+  always returns an Array.
+* Elements yielded to a block within a route no longer have the
+  BlockFilterElement extension added to them. In the rare occasion that
+  you need to use the `#back` or `#vars` methods within a block, they
+  can be easily accessed by referring to the route itself within the
+  block. To see an example, refer to the `as_spec.rb` file.
+

data/lib/pacer/core/graph/element_route.rb

@@ -75,9 +75,13 @@ module Pacer::Core::Graph
     def [](prop_or_subset)
       case prop_or_subset
       when String, Symbol
-        chain_route(:element_type => :object,
+        route = chain_route(:element_type => :object,
                     :pipe_class => Pacer::Pipes::PropertyPipe,
-                    :pipe_args => [prop_or_subset.to_s])
+                    :pipe_args => [prop_or_subset.to_s],
+                    :lookahead_replacement => proc { |r| r.back.property?(prop_or_subset) })
+        route.map(route_name: 'decode', remove_from_lookahead: true) do |v|
+          graph.decode_property(v)
+        end
       when Fixnum
         range(prop_or_subset, prop_or_subset)
       when Range
@@ -85,7 +89,7 @@ module Pacer::Core::Graph
       when Array
         if prop_or_subset.all? { |i| i.is_a? String or i.is_a? Symbol }
           map do |element|
-            prop_or_subset.collect { |i| element.getProperty(i.to_s) }
+            element[prop_or_subset]
           end
         end
       end
@@ -98,7 +102,7 @@ module Pacer::Core::Graph
     def property?(name)
       chain_route(:element_type => :object,
                   :pipe_class => Pacer::Pipes::PropertyPipe,
-                  :pipe_args => [name.to_s, true])
+                  :pipe_args => [name.to_s, false])
     end
 
     # Attach a route to the element id for each element emitted by the

data/lib/pacer/core/graph/vertices_route.rb

@@ -174,7 +174,7 @@ module Pacer::Core::Graph
       has_props = !props.empty?
       edge_ids = []
       counter = 0
-      graph.transaction do |commit, rollback|
+      graph.transaction(nesting: true) do |commit, rollback|
         v.each do |from_v|
           to_vertices.each do |to_v|
             counter += 1

data/lib/pacer/core/route.rb

@@ -31,6 +31,22 @@ module Pacer
       # @todo move this to graph routes.
       attr_writer :graph
 
+      # If this piece of the route is useless in a lookahead, set this to true
+      # and when it is at the tail of a lookahead, it will be removed
+      # automatically. (for instance the property decoder, or wrap/unwrap)
+      attr_accessor :remove_from_lookahead
+
+      # If a route's function won't do the expected thing in a lookahead, set a
+      # proc here that will correct the route. For instance:
+      #
+      #   g.v.lookahead { |v| v[:prop] }
+      #
+      # gets transformed to:
+      #
+      #   g.v.lookahead { |v| v.property?(:prop) }
+      #
+      attr_accessor :lookahead_replacement
+
       # Return which graph this route operates on.
       #
       # @todo move this to graph routes.
@@ -46,6 +62,10 @@ module Pacer
         graph.equals g
       end
 
+      def chain_route(args_hash)
+        Pacer::RouteBuilder.current.chain self, args_hash
+      end
+
       # The arguments passed to the pipe constructor.
       #
       # @overload pipe_args

data/lib/pacer/exceptions.rb

@@ -7,6 +7,7 @@ module Pacer
     class LogicError < Error; end
       class ClientError < LogicError; end
         class TransactionConcludedError < ClientError; end
+        class NestedTransactionError < ClientError; end
         class NestedTransactionRollback < ClientError; end
         class NestedMockTransactionRollback < NestedTransactionRollback; end
         class MockTransactionRollback < ClientError; end

data/lib/pacer/filter/future_filter.rb

@@ -44,6 +44,8 @@ module Pacer
           block, negate = @future_filter
           @future_filter = nil
           route = block.call(Pacer::Route.empty(self))
+          route = route.back while route.remove_from_lookahead
+          route = route.lookahead_replacement.call(route) if route.lookahead_replacement
           if min or max
             route = route.has_count_route(:min => min, :max => max).is(true)
           end

data/lib/pacer/graph/graph_transactions_mixin.rb

@@ -1,14 +1,9 @@
 module Pacer
   import com.tinkerpop.blueprints.TransactionalGraph
 
-  # Add features to transactions to allow them to be more rubyish and
-  # more convenient to use.
-  #
-  # TODO: the method names in this module need to be cleaned up.
-  # TODO: some methods may be able to be eliminated.
   module GraphTransactionsMixin
     def in_transaction?
-      threadlocal_graph_info[:tx_depth] > 0
+      threadlocal_graph_info.fetch(:tx_depth, 0) > 0
     end
 
     # Basic usage:
@@ -34,8 +29,12 @@ module Pacer
     # Also considering a 3rd callback that could be used to get info about the
     # current transaction stack like depth, number of commits/rollbacks, possibly the number of
     # mutations it wraps and even some event registration stuff could be made available.
-    def transaction
-      commit, rollback = start_transaction!
+    #
+    # opts:
+    #   nesting: true  -- allow mock nested transactions
+    #   nesting: false -- (default) raise an exception instead of starting a nested transaction
+    def transaction(opts = {})
+      commit, rollback = start_transaction! opts
       begin
         r = yield commit, rollback
         commit.call
@@ -55,15 +54,17 @@ module Pacer
       graphs[blueprints_graph.object_id] ||= {}
     end
 
-    def start_transaction!
+    def start_transaction!(opts)
       tgi = threadlocal_graph_info
       tx_depth = tgi[:tx_depth] ||= 0
       tgi[:tx_depth] += 1
       if blueprints_graph.is_a? TransactionalGraph
         if tx_depth == 0
           base_tx_finalizers
-        else
+        elsif opts[:nesting] == true
           nested_tx_finalizers
+        else
+          fail NestedTransactionError, "To use nested transactions, use nesting: true"
         end
       else
         if tx_depth == 0
@@ -75,14 +76,14 @@ module Pacer
     end
 
     def finish_transaction!
-      threadlocal_graph_info[:tx_depth] -= 1
+      threadlocal_graph_info[:tx_depth] -= 1 rescue nil
     end
 
     def base_tx_finalizers
       tx_id = threadlocal_graph_info[:tx_id] = rand
       commit = -> do
         if tx_id != threadlocal_graph_info[:tx_id]
-          fail InternalError
+          fail InternalError, 'Can not commit transaction outside its original block'
         end
         puts "transaction committed" if Pacer.verbose == :very
         blueprints_graph.stopTransaction TransactionalGraph::Conclusion::SUCCESS

data/lib/pacer/graph/hash_index.rb

@@ -0,0 +1,29 @@
+module Pacer::Graph
+  class HashIndex
+    attr_reader :name, :type
+
+    def initialize(element_type, name)
+      @type = type
+      @name = name
+      @data = Hash.new do |h, k|
+        h[k] = Hash.new { |h, k| h[k] = Set[] }
+      end
+    end
+
+    def get(key, value)
+      Pacer::Pipes::EnumerablePipe.new data[key][value]
+    end
+
+    def put(key, value, element)
+      data[key][value] << element
+    end
+
+    def count(key, value)
+      data[key][value].count
+    end
+
+    private
+
+    attr_reader :data
+  end
+end

data/lib/pacer/graph/pacer_graph.rb

@@ -188,10 +188,6 @@ module Pacer
     end
 
     module Encoding
-      def sanitize_properties(props)
-        encoder.sanitize_properties props
-      end
-
       def encode_property(value)
         encoder.encode_property value
       end
@@ -270,7 +266,7 @@ module Pacer
       # @option opts [true] :create create the index if it doesn't exist
       # @return [Pacer::IndexMixin]
       def index(name, type = nil, opts = {})
-        return unless features.supportsIndices
+        return temp_index(name, type, opts) unless features.supportsIndices
         name = name.to_s
         if type
           type = index_class element_type type
@@ -285,7 +281,7 @@ module Pacer
       end
 
       def drop_index(idx)
-        return unless features.supportsIndices
+        return drop_temp_index(idx) unless features.supportsIndices
         if idx.is_a? String or idx.is_a? Symbol
           blueprints_graph.dropIndex idx
         else
@@ -293,6 +289,27 @@ module Pacer
         end
       end
 
+      def temp_index(name, type = nil, opts = {})
+        @temp_indices ||= {}
+        idx = @temp_indices[name]
+        unless idx
+          if name and type and opts[:create]
+            idx = @temp_indices[name] = Pacer::Graph::HashIndex.new type, name
+          elsif opts[:create]
+            idx = Pacer::Graph::HashIndex.new type, nil
+          end
+        end
+        Pacer::Wrappers::IndexWrapper.new self, idx, idx.type if idx
+      end
+
+      def drop_temp_index(idx)
+        @temp_indices.delete idx.name
+      end
+
+      def drop_temp_indices
+        @temp_indices = {}
+      end
+
       # Return an object that can be compared to the return value of
       # Index#index_class.
       def index_class(et)

data/lib/pacer/graph/simple_encoder.rb

@@ -1,20 +1,25 @@
 module Pacer
   class SimpleEncoder
-    def self.sanitize_properties(props)
-      props
-    end
+    JBoolean = java.lang.Boolean
+    JFalse = false.to_java
 
     def self.encode_property(value)
       if value.is_a? String
         value = value.strip
         value unless value == ''
+      elsif false == value
+        JFalse
       else
         value
       end
     end
 
     def self.decode_property(value)
-      value
+      if value.is_a? JBoolean and value == JFalse
+        false
+      else
+        value
+      end
     end
   end
 end

data/lib/pacer/graph/yaml_encoder.rb

@@ -5,13 +5,6 @@ module Pacer
   # Neo4j could and for everything else it uses (slow (but easy))
   # human-readable YAML encoding.
   class YamlEncoder
-    def self.sanitize_properties(props)
-      pairs = props.map do |name, value|
-        [name, encode_property(value)]
-      end
-      Hash[pairs]
-    end
-
     def self.encode_property(value)
       case value
       when nil
@@ -22,12 +15,12 @@ module Pacer
         value
       when Numeric
         if value.is_a? Bignum
-          value.to_yaml
+          dump value
         else
-          value
+          value.to_java
         end
       when true, false
-        value
+        value.to_java
       when Array
         if value.length == 0
           value_type = Fixnum
@@ -51,33 +44,27 @@ module Pacer
         when String
           value.to_java :string
         else
-          value.to_yaml
+          dump value
         end
       else
-        value.to_yaml
+        dump value
       end
     end
 
-    if 'x'.to_yaml[0, 5] == '%YAML'
-      def self.decode_property(value)
-        if value.is_a? String and value[0, 5] == '%YAML'
-          YAML.load(value)
-        elsif value.is_a? ArrayJavaProxy
-          value.to_a
-        else
-          value
-        end
-      end
-    else
-      def self.decode_property(value)
-        if value.is_a? String and value[0, 3] == '---'
-          YAML.load(value)
-        elsif value.is_a? ArrayJavaProxy
-          value.to_a
-        else
-          value
-        end
+    def self.decode_property(value)
+      if value.is_a? String and value[0, 1] == ' '
+        YAML.load(value[1..-1])
+      elsif value.is_a? ArrayJavaProxy
+        value.to_a
+      else
+        value
       end
     end
+
+    private
+
+    def self.dump(value)
+      " #{ YAML.dump value }"
+    end
   end
 end

data/lib/pacer/loader.rb

@@ -0,0 +1,93 @@
+module Pacer
+  import com.tinkerpop.blueprints.Graph
+  import com.tinkerpop.blueprints.Element
+  import com.tinkerpop.blueprints.Vertex
+  import com.tinkerpop.blueprints.Edge
+
+  module Core       end
+  module Routes     end
+  module Wrappers   end
+  module Support    end
+  module Visitors   end
+  module Filter     end
+  module Transform  end
+  module SideEffect end
+end
+
+
+require 'pacer/exceptions'
+require 'pacer/pipes'
+
+require 'pacer/core/route'
+require 'pacer/core/graph'
+require 'pacer/core/side_effect'
+
+require 'pacer/graph/graph_transactions_mixin'
+require 'pacer/graph/pacer_graph'
+require 'pacer/graph/simple_encoder'
+require 'pacer/graph/yaml_encoder'
+require 'pacer/graph/graph_ml'
+require 'pacer/graph/hash_index'
+
+require 'pacer/route/mixin/bulk_operations'
+require 'pacer/route/mixin/route_operations'
+
+require 'forwardable'
+require 'pacer/wrappers/element_wrapper'
+require 'pacer/wrappers/vertex_wrapper'
+require 'pacer/wrappers/edge_wrapper'
+require 'pacer/wrappers/index_wrapper'
+require 'pacer/wrappers/wrapper_selector'
+require 'pacer/wrappers/wrapping_pipe_function'
+
+require 'pacer/route_builder'
+require 'pacer/function_resolver'
+require 'pacer/route'
+
+require 'pacer/blueprints/tg'
+require 'pacer/blueprints/ruby_graph'
+require 'pacer/blueprints/multi_graph'
+
+require 'pacer/support/array_list'
+require 'pacer/support/enumerable'
+require 'pacer/support/proc'
+require 'pacer/support/hash'
+require 'pacer/support/native_exception'
+require 'pacer/support/nil_class'
+
+require 'pacer/utils'
+
+require 'pacer/visitors/visits_section'
+require 'pacer/visitors/section'
+
+require 'pacer/filter/collection_filter'
+require 'pacer/filter/empty_filter'
+require 'pacer/filter/future_filter'
+require 'pacer/filter/property_filter'
+require 'pacer/filter/range_filter'
+require 'pacer/filter/uniq_filter'
+require 'pacer/filter/index_filter'
+require 'pacer/filter/loop_filter'
+require 'pacer/filter/block_filter'
+require 'pacer/filter/object_filter'
+require 'pacer/filter/where_filter'
+require 'pacer/filter/random_filter'
+
+require 'pacer/transform/cap'
+require 'pacer/transform/stream_sort'
+require 'pacer/transform/stream_uniq'
+require 'pacer/transform/gather'
+require 'pacer/transform/map'
+require 'pacer/transform/process'
+require 'pacer/transform/join'
+require 'pacer/transform/path'
+require 'pacer/transform/scatter'
+require 'pacer/transform/has_count_cap'
+require 'pacer/transform/sort_section'
+
+require 'pacer/side_effect/aggregate'
+require 'pacer/side_effect/as'
+require 'pacer/side_effect/group_count'
+require 'pacer/side_effect/is_unique'
+require 'pacer/side_effect/counted'
+require 'pacer/side_effect/visitor'

data/lib/pacer/route/mixin/bulk_operations.rb

@@ -24,7 +24,7 @@ module Pacer::Routes
           target_graph.in_bulk_job = true
           size ||= target_graph.bulk_job_size
           counter = 0
-          target_graph.transaction do |commit, rollback|
+          target_graph.transaction(nesting: true) do |commit, rollback|
             each_slice(size) do |slice|
               print counter if Pacer.verbose?
               counter += size

data/lib/pacer/route.rb

@@ -1,155 +1,4 @@
-[Pacer::Core::Route, Pacer::Wrappers::ElementWrapper].each do |klass|
-  klass.class_eval %{
-    def chain_route(args_hash)
-      Pacer::RouteBuilder.current.chain self, args_hash
-    end
-  }
-end
-
 module Pacer
-
-  class RouteBuilder
-    class << self
-      attr_writer :current
-
-      def current
-        @current ||= RouteBuilder.new
-      end
-    end
-
-    attr_reader :types
-
-    def initialize
-      @types = Hash.new do |h, type_def|
-        h[type_def] = Class.new(Route) do
-          type_def.each do |mods|
-            mods.each do |mod|
-              include mod
-            end
-          end
-        end
-      end
-    end
-
-    def chain(source, args)
-      types[type_def(source, args)].new source, configuration(source, args), arguments(source, args)
-    end
-
-    protected
-
-    def source_value(source, name)
-      source.send name if source.respond_to? name
-    end
-
-    def type_def(source, args)
-      [
-        type_modules(source, args),
-        function_modules(source, args),
-        other_modules(source, args),
-        extension_modules(source, args)
-      ]
-    end
-
-    def configuration(source, args)
-      {
-        element_type: element_type(source, args),
-        graph: graph(source, args),
-        extensions: extensions(source, args),
-        wrapper: wrapper(source, args),
-        function: function_modules(source, args).first
-      }
-    end
-
-    def arguments(source, args)
-      args.reject do |key, val|
-        Set[:element_type, :wrapper, :extensions, :modules, :graph, :back, :filter, :side_effect, :transform, :visitor].include? key
-      end
-    end
-
-    def graph(source, args)
-      args[:graph] || source_value(source, :graph)
-    end
-
-    def element_type(source, args)
-      et = args[:element_type] || source_value(source, :element_type)
-      if not et
-        fail ClientError, "No element_type specified or inferred"
-      end
-      if not graph(source, args) and (et == :vertex or et == :edge or et == :mixed)
-        fail ClientError, "Element type #{ et.inspect } specified, but no graph specified."
-      end
-      et
-    end
-
-    def type_modules(source, args)
-      case element_type source, args
-      when :vertex
-        [Pacer::Core::Graph::ElementRoute, Pacer::Core::Graph::VerticesRoute]
-      when :edge
-        [Pacer::Core::Graph::ElementRoute, Pacer::Core::Graph::EdgesRoute]
-      when :mixed
-        [Pacer::Core::Graph::ElementRoute, Pacer::Core::Graph::MixedRoute]
-      else
-        []
-      end
-    end
-
-    def other_modules(source, args)
-      [*args[:modules]]
-    end
-
-    def type_from_source?(source, args)
-      element_type(source, args) == source_value(source, :element_type)
-    end
-
-    def wrapper(source, args)
-      args.fetch(:wrapper) do
-        source_value(source, :wrapper) if type_from_source? source, args
-      end
-    end
-
-    def extensions(source, args)
-      exts = args.fetch(:extensions) do
-        source_value(source, :extensions) if type_from_source? source, args
-      end
-      if exts.is_a? Set
-        exts.to_a
-      elsif exts.is_a? Module
-        [exts]
-      elsif exts.is_a? Array
-        exts.uniq
-      else
-        []
-      end
-    end
-
-    def all_extensions(source, args)
-      w = wrapper source, args
-      exts = extensions source, args
-      if w and exts
-        (w.extensions + exts).uniq
-      elsif w
-        w.extensions
-      elsif exts
-        exts
-      else
-        []
-      end
-    end
-
-    def extension_modules(source, args)
-      all_extensions(source, args).uniq.select do |mod|
-        mod.respond_to?(:const_defined?) and mod.const_defined? :Route
-      end.map { |mod| mod::Route }
-    end
-
-
-    def function_modules(source, args)
-      FunctionResolver.function(args).compact
-    end
-  end
-
-
   # The base class for almost everything in Pacer. Every route is an
   # instance of this class with a variety of modules mixed into the
   # instance at runtime.