neoid 0.0.2 → 0.0.5.alpha

data/.travis.yml

@@ -0,0 +1,4 @@
+script: "bundle exec rake neo4j:install neo4j:start spec --trace"
+rvm:
+  - 1.9.2
+  - 1.9.3

data/CHANGELOG.md

@@ -1,3 +1,17 @@
+## v0.0.41
+
+* fixed really annoying bug caused by Rails design -- Rails doesn't call `after_destroy` when assigning many to many relationships to a model, like `user.movies = [m1, m2, m3]` or `user.update_attributes(params[:user])` where it contains `params[:user][:movie_ids]` list (say from checkboxes), but it DOES CALL after_create for the new relationships. the fix adds after_remove callback to the has_many relationships, ensuring neo4j is up to date with all changes, no matter how they were committed
+
+## v0.0.4
+
+* rewrote seacrch. one index for all types instead of one for type. please run neo_search_index on all of your models.
+  search in multiple types at once with `Neoid.search(types_array, term)
+
+## v0.0.3
+
+* new configuration syntax (backwards compatible)
+* full text search index
+
 ## v0.0.2
 
 * create node immediately after active record create

data/README.md

@@ -1,5 +1,9 @@
 # Neoid
 
+[![Build Status](https://secure.travis-ci.org/elado/neoid.png)](http://travis-ci.org/elado/neoid)
+
+
+
 Make your ActiveRecords stored and searchable on Neo4j graph database, in order to make fast graph queries that MySQL would crawl while doing them.
 
 Neoid to Neo4j is like Sunspot to Solr. You get the benefits of Neo4j speed while keeping your schema on your plain old RDBMS.
@@ -14,38 +18,40 @@ Neoid offers querying Neo4j for IDs of objects and then fetch them from your RDB
 
 Add to your Gemfile and run the `bundle` command to install it.
 
-	gem 'neoid'
-
+```ruby
+gem 'neoid', git: 'git://github.com/elado/neoid.git'
+```
 
 **Requires Ruby 1.9.2 or later.**
 
 ## Usage
 
-### First app configuration:
+### Rails app configuration:
 
 In an initializer, such as `config/initializers/01_neo4j.rb`:
 
-	ENV["NEO4J_URL"] ||= "http://localhost:7474"
-
-	uri = URI.parse(ENV["NEO4J_URL"])
+```ruby
+ENV["NEO4J_URL"] ||= "http://localhost:7474"
 
-    $neo = Neography::Rest.new(neo4j_uri.to_s)
+uri = URI.parse(ENV["NEO4J_URL"])
 
-    Neography::Config.tap do |c|
-      c.server = uri.host
-      c.port = uri.port
+$neo = Neography::Rest.new(uri.to_s)
 
-      if uri.user && uri.password
-        c.authentication = 'basic'
-        c.username = uri.user
-        c.password = uri.password
-      end
-    end
+Neography.configure do |c|
+  c.server = uri.host
+  c.port = uri.port
 
-    Neoid.db = $neo
+  if uri.user && uri.password
+    c.authentication = 'basic'
+    c.username = uri.user
+    c.password = uri.password
+  end
+end
 
+Neoid.db = $neo
+```
 
-`01_` in the file name is in order to get this file loaded first, before the models (files are loaded alphabetically).
+`01_` in the file name is in order to get this file loaded first, before the models (initializers are loaded alphabetically).
 
 If you have a better idea (I bet you do!) please let me know.
 
@@ -57,105 +63,137 @@ If you have a better idea (I bet you do!) please let me know.
 For nodes, first include the `Neoid::Node` module in your model:
 
 
-	class User < ActiveRecord::Base
-      include Neoid::Node
-	end
-
+```ruby
+class User < ActiveRecord::Base
+  include Neoid::Node
+end
+```
 
 This will help to create a corresponding node on Neo4j when a user is created, delete it when a user is destroyed, and update it if needed.
 
-Then, you can customize what fields will be saved on the node in Neo4j, by implementing `to_neo` method:
-
-
-	class User < ActiveRecord::Base
-      include Neoid::Node
-    
-	  def to_neo
-        {
-          slug: slug,
-          display_name: display_name
-	    }
-      end
-	end
+Then, you can customize what fields will be saved on the node in Neo4j, inside neoidable configuration:
+
+```ruby
+class User < ActiveRecord::Base
+  include Neoid::Node
+  
+  neoidable do |c|
+    c.field :slug
+    c.field :display_name
+    c.field :display_name_length do
+      self.display_name.length
+    end
+  end
+end
+```
 
-You can use `neo_properties_to_hash`, a helper method to make  things shorter:
 
+#### Relationships
 
-	def to_neo
-	  neo_properties_to_hash(%w(slug display_name))
-	end
+Let's assume that a `User` can `Like` `Movie`s:
 
 
-#### Relationships
+```ruby
+# user.rb
 
-Let's assume that a `User` can `Like` `Movie`s:
+class User < ActiveRecord::Base
+  include Neoid::Node
 
+  has_many :likes
+  has_many :movies, through: :likes
 
-	# user.rb
+  neoidable do |c|
+    c.field :slug
+    c.field :display_name
+  end
+end
 
-	class User < ActiveRecord::Base
-      include Neoid::Node
-    
-	  has_many :likes
-      has_many :movies, through: :likes
-    
-	  def to_neo
-        neo_properties_to_hash(%w(slug display_name))
-	  end
-	end
 
+# movie.rb
 
-	# movie.rb
+class Movie < ActiveRecord::Base
+  include Neoid::Node
 
-	class Movie < ActiveRecord::Base
-      include Neoid::Node
-    
-	  has_many :likes
-      has_many :users, through: :likes
-    
-	  def to_neo
-        neo_properties_to_hash(%w(slug name))
-	  end
-	end
+  has_many :likes
+  has_many :users, through: :likes
 
+  neoidable do |c|
+    c.field :slug
+    c.field :name
+  end
+end
 
-	# like.rb
 
-	class Like < ActiveRecord::Base
-	  belongs_to :user
-      belongs_to :movie
-	end
+# like.rb
 
+class Like < ActiveRecord::Base
+  belongs_to :user
+  belongs_to :movie
+end
+```
 
 
-Now let's make the `Like` model a Neoid, by including the `Neoid::Relationship` module, and define the relationship (start & end nodes and relationship type) options with `neoidable` method:
+Now let's make the `Like` model a Neoid, by including the `Neoid::Relationship` module, and define the relationship (start & end nodes and relationship type) options with `neoidable` config and `relationship` method:
 
 
-	class Like < ActiveRecord::Base
-	  belongs_to :user
-	  belongs_to :movie
+```ruby
+class Like < ActiveRecord::Base
+  belongs_to :user
+  belongs_to :movie
 
-	  include Neoid::Relationship
-	  neoidable start_node: :user, end_node: :movie, type: :likes
-	end
+  include Neoid::Relationship
 
+  neoidable do |c|
+    c.relationship start_node: :user, end_node: :movie, type: :likes
+  end
+end
+```
 
 Neoid adds `neo_node` and `neo_relationships` to nodes and relationships, respectively.
 
 So you could do:
 
-	user = User.create!(display_name: "elado")
-	user.movies << Movie.create("Memento")
-	user.movies << Movie.create("Inception")
+```ruby
+user = User.create!(display_name: "elado")
+user.movies << Movie.create("Memento")
+user.movies << Movie.create("Inception")
+
+user.neo_node                # => #<Neography::Node…>
+user.neo_node.display_name   # => "elado"
+
+rel = user.likes.first.neo_relationship
+rel.start_node  # user.neo_node
+rel.end_node    # user.movies.first.neo_node
+rel.rel_type    # 'likes'
+```
+
+## Index for Full-Text Search
+
+Using `search` block inside a `neoidable` block, you can store certain fields.
+
+```ruby
+# movie.rb
+
+class Movie < ActiveRecord::Base
+  include Neoid::Node
+
+  neoidable do |c|
+    c.field :slug
+    c.field :name
 
-	user.neo_node                # => #<Neography::Node…>
-	user.neo_node.display_name   # => "elado"
+    c.search do |s|
+      # full-text index fields
+      s.fulltext :name
+      s.fulltext :description
 
-	rel = user.likes.first.neo_relationship
-	rel.start_node  # user.neo_node
-	rel.end_node    # user.movies.first.neo_node
-	rel.rel_type    # 'likes'
+      # just index for exact matches
+      s.index :year
+    end
+  end
+end
+```
 
+Records will be automatically indexed when inserted or updated.
 
 ## Querying
 
@@ -165,51 +203,86 @@ You can query with all [Neography](https://github.com/maxdemarzi/neography)'s AP
 
 These examples query Neo4j using Gremlin for IDs of objects, and then fetches them from ActiveRecord with an `in` query.
 
-Of course, you can store using the `to_neo` all the data you need in Neo4j and avoid querying ActiveRecord.
+Of course, you can store using the `neoidable do |c| c.field ... end` all the data you need in Neo4j and avoid querying ActiveRecord.
 
 
 **Most popular categories**
 
-	gremlin_query = <<-GREMLIN
-	  m = [:]
-
-	  g.v(0)
-	    .out('movies_subref').out
-          .inE('likes')
-          .inV
-          .groupCount(m).iterate()
+```ruby
+gremlin_query = <<-GREMLIN
+  m = [:]
 
-	  m.sort{-it.value}.collect{it.key.ar_id}
-	GREMLIN
+  g.v(0)
+    .out('movies_subref').out
+      .inE('likes')
+      .inV
+      .groupCount(m).iterate()
 
-	movie_ids = Neoid.db.execute_script(gremlin_query)
+  m.sort{-it.value}.collect{it.key.ar_id}
+GREMLIN
 
-	Movie.where(id: movie_ids)
+movie_ids = Neoid.db.execute_script(gremlin_query)
 
+Movie.where(id: movie_ids)
+```
 
 Assuming we have another `Friendship` model which is a relationship with start/end nodes of `user` and type of `friends`,
 
 **Movies of user friends that the user doesn't have**
 
-	user = User.find(1)
+```ruby
+user = User.find(1)
+
+gremlin_query = <<-GREMLIN
+  u = g.idx('users_index')[[ar_id:user_id]].next()
+  movies = []
+
+  u
+    .out('likes').aggregate(movies).back(2)
+    .out('friends').out('likes')
+    .dedup
+    .except(movies).collect{it.ar_id}
+GREMLIN
+
+movie_ids = Neoid.db.execute_script(gremlin_query, user_id: user.id)
+
+Movie.where(id: movie_ids)
+```
+
+`.next()` is in order to get a vertex object which we can actually query on.
+
+
+### Full Text Search
+
+```ruby
+# will match all movies with full-text match for name/description. returns ActiveRecord instanced
+Movie.neo_search("*hello*").results
+
+# same as above but returns hashes with the values that were indexed on Neo4j
+Movie.search("*hello*").hits
 
-	gremlin_query = <<-GREMLIN
-	  u = g.idx('users_index')[[ar_id:'#{user.id}']][0].toList()[0]
-	  movies = []
+# search in multiple types
+Neoid.neo_search([Movie, User], "hello")
 
-	  u
-		.out('likes').aggregate(movies).back(2)
-	    .out('friends').out('likes')
-		.dedup
-		.except(movies).collect{it.ar_id}
-	GREMLIN
+# search with exact matches (pass a hash of field/value)
+Movie.neo_search(year: 2013).results
+```
 
-	movie_ids = Neoid.db.execute_script(gremlin_query)
+## Inserting records of existing app
 
-	Movie.where(id: movie_ids)
+If you have an existing database and just want to integrate Neoid, configure the `neoidable`s and run in a rake task or console
 
+```ruby
+[ Like.includes(:user).includes(:movie), OtherRelationshipModel ].each { |model| model.all.each(&:neo_update) }
 
-`[0].toList()[0]` is in order to get a pipeline object which we can actually query on.
+NodeModel.all.each(&:neo_update)
+```
+
+This will loop through all of your relationship records and generate the two edge nodes along with a relationship (eager loading for better performance).
+The second line is for nodes without relationships.
+
+For large data sets use pagination.
+Better interface for that in the future.
 
 
 ## Behind The Scenes
@@ -219,7 +292,7 @@ Whenever the `neo_node` on nodes or `neo_relationship` on relationships is calle
 ### For Nodes:
 
 1. Ensures there's a sub reference node (read [here](http://docs.neo4j.org/chunked/stable/tutorials-java-embedded-index.html) about sub reference nodes)
-2. Creates a node based on the ActiveRecord, with the `id` attribute and all other attributes from `to_neo`
+2. Creates a node based on the ActiveRecord, with the `id` attribute and all other attributes from `neoidable`'s field list
 3. Creates a relationship between the sub reference node and the newly created node
 4. Adds the ActiveRecord `id` to a node index, pointing to the Neo4j node id, for fast lookup in the future
 
@@ -235,35 +308,57 @@ Like Nodes, it uses an index (relationship index) to look up a relationship by A
 
 ## Testing
 
-Neoid tests run on a regular Neo4j database, on port 7574. You probably want to have it running on a different instance than your development one.
+In order to test your app or this gem, you need a running Neo4j database, dedicated to tests.
 
-In order to do that:
+I use port 7574 for this. To run another database locally:
 
-Copy the Neo4j folder to a different location,
+Copy the entire Neo4j database folder to a different location,
 
 **or**
 
-symlink `bin`, `lib`, `plugins`, `system`, copy `conf` and create an empty `data` folder.
+symlink `bin`, `lib`, `plugins`, `system`, copy `conf` to a single folder, and create an empty `data` folder.
 
 Then, edit `conf/neo4j-server.properties` and set the port (`org.neo4j.server.webserver.port`) from 7474 to 7574 and run the server with `bin/neo4j start`
 
+## Testing Your App with Neoid (RSpec)
+
+In `environments/test.rb`, add:
+
+```ruby
+ENV["NEO4J_URL"] = 'http://localhost:7574'
+```
+
+In your `spec_helper.rb`, add the following configurations:
 
-Download, install and configure [neo4j-clean-remote-db-addon](https://github.com/jexp/neo4j-clean-remote-db-addon). For the test database, leave the default `secret-key` key.
+```ruby
+config.before :all do
+  Neoid.clean_db(:yes_i_am_sure)
+end
 
+config.before :each do
+  Neoid.reset_cached_variables
+end
+```
+
+## Testing This Gem
+
+Just run `rake` from the gem folder.
 
 ## Contributing
 
 Please create a [new issue](https://github.com/elado/neoid/issues) if you run into any bugs. Contribute patches via pull requests. Write tests and make sure all tests pass.
 
 
+## Heroku Support
+
+Unfortunately, as for now, Neo4j add-on on Heroku doesn't support Gremlin. Therefore, this gem won't work on Heroku's add on. You should self-host a Neo4j instance on an EC2 or any other server.
+
 
 ## To Do
 
-* `after_update` to update a node/relationship.
-* Allow to disable sub reference nodes through options
-* Execute queries/scripts from model and not Neography (e.g. `Movie.neo_gremlin(gremlin_query)` with query that outputs IDs, returns a list of `Movie`s)
-* Rake task to index all nodes and relatiohsips in Neo4j
+[To Do](https://github.com/elado/neoid/blob/master/TODO.md)
+
 
 ---
 
-Developed by [@elado](http://twitter.com/elado)
+Developed by [@elado](http://twitter.com/elado)

data/Rakefile

@@ -1,5 +1,6 @@
 require "bundler/gem_tasks"
 require 'rspec/core/rake_task'
+require 'neography/tasks'
 
 RSpec::Core::RakeTask.new(:spec)
 

data/TODO.md

@@ -0,0 +1,6 @@
+# Neoid - To Do
+
+* Allow to disable sub reference nodes through options
+* Execute queries/scripts from model and not Neography (e.g. `Movie.neo_gremlin(gremlin_query)` with query that outputs IDs, returns a list of `Movie`s)
+* Rake task to index all nodes and relatiohsips in Neo4j
+* Test update node

data/lib/neoid/database_cleaner.rb

@@ -0,0 +1,12 @@
+module Neoid
+  class NeoDatabaseCleaner
+    def self.clean_db(start_node = Neoid.db.get_root)
+      Neoid.db.execute_script <<-GREMLIN
+        g.V.toList().each { if (it.id != 0) g.removeVertex(it) }
+        g.indices.each { g.dropIndex(it.indexName); }
+      GREMLIN
+
+      true
+    end
+  end
+end

data/lib/neoid/middleware.rb

@@ -0,0 +1,14 @@
+module Ndoid
+  class Middleware
+    def initialize(app)
+      @app = app
+    end
+
+    def call(env)
+      old, Thread.current[:neoid_enabled] = Thread.current[:neoid_enabled], true
+      @app.call(env)
+    ensure
+      Thread.current[:neoid_enabled] = old
+    end
+  end
+end

data/lib/neoid/model_additions.rb

@@ -1,14 +1,16 @@
 module Neoid
   module ModelAdditions
     module ClassMethods
-      def neoidable(options)
-        @config = Neoid::ModelConfig.new
-        yield(@config) if block_given?
-        @neoidable_options = options
+      attr_reader :neoid_config
+      attr_reader :neoid_options
+      
+      def neoid_config
+        @neoid_config ||= Neoid::ModelConfig.new(self)
       end
-    
-      def neoidable_options
-        @neoidable_options
+      
+      def neoidable(options = {})
+        yield(neoid_config) if block_given?
+        @neoid_options = options
       end
     
       def neo_index_name
@@ -18,13 +20,32 @@ module Neoid
   
     module InstanceMethods
       def to_neo
-        {}
+        if self.class.neoid_config.stored_fields
+          hash = self.class.neoid_config.stored_fields.inject({}) do |all, (field, block)|
+            all[field] = if block
+              instance_eval(&block)
+            else
+              self.send(field) rescue (raise "No field #{field} for #{self.class.name}")
+            end
+            
+            all
+          end
+
+          hash.reject { |k, v| v.nil? }
+        else
+          {}
+        end
+      end
+
+      def neo_resave
+        _reset_neo_representation
+        neo_update
       end
 
       protected
-      def neo_properties_to_hash(*property_list)
-        property_list.flatten.inject({}) { |all, property|
-          all[property] = self.attributes[property]
+      def neo_properties_to_hash(*attribute_list)
+        attribute_list.flatten.inject({}) { |all, property|
+          all[property] = self.send(property)
           all
         }
       end
@@ -36,11 +57,20 @@ module Neoid
           if results
             neo_load(results.first['self'])
           else
-            node = neo_create
-            node
+            neo_create
           end
         end
       end
+
+      def _reset_neo_representation
+        @_neo_representation = nil
+      end
+    end
+
+    def self.included(receiver)
+      receiver.extend         ClassMethods
+      receiver.send :include, InstanceMethods
+      Neoid.models << receiver
     end
   end
 end

data/lib/neoid/model_config.rb

@@ -1,11 +1,55 @@
 module Neoid
   class ModelConfig
-    @properties = []
-  
-    attr_accessor :properties
+    attr_reader :properties
+    attr_reader :search_options
+    attr_reader :relationship_options
+    
+    def initialize(klass)
+      @klass = klass
+    end
+    
+    def stored_fields
+      @stored_fields ||= {}
+    end
+    
+    def field(name, &block)
+      self.stored_fields[name] = block
+    end
+    
+    def relationship(options)
+      @relationship_options = options
+    end
+    
+    def search(&block)
+      raise "search needs a block" unless block_given?
+      @search_options = SearchConfig.new
+      block.(@search_options)
+    end
+    
+    def inspect
+      "#<Neoid::ModelConfig @properties=#{properties.inspect} @search_options=#{@search_options.inspect}>"
+    end
+  end
   
-    def property(name)
-      @properties << name
+  class SearchConfig
+    def index_fields
+      @index_fields ||= {}
+    end
+
+    def fulltext_fields
+      @fulltext_fields ||= {}
+    end
+    
+    def index(field, options = {}, &block)
+      index_fields[field] = options.merge(block: block)
+    end
+    
+    def fulltext(field, options = {}, &block)
+      fulltext_fields[field] = options.merge(block: block)
+    end
+    
+    def inspect
+      "#<Neoid::SearchConfig @index_fields=#{index_fields.inspect} @fulltext_fields=#{fulltext_fields.inspect}>"
     end
   end
 end