neoid 0.0.2 → 0.0.5.alpha

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.travis.yml ADDED
@@ -0,0 +1,4 @@
1
+ script: "bundle exec rake neo4j:install neo4j:start spec --trace"
2
+ rvm:
3
+ - 1.9.2
4
+ - 1.9.3
data/CHANGELOG.md CHANGED
@@ -1,3 +1,17 @@
1
+ ## v0.0.41
2
+
3
+ * fixed really annoying bug caused by Rails design -- Rails doesn't call `after_destroy` when assigning many to many relationships to a model, like `user.movies = [m1, m2, m3]` or `user.update_attributes(params[:user])` where it contains `params[:user][:movie_ids]` list (say from checkboxes), but it DOES CALL after_create for the new relationships. the fix adds after_remove callback to the has_many relationships, ensuring neo4j is up to date with all changes, no matter how they were committed
4
+
5
+ ## v0.0.4
6
+
7
+ * rewrote seacrch. one index for all types instead of one for type. please run neo_search_index on all of your models.
8
+ search in multiple types at once with `Neoid.search(types_array, term)
9
+
10
+ ## v0.0.3
11
+
12
+ * new configuration syntax (backwards compatible)
13
+ * full text search index
14
+
1
15
  ## v0.0.2
2
16
 
3
17
  * create node immediately after active record create
data/README.md CHANGED
@@ -1,5 +1,9 @@
1
1
  # Neoid
2
2
 
3
+ [![Build Status](https://secure.travis-ci.org/elado/neoid.png)](http://travis-ci.org/elado/neoid)
4
+
5
+
6
+
3
7
  Make your ActiveRecords stored and searchable on Neo4j graph database, in order to make fast graph queries that MySQL would crawl while doing them.
4
8
 
5
9
  Neoid to Neo4j is like Sunspot to Solr. You get the benefits of Neo4j speed while keeping your schema on your plain old RDBMS.
@@ -14,38 +18,40 @@ Neoid offers querying Neo4j for IDs of objects and then fetch them from your RDB
14
18
 
15
19
  Add to your Gemfile and run the `bundle` command to install it.
16
20
 
17
- gem 'neoid'
18
-
21
+ ```ruby
22
+ gem 'neoid', git: 'git://github.com/elado/neoid.git'
23
+ ```
19
24
 
20
25
  **Requires Ruby 1.9.2 or later.**
21
26
 
22
27
  ## Usage
23
28
 
24
- ### First app configuration:
29
+ ### Rails app configuration:
25
30
 
26
31
  In an initializer, such as `config/initializers/01_neo4j.rb`:
27
32
 
28
- ENV["NEO4J_URL"] ||= "http://localhost:7474"
29
-
30
- uri = URI.parse(ENV["NEO4J_URL"])
33
+ ```ruby
34
+ ENV["NEO4J_URL"] ||= "http://localhost:7474"
31
35
 
32
- $neo = Neography::Rest.new(neo4j_uri.to_s)
36
+ uri = URI.parse(ENV["NEO4J_URL"])
33
37
 
34
- Neography::Config.tap do |c|
35
- c.server = uri.host
36
- c.port = uri.port
38
+ $neo = Neography::Rest.new(uri.to_s)
37
39
 
38
- if uri.user && uri.password
39
- c.authentication = 'basic'
40
- c.username = uri.user
41
- c.password = uri.password
42
- end
43
- end
40
+ Neography.configure do |c|
41
+ c.server = uri.host
42
+ c.port = uri.port
44
43
 
45
- Neoid.db = $neo
44
+ if uri.user && uri.password
45
+ c.authentication = 'basic'
46
+ c.username = uri.user
47
+ c.password = uri.password
48
+ end
49
+ end
46
50
 
51
+ Neoid.db = $neo
52
+ ```
47
53
 
48
- `01_` in the file name is in order to get this file loaded first, before the models (files are loaded alphabetically).
54
+ `01_` in the file name is in order to get this file loaded first, before the models (initializers are loaded alphabetically).
49
55
 
50
56
  If you have a better idea (I bet you do!) please let me know.
51
57
 
@@ -57,105 +63,137 @@ If you have a better idea (I bet you do!) please let me know.
57
63
  For nodes, first include the `Neoid::Node` module in your model:
58
64
 
59
65
 
60
- class User < ActiveRecord::Base
61
- include Neoid::Node
62
- end
63
-
66
+ ```ruby
67
+ class User < ActiveRecord::Base
68
+ include Neoid::Node
69
+ end
70
+ ```
64
71
 
65
72
  This will help to create a corresponding node on Neo4j when a user is created, delete it when a user is destroyed, and update it if needed.
66
73
 
67
- Then, you can customize what fields will be saved on the node in Neo4j, by implementing `to_neo` method:
68
-
69
-
70
- class User < ActiveRecord::Base
71
- include Neoid::Node
72
-
73
- def to_neo
74
- {
75
- slug: slug,
76
- display_name: display_name
77
- }
78
- end
79
- end
74
+ Then, you can customize what fields will be saved on the node in Neo4j, inside neoidable configuration:
75
+
76
+ ```ruby
77
+ class User < ActiveRecord::Base
78
+ include Neoid::Node
79
+
80
+ neoidable do |c|
81
+ c.field :slug
82
+ c.field :display_name
83
+ c.field :display_name_length do
84
+ self.display_name.length
85
+ end
86
+ end
87
+ end
88
+ ```
80
89
 
81
- You can use `neo_properties_to_hash`, a helper method to make things shorter:
82
90
 
91
+ #### Relationships
83
92
 
84
- def to_neo
85
- neo_properties_to_hash(%w(slug display_name))
86
- end
93
+ Let's assume that a `User` can `Like` `Movie`s:
87
94
 
88
95
 
89
- #### Relationships
96
+ ```ruby
97
+ # user.rb
90
98
 
91
- Let's assume that a `User` can `Like` `Movie`s:
99
+ class User < ActiveRecord::Base
100
+ include Neoid::Node
92
101
 
102
+ has_many :likes
103
+ has_many :movies, through: :likes
93
104
 
94
- # user.rb
105
+ neoidable do |c|
106
+ c.field :slug
107
+ c.field :display_name
108
+ end
109
+ end
95
110
 
96
- class User < ActiveRecord::Base
97
- include Neoid::Node
98
-
99
- has_many :likes
100
- has_many :movies, through: :likes
101
-
102
- def to_neo
103
- neo_properties_to_hash(%w(slug display_name))
104
- end
105
- end
106
111
 
112
+ # movie.rb
107
113
 
108
- # movie.rb
114
+ class Movie < ActiveRecord::Base
115
+ include Neoid::Node
109
116
 
110
- class Movie < ActiveRecord::Base
111
- include Neoid::Node
112
-
113
- has_many :likes
114
- has_many :users, through: :likes
115
-
116
- def to_neo
117
- neo_properties_to_hash(%w(slug name))
118
- end
119
- end
117
+ has_many :likes
118
+ has_many :users, through: :likes
120
119
 
120
+ neoidable do |c|
121
+ c.field :slug
122
+ c.field :name
123
+ end
124
+ end
121
125
 
122
- # like.rb
123
126
 
124
- class Like < ActiveRecord::Base
125
- belongs_to :user
126
- belongs_to :movie
127
- end
127
+ # like.rb
128
128
 
129
+ class Like < ActiveRecord::Base
130
+ belongs_to :user
131
+ belongs_to :movie
132
+ end
133
+ ```
129
134
 
130
135
 
131
- Now let's make the `Like` model a Neoid, by including the `Neoid::Relationship` module, and define the relationship (start & end nodes and relationship type) options with `neoidable` method:
136
+ Now let's make the `Like` model a Neoid, by including the `Neoid::Relationship` module, and define the relationship (start & end nodes and relationship type) options with `neoidable` config and `relationship` method:
132
137
 
133
138
 
134
- class Like < ActiveRecord::Base
135
- belongs_to :user
136
- belongs_to :movie
139
+ ```ruby
140
+ class Like < ActiveRecord::Base
141
+ belongs_to :user
142
+ belongs_to :movie
137
143
 
138
- include Neoid::Relationship
139
- neoidable start_node: :user, end_node: :movie, type: :likes
140
- end
144
+ include Neoid::Relationship
141
145
 
146
+ neoidable do |c|
147
+ c.relationship start_node: :user, end_node: :movie, type: :likes
148
+ end
149
+ end
150
+ ```
142
151
 
143
152
  Neoid adds `neo_node` and `neo_relationships` to nodes and relationships, respectively.
144
153
 
145
154
  So you could do:
146
155
 
147
- user = User.create!(display_name: "elado")
148
- user.movies << Movie.create("Memento")
149
- user.movies << Movie.create("Inception")
156
+ ```ruby
157
+ user = User.create!(display_name: "elado")
158
+ user.movies << Movie.create("Memento")
159
+ user.movies << Movie.create("Inception")
160
+
161
+ user.neo_node # => #<Neography::Node…>
162
+ user.neo_node.display_name # => "elado"
163
+
164
+ rel = user.likes.first.neo_relationship
165
+ rel.start_node # user.neo_node
166
+ rel.end_node # user.movies.first.neo_node
167
+ rel.rel_type # 'likes'
168
+ ```
169
+
170
+ ## Index for Full-Text Search
171
+
172
+ Using `search` block inside a `neoidable` block, you can store certain fields.
173
+
174
+ ```ruby
175
+ # movie.rb
176
+
177
+ class Movie < ActiveRecord::Base
178
+ include Neoid::Node
179
+
180
+ neoidable do |c|
181
+ c.field :slug
182
+ c.field :name
150
183
 
151
- user.neo_node # => #<Neography::Node…>
152
- user.neo_node.display_name # => "elado"
184
+ c.search do |s|
185
+ # full-text index fields
186
+ s.fulltext :name
187
+ s.fulltext :description
153
188
 
154
- rel = user.likes.first.neo_relationship
155
- rel.start_node # user.neo_node
156
- rel.end_node # user.movies.first.neo_node
157
- rel.rel_type # 'likes'
189
+ # just index for exact matches
190
+ s.index :year
191
+ end
192
+ end
193
+ end
194
+ ```
158
195
 
196
+ Records will be automatically indexed when inserted or updated.
159
197
 
160
198
  ## Querying
161
199
 
@@ -165,51 +203,86 @@ You can query with all [Neography](https://github.com/maxdemarzi/neography)'s AP
165
203
 
166
204
  These examples query Neo4j using Gremlin for IDs of objects, and then fetches them from ActiveRecord with an `in` query.
167
205
 
168
- Of course, you can store using the `to_neo` all the data you need in Neo4j and avoid querying ActiveRecord.
206
+ Of course, you can store using the `neoidable do |c| c.field ... end` all the data you need in Neo4j and avoid querying ActiveRecord.
169
207
 
170
208
 
171
209
  **Most popular categories**
172
210
 
173
- gremlin_query = <<-GREMLIN
174
- m = [:]
175
-
176
- g.v(0)
177
- .out('movies_subref').out
178
- .inE('likes')
179
- .inV
180
- .groupCount(m).iterate()
211
+ ```ruby
212
+ gremlin_query = <<-GREMLIN
213
+ m = [:]
181
214
 
182
- m.sort{-it.value}.collect{it.key.ar_id}
183
- GREMLIN
215
+ g.v(0)
216
+ .out('movies_subref').out
217
+ .inE('likes')
218
+ .inV
219
+ .groupCount(m).iterate()
184
220
 
185
- movie_ids = Neoid.db.execute_script(gremlin_query)
221
+ m.sort{-it.value}.collect{it.key.ar_id}
222
+ GREMLIN
186
223
 
187
- Movie.where(id: movie_ids)
224
+ movie_ids = Neoid.db.execute_script(gremlin_query)
188
225
 
226
+ Movie.where(id: movie_ids)
227
+ ```
189
228
 
190
229
  Assuming we have another `Friendship` model which is a relationship with start/end nodes of `user` and type of `friends`,
191
230
 
192
231
  **Movies of user friends that the user doesn't have**
193
232
 
194
- user = User.find(1)
233
+ ```ruby
234
+ user = User.find(1)
235
+
236
+ gremlin_query = <<-GREMLIN
237
+ u = g.idx('users_index')[[ar_id:user_id]].next()
238
+ movies = []
239
+
240
+ u
241
+ .out('likes').aggregate(movies).back(2)
242
+ .out('friends').out('likes')
243
+ .dedup
244
+ .except(movies).collect{it.ar_id}
245
+ GREMLIN
246
+
247
+ movie_ids = Neoid.db.execute_script(gremlin_query, user_id: user.id)
248
+
249
+ Movie.where(id: movie_ids)
250
+ ```
251
+
252
+ `.next()` is in order to get a vertex object which we can actually query on.
253
+
254
+
255
+ ### Full Text Search
256
+
257
+ ```ruby
258
+ # will match all movies with full-text match for name/description. returns ActiveRecord instanced
259
+ Movie.neo_search("*hello*").results
260
+
261
+ # same as above but returns hashes with the values that were indexed on Neo4j
262
+ Movie.search("*hello*").hits
195
263
 
196
- gremlin_query = <<-GREMLIN
197
- u = g.idx('users_index')[[ar_id:'#{user.id}']][0].toList()[0]
198
- movies = []
264
+ # search in multiple types
265
+ Neoid.neo_search([Movie, User], "hello")
199
266
 
200
- u
201
- .out('likes').aggregate(movies).back(2)
202
- .out('friends').out('likes')
203
- .dedup
204
- .except(movies).collect{it.ar_id}
205
- GREMLIN
267
+ # search with exact matches (pass a hash of field/value)
268
+ Movie.neo_search(year: 2013).results
269
+ ```
206
270
 
207
- movie_ids = Neoid.db.execute_script(gremlin_query)
271
+ ## Inserting records of existing app
208
272
 
209
- Movie.where(id: movie_ids)
273
+ If you have an existing database and just want to integrate Neoid, configure the `neoidable`s and run in a rake task or console
210
274
 
275
+ ```ruby
276
+ [ Like.includes(:user).includes(:movie), OtherRelationshipModel ].each { |model| model.all.each(&:neo_update) }
211
277
 
212
- `[0].toList()[0]` is in order to get a pipeline object which we can actually query on.
278
+ NodeModel.all.each(&:neo_update)
279
+ ```
280
+
281
+ This will loop through all of your relationship records and generate the two edge nodes along with a relationship (eager loading for better performance).
282
+ The second line is for nodes without relationships.
283
+
284
+ For large data sets use pagination.
285
+ Better interface for that in the future.
213
286
 
214
287
 
215
288
  ## Behind The Scenes
@@ -219,7 +292,7 @@ Whenever the `neo_node` on nodes or `neo_relationship` on relationships is calle
219
292
  ### For Nodes:
220
293
 
221
294
  1. Ensures there's a sub reference node (read [here](http://docs.neo4j.org/chunked/stable/tutorials-java-embedded-index.html) about sub reference nodes)
222
- 2. Creates a node based on the ActiveRecord, with the `id` attribute and all other attributes from `to_neo`
295
+ 2. Creates a node based on the ActiveRecord, with the `id` attribute and all other attributes from `neoidable`'s field list
223
296
  3. Creates a relationship between the sub reference node and the newly created node
224
297
  4. Adds the ActiveRecord `id` to a node index, pointing to the Neo4j node id, for fast lookup in the future
225
298
 
@@ -235,35 +308,57 @@ Like Nodes, it uses an index (relationship index) to look up a relationship by A
235
308
 
236
309
  ## Testing
237
310
 
238
- Neoid tests run on a regular Neo4j database, on port 7574. You probably want to have it running on a different instance than your development one.
311
+ In order to test your app or this gem, you need a running Neo4j database, dedicated to tests.
239
312
 
240
- In order to do that:
313
+ I use port 7574 for this. To run another database locally:
241
314
 
242
- Copy the Neo4j folder to a different location,
315
+ Copy the entire Neo4j database folder to a different location,
243
316
 
244
317
  **or**
245
318
 
246
- symlink `bin`, `lib`, `plugins`, `system`, copy `conf` and create an empty `data` folder.
319
+ symlink `bin`, `lib`, `plugins`, `system`, copy `conf` to a single folder, and create an empty `data` folder.
247
320
 
248
321
  Then, edit `conf/neo4j-server.properties` and set the port (`org.neo4j.server.webserver.port`) from 7474 to 7574 and run the server with `bin/neo4j start`
249
322
 
323
+ ## Testing Your App with Neoid (RSpec)
324
+
325
+ In `environments/test.rb`, add:
326
+
327
+ ```ruby
328
+ ENV["NEO4J_URL"] = 'http://localhost:7574'
329
+ ```
330
+
331
+ In your `spec_helper.rb`, add the following configurations:
250
332
 
251
- Download, install and configure [neo4j-clean-remote-db-addon](https://github.com/jexp/neo4j-clean-remote-db-addon). For the test database, leave the default `secret-key` key.
333
+ ```ruby
334
+ config.before :all do
335
+ Neoid.clean_db(:yes_i_am_sure)
336
+ end
252
337
 
338
+ config.before :each do
339
+ Neoid.reset_cached_variables
340
+ end
341
+ ```
342
+
343
+ ## Testing This Gem
344
+
345
+ Just run `rake` from the gem folder.
253
346
 
254
347
  ## Contributing
255
348
 
256
349
  Please create a [new issue](https://github.com/elado/neoid/issues) if you run into any bugs. Contribute patches via pull requests. Write tests and make sure all tests pass.
257
350
 
258
351
 
352
+ ## Heroku Support
353
+
354
+ Unfortunately, as for now, Neo4j add-on on Heroku doesn't support Gremlin. Therefore, this gem won't work on Heroku's add on. You should self-host a Neo4j instance on an EC2 or any other server.
355
+
259
356
 
260
357
  ## To Do
261
358
 
262
- * `after_update` to update a node/relationship.
263
- * Allow to disable sub reference nodes through options
264
- * Execute queries/scripts from model and not Neography (e.g. `Movie.neo_gremlin(gremlin_query)` with query that outputs IDs, returns a list of `Movie`s)
265
- * Rake task to index all nodes and relatiohsips in Neo4j
359
+ [To Do](https://github.com/elado/neoid/blob/master/TODO.md)
360
+
266
361
 
267
362
  ---
268
363
 
269
- Developed by [@elado](http://twitter.com/elado)
364
+ Developed by [@elado](http://twitter.com/elado)
data/Rakefile CHANGED
@@ -1,5 +1,6 @@
1
1
  require "bundler/gem_tasks"
2
2
  require 'rspec/core/rake_task'
3
+ require 'neography/tasks'
3
4
 
4
5
  RSpec::Core::RakeTask.new(:spec)
5
6
 
data/TODO.md ADDED
@@ -0,0 +1,6 @@
1
+ # Neoid - To Do
2
+
3
+ * Allow to disable sub reference nodes through options
4
+ * Execute queries/scripts from model and not Neography (e.g. `Movie.neo_gremlin(gremlin_query)` with query that outputs IDs, returns a list of `Movie`s)
5
+ * Rake task to index all nodes and relatiohsips in Neo4j
6
+ * Test update node
@@ -0,0 +1,12 @@
1
+ module Neoid
2
+ class NeoDatabaseCleaner
3
+ def self.clean_db(start_node = Neoid.db.get_root)
4
+ Neoid.db.execute_script <<-GREMLIN
5
+ g.V.toList().each { if (it.id != 0) g.removeVertex(it) }
6
+ g.indices.each { g.dropIndex(it.indexName); }
7
+ GREMLIN
8
+
9
+ true
10
+ end
11
+ end
12
+ end
@@ -0,0 +1,14 @@
1
+ module Ndoid
2
+ class Middleware
3
+ def initialize(app)
4
+ @app = app
5
+ end
6
+
7
+ def call(env)
8
+ old, Thread.current[:neoid_enabled] = Thread.current[:neoid_enabled], true
9
+ @app.call(env)
10
+ ensure
11
+ Thread.current[:neoid_enabled] = old
12
+ end
13
+ end
14
+ end
@@ -1,14 +1,16 @@
1
1
  module Neoid
2
2
  module ModelAdditions
3
3
  module ClassMethods
4
- def neoidable(options)
5
- @config = Neoid::ModelConfig.new
6
- yield(@config) if block_given?
7
- @neoidable_options = options
4
+ attr_reader :neoid_config
5
+ attr_reader :neoid_options
6
+
7
+ def neoid_config
8
+ @neoid_config ||= Neoid::ModelConfig.new(self)
8
9
  end
9
-
10
- def neoidable_options
11
- @neoidable_options
10
+
11
+ def neoidable(options = {})
12
+ yield(neoid_config) if block_given?
13
+ @neoid_options = options
12
14
  end
13
15
 
14
16
  def neo_index_name
@@ -18,13 +20,32 @@ module Neoid
18
20
 
19
21
  module InstanceMethods
20
22
  def to_neo
21
- {}
23
+ if self.class.neoid_config.stored_fields
24
+ hash = self.class.neoid_config.stored_fields.inject({}) do |all, (field, block)|
25
+ all[field] = if block
26
+ instance_eval(&block)
27
+ else
28
+ self.send(field) rescue (raise "No field #{field} for #{self.class.name}")
29
+ end
30
+
31
+ all
32
+ end
33
+
34
+ hash.reject { |k, v| v.nil? }
35
+ else
36
+ {}
37
+ end
38
+ end
39
+
40
+ def neo_resave
41
+ _reset_neo_representation
42
+ neo_update
22
43
  end
23
44
 
24
45
  protected
25
- def neo_properties_to_hash(*property_list)
26
- property_list.flatten.inject({}) { |all, property|
27
- all[property] = self.attributes[property]
46
+ def neo_properties_to_hash(*attribute_list)
47
+ attribute_list.flatten.inject({}) { |all, property|
48
+ all[property] = self.send(property)
28
49
  all
29
50
  }
30
51
  end
@@ -36,11 +57,20 @@ module Neoid
36
57
  if results
37
58
  neo_load(results.first['self'])
38
59
  else
39
- node = neo_create
40
- node
60
+ neo_create
41
61
  end
42
62
  end
43
63
  end
64
+
65
+ def _reset_neo_representation
66
+ @_neo_representation = nil
67
+ end
68
+ end
69
+
70
+ def self.included(receiver)
71
+ receiver.extend ClassMethods
72
+ receiver.send :include, InstanceMethods
73
+ Neoid.models << receiver
44
74
  end
45
75
  end
46
76
  end
@@ -1,11 +1,55 @@
1
1
  module Neoid
2
2
  class ModelConfig
3
- @properties = []
4
-
5
- attr_accessor :properties
3
+ attr_reader :properties
4
+ attr_reader :search_options
5
+ attr_reader :relationship_options
6
+
7
+ def initialize(klass)
8
+ @klass = klass
9
+ end
10
+
11
+ def stored_fields
12
+ @stored_fields ||= {}
13
+ end
14
+
15
+ def field(name, &block)
16
+ self.stored_fields[name] = block
17
+ end
18
+
19
+ def relationship(options)
20
+ @relationship_options = options
21
+ end
22
+
23
+ def search(&block)
24
+ raise "search needs a block" unless block_given?
25
+ @search_options = SearchConfig.new
26
+ block.(@search_options)
27
+ end
28
+
29
+ def inspect
30
+ "#<Neoid::ModelConfig @properties=#{properties.inspect} @search_options=#{@search_options.inspect}>"
31
+ end
32
+ end
6
33
 
7
- def property(name)
8
- @properties << name
34
+ class SearchConfig
35
+ def index_fields
36
+ @index_fields ||= {}
37
+ end
38
+
39
+ def fulltext_fields
40
+ @fulltext_fields ||= {}
41
+ end
42
+
43
+ def index(field, options = {}, &block)
44
+ index_fields[field] = options.merge(block: block)
45
+ end
46
+
47
+ def fulltext(field, options = {}, &block)
48
+ fulltext_fields[field] = options.merge(block: block)
49
+ end
50
+
51
+ def inspect
52
+ "#<Neoid::SearchConfig @index_fields=#{index_fields.inspect} @fulltext_fields=#{fulltext_fields.inspect}>"
9
53
  end
10
54
  end
11
55
  end