neoid 0.1.2 → 0.2.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +15 -0
- data/LICENSE +1 -1
- data/README.md +27 -25
- data/Rakefile +2 -2
- data/lib/neoid.rb +74 -61
- data/lib/neoid/batch.rb +27 -27
- data/lib/neoid/database_cleaner.rb +1 -1
- data/lib/neoid/middleware.rb +1 -1
- data/lib/neoid/model_additions.rb +13 -11
- data/lib/neoid/model_config.rb +13 -13
- data/lib/neoid/node.rb +21 -21
- data/lib/neoid/railtie.rb +1 -1
- data/lib/neoid/relationship.rb +78 -78
- data/lib/neoid/search_session.rb +3 -3
- data/lib/neoid/version.rb +1 -1
- data/spec/factories.rb +12 -0
- data/spec/neoid/batch_spec.rb +68 -89
- data/spec/neoid/config_spec.rb +6 -6
- data/spec/neoid/model_config_spec.rb +14 -13
- data/spec/neoid/node_spec.rb +60 -79
- data/spec/neoid/relationship_spec.rb +39 -37
- data/spec/neoid/search_spec.rb +71 -46
- data/spec/neoid_spec.rb +11 -4
- data/spec/spec_helper.rb +23 -4
- data/spec/support/database.yml +1 -1
- data/spec/support/models.rb +16 -16
- data/spec/support/schema.rb +1 -2
- metadata +84 -31
- data/.gitignore +0 -5
- data/.rspec +0 -1
- data/.travis.yml +0 -4
- data/CHANGELOG.md +0 -58
- data/Gemfile +0 -4
- data/TODO.md +0 -4
- data/neoid.gemspec +0 -28
checksums.yaml
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
---
|
2
|
+
!binary "U0hBMQ==":
|
3
|
+
metadata.gz: !binary |-
|
4
|
+
YTViMzkyM2ZlZmY5NDliZTcxN2I4ODAzODAwN2Y1ZWEzYjVkOGZiMQ==
|
5
|
+
data.tar.gz: !binary |-
|
6
|
+
MTczODYzMmJjODQzOTA5MjIxYjg2ZWY5OTBiM2I0N2JlMDg3OWE5OQ==
|
7
|
+
SHA512:
|
8
|
+
metadata.gz: !binary |-
|
9
|
+
MWY4ZWZhZTJhMTJhN2VjZjg5ODA1NWE1ODI4NDY1OTdhNTI2M2MyOGI0MjI2
|
10
|
+
MmEzNjg2OTgzMzRiNDA3NDgwYjY4NWY1NzU3OWJlMTA5YjlhOTA0NjY2OWE4
|
11
|
+
YzY5YTIwYWU4MjQyNTgxZWE2YzYwYTkwOWE1Njc0OWY4ODExMzk=
|
12
|
+
data.tar.gz: !binary |-
|
13
|
+
ZGYyMTEwOGM5MjY0ZGVlMjljZDIzYTQ1NWQ1OTQ0ZmY4ZmUxMzFiOTk0OTYx
|
14
|
+
NzU1M2ZmYjhlNjQ5OWY3YzA1MTUwMTY0YzhjNDQxMTc3NjAwNmYzMDc0OWIw
|
15
|
+
NWM1YWQxNTc4MmY5NzRiOGM2MGM3ODVmNjNmNGI0YWY3ZTE5YWQ=
|
data/LICENSE
CHANGED
@@ -16,4 +16,4 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
16
16
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
17
17
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
18
18
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
19
|
-
THE SOFTWARE.
|
19
|
+
THE SOFTWARE.
|
data/README.md
CHANGED
@@ -1,17 +1,20 @@
|
|
1
1
|
# Neoid
|
2
2
|
|
3
|
-
[![
|
3
|
+
[![Gem Version](https://badge.fury.io/rb/neoid.svg)](http://badge.fury.io/rb/neoid)
|
4
|
+
[![Code Climate](https://codeclimate.com/github/neoid-gem/neoid.png)](https://codeclimate.com/github/neoid-gem/neoid)
|
5
|
+
[![Build Status](https://secure.travis-ci.org/neoid-gem/neoid.png)](http://travis-ci.org/neoid-gem/neoid)
|
4
6
|
|
7
|
+
__This gem is not stable. There are currently no stable versions. We're working on fixing this right now. Apologies.__
|
5
8
|
|
6
|
-
Make your ActiveRecords stored and searchable on Neo4j graph database, in order to make fast graph queries that MySQL would crawl while doing them.
|
9
|
+
Make your ActiveRecords stored and searchable on Neo4j graph database, in order to make fast graph queries that MySQL would crawl while doing them. Originally by [@elado](http://twitter.com/elado).
|
7
10
|
|
8
|
-
Neoid to Neo4j
|
11
|
+
Neoid is to Neo4j as Sunspot is to Solr. You get the benefits of Neo4j's speed while keeping your schema on your RDBMS.
|
9
12
|
|
10
|
-
Neoid
|
13
|
+
Neoid does not require JRuby. It's based on the [Neography](https://github.com/maxdemarzi/neography) gem which uses Neo4j's REST API.
|
11
14
|
|
12
15
|
Neoid offers querying Neo4j for IDs of objects and then fetch them from your RDBMS, or storing all desired data on Neo4j.
|
13
16
|
|
14
|
-
|
17
|
+
__Important: If you are hosting your application on Heroku with Neoid, [GrapheneDB](http://www.graphenedb.com/) does support Gremlin code; their add-on is [located here](https://addons.heroku.com/graphenedb). Also be reminded that the Gremlin code is actively being refactored into Cypher.__
|
15
18
|
|
16
19
|
## Changelog
|
17
20
|
|
@@ -23,16 +26,26 @@ Neoid offers querying Neo4j for IDs of objects and then fetch them from your RDB
|
|
23
26
|
Add to your Gemfile and run the `bundle` command to install it.
|
24
27
|
|
25
28
|
```ruby
|
26
|
-
gem 'neoid', '~> 0.
|
29
|
+
gem 'neoid', '~> 0.2.0'
|
27
30
|
```
|
28
31
|
|
29
|
-
**Requires Ruby 1.9.
|
32
|
+
**Requires Ruby 1.9.3 or later and Neo4j 1.9.8.**
|
33
|
+
|
34
|
+
### Installing Neo4j 1.9.8 for your project
|
35
|
+
|
36
|
+
We're currently working to bump to 2.1.x land, but for now, you have to use 1.9.8. To get started, install neo4j locally in your project with:
|
37
|
+
|
38
|
+
```bash
|
39
|
+
gem install neo4j-core --pre
|
40
|
+
rake neo4j:install[community,1.9.8]
|
41
|
+
rake neo4j:start
|
42
|
+
```
|
30
43
|
|
31
44
|
## Usage
|
32
45
|
|
33
46
|
### Rails app configuration:
|
34
47
|
|
35
|
-
|
48
|
+
Initializer neography and neoid in an initializer that is prefixed with `01_`, such as `config/initializers/01_neo4j.rb`:
|
36
49
|
|
37
50
|
```ruby
|
38
51
|
ENV["NEO4J_URL"] ||= "http://localhost:7474"
|
@@ -60,11 +73,6 @@ Neoid.configure do |c|
|
|
60
73
|
end
|
61
74
|
```
|
62
75
|
|
63
|
-
`01_` in the file name is in order to get this file loaded first, before the models (initializers are loaded alphabetically).
|
64
|
-
|
65
|
-
If you have a better idea (I bet you do!) please let me know.
|
66
|
-
|
67
|
-
|
68
76
|
### ActiveRecord configuration
|
69
77
|
|
70
78
|
#### Nodes
|
@@ -157,7 +165,7 @@ class Like < ActiveRecord::Base
|
|
157
165
|
end
|
158
166
|
```
|
159
167
|
|
160
|
-
Neoid adds the
|
168
|
+
Neoid adds the methods `neo_node` and `neo_relationships` to instances of nodes and relationships, respectively.
|
161
169
|
|
162
170
|
So you could do:
|
163
171
|
|
@@ -413,7 +421,8 @@ In order to test your app or this gem, you need a running Neo4j database, dedica
|
|
413
421
|
|
414
422
|
I use port 7574 for testing.
|
415
423
|
|
416
|
-
To run another database locally (read
|
424
|
+
To run another database locally (read
|
425
|
+
[here](http://docs.neo4j.org/chunked/stable/ha-setup-tutorial.html#ha-local-cluster) too):
|
417
426
|
|
418
427
|
Copy the entire Neo4j database folder to a different location,
|
419
428
|
|
@@ -445,23 +454,16 @@ end
|
|
445
454
|
|
446
455
|
## Testing This Gem
|
447
456
|
|
448
|
-
Run the Neo4j DB on port
|
457
|
+
Run the Neo4j DB on port 7474, and run `rake` from the gem folder.
|
449
458
|
|
450
459
|
## Contributing
|
451
460
|
|
452
461
|
Please create a [new issue](https://github.com/elado/neoid/issues) if you run into any bugs. Contribute patches via pull requests. Write tests and make sure all tests pass.
|
453
462
|
|
454
|
-
|
455
|
-
## Heroku Support
|
456
|
-
|
457
|
-
Unfortunately, as for now, Neo4j add-on on Heroku doesn't support Gremlin. Therefore, this gem won't work on Heroku's add on. You should self-host a Neo4j instance on an EC2 or any other server.
|
458
|
-
|
459
|
-
|
460
463
|
## TO DO
|
461
464
|
|
462
|
-
[TO DO](
|
463
|
-
|
465
|
+
[TO DO](https://github.com/elado/neoid/blob/master/TODO.md)
|
464
466
|
|
465
467
|
---
|
466
468
|
|
467
|
-
Developed by [@elado](http://twitter.com/elado)
|
469
|
+
Developed by [@elado](http://twitter.com/elado) and [@BenMorganIO](http://twitter.com/BenMorganIO)
|
data/Rakefile
CHANGED
data/lib/neoid.rb
CHANGED
@@ -22,11 +22,11 @@ module Neoid
|
|
22
22
|
attr_accessor :ref_node
|
23
23
|
attr_accessor :env_loaded
|
24
24
|
attr_reader :config
|
25
|
-
|
25
|
+
|
26
26
|
def node_models
|
27
27
|
@node_models ||= []
|
28
28
|
end
|
29
|
-
|
29
|
+
|
30
30
|
def relationship_models
|
31
31
|
@relationship_models ||= []
|
32
32
|
end
|
@@ -49,7 +49,7 @@ module Neoid
|
|
49
49
|
|
50
50
|
def initialize_all
|
51
51
|
@env_loaded = true
|
52
|
-
logger.info
|
52
|
+
logger.info 'Neoid initialize_all'
|
53
53
|
initialize_relationships
|
54
54
|
initialize_server
|
55
55
|
end
|
@@ -58,34 +58,43 @@ module Neoid
|
|
58
58
|
initialize_auto_index
|
59
59
|
initialize_subrefs
|
60
60
|
initialize_per_model_indexes
|
61
|
+
# begin
|
62
|
+
# @initialized_server = true
|
63
|
+
# initialize_auto_index
|
64
|
+
# initialize_subrefs
|
65
|
+
# initialize_per_model_indexes
|
66
|
+
# rescue Exception => e
|
67
|
+
# @initialized_server = false
|
68
|
+
# logger.error "Failed to initialize neoid: #{e.message}"
|
69
|
+
# end
|
61
70
|
end
|
62
|
-
|
71
|
+
|
63
72
|
def db
|
64
|
-
raise
|
73
|
+
raise 'Must set Neoid.db with a Neography::Rest instance' unless @db
|
74
|
+
# initialize_server unless @initialized_server
|
65
75
|
@db
|
66
76
|
end
|
67
77
|
|
68
78
|
def batch(options={}, &block)
|
69
79
|
Neoid::Batch.new(options, &block).run
|
70
80
|
end
|
71
|
-
|
81
|
+
|
72
82
|
def logger
|
73
83
|
@logger ||= Logger.new(ENV['NEOID_LOG'] ? ENV['NEOID_LOG_FILE'] || $stdout : '/dev/null')
|
74
84
|
end
|
75
|
-
|
85
|
+
|
76
86
|
def ref_node
|
77
87
|
@ref_node ||= Neography::Node.load(Neoid.db.get_root['self'])
|
78
88
|
end
|
79
|
-
|
89
|
+
|
80
90
|
def reset_cached_variables
|
81
91
|
initialize_subrefs
|
82
92
|
end
|
83
|
-
|
93
|
+
|
84
94
|
def clean_db(confirm)
|
85
|
-
puts
|
95
|
+
puts 'must call with confirm: Neoid.clean_db(:yes_i_am_sure)' and return unless confirm == :yes_i_am_sure
|
86
96
|
Neoid::NeoDatabaseCleaner.clean_db
|
87
97
|
end
|
88
|
-
|
89
98
|
|
90
99
|
def enabled=(flag)
|
91
100
|
Thread.current[:neoid_enabled] = flag
|
@@ -124,7 +133,7 @@ module Neoid
|
|
124
133
|
end
|
125
134
|
|
126
135
|
def search(types, term, options = {})
|
127
|
-
options = options.reverse_merge(limit: 15)
|
136
|
+
options = options.reverse_merge(limit: 15, match_type: 'AND')
|
128
137
|
|
129
138
|
types = [*types]
|
130
139
|
|
@@ -139,17 +148,17 @@ module Neoid
|
|
139
148
|
when String
|
140
149
|
search_in_fields = type.neoid_config.search_options.fulltext_fields.keys
|
141
150
|
next if search_in_fields.empty?
|
142
|
-
query_for_type << search_in_fields.map{ |field| generate_field_query(field, term, true) }.join(
|
151
|
+
query_for_type << search_in_fields.map{ |field| generate_field_query(field, term, true, options[:match_type]) }.join(' OR ')
|
143
152
|
when Hash
|
144
153
|
term.each do |field, value|
|
145
154
|
query_for_type << generate_field_query(field, value, false)
|
146
155
|
end
|
147
156
|
end
|
148
157
|
|
149
|
-
query << "(#{query_for_type.join(
|
158
|
+
query << "(#{query_for_type.join(') AND (')})"
|
150
159
|
end
|
151
160
|
|
152
|
-
query = "(#{query.join(
|
161
|
+
query = "(#{query.join(') OR (')})"
|
153
162
|
|
154
163
|
logger.info "Neoid query #{query}"
|
155
164
|
|
@@ -159,7 +168,7 @@ module Neoid
|
|
159
168
|
idx = g.getRawGraph().index().forNodes('#{DEFAULT_FULLTEXT_SEARCH_INDEX_NAME}')
|
160
169
|
hits = idx.query('#{sanitize_query_for_gremlin(query)}')
|
161
170
|
|
162
|
-
hits = #{options[:limit] ? "hits.take(#{options[:limit]})" :
|
171
|
+
hits = #{options[:limit] ? "hits.take(#{options[:limit]})" : 'hits'}
|
163
172
|
|
164
173
|
#{options[:after_query]}
|
165
174
|
GREMLIN
|
@@ -172,65 +181,69 @@ module Neoid
|
|
172
181
|
end
|
173
182
|
|
174
183
|
private
|
175
|
-
def sanitize_term(term)
|
176
|
-
# TODO - case sensitive?
|
177
|
-
term.downcase
|
178
|
-
end
|
179
184
|
|
180
|
-
|
181
|
-
|
182
|
-
|
183
|
-
|
185
|
+
def sanitize_term(term)
|
186
|
+
# TODO - case sensitive?
|
187
|
+
term.downcase
|
188
|
+
end
|
184
189
|
|
185
|
-
|
186
|
-
|
187
|
-
|
190
|
+
def sanitize_query_for_gremlin(query)
|
191
|
+
# TODO - case sensitive?
|
192
|
+
query.gsub("'", "\\\\'")
|
193
|
+
end
|
188
194
|
|
189
|
-
|
195
|
+
def generate_field_query(field, term, fulltext = false, match_type = 'AND')
|
196
|
+
term = term.to_s if term
|
197
|
+
return '' if term.nil? || term.empty?
|
190
198
|
|
191
|
-
|
192
|
-
|
199
|
+
fulltext = fulltext ? '_fulltext' : nil
|
200
|
+
valid_match_types = %w( AND OR )
|
201
|
+
match_type = valid_match_types.delete(match_type)
|
202
|
+
raise "Invalid match_type option. Valid values are #{valid_match_types.join(',')}" unless match_type
|
193
203
|
|
194
|
-
|
195
|
-
|
196
|
-
|
197
|
-
|
198
|
-
|
204
|
+
'(' + term.split(/\s+/).reject(&:empty?).map{ |t| "#{field}#{fulltext}:#{sanitize_term(t)}" }.join(" #{match_type} ") + ')'
|
205
|
+
end
|
206
|
+
|
207
|
+
def initialize_relationships
|
208
|
+
logger.info 'Neoid initialize_relationships'
|
209
|
+
relationship_models.each do |rel_model|
|
210
|
+
Relationship.initialize_relationship(rel_model)
|
199
211
|
end
|
212
|
+
end
|
200
213
|
|
201
|
-
|
202
|
-
|
203
|
-
|
204
|
-
|
214
|
+
def initialize_auto_index
|
215
|
+
logger.info 'Neoid initialize_auto_index'
|
216
|
+
Neoid.db.set_node_auto_index_status(true)
|
217
|
+
Neoid.db.add_node_auto_index_property(UNIQUE_ID_KEY)
|
205
218
|
|
206
|
-
|
207
|
-
|
208
|
-
|
219
|
+
Neoid.db.set_relationship_auto_index_status(true)
|
220
|
+
Neoid.db.add_relationship_auto_index_property(UNIQUE_ID_KEY)
|
221
|
+
end
|
209
222
|
|
210
|
-
|
211
|
-
|
212
|
-
|
213
|
-
node_models.each do |klass|
|
214
|
-
klass.reset_neo_subref_node
|
215
|
-
end
|
223
|
+
def initialize_subrefs
|
224
|
+
return unless config.enable_subrefs
|
216
225
|
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
223
|
-
|
226
|
+
node_models.each do |klass|
|
227
|
+
klass.reset_neo_subref_node
|
228
|
+
end
|
229
|
+
|
230
|
+
logger.info 'Neoid initialize_subrefs'
|
231
|
+
batch do
|
232
|
+
node_models.each(&:neo_subref_node)
|
233
|
+
end.then do |results|
|
234
|
+
node_models.zip(results).each do |klass, subref|
|
235
|
+
klass.neo_subref_node = subref
|
224
236
|
end
|
225
237
|
end
|
238
|
+
end
|
226
239
|
|
227
|
-
|
228
|
-
|
240
|
+
def initialize_per_model_indexes
|
241
|
+
return unless config.enable_per_model_indexes
|
229
242
|
|
230
|
-
|
231
|
-
|
232
|
-
|
233
|
-
end
|
243
|
+
logger.info 'Neoid initialize_subrefs'
|
244
|
+
batch do
|
245
|
+
node_models.each(&:neo_model_index)
|
234
246
|
end
|
247
|
+
end
|
235
248
|
end
|
236
249
|
end
|
data/lib/neoid/batch.rb
CHANGED
@@ -67,7 +67,7 @@ module Neoid
|
|
67
67
|
|
68
68
|
begin
|
69
69
|
@block.call(self)
|
70
|
-
ensure
|
70
|
+
ensure
|
71
71
|
self.class.reset_current_batch
|
72
72
|
end
|
73
73
|
|
@@ -79,40 +79,40 @@ module Neoid
|
|
79
79
|
end
|
80
80
|
|
81
81
|
private
|
82
|
-
def flush_batch
|
83
|
-
return [] if commands.empty?
|
84
|
-
current_results = nil
|
85
82
|
|
86
|
-
|
83
|
+
def flush_batch
|
84
|
+
return [] if commands.empty?
|
85
|
+
current_results = nil
|
87
86
|
|
88
|
-
|
89
|
-
|
90
|
-
}
|
91
|
-
|
92
|
-
|
87
|
+
benchmark = Benchmark.measure {
|
88
|
+
current_results = Neoid.db.batch(*commands)
|
89
|
+
current_results.map { |result| result['body'] } if current_results.respond_to?(:map)
|
90
|
+
}
|
91
|
+
Neoid.logger.info "Neoid batch (#{commands.length} commands) - #{benchmark}"
|
92
|
+
commands.clear
|
93
93
|
|
94
|
-
|
94
|
+
process_results(current_results)
|
95
95
|
|
96
|
-
|
96
|
+
thens.zip(current_results).each { |t, result| t.perform(result) }
|
97
97
|
|
98
|
-
|
98
|
+
thens.clear
|
99
99
|
|
100
|
-
|
101
|
-
|
100
|
+
results.concat current_results
|
101
|
+
end
|
102
102
|
|
103
|
-
|
104
|
-
|
105
|
-
|
103
|
+
def process_results(results)
|
104
|
+
results.map! do |result|
|
105
|
+
return result unless result.is_a?(Hash) && result['self'] && result['self'][%r[^https?://.*/(node|relationship)/\d+]]
|
106
106
|
|
107
|
-
|
108
|
-
|
109
|
-
|
110
|
-
|
111
|
-
|
107
|
+
type = case $1
|
108
|
+
when 'node' then Neoid::Node
|
109
|
+
when 'relationship' then Neoid::Relationship
|
110
|
+
else return result
|
111
|
+
end
|
112
112
|
|
113
|
-
|
114
|
-
end
|
113
|
+
type.from_hash(result)
|
115
114
|
end
|
115
|
+
end
|
116
116
|
end
|
117
117
|
|
118
118
|
# returned from a full batch, after it has been executed,
|
@@ -140,13 +140,13 @@ module Neoid
|
|
140
140
|
# it proxies all methods to the result, so in case it is returned (like in Neoid.execute_script_or_add_to_batch)
|
141
141
|
# the result of the method will be proxied to the result from the batch. See Node#neo_save
|
142
142
|
class SingleResultPromiseProxy
|
143
|
-
def initialize(*
|
143
|
+
def initialize(*)
|
144
144
|
end
|
145
145
|
|
146
146
|
attr_accessor :result
|
147
147
|
|
148
148
|
def result
|
149
|
-
raise
|
149
|
+
raise 'Accessed result too soon' unless @result
|
150
150
|
@result
|
151
151
|
end
|
152
152
|
|