elasticsearch-documents 0.0.1 → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.travis.yml +6 -0
- data/README.md +36 -14
- data/elasticsearch-documents.gemspec +1 -1
- data/lib/elasticsearch/extensions/documents.rb +4 -15
- data/lib/elasticsearch/extensions/documents/indexer.rb +5 -2
- data/lib/elasticsearch/extensions/documents/version.rb +1 -1
- data/spec/elasticsearch/extensions/documents/indexer_spec.rb +14 -4
- data/spec/elasticsearch/extensions/documents_spec.rb +58 -0
- data/spec/spec_helper.rb +17 -6
- metadata +10 -9
- data/spec/elasticsearch/extensions/documentor_spec.rb +0 -50
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: d5c254c644e1b8f9f4649a463816664305479ce7
|
4
|
+
data.tar.gz: d37eb314977344a51a554bd20799f924c954e4cc
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5101e3b59ff7ef130591279d0076c0480eeabe93cff21c19f69fffc5f3ba0bed89f87ba1f92c1ed90000c347d37725cb28098c1dbbd6b338e43318231eeccc9c
|
7
|
+
data.tar.gz: e6a6e3980de1d94f54a4976ef558371e468dd1b126c7260c01ed500a1a2389670ced05b63c96da99e50ce57e0e473710e3175706b57d53a6c365d04315a16cf3
|
data/.travis.yml
ADDED
data/README.md
CHANGED
@@ -1,6 +1,9 @@
|
|
1
1
|
# Elasticsearch::Extensions::Documents
|
2
2
|
|
3
|
-
A service wrapper to manage Elasticsearch index documents
|
3
|
+
A service wrapper to manage Elasticsearch index documents. Built on the
|
4
|
+
[`elasticsearch-ruby`][es-ruby-gem] Gem.
|
5
|
+
|
6
|
+
[](https://travis-ci.org/ryanhouston/elasticsearch-documents)
|
4
7
|
|
5
8
|
## Installation
|
6
9
|
|
@@ -19,7 +22,8 @@ Or install it yourself as:
|
|
19
22
|
## Configuration
|
20
23
|
|
21
24
|
Before making any calls to Elasticsearch you need to configure the `Documents`
|
22
|
-
extension.
|
25
|
+
extension. Configuration options namespaced under 'client' are passed through to
|
26
|
+
[`Elasticsearch::Client`](https://github.com/elasticsearch/elasticsearch-ruby/blob/a7bbdbf2a96168c1b33dca46ee160d2d4d75ada0/elasticsearch-transport/lib/elasticsearch/transport/client.rb).
|
23
27
|
|
24
28
|
```ruby
|
25
29
|
ES_MAPPINGS = {
|
@@ -42,12 +46,11 @@ ES_SETTINGS = {
|
|
42
46
|
}
|
43
47
|
|
44
48
|
Elasticsearch::Extensions::Documents.configure do |config|
|
45
|
-
config.url = 'http://example.com:9200' # your elasticsearch endpoint
|
46
49
|
config.index_name = 'test_index' # the name of your index
|
47
50
|
config.mappings = ES_MAPPINGS # a hash containing your index mappings
|
48
51
|
config.settings = ES_SETTINGS # a hash containing your index settings
|
49
|
-
config.
|
50
|
-
config.
|
52
|
+
config.client.url = 'http://example.com:9200' # your elasticsearch endpoint
|
53
|
+
config.client.logger = Rails.logger # the logger to use. (defaults to Logger.new(STDERR))
|
51
54
|
end
|
52
55
|
```
|
53
56
|
|
@@ -56,16 +59,16 @@ could live in an initializer like `config/initializers/elasticsearch.rb`.
|
|
56
59
|
|
57
60
|
## Usage
|
58
61
|
|
59
|
-
The `Documents` extension builds on the
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
-
|
64
|
-
`elasticsearch-ruby`
|
62
|
+
The `Documents` extension builds on the [`elasticsearch-ruby`][es-ruby-gem] Gem
|
63
|
+
adding conventions and helper classes to aide in the serialization and flow of
|
64
|
+
data between your application code and the elasticsearch-ruby client. To
|
65
|
+
accomplish this the application data models will be serialized into instances
|
66
|
+
of `Document` classes. These `Document` instances are then indexed and searched
|
67
|
+
with wrappers around the `elasticsearch-ruby` client.
|
65
68
|
|
66
69
|
### Saving a Document
|
67
|
-
|
68
|
-
|
70
|
+
Assume your application has a `User` model. To index the `User` records you
|
71
|
+
define a `Document` that maps `User` records to a search index mapping.
|
69
72
|
|
70
73
|
```ruby
|
71
74
|
class UserDocument < Elasticsearch::Extensions::Documents::Document
|
@@ -124,7 +127,8 @@ end
|
|
124
127
|
```
|
125
128
|
|
126
129
|
You could elaborate on this class with a constructor that takes the search
|
127
|
-
term and other options specific to your use case as arguments.
|
130
|
+
term and other options specific to your use case as arguments. The impoortant
|
131
|
+
part is to define the `#as_hash` method.
|
128
132
|
|
129
133
|
You can then call the `#execute` method to run the query. The Elasticsearch JSON
|
130
134
|
response will be returned in whole wrapped in a
|
@@ -203,6 +207,23 @@ indexer.reindex do |indexer|
|
|
203
207
|
indexer.bulk_index(documents)
|
204
208
|
end
|
205
209
|
```
|
210
|
+
|
211
|
+
By default the call to `#reindex` will create the index if it does not yet
|
212
|
+
exist. If the index already exists it will be left in place and the documents
|
213
|
+
provided to be indexed will be added or updated as needed. You can force the
|
214
|
+
index to be dropped and recreated during the reindex process by passing the
|
215
|
+
`force_create: true` option:
|
216
|
+
|
217
|
+
```ruby
|
218
|
+
indexer.reindex(force_create: true) do |indexer|
|
219
|
+
# bulk index those documents into a fresh index
|
220
|
+
end
|
221
|
+
```
|
222
|
+
|
223
|
+
Different reindexing strategies may be added in the future to allow "zero
|
224
|
+
downtime reindexing". This could be accomplished using index names with a
|
225
|
+
timestamp appended and index aliases.
|
226
|
+
|
206
227
|
## Contributing
|
207
228
|
|
208
229
|
1. Fork it
|
@@ -213,5 +234,6 @@ end
|
|
213
234
|
|
214
235
|
|
215
236
|
[es-query-dsl]: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
|
237
|
+
[es-ruby-gem]: https://github.com/elasticsearch/elasticsearch-ruby
|
216
238
|
[es-ruby-search-src]: https://github.com/elasticsearch/elasticsearch-ruby/blob/master/elasticsearch-api/lib/elasticsearch/api/actions/search.rb
|
217
239
|
|
@@ -21,7 +21,7 @@ Gem::Specification.new do |spec|
|
|
21
21
|
|
22
22
|
spec.add_development_dependency "bundler", "~> 1.3"
|
23
23
|
spec.add_development_dependency "rake"
|
24
|
-
spec.add_development_dependency "rspec"
|
24
|
+
spec.add_development_dependency "rspec", "~> 2.14"
|
25
25
|
|
26
26
|
spec.add_runtime_dependency "elasticsearch"
|
27
27
|
spec.add_runtime_dependency "hashie"
|
@@ -1,5 +1,6 @@
|
|
1
1
|
require "elasticsearch"
|
2
2
|
require "logger"
|
3
|
+
require "ostruct"
|
3
4
|
require "elasticsearch/extensions/documents/version"
|
4
5
|
require "elasticsearch/extensions/documents/document"
|
5
6
|
require "elasticsearch/extensions/documents/index"
|
@@ -15,14 +16,11 @@ module Elasticsearch
|
|
15
16
|
attr_accessor :client, :configuration
|
16
17
|
|
17
18
|
def client
|
18
|
-
|
19
|
-
host: self.configuration.url,
|
20
|
-
log: self.configuration.log,
|
21
|
-
})
|
19
|
+
Elasticsearch::Client.new(configuration.client.marshal_dump)
|
22
20
|
end
|
23
21
|
|
24
22
|
def configure
|
25
|
-
self.configuration ||=
|
23
|
+
self.configuration ||= OpenStruct.new(client: OpenStruct.new)
|
26
24
|
yield configuration
|
27
25
|
end
|
28
26
|
|
@@ -31,16 +29,7 @@ module Elasticsearch
|
|
31
29
|
end
|
32
30
|
|
33
31
|
def logger
|
34
|
-
self.configuration.logger
|
35
|
-
end
|
36
|
-
end
|
37
|
-
|
38
|
-
class Configuration
|
39
|
-
attr_accessor :url, :index_name, :mappings, :settings, :log, :logger
|
40
|
-
|
41
|
-
def initialize
|
42
|
-
@logger = Logger.new(STDOUT)
|
43
|
-
@log = true
|
32
|
+
self.configuration.client.logger ||= Logger.new(STDERR)
|
44
33
|
end
|
45
34
|
end
|
46
35
|
|
@@ -28,9 +28,12 @@ module Elasticsearch
|
|
28
28
|
end
|
29
29
|
end
|
30
30
|
|
31
|
-
def reindex(&block)
|
32
|
-
|
31
|
+
def reindex(options = {}, &block)
|
32
|
+
force_create = options.fetch(:force_create) { false }
|
33
|
+
|
34
|
+
drop_index if force_create
|
33
35
|
create_index
|
36
|
+
|
34
37
|
block.call(self) if block_given?
|
35
38
|
end
|
36
39
|
|
@@ -83,10 +83,20 @@ module Elasticsearch
|
|
83
83
|
|
84
84
|
describe '#reindex' do
|
85
85
|
|
86
|
-
|
87
|
-
|
88
|
-
|
89
|
-
|
86
|
+
context 'with the :force_create option' do
|
87
|
+
it 'drops the index if exists' do
|
88
|
+
indices.stub(:exists).and_return(true)
|
89
|
+
expect(indexer).to receive(:drop_index)
|
90
|
+
indexer.reindex(force_create: true)
|
91
|
+
end
|
92
|
+
end
|
93
|
+
|
94
|
+
context 'without the :force_create option' do
|
95
|
+
it 'does not drop the index if exists' do
|
96
|
+
indices.stub(:exists).and_return(true)
|
97
|
+
expect(indexer).not_to receive(:drop_index)
|
98
|
+
indexer.reindex
|
99
|
+
end
|
90
100
|
end
|
91
101
|
|
92
102
|
it 'creates a new index' do
|
@@ -0,0 +1,58 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
require 'logger'
|
3
|
+
|
4
|
+
module Elasticsearch::Extensions
|
5
|
+
describe Documents do
|
6
|
+
|
7
|
+
before(:each) { reset_configuration }
|
8
|
+
|
9
|
+
describe '.configuration' do
|
10
|
+
subject(:config) { Documents.configuration }
|
11
|
+
|
12
|
+
its(:index_name) { should eql 'test_index' }
|
13
|
+
its(:mappings) { should eql :fake_mappings }
|
14
|
+
its(:settings) { should eql :fake_settings }
|
15
|
+
end
|
16
|
+
|
17
|
+
describe '.client' do
|
18
|
+
subject(:client) { Documents.client }
|
19
|
+
|
20
|
+
it 'creates an instance of Elasticsearch::Transport::Client' do
|
21
|
+
expect(client).to be_instance_of Elasticsearch::Transport::Client
|
22
|
+
end
|
23
|
+
|
24
|
+
it 'provides a new client instance for each call' do
|
25
|
+
c1 = Documents.client
|
26
|
+
c2 = Documents.client
|
27
|
+
expect(c1).not_to equal c2
|
28
|
+
end
|
29
|
+
|
30
|
+
it 'instantiates the client with the config.client settings' do
|
31
|
+
Documents.configure do |config|
|
32
|
+
config.client.url = 'http://example.com:9200/es'
|
33
|
+
config.client.logger = :app_logger
|
34
|
+
config.client.tracer = :app_tracer
|
35
|
+
end
|
36
|
+
|
37
|
+
expect(Elasticsearch::Client).to receive(:new) do |options|
|
38
|
+
expect(options[:url]).to eq 'http://example.com:9200/es'
|
39
|
+
expect(options[:logger]).to eq :app_logger
|
40
|
+
expect(options[:tracer]).to eq :app_tracer
|
41
|
+
end
|
42
|
+
Documents.client
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
describe '.index_name' do
|
47
|
+
specify { expect(Documents.index_name).to eq 'test_index' }
|
48
|
+
end
|
49
|
+
|
50
|
+
describe '.logger' do
|
51
|
+
it 'provides a default logger if none is given' do
|
52
|
+
expect(Documents.logger).to be_kind_of Logger
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
data/spec/spec_helper.rb
CHANGED
@@ -1,18 +1,29 @@
|
|
1
1
|
require 'elasticsearch-documents'
|
2
2
|
|
3
|
+
def reset_configuration
|
4
|
+
Elasticsearch::Extensions::Documents.configuration = nil
|
5
|
+
configure_documents
|
6
|
+
end
|
7
|
+
|
8
|
+
def configure_documents
|
9
|
+
Elasticsearch::Extensions::Documents.configure do |config|
|
10
|
+
config.index_name = 'test_index'
|
11
|
+
config.settings = :fake_settings
|
12
|
+
config.mappings = :fake_mappings
|
13
|
+
config.client.url = 'http://example.com:9200'
|
14
|
+
end
|
15
|
+
end
|
16
|
+
|
3
17
|
RSpec.configure do |config|
|
4
18
|
config.treat_symbols_as_metadata_keys_with_true_values = true
|
5
19
|
config.run_all_when_everything_filtered = true
|
6
20
|
config.filter_run :focus
|
7
21
|
|
8
22
|
config.order = 'random'
|
9
|
-
end
|
10
23
|
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
config.settings = :fake_settings
|
15
|
-
config.mappings = :fake_mappings
|
24
|
+
config.before(:suite) do
|
25
|
+
configure_documents
|
26
|
+
end
|
16
27
|
end
|
17
28
|
|
18
29
|
class TestDocumentsDocument < Elasticsearch::Extensions::Documents::Document
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: elasticsearch-documents
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0
|
4
|
+
version: 0.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ryan Houston
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2014-
|
11
|
+
date: 2014-07-15 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: bundler
|
@@ -42,16 +42,16 @@ dependencies:
|
|
42
42
|
name: rspec
|
43
43
|
requirement: !ruby/object:Gem::Requirement
|
44
44
|
requirements:
|
45
|
-
- -
|
45
|
+
- - ~>
|
46
46
|
- !ruby/object:Gem::Version
|
47
|
-
version: '
|
47
|
+
version: '2.14'
|
48
48
|
type: :development
|
49
49
|
prerelease: false
|
50
50
|
version_requirements: !ruby/object:Gem::Requirement
|
51
51
|
requirements:
|
52
|
-
- -
|
52
|
+
- - ~>
|
53
53
|
- !ruby/object:Gem::Version
|
54
|
-
version: '
|
54
|
+
version: '2.14'
|
55
55
|
- !ruby/object:Gem::Dependency
|
56
56
|
name: elasticsearch
|
57
57
|
requirement: !ruby/object:Gem::Requirement
|
@@ -89,6 +89,7 @@ extra_rdoc_files: []
|
|
89
89
|
files:
|
90
90
|
- .gitignore
|
91
91
|
- .rspec
|
92
|
+
- .travis.yml
|
92
93
|
- Gemfile
|
93
94
|
- LICENSE.txt
|
94
95
|
- README.md
|
@@ -102,12 +103,12 @@ files:
|
|
102
103
|
- lib/elasticsearch/extensions/documents/queryable.rb
|
103
104
|
- lib/elasticsearch/extensions/documents/utils.rb
|
104
105
|
- lib/elasticsearch/extensions/documents/version.rb
|
105
|
-
- spec/elasticsearch/extensions/documentor_spec.rb
|
106
106
|
- spec/elasticsearch/extensions/documents/document_spec.rb
|
107
107
|
- spec/elasticsearch/extensions/documents/index_spec.rb
|
108
108
|
- spec/elasticsearch/extensions/documents/indexer_spec.rb
|
109
109
|
- spec/elasticsearch/extensions/documents/queryable_spec.rb
|
110
110
|
- spec/elasticsearch/extensions/documents/utils_spec.rb
|
111
|
+
- spec/elasticsearch/extensions/documents_spec.rb
|
111
112
|
- spec/spec_helper.rb
|
112
113
|
homepage: http://github.com/RyanHouston/elasticsearch-documents
|
113
114
|
licenses:
|
@@ -129,15 +130,15 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
129
130
|
version: '0'
|
130
131
|
requirements: []
|
131
132
|
rubyforge_project:
|
132
|
-
rubygems_version: 2.2.
|
133
|
+
rubygems_version: 2.2.2
|
133
134
|
signing_key:
|
134
135
|
specification_version: 4
|
135
136
|
summary: A service wrapper to manage elasticsearch index documents
|
136
137
|
test_files:
|
137
|
-
- spec/elasticsearch/extensions/documentor_spec.rb
|
138
138
|
- spec/elasticsearch/extensions/documents/document_spec.rb
|
139
139
|
- spec/elasticsearch/extensions/documents/index_spec.rb
|
140
140
|
- spec/elasticsearch/extensions/documents/indexer_spec.rb
|
141
141
|
- spec/elasticsearch/extensions/documents/queryable_spec.rb
|
142
142
|
- spec/elasticsearch/extensions/documents/utils_spec.rb
|
143
|
+
- spec/elasticsearch/extensions/documents_spec.rb
|
143
144
|
- spec/spec_helper.rb
|
@@ -1,50 +0,0 @@
|
|
1
|
-
require 'spec_helper'
|
2
|
-
require 'logger'
|
3
|
-
|
4
|
-
module Elasticsearch::Extensions
|
5
|
-
describe Documents do
|
6
|
-
|
7
|
-
let(:logger) { Logger.new(STDOUT) }
|
8
|
-
before do
|
9
|
-
Documents.configure do |config|
|
10
|
-
config.logger = logger
|
11
|
-
config.log = true
|
12
|
-
end
|
13
|
-
end
|
14
|
-
|
15
|
-
describe '.configuration' do
|
16
|
-
subject(:config) { Documents.configuration }
|
17
|
-
|
18
|
-
specify { expect(config.url).to eql 'http://example.com:9200' }
|
19
|
-
specify { expect(config.index_name).to eql 'test_index' }
|
20
|
-
specify { expect(config.mappings).to eql( :fake_mappings ) }
|
21
|
-
specify { expect(config.settings).to eql( :fake_settings ) }
|
22
|
-
specify { expect(config.logger).to equal logger }
|
23
|
-
specify { expect(config.logger).to be_true }
|
24
|
-
end
|
25
|
-
|
26
|
-
describe '.client' do
|
27
|
-
subject(:client) { Documents.client }
|
28
|
-
|
29
|
-
it 'creates an instance of Elasticsearch::Transport::Client' do
|
30
|
-
expect(client).to be_instance_of Elasticsearch::Transport::Client
|
31
|
-
end
|
32
|
-
|
33
|
-
it 'caches the client instance' do
|
34
|
-
c1 = Documents.client
|
35
|
-
c2 = Documents.client
|
36
|
-
expect(c1).to equal c2
|
37
|
-
end
|
38
|
-
end
|
39
|
-
|
40
|
-
describe '.index_name' do
|
41
|
-
specify { expect(Documents.index_name).to eq 'test_index' }
|
42
|
-
end
|
43
|
-
|
44
|
-
describe '.logger' do
|
45
|
-
specify { expect(Documents.logger).to equal logger }
|
46
|
-
end
|
47
|
-
|
48
|
-
end
|
49
|
-
end
|
50
|
-
|