es_tractor 0.0.2 → 0.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA1:
3
- metadata.gz: 8c7ccbb02ef8a6f0bb4b970cc28f963c55952a2e
4
- data.tar.gz: ce0bd10580ac1d4adc9d4ff0c48acf21e5bfaa0c
3
+ metadata.gz: b0c7612550a240d22377140293d1360e419f7223
4
+ data.tar.gz: 2037664523b539a9f40c20f9a7672c7736293fe3
5
5
  SHA512:
6
- metadata.gz: 198ab52863d622b2fcbb9e8a9d752a72e8abe00f6846aebebd56680e7bebb707dcedc7b49629205cae656fdd8951279b115afe4a6ccdadc25b4b2e897a36b25b
7
- data.tar.gz: b8e4d5d4925becc313fa08cf8eb3ebb3f42710b168ba47845646739998a6b3ac854bc2dc7f8419f99a292909e5c2a504e6a3567b2c316436dbfa4ecf51a2b7d9
6
+ metadata.gz: eb989abd3b4a9340740f2e5c6da3a9a17779296b304bd95a823b4a44b219b5ca2c7a422e898175d99a51a338ff56ce1caee427e67bbc62b3b36c9cb80e97730c
7
+ data.tar.gz: ee3e75992f0fd8503b96c08f0cfa0ab2e1879f8c5e729ca68948c9e17891634e4598b78089bcc63378bf4d487634422347d920bf70d3a8b097a1d5505e723f33
data/History.rdoc ADDED
@@ -0,0 +1,21 @@
1
+ === 0.0.3 / 2017-09-09
2
+
3
+ * A few enhancements
4
+
5
+ * Add most metrics aggregations
6
+ * Specify min Ruby version
7
+ * Documentation cleanup
8
+ * History catchup
9
+
10
+ === 0.0.2 / 2017-09-04
11
+
12
+ * 1 mintor enhancement
13
+
14
+ * Version bump
15
+
16
+ === 0.0.1 / 2017-08-31
17
+
18
+ * 1 major enhancement
19
+
20
+ * Birthday!
21
+
data/Manifest.txt CHANGED
@@ -1,8 +1,8 @@
1
1
  .autotest
2
2
  .rubocop.yml
3
- History.txt
3
+ History.rdoc
4
4
  Manifest.txt
5
- README.txt
5
+ README.rdoc
6
6
  Rakefile
7
7
  bin/console
8
8
  lib/es_tractor.rb
data/README.rdoc ADDED
@@ -0,0 +1,104 @@
1
+ = EsTractor: Simplified data extracton from Elasticsearch
2
+
3
+ home :: https://github.com/utilum/es_tractor
4
+ bugs :: https://github.com/utilum/es_tractor/issues
5
+ rdoc :: http://www.rubydoc.info/gems/es_tractor
6
+
7
+ == DESCRIPTION:
8
+
9
+ es_tractor provides a reduced, simplified and more DRY search API to
10
+ Elasticsearch. It aims to facilitate data extraction and preparation
11
+ without server scripting.
12
+
13
+ == FEATURES/PROBLEMS:
14
+
15
+ * Subset of Search APIs: count, search.
16
+ * Most metrics aggregations without optional parameters
17
+ * Minimal DRY query builder maps root elements of a Hash argument into boolean
18
+ filters.
19
+
20
+ * TODO:
21
+ * Aggregations:
22
+ * Bucket
23
+ * Nesting
24
+ * Optional parameters
25
+ * Add Search APIs:
26
+ * Fields
27
+ * Sort
28
+ * Scroll
29
+ * Report formats: CSV, JSON.
30
+ * Extraction: Aggregation to flat records (separate project?).
31
+
32
+ == SYNOPSIS:
33
+
34
+ include EsTractor
35
+ tractor = Client.new
36
+
37
+ tractor.count
38
+ # => (Integer) number of documents on the server
39
+
40
+ tractor.count(term: { my_field: 'my precise term' })
41
+ # => (Integer) number of documents where my_field == 'my precise term'
42
+
43
+ tractor.search(size: 1, term: { my_field: 'my precise term' })
44
+ # =>
45
+ # {
46
+ # "took"=>29,
47
+ # "timed_out"=>false,
48
+ # "_shards"=>{"total"=>542, "successful"=>542, "failed"=>0},
49
+ # "hits"=> {
50
+ # "total"=>279271,
51
+ # "max_score"=>0.0,
52
+ # "hits"=> [
53
+ # {
54
+ # "_index"=>"my_index",
55
+ # "_id"=>"bc596ff2-955b-11e7-a7d2-001ed3f963a9",
56
+ # "_score"=>1.6272357,
57
+ # "_source"=> {
58
+ # "my_field"=>"my precise term",
59
+ # "timestamp"=>"2017-09-09 12:38:09.263",
60
+ # },
61
+ # },
62
+ # ],
63
+ # },
64
+ # }
65
+
66
+ == REQUIREMENTS:
67
+
68
+ Some environment variables are expected:
69
+ * ES_TRACTOR_ELASTICSEARCH_HOST
70
+ * ES_TRACTOR_ELASTICSEARCH_INDEX
71
+
72
+ == DEVELOPERS:
73
+
74
+ After checking out the source, run:
75
+
76
+ $ rake newb
77
+
78
+ This task will install any missing dependencies, run the tests, and
79
+ generate the RDoc.
80
+
81
+ == LICENSE:
82
+
83
+ (The MIT License)
84
+
85
+ Copyright (c) 2017 FIX
86
+
87
+ Permission is hereby granted, free of charge, to any person obtaining
88
+ a copy of this software and associated documentation files (the
89
+ 'Software'), to deal in the Software without restriction, including
90
+ without limitation the rights to use, copy, modify, merge, publish,
91
+ distribute, sublicense, and/or sell copies of the Software, and to
92
+ permit persons to whom the Software is furnished to do so, subject to
93
+ the following conditions:
94
+
95
+ The above copyright notice and this permission notice shall be
96
+ included in all copies or substantial portions of the Software.
97
+
98
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
99
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
100
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
101
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
102
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
103
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
104
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/Rakefile CHANGED
@@ -7,10 +7,11 @@ require 'hoe'
7
7
  Hoe.plugin :yard
8
8
 
9
9
  Hoe.spec 'es_tractor' do
10
- developer('utilum', 'oz@utilum.com')
10
+ developer('oz', 'oz@utilum.com')
11
11
 
12
12
  license 'MIT'
13
13
 
14
+ require_ruby_version '~> 2.0'
14
15
  extra_deps << ['elasticsearch', '~> 5.0', '>= 5.0.4']
15
16
 
16
17
  extra_dev_deps << ['hoe-yard', '~> 0.1', '>=0.1.3']
@@ -89,6 +89,28 @@ module EsTractor
89
89
 
90
90
  private
91
91
 
92
+ def supported_aggs
93
+ metrics_aggs
94
+ end
95
+
96
+ def metrics_aggs
97
+ %i[avg cardinality extended_stats geo_bounds geo_centroid max min
98
+ percentiles stats sum value_count]
99
+ end
100
+
101
+ def aggs(opts)
102
+ aggregations = {}
103
+ (supported_aggs & opts.keys).each do |aggregation|
104
+ if metrics_aggs.include?(aggregation)
105
+ name = [aggregation, opts[aggregation]].join('-').to_sym
106
+ aggregations[name] = {
107
+ aggregation => { field: opts[aggregation] }
108
+ }
109
+ end
110
+ end
111
+ aggregations
112
+ end
113
+
92
114
  def array_or_hash(name, filter)
93
115
  case filter
94
116
  when Array
@@ -101,7 +123,9 @@ module EsTractor
101
123
  end
102
124
 
103
125
  def body(opts = {})
104
- { query: query(opts) }
126
+ body = { query: query(opts) }
127
+ body[:aggs] = aggs(opts) if (supported_aggs & opts.keys).any?
128
+ body
105
129
  end
106
130
 
107
131
  def query(opts = {})
data/lib/es_tractor.rb CHANGED
@@ -9,5 +9,5 @@ require 'client'
9
9
  # EsTractor provides tools to query Elasticsearch
10
10
 
11
11
  module EsTractor
12
- VERSION = '0.0.2'
12
+ VERSION = '0.0.3' # :nodoc:
13
13
  end
data/test/test_client.rb CHANGED
@@ -126,6 +126,39 @@ module EsTractor
126
126
 
127
127
  @tractor.send(action, opts)
128
128
  end
129
+
130
+ next unless action == 'search'
131
+
132
+ %i[avg cardinality extended_stats geo_bounds geo_centroid max min
133
+ percentiles stats sum value_count].each do |aggregation|
134
+ define_method "test_search_with_#{aggregation}_agg" do
135
+ opts = {
136
+ query_string: 'My query string',
137
+ aggregation => 'grade',
138
+ }
139
+ exp = {
140
+ from: 0,
141
+ size: 0,
142
+ body: {
143
+ query: {
144
+ bool: {
145
+ must: [query_string: { query: 'My query string' }],
146
+ filter: [],
147
+ },
148
+ },
149
+ aggs: {
150
+ "#{aggregation}-grade".to_sym => {
151
+ aggregation => { field: 'grade' }
152
+ },
153
+ },
154
+ },
155
+ }
156
+
157
+ @tractor.client.expects(:search).with(exp)
158
+
159
+ @tractor.search opts
160
+ end
161
+ end
129
162
  end
130
163
  end
131
164
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: es_tractor
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.2
4
+ version: 0.0.3
5
5
  platform: ruby
6
6
  authors:
7
- - utilum
7
+ - oz
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2017-09-04 00:00:00.000000000 Z
11
+ date: 2017-09-09 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: elasticsearch
@@ -112,22 +112,25 @@ dependencies:
112
112
  - - "~>"
113
113
  - !ruby/object:Gem::Version
114
114
  version: '3.16'
115
- description: Sub-set of Search APIs with DRY query building.
115
+ description: |-
116
+ es_tractor provides a reduced, simplified and more DRY search API to
117
+ Elasticsearch. It aims to facilitate data extraction and preparation
118
+ without server scripting.
116
119
  email:
117
120
  - oz@utilum.com
118
121
  executables:
119
122
  - console
120
123
  extensions: []
121
124
  extra_rdoc_files:
122
- - History.txt
125
+ - History.rdoc
123
126
  - Manifest.txt
124
- - README.txt
127
+ - README.rdoc
125
128
  files:
126
129
  - ".autotest"
127
130
  - ".rubocop.yml"
128
- - History.txt
131
+ - History.rdoc
129
132
  - Manifest.txt
130
- - README.txt
133
+ - README.rdoc
131
134
  - Rakefile
132
135
  - bin/console
133
136
  - lib/es_tractor.rb
@@ -135,7 +138,7 @@ files:
135
138
  - test/helper.rb
136
139
  - test/test_client.rb
137
140
  - test/test_es_tractor.rb
138
- homepage:
141
+ homepage: https://github.com/utilum/es_tractor
139
142
  licenses:
140
143
  - MIT
141
144
  metadata: {}
@@ -148,9 +151,9 @@ require_paths:
148
151
  - lib
149
152
  required_ruby_version: !ruby/object:Gem::Requirement
150
153
  requirements:
151
- - - ">="
154
+ - - "~>"
152
155
  - !ruby/object:Gem::Version
153
- version: '0'
156
+ version: '2.0'
154
157
  required_rubygems_version: !ruby/object:Gem::Requirement
155
158
  requirements:
156
159
  - - ">="
@@ -161,5 +164,5 @@ rubyforge_project:
161
164
  rubygems_version: 2.6.13
162
165
  signing_key:
163
166
  specification_version: 4
164
- summary: Sub-set of Search APIs with DRY query building.
167
+ summary: es_tractor provides a reduced, simplified and more DRY search API to Elasticsearch
165
168
  test_files: []
data/History.txt DELETED
@@ -1,12 +0,0 @@
1
- === 0.0.1 / 2017-08-31
2
-
3
- * 1 major enhancement
4
-
5
- * Birthday!
6
-
7
- === 0.0.2 / 2017-09-04
8
-
9
- * 1 major enhancement
10
-
11
- * Search, count, filter
12
-
data/README.txt DELETED
@@ -1,76 +0,0 @@
1
- # @markup markdown
2
- # EsTractor: Simplified data extracton from Elasticsearch
3
-
4
- [[Home]](https://github.com/utilum/es_tractor)
5
-
6
- ## DESCRIPTION:
7
-
8
- Sub-set of Search APIs with DRY query building.
9
-
10
- ## FEATURES/PROBLEMS:
11
-
12
- - Subset of Search APIs: count, search.
13
- - Minimal DRY query builder maps root elements of a Hash argument into boolean
14
- filters.
15
-
16
- - TODO:
17
- - Add Search APIs:
18
- - Fields
19
- - Sort
20
- - Scroll
21
- - Aggregations
22
- - Report formats: CSV, JSON.
23
- - Extraction: Aggregation to flat records (separate project?).
24
-
25
- ## SYNOPSIS:
26
-
27
- ```
28
- include EsTractor
29
- tractor = Client.new
30
-
31
- tractor.count
32
- # => (Integer) number of documents on the server
33
-
34
- tractor.count(term: { my_field: 'my precise term' })
35
- # => (Integer) number of documents where my_field == 'my precise term'
36
- ```
37
-
38
- ## REQUIREMENTS:
39
-
40
- Some environment variables are expected:
41
- - L2B_ELASTICSEARCH_HOST
42
- - L2B_ELASTICSEARCH_INDEX
43
-
44
- ## DEVELOPERS:
45
-
46
- After checking out the source, run:
47
-
48
- $ rake newb
49
-
50
- This task will install any missing dependencies, run the tests/specs,
51
- and generate the RDoc.
52
-
53
- ## LICENSE:
54
-
55
- (The MIT License)
56
-
57
- Copyright (c) 2017 FIX
58
-
59
- Permission is hereby granted, free of charge, to any person obtaining
60
- a copy of this software and associated documentation files (the
61
- 'Software'), to deal in the Software without restriction, including
62
- without limitation the rights to use, copy, modify, merge, publish,
63
- distribute, sublicense, and/or sell copies of the Software, and to
64
- permit persons to whom the Software is furnished to do so, subject to
65
- the following conditions:
66
-
67
- The above copyright notice and this permission notice shall be
68
- included in all copies or substantial portions of the Software.
69
-
70
- THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
71
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
72
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
73
- IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
74
- CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
75
- TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
76
- SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.