ruby-druid 0.10.0 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +41 -51
- data/lib/druid/data_source.rb +31 -4
- data/lib/druid/dimension.rb +2 -2
- data/lib/druid/query.rb +1 -1
- data/lib/druid/version.rb +1 -1
- data/spec/lib/data_source_spec.rb +26 -11
- metadata +4 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 62a8990d73fd742e2fbe2eb50bea8e6867d86e76
|
4
|
+
data.tar.gz: 5e17079e581ecce9e6a013b3e1dd44c25b8c2555
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: d08040c0aefb284858e323275fc86c18c7f99290f4463e6c309f172efa7086c43ff56d06869fe70c9563be423b86f6243ef203db06dcbaa9e4ade5515ff5328c
|
7
|
+
data.tar.gz: 3dcf38b1a373577940f2de02e44bf14252e09fc7a2c760f5c8be6d7d1e6b09c2f80412e4341491708759250e306ecf62ce8f5986db5468d5c87e6cd4a69db830
|
data/README.md
CHANGED
@@ -29,32 +29,22 @@ gem install ruby-druid
|
|
29
29
|
|
30
30
|
## Usage
|
31
31
|
|
32
|
-
|
33
|
-
Druid::Client.new('zk1:2181,zk2:2181/druid').query('service/source')
|
34
|
-
```
|
35
|
-
|
36
|
-
returns a query object on which all other methods can be called to create a full and valid Druid query.
|
37
|
-
|
38
|
-
A query object can be sent like this:
|
32
|
+
A query can be constructed and sent like so:
|
39
33
|
|
40
34
|
```ruby
|
41
|
-
|
42
|
-
query = Druid::Query.new(
|
43
|
-
|
35
|
+
data_source = Druid::Client.new('zk1:2181,zk2:2181/druid').data_source('service/source')
|
36
|
+
query = Druid::Query::Builder.new.long_sum(:aggregate1).last(1.day).granularity(:all)
|
37
|
+
result = data_source.post(query)
|
44
38
|
```
|
45
39
|
|
46
|
-
The `
|
40
|
+
The `post` method on the `DataSource` returns the parsed response from the Druid server as an array.
|
47
41
|
|
48
|
-
|
42
|
+
If you don't want to use ZooKeeper for broker discovery, you can explicitly construct a `DataSource`:
|
49
43
|
|
50
44
|
```ruby
|
51
|
-
|
45
|
+
data_source = Druid::DataSource.new('service/source', 'http://localhost:8080/druid/v2')
|
52
46
|
```
|
53
47
|
|
54
|
-
Supported options are:
|
55
|
-
* `static_setup` to explicitly specify a broker url, e.g. `static_setup: { 'my/source_name' => 'http://1.2.3.4:8080/druid/v2/' }`
|
56
|
-
* `http_timeout` to define a timeout for sending http queries to a broker (in minutes, default value is 2)
|
57
|
-
|
58
48
|
### GroupBy
|
59
49
|
|
60
50
|
A [GroupByQuery](http://druid.io/docs/latest/querying/groupbyquery.html) sets the
|
@@ -63,7 +53,7 @@ dimensions to group the data.
|
|
63
53
|
`queryType` is set automatically to `groupBy`.
|
64
54
|
|
65
55
|
```ruby
|
66
|
-
Druid::Query.new
|
56
|
+
Druid::Query::Builder.new.group_by([:dimension1, :dimension2])
|
67
57
|
```
|
68
58
|
|
69
59
|
### TimeSeries
|
@@ -71,7 +61,7 @@ Druid::Query.new('service/source').group_by([:dimension1, :dimension2])
|
|
71
61
|
A [TimeSeriesQuery](http://druid.io/docs/latest/querying/timeseriesquery.html) returns an array of JSON objects where each object represents a value asked for by the timeseries query.
|
72
62
|
|
73
63
|
```ruby
|
74
|
-
Druid::Query.new
|
64
|
+
Druid::Query::Builder.new.time_series([:aggregate1, :aggregate2])
|
75
65
|
```
|
76
66
|
|
77
67
|
### Aggregations
|
@@ -79,7 +69,7 @@ Druid::Query.new('service/source').time_series([:aggregate1, :aggregate2])
|
|
79
69
|
#### longSum, doubleSum, count, min, max, hyperUnique
|
80
70
|
|
81
71
|
```ruby
|
82
|
-
Druid::Query.new
|
72
|
+
Druid::Query::Builder.new.long_sum([:aggregate1, :aggregate2])
|
83
73
|
```
|
84
74
|
|
85
75
|
In the same way could be used the following methods for [aggregations](http://druid.io/docs/latest/querying/aggregations.html) adding: `double_sum, count, min, max, hyper_unique`
|
@@ -87,7 +77,7 @@ In the same way could be used the following methods for [aggregations](http://dr
|
|
87
77
|
#### cardinality
|
88
78
|
|
89
79
|
```ruby
|
90
|
-
Druid::Query.new
|
80
|
+
Druid::Query::Builder.new.cardinality(:aggregate, [:dimension1, dimension2], <by_row: true | false>)
|
91
81
|
```
|
92
82
|
|
93
83
|
#### javascript
|
@@ -95,7 +85,7 @@ Druid::Query.new('service/source').cardinality(:aggregate, [:dimension1, dimensi
|
|
95
85
|
For example calculation for `sum(log(x)/y) + 10`:
|
96
86
|
|
97
87
|
```ruby
|
98
|
-
Druid::Query.new
|
88
|
+
Druid::Query::Builder.new.js_aggregation(:aggregate, [:x, :y],
|
99
89
|
aggregate: "function(current, a, b) { return current + (Math.log(a) * b); }",
|
100
90
|
combine: "function(partialA, partialB) { return partialA + partialB; }",
|
101
91
|
reset: "function() { return 10; }"
|
@@ -107,7 +97,7 @@ Druid::Query.new('service/source').js_aggregation(:aggregate, [:x, :y],
|
|
107
97
|
A simple syntax for post aggregations with +,-,/,* can be used like:
|
108
98
|
|
109
99
|
```ruby
|
110
|
-
query = Druid::Query.new
|
100
|
+
query = Druid::Query::Builder.new.long_sum([:aggregate1, :aggregate2])
|
111
101
|
query.postagg { (aggregate2 + aggregate2).as output_field_name }
|
112
102
|
```
|
113
103
|
|
@@ -124,7 +114,7 @@ query.postagg { js('function(aggregate1, aggregate2) { return aggregate1 + aggre
|
|
124
114
|
The interval for the query takes a string with date and time or objects that provide an `iso8601` method.
|
125
115
|
|
126
116
|
```ruby
|
127
|
-
query = Druid::Query.new
|
117
|
+
query = Druid::Query::Builder.new.long_sum(:aggregate1)
|
128
118
|
query.interval("2013-01-01T00", Time.now)
|
129
119
|
```
|
130
120
|
|
@@ -139,14 +129,14 @@ The period `'day'` or `:day` will be interpreted as `'P1D'`.
|
|
139
129
|
If a period granularity is specifed, the (optional) second parameter is a time zone. It defaults to the machines local time zone. i.e.
|
140
130
|
|
141
131
|
```ruby
|
142
|
-
query = Druid::Query.new
|
132
|
+
query = Druid::Query::Builder.new.long_sum(:aggregate1)
|
143
133
|
query.granularity(:day)
|
144
134
|
```
|
145
135
|
|
146
136
|
is (on my box) the same as
|
147
137
|
|
148
138
|
```ruby
|
149
|
-
query = Druid::Query.new
|
139
|
+
query = Druid::Query::Builder.new.long_sum(:aggregate1)
|
150
140
|
query.granularity('P1D', 'Europe/Berlin')
|
151
141
|
```
|
152
142
|
|
@@ -154,18 +144,18 @@ query.granularity('P1D', 'Europe/Berlin')
|
|
154
144
|
|
155
145
|
```ruby
|
156
146
|
# equality
|
157
|
-
Druid::Query.new
|
147
|
+
Druid::Query::Builder.new.having { metric == 10 }
|
158
148
|
```
|
159
149
|
|
160
150
|
```ruby
|
161
151
|
# inequality
|
162
|
-
Druid::Query.new
|
152
|
+
Druid::Query::Builder.new.having { metric != 10 }
|
163
153
|
```
|
164
154
|
|
165
155
|
```ruby
|
166
156
|
# greater, less
|
167
|
-
Druid::Query.new
|
168
|
-
Druid::Query.new
|
157
|
+
Druid::Query::Builder.new.having { metric > 10 }
|
158
|
+
Druid::Query::Builder.new.having { metric < 10 }
|
169
159
|
```
|
170
160
|
|
171
161
|
#### Compound having filters
|
@@ -174,17 +164,17 @@ Having filters can be combined with boolean logic.
|
|
174
164
|
|
175
165
|
```ruby
|
176
166
|
# and
|
177
|
-
Druid::Query.new
|
167
|
+
Druid::Query::Builder.new.having { (metric != 1) & (metric2 != 2) }
|
178
168
|
```
|
179
169
|
|
180
170
|
```ruby
|
181
171
|
# or
|
182
|
-
Druid::Query.new
|
172
|
+
Druid::Query::Builder.new.having { (metric == 1) | (metric2 == 2) }
|
183
173
|
```
|
184
174
|
|
185
175
|
```ruby
|
186
176
|
# not
|
187
|
-
Druid::Query.new
|
177
|
+
Druid::Query::Builder.new.having{ !metric.eq(1) }
|
188
178
|
```
|
189
179
|
|
190
180
|
### Filters
|
@@ -197,27 +187,27 @@ Filters can be chained `filter{...}.filter{...}`
|
|
197
187
|
|
198
188
|
```ruby
|
199
189
|
# equality
|
200
|
-
Druid::Query.new
|
201
|
-
Druid::Query.new
|
190
|
+
Druid::Query::Builder.new.filter{dimension.eq 1}
|
191
|
+
Druid::Query::Builder.new.filter{dimension == 1}
|
202
192
|
```
|
203
193
|
|
204
194
|
```ruby
|
205
195
|
# inequality
|
206
|
-
Druid::Query.new
|
207
|
-
Druid::Query.new
|
196
|
+
Druid::Query::Builder.new.filter{dimension.neq 1}
|
197
|
+
Druid::Query::Builder.new.filter{dimension != 1}
|
208
198
|
```
|
209
199
|
|
210
200
|
```ruby
|
211
201
|
# greater, less
|
212
|
-
Druid::Query.new
|
213
|
-
Druid::Query.new
|
214
|
-
Druid::Query.new
|
215
|
-
Druid::Query.new
|
202
|
+
Druid::Query::Builder.new.filter{dimension > 1}
|
203
|
+
Druid::Query::Builder.new.filter{dimension >= 1}
|
204
|
+
Druid::Query::Builder.new.filter{dimension < 1}
|
205
|
+
Druid::Query::Builder.new.filter{dimension <= 1}
|
216
206
|
```
|
217
207
|
|
218
208
|
```ruby
|
219
209
|
# JavaScript
|
220
|
-
Druid::Query.new
|
210
|
+
Druid::Query::Builder.new.filter{a.javascript('dimension >= 1 && dimension < 5')}
|
221
211
|
```
|
222
212
|
|
223
213
|
#### Compound Filters
|
@@ -226,17 +216,17 @@ Filters can be combined with boolean logic.
|
|
226
216
|
|
227
217
|
```ruby
|
228
218
|
# and
|
229
|
-
Druid::Query.new
|
219
|
+
Druid::Query::Builder.new.filter{dimension.neq 1 & dimension2.neq 2}
|
230
220
|
```
|
231
221
|
|
232
222
|
```ruby
|
233
223
|
# or
|
234
|
-
Druid::Query.new
|
224
|
+
Druid::Query::Builder.new.filter{dimension.neq 1 | dimension2.neq 2}
|
235
225
|
```
|
236
226
|
|
237
227
|
```ruby
|
238
228
|
# not
|
239
|
-
Druid::Query.new
|
229
|
+
Druid::Query::Builder.new.filter{!dimension.eq(1)}
|
240
230
|
```
|
241
231
|
|
242
232
|
#### Inclusion Filter
|
@@ -244,18 +234,18 @@ Druid::Query.new('service/source').filter{!dimension.eq(1)}
|
|
244
234
|
This filter creates a set of equals filters in an or filter.
|
245
235
|
|
246
236
|
```ruby
|
247
|
-
Druid::Query.new
|
237
|
+
Druid::Query::Builder.new.filter{dimension.in(1,2,3)}
|
248
238
|
```
|
249
239
|
#### Geographic filter
|
250
240
|
|
251
241
|
These filters have to be combined with time_series and do only work when coordinates is a spatial dimension [GeographicQueries](http://druid.io/docs/latest/development/geo.html)
|
252
242
|
|
253
243
|
```ruby
|
254
|
-
Druid::Query.new
|
244
|
+
Druid::Query::Builder.new.time_series().long_sum([:aggregate1]).filter{coordinates.in_rec [[50.0,13.0],[54.0,15.0]]}
|
255
245
|
```
|
256
246
|
|
257
247
|
```ruby
|
258
|
-
Druid::Query.new
|
248
|
+
Druid::Query::Builder.new.time_series().long_sum([:aggregate1]).filter{coordinates.in_circ [[53.0,13.0], 5.0]}
|
259
249
|
```
|
260
250
|
|
261
251
|
#### Exclusion Filter
|
@@ -263,7 +253,7 @@ Druid::Query.new('service/source').time_series().long_sum([:aggregate1]).filter{
|
|
263
253
|
This filter creates a set of not-equals fitlers in an and filter.
|
264
254
|
|
265
255
|
```ruby
|
266
|
-
Druid::Query.new
|
256
|
+
Druid::Query::Builder.new.filter{dimension.nin(1,2,3)}
|
267
257
|
```
|
268
258
|
|
269
259
|
#### Hash syntax
|
@@ -271,9 +261,9 @@ Druid::Query.new('service/source').filter{dimension.nin(1,2,3)}
|
|
271
261
|
Sometimes it can be useful to use a hash syntax for filtering for example if you already get them from a list or parameter hash.
|
272
262
|
|
273
263
|
```ruby
|
274
|
-
Druid::Query.new
|
264
|
+
Druid::Query::Builder.new.filter{dimension => 1, dimension1 =>2, dimension2 => 3}
|
275
265
|
# which is equivalent to
|
276
|
-
Druid::Query.new
|
266
|
+
Druid::Query::Builder.new.filter{dimension.eq(1) & dimension1.eq(2) & dimension2.eq(3)}
|
277
267
|
```
|
278
268
|
|
279
269
|
## Contributing
|
data/lib/druid/data_source.rb
CHANGED
@@ -74,22 +74,49 @@ module Druid
|
|
74
74
|
return self.post(query)
|
75
75
|
end
|
76
76
|
|
77
|
-
raise Error.new(response)
|
77
|
+
raise Error.new(response)
|
78
78
|
end
|
79
79
|
|
80
80
|
MultiJson.load(response.body)
|
81
81
|
end
|
82
82
|
|
83
83
|
class Error < StandardError
|
84
|
-
attr_reader :response
|
84
|
+
attr_reader :error, :error_message, :error_class, :host, :response
|
85
|
+
|
85
86
|
def initialize(response)
|
86
87
|
@response = response
|
88
|
+
parsed_body = MultiJson.load(response.body)
|
89
|
+
@error, @error_message, @error_class, @host = parsed_body.values_at(*%w(
|
90
|
+
error
|
91
|
+
errorMessage
|
92
|
+
errorClass
|
93
|
+
host
|
94
|
+
))
|
87
95
|
end
|
88
96
|
|
89
97
|
def message
|
90
|
-
|
98
|
+
error
|
99
|
+
end
|
100
|
+
|
101
|
+
def query_timeout?
|
102
|
+
error == 'Query timeout'.freeze
|
103
|
+
end
|
104
|
+
|
105
|
+
def query_interrupted?
|
106
|
+
error == 'Query interrupted'.freeze
|
91
107
|
end
|
92
|
-
end
|
93
108
|
|
109
|
+
def query_cancelled?
|
110
|
+
error == 'Query cancelled'.freeze
|
111
|
+
end
|
112
|
+
|
113
|
+
def resource_limit_exceeded?
|
114
|
+
error == 'Resource limit exceeded'.freeze
|
115
|
+
end
|
116
|
+
|
117
|
+
def unknown_exception?
|
118
|
+
error == 'Unknown exception'.freeze
|
119
|
+
end
|
120
|
+
end
|
94
121
|
end
|
95
122
|
end
|
data/lib/druid/dimension.rb
CHANGED
@@ -37,11 +37,11 @@ module Druid
|
|
37
37
|
super(options.merge(except: %w(errors validation_context)))
|
38
38
|
end
|
39
39
|
|
40
|
-
def self.lookup(dimension, namespace, retain: true, injective: false)
|
40
|
+
def self.lookup(dimension, namespace, outputName: nil, retain: true, injective: false)
|
41
41
|
new({
|
42
42
|
type: 'extraction',
|
43
43
|
dimension: dimension,
|
44
|
-
outputName:
|
44
|
+
outputName: outputName || namespace,
|
45
45
|
extractionFn: {
|
46
46
|
type: 'registeredLookup',
|
47
47
|
lookup: namespace,
|
data/lib/druid/query.rb
CHANGED
@@ -365,7 +365,7 @@ module Druid
|
|
365
365
|
|
366
366
|
def group_by(*dimensions)
|
367
367
|
query_type(:groupBy)
|
368
|
-
@query.dimensions = dimensions.map do |dimension|
|
368
|
+
@query.dimensions = dimensions.flatten.map do |dimension|
|
369
369
|
dimension.is_a?(Dimension) ? dimension : Dimension.new(dimension)
|
370
370
|
end
|
371
371
|
self
|
data/lib/druid/version.rb
CHANGED
@@ -19,20 +19,35 @@ describe Druid::DataSource do
|
|
19
19
|
end
|
20
20
|
|
21
21
|
it 'raises on request failure' do
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
:headers => {
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
22
|
+
stub_request(:post, 'http://www.example.com/druid/v2')
|
23
|
+
.with(
|
24
|
+
:body => %q({"context":{"queryId":null},"queryType":"timeseries","intervals":["2013-04-04T00:00:00+00:00/2013-04-04T00:00:00+00:00"],"granularity":"all","dataSource":"test"}),
|
25
|
+
:headers => {
|
26
|
+
'Accept' => '*/*',
|
27
|
+
'Accept-Encoding' => 'gzip;q=1.0,deflate;q=0.6,identity;q=0.3',
|
28
|
+
'Content-Type' => 'application/json',
|
29
|
+
'User-Agent' => 'Ruby',
|
30
|
+
}
|
31
|
+
)
|
32
|
+
.to_return(
|
33
|
+
:status => 500,
|
34
|
+
:body => %q({"error":"Unknown exception","errorMessage":"NullPointerException","errorClass":"java.lang.NullPointerException","host":"www.example.com"}),
|
35
|
+
:headers => {},
|
36
|
+
)
|
37
|
+
|
32
38
|
ds = Druid::DataSource.new('test/test', 'http://www.example.com/druid/v2')
|
33
39
|
query = Druid::Query::Builder.new.interval('2013-04-04', '2013-04-04').granularity(:all).query
|
34
40
|
query.context.queryId = nil
|
35
|
-
|
41
|
+
|
42
|
+
expect { ds.post(query) }.to raise_error { |error|
|
43
|
+
expect(error).to be_a(Druid::DataSource::Error)
|
44
|
+
expect(error.message).to eq(error.error)
|
45
|
+
expect(error.error).to eq('Unknown exception')
|
46
|
+
expect(error.error_message).to eq('NullPointerException')
|
47
|
+
expect(error.error_class).to eq('java.lang.NullPointerException')
|
48
|
+
expect(error.host).to eq('www.example.com')
|
49
|
+
expect(error).to be_unknown_exception
|
50
|
+
}
|
36
51
|
end
|
37
52
|
end
|
38
53
|
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: ruby-druid
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.10.
|
4
|
+
version: 0.10.1
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Ruby Druid Community
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2018-02-26 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: activesupport
|
@@ -211,13 +211,13 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
211
211
|
version: '0'
|
212
212
|
requirements: []
|
213
213
|
rubyforge_project:
|
214
|
-
rubygems_version: 2.
|
214
|
+
rubygems_version: 2.6.11
|
215
215
|
signing_key:
|
216
216
|
specification_version: 4
|
217
217
|
summary: A Ruby client for Druid
|
218
218
|
test_files:
|
219
219
|
- spec/spec_helper.rb
|
220
220
|
- spec/lib/client_spec.rb
|
221
|
-
- spec/lib/data_source_spec.rb
|
222
221
|
- spec/lib/zk_spec.rb
|
222
|
+
- spec/lib/data_source_spec.rb
|
223
223
|
- spec/lib/query_spec.rb
|