presto-client 0.6.2 → 0.6.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.github/PULL_REQUEST_TEMPLATE.md +18 -0
- data/.travis.yml +4 -2
- data/ChangeLog.md +6 -0
- data/Gemfile +4 -0
- data/LICENSE +202 -0
- data/README.md +19 -6
- data/lib/presto/client/query.rb +1 -4
- data/lib/presto/client/statement_client.rb +36 -29
- data/lib/presto/client/version.rb +1 -1
- data/release.rb +0 -2
- data/spec/basic_query_spec.rb +82 -0
- data/spec/spec_helper.rb +24 -0
- data/spec/statement_client_spec.rb +25 -2
- data/spec/tpch/q01.sql +21 -0
- data/spec/tpch/q02.sql +43 -0
- data/spec/tpch_query_spec.rb +41 -0
- metadata +13 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 45c1f4293f282e8926a9aba438e42d15b0398a06f6f868dd467a1f905874e88e
|
4
|
+
data.tar.gz: add5e079212611da30f8bd66a264bc3586c5ee8c2e66d66b8bca17e03c1b0ad0
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 8198a307d376bc8e81030aa296b11dc77fed2390f9abf69743a4d3a92af5ac0a8101af1c0cd38386bf05f55117ce15a203ecc64a1e68b0f260fade9ebd26142f
|
7
|
+
data.tar.gz: d2ff31ccc7a069a0a1fc5de6236bcfdb3d5b43efaebaec56d49cda988ca9805aca9d6140a25d31ae49f45144a812e3f731942742fb6b395521df7c7371acd91b
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# Purpose
|
2
|
+
|
3
|
+
Briefly describe the purpose of the pull request.
|
4
|
+
|
5
|
+
# Overview
|
6
|
+
|
7
|
+
Describe the list of work you have done in this pull request.
|
8
|
+
|
9
|
+
Example:
|
10
|
+
- Add something
|
11
|
+
- Remove other things
|
12
|
+
|
13
|
+
# Checklist
|
14
|
+
|
15
|
+
- [ ] Code compiles correctly
|
16
|
+
- [ ] Created tests which fail without the change (if possible)
|
17
|
+
- [ ] All tests passing
|
18
|
+
- [ ] Extended the README / documentation, if necessary
|
data/.travis.yml
CHANGED
data/ChangeLog.md
CHANGED
@@ -1,5 +1,11 @@
|
|
1
1
|
presto-client-ruby
|
2
2
|
====
|
3
|
+
## 0.6.3
|
4
|
+
- Merge pull request [#56](https://github.com/treasure-data/presto-client-ruby/issues/56) from miniway/more_recent_statementclient [[423ff42](https://github.com/treasure-data/presto-client-ruby/commit/423ff42)]
|
5
|
+
- Make more similar to recent java StatementClient [[94f5193](https://github.com/treasure-data/presto-client-ruby/commit/94f5193)]
|
6
|
+
- Test with presto docker container ([#62](https://github.com/treasure-data/presto-client-ruby/issues/62)) [[fe8af11](https://github.com/treasure-data/presto-client-ruby/commit/fe8af11)]
|
7
|
+
- Add pull request template [[7b286cc](https://github.com/treasure-data/presto-client-ruby/commit/7b286cc)]
|
8
|
+
|
3
9
|
## 0.6.2
|
4
10
|
- Release 0.6.2 [[68f5b27](https://github.com/treasure-data/presto-client-ruby/commit/68f5b27)]
|
5
11
|
- Merge pull request [#61](https://github.com/treasure-data/presto-client-ruby/issues/61) from treasure-data/implement_query_id [[6f3f236](https://github.com/treasure-data/presto-client-ruby/commit/6f3f236)]
|
data/Gemfile
CHANGED
data/LICENSE
ADDED
@@ -0,0 +1,202 @@
|
|
1
|
+
|
2
|
+
Apache License
|
3
|
+
Version 2.0, January 2004
|
4
|
+
http://www.apache.org/licenses/
|
5
|
+
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
7
|
+
|
8
|
+
1. Definitions.
|
9
|
+
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
12
|
+
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
14
|
+
the copyright owner that is granting the License.
|
15
|
+
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
17
|
+
other entities that control, are controlled by, or are under common
|
18
|
+
control with that entity. For the purposes of this definition,
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
20
|
+
direction or management of such entity, whether by contract or
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
23
|
+
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
25
|
+
exercising permissions granted by this License.
|
26
|
+
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
28
|
+
including but not limited to software source code, documentation
|
29
|
+
source, and configuration files.
|
30
|
+
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
32
|
+
transformation or translation of a Source form, including but
|
33
|
+
not limited to compiled object code, generated documentation,
|
34
|
+
and conversions to other media types.
|
35
|
+
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
37
|
+
Object form, made available under the License, as indicated by a
|
38
|
+
copyright notice that is included in or attached to the work
|
39
|
+
(an example is provided in the Appendix below).
|
40
|
+
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
47
|
+
the Work and Derivative Works thereof.
|
48
|
+
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
50
|
+
the original version of the Work and any modifications or additions
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
62
|
+
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
65
|
+
subsequently incorporated within the Work.
|
66
|
+
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
73
|
+
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
79
|
+
where such license applies only to those patent claims licensable
|
80
|
+
by such Contributor that are necessarily infringed by their
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
83
|
+
institute patent litigation against any entity (including a
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
86
|
+
or contributory patent infringement, then any patent licenses
|
87
|
+
granted to You under this License for that Work shall terminate
|
88
|
+
as of the date such litigation is filed.
|
89
|
+
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
92
|
+
modifications, and in Source or Object form, provided that You
|
93
|
+
meet the following conditions:
|
94
|
+
|
95
|
+
(a) You must give any other recipients of the Work or
|
96
|
+
Derivative Works a copy of this License; and
|
97
|
+
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
99
|
+
stating that You changed the files; and
|
100
|
+
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
103
|
+
attribution notices from the Source form of the Work,
|
104
|
+
excluding those notices that do not pertain to any part of
|
105
|
+
the Derivative Works; and
|
106
|
+
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
109
|
+
include a readable copy of the attribution notices contained
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
112
|
+
of the following places: within a NOTICE text file distributed
|
113
|
+
as part of the Derivative Works; within the Source form or
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
115
|
+
within a display generated by the Derivative Works, if and
|
116
|
+
wherever such third-party notices normally appear. The contents
|
117
|
+
of the NOTICE file are for informational purposes only and
|
118
|
+
do not modify the License. You may add Your own attribution
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
121
|
+
that such additional attribution notices cannot be construed
|
122
|
+
as modifying the License.
|
123
|
+
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
125
|
+
may provide additional or different license terms and conditions
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
129
|
+
the conditions stated in this License.
|
130
|
+
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
134
|
+
this License, without any additional terms or conditions.
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
136
|
+
the terms of any separate license agreement you may have executed
|
137
|
+
with Licensor regarding such Contributions.
|
138
|
+
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
141
|
+
except as required for reasonable and customary use in describing the
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
143
|
+
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
153
|
+
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
159
|
+
incidental, or consequential damages of any character arising as a
|
160
|
+
result of this License or out of the use or inability to use the
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
163
|
+
other commercial damages or losses), even if such Contributor
|
164
|
+
has been advised of the possibility of such damages.
|
165
|
+
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
169
|
+
or other liability obligations and/or rights consistent with this
|
170
|
+
License. However, in accepting such obligations, You may act only
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
175
|
+
of your accepting any such warranty or additional liability.
|
176
|
+
|
177
|
+
END OF TERMS AND CONDITIONS
|
178
|
+
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
180
|
+
|
181
|
+
To apply the Apache License to your work, attach the following
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
183
|
+
replaced with your own identifying information. (Don't include
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
185
|
+
comment syntax for the file format. We also recommend that a
|
186
|
+
file or class name and description of purpose be included on the
|
187
|
+
same "printed page" as the copyright notice for easier
|
188
|
+
identification within third-party archives.
|
189
|
+
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
191
|
+
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
193
|
+
you may not use this file except in compliance with the License.
|
194
|
+
You may obtain a copy of the License at
|
195
|
+
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
197
|
+
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
201
|
+
See the License for the specific language governing permissions and
|
202
|
+
limitations under the License.
|
data/README.md
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# Presto client library for Ruby
|
2
2
|
|
3
|
-
[![Build Status](https://travis-ci.org/treasure-data/presto-client-ruby.svg?branch=master)](https://travis-ci.org/treasure-data/presto-client-ruby)
|
3
|
+
[![Build Status](https://travis-ci.org/treasure-data/presto-client-ruby.svg?branch=master)](https://travis-ci.org/treasure-data/presto-client-ruby) [![Gem](https://img.shields.io/gem/v/presto-client)](https://rubygems.org/gems/presto-client) [![Gem](https://img.shields.io/gem/dt/presto-client)](https://rubygems.org/gems/presto-client) [![GitHub](https://img.shields.io/github/license/treasure-data/presto-client-ruby)]()
|
4
4
|
|
5
5
|
Presto is a distributed SQL query engine for big data:
|
6
6
|
https://prestosql.io/
|
@@ -109,8 +109,21 @@ See [RDoc](http://www.rubydoc.info/gems/presto-client/) for the full documentati
|
|
109
109
|
|
110
110
|
### Releasing a new version
|
111
111
|
|
112
|
-
1.
|
113
|
-
2.
|
114
|
-
|
115
|
-
|
116
|
-
|
112
|
+
1. First update `lib/presto/client/version.rb` to the next version.
|
113
|
+
2. Run the following command which will update `ChangeLog.md` file automatically.
|
114
|
+
```
|
115
|
+
$ ruby release.rb
|
116
|
+
```
|
117
|
+
|
118
|
+
3. Create tag
|
119
|
+
```
|
120
|
+
$ git commit -am "vX.Y.Z"
|
121
|
+
$ git tag "vX.Y.Z"
|
122
|
+
% git push --tags
|
123
|
+
```
|
124
|
+
|
125
|
+
4. Push package
|
126
|
+
```
|
127
|
+
$ gem build presto-client.gemspec
|
128
|
+
$ gem push presto-client-X.Y.Z.gem
|
129
|
+
```
|
data/lib/presto/client/query.rb
CHANGED
@@ -130,11 +130,8 @@ module Presto::Client
|
|
130
130
|
end
|
131
131
|
|
132
132
|
def raise_if_failed
|
133
|
-
if @api.
|
133
|
+
if @api.client_aborted?
|
134
134
|
raise PrestoClientError, "Query aborted by user"
|
135
|
-
elsif @api.exception?
|
136
|
-
# query is gone
|
137
|
-
raise @api.exception
|
138
135
|
elsif @api.query_failed?
|
139
136
|
results = @api.current_results
|
140
137
|
error = results.error
|
@@ -31,8 +31,7 @@ module Presto::Client
|
|
31
31
|
|
32
32
|
@options = options
|
33
33
|
@query = query
|
34
|
-
@
|
35
|
-
@exception = nil
|
34
|
+
@state = :running
|
36
35
|
@retry_timeout = options[:retry_timeout] || 120
|
37
36
|
if model_version = @options[:model_version]
|
38
37
|
@models = ModelVersions.const_get("V#{model_version.gsub(".", "_")}")
|
@@ -76,7 +75,7 @@ module Presto::Client
|
|
76
75
|
|
77
76
|
# TODO error handling
|
78
77
|
if response.status != 200
|
79
|
-
|
78
|
+
exception! PrestoHttpError.new(response.status, "Failed to start query: #{response.body} (#{response.status})")
|
80
79
|
end
|
81
80
|
|
82
81
|
@results_headers = response.headers
|
@@ -91,14 +90,20 @@ module Presto::Client
|
|
91
90
|
!!@options[:debug]
|
92
91
|
end
|
93
92
|
|
94
|
-
def
|
95
|
-
@
|
93
|
+
def running?
|
94
|
+
@state == :running
|
96
95
|
end
|
97
96
|
|
98
|
-
|
97
|
+
def client_aborted?
|
98
|
+
@state == :client_aborted
|
99
|
+
end
|
100
|
+
|
101
|
+
def client_error?
|
102
|
+
@state == :client_error
|
103
|
+
end
|
99
104
|
|
100
|
-
def
|
101
|
-
@
|
105
|
+
def finished?
|
106
|
+
@state == :finished
|
102
107
|
end
|
103
108
|
|
104
109
|
def query_failed?
|
@@ -106,7 +111,7 @@ module Presto::Client
|
|
106
111
|
end
|
107
112
|
|
108
113
|
def query_succeeded?
|
109
|
-
@results.error == nil &&
|
114
|
+
@results.error == nil && finished?
|
110
115
|
end
|
111
116
|
|
112
117
|
def current_results
|
@@ -125,12 +130,21 @@ module Presto::Client
|
|
125
130
|
!!@results.next_uri
|
126
131
|
end
|
127
132
|
|
133
|
+
def exception!(e)
|
134
|
+
@state = :client_error
|
135
|
+
raise e
|
136
|
+
end
|
137
|
+
|
128
138
|
def advance
|
129
|
-
|
139
|
+
return false unless running?
|
140
|
+
|
141
|
+
unless has_next?
|
142
|
+
@state = :finished
|
130
143
|
return false
|
131
144
|
end
|
132
145
|
|
133
146
|
uri = @results.next_uri
|
147
|
+
|
134
148
|
response = faraday_get_with_retry(uri)
|
135
149
|
@results_headers = response.headers
|
136
150
|
@results = decode_model(uri, parse_body(response), @models::QueryResults)
|
@@ -154,8 +168,7 @@ module Presto::Client
|
|
154
168
|
if body.size > 1024 + 3
|
155
169
|
body = "#{body[0, 1024]}..."
|
156
170
|
end
|
157
|
-
|
158
|
-
raise @exception
|
171
|
+
exception! PrestoHttpError.new(500, "Presto API returned unexpected structure at #{uri}. Expected #{body_class} but got #{body}: #{e}")
|
159
172
|
end
|
160
173
|
end
|
161
174
|
|
@@ -170,8 +183,7 @@ module Presto::Client
|
|
170
183
|
JSON.parse(response.body, opts = JSON_OPTIONS)
|
171
184
|
end
|
172
185
|
rescue => e
|
173
|
-
|
174
|
-
raise @exception
|
186
|
+
exception! PrestoHttpError.new(500, "Presto API returned unexpected data format. #{e}")
|
175
187
|
end
|
176
188
|
end
|
177
189
|
|
@@ -188,8 +200,7 @@ module Presto::Client
|
|
188
200
|
# temporally error to retry
|
189
201
|
response = nil
|
190
202
|
rescue => e
|
191
|
-
|
192
|
-
raise @exception
|
203
|
+
exception! e
|
193
204
|
end
|
194
205
|
|
195
206
|
if response
|
@@ -199,8 +210,7 @@ module Presto::Client
|
|
199
210
|
|
200
211
|
if response.status != 503 # retry only if 503 Service Unavailable
|
201
212
|
# deterministic error
|
202
|
-
|
203
|
-
raise @exception
|
213
|
+
exception! PrestoHttpError.new(response.status, "Presto API error at #{uri} returned #{response.status}: #{response.body}")
|
204
214
|
end
|
205
215
|
end
|
206
216
|
|
@@ -208,18 +218,14 @@ module Presto::Client
|
|
208
218
|
|
209
219
|
attempts += 1
|
210
220
|
sleep attempts * 0.1
|
211
|
-
end while (Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) < @retry_timeout &&
|
221
|
+
end while (Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) < @retry_timeout && !client_aborted?
|
212
222
|
|
213
|
-
|
214
|
-
raise @exception
|
223
|
+
exception! PrestoHttpError.new(408, "Presto API error due to timeout")
|
215
224
|
end
|
216
225
|
|
217
226
|
def raise_if_timeout!
|
218
227
|
if @started_at
|
219
|
-
if
|
220
|
-
# query is already done
|
221
|
-
return
|
222
|
-
end
|
228
|
+
return if finished?
|
223
229
|
|
224
230
|
elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - @started_at
|
225
231
|
|
@@ -238,9 +244,9 @@ module Presto::Client
|
|
238
244
|
|
239
245
|
def raise_timeout_error!
|
240
246
|
if query_id = @results && @results.id
|
241
|
-
|
247
|
+
exception! PrestoQueryTimeoutError.new("Query #{query_id} timed out")
|
242
248
|
else
|
243
|
-
|
249
|
+
exception! PrestoQueryTimeoutError.new("Query timed out")
|
244
250
|
end
|
245
251
|
end
|
246
252
|
|
@@ -253,7 +259,9 @@ module Presto::Client
|
|
253
259
|
end
|
254
260
|
|
255
261
|
def close
|
256
|
-
return
|
262
|
+
return unless running?
|
263
|
+
|
264
|
+
@state = :client_aborted
|
257
265
|
|
258
266
|
begin
|
259
267
|
if uri = @results.next_uri
|
@@ -264,7 +272,6 @@ module Presto::Client
|
|
264
272
|
rescue => e
|
265
273
|
end
|
266
274
|
|
267
|
-
@closed = true
|
268
275
|
nil
|
269
276
|
end
|
270
277
|
end
|
data/release.rb
CHANGED
@@ -17,8 +17,6 @@ abort("Can't use empty version string") if next_version.empty?
|
|
17
17
|
logs = `git log #{last_tag}..HEAD --pretty=format:'%h %s'`
|
18
18
|
# Add links to GitHub issues
|
19
19
|
logs = logs.gsub(/\#([0-9]+)/, "[#\\1](#{PREFIX}/issues/\\1)")
|
20
|
-
# Add links to JIRA tickets (e.g,. MPP-xxxx, PLT-xxxx)
|
21
|
-
logs = logs.gsub(/^([A-Z]{2,}+-[0-9]+):/, '[\1](https://treasure-data.atlassian.net/browse/\1):')
|
22
20
|
|
23
21
|
new_release_notes = []
|
24
22
|
new_release_notes <<= "\#\# #{next_version}\n"
|
@@ -0,0 +1,82 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Presto::Client::Client do
|
4
|
+
before(:all) do
|
5
|
+
WebMock.disable!
|
6
|
+
@cluster = TinyPresto::Cluster.new('316')
|
7
|
+
@container = @cluster.run
|
8
|
+
@client = Presto::Client.new(server: 'localhost:8080', catalog: 'memory', user: 'test-user', schema: 'default')
|
9
|
+
loop do
|
10
|
+
begin
|
11
|
+
@client.run('show schemas')
|
12
|
+
break
|
13
|
+
rescue StandardError => exception
|
14
|
+
puts "Waiting for cluster ready... #{exception}"
|
15
|
+
sleep(3)
|
16
|
+
end
|
17
|
+
end
|
18
|
+
puts 'Cluster is ready'
|
19
|
+
end
|
20
|
+
|
21
|
+
after(:all) do
|
22
|
+
@cluster.stop
|
23
|
+
WebMock.enable!
|
24
|
+
end
|
25
|
+
|
26
|
+
it 'show schemas' do
|
27
|
+
columns, rows = run_with_retry(@client, 'show schemas')
|
28
|
+
expect(columns.length).to be(1)
|
29
|
+
expect(rows.length).to be(2)
|
30
|
+
end
|
31
|
+
|
32
|
+
it 'ctas' do
|
33
|
+
expected = [[1, 'a'], [2, 'b']]
|
34
|
+
run_with_retry(@client, "create table ctas1 as select * from (values (1, 'a'), (2, 'b')) t(c1, c2)")
|
35
|
+
columns, rows = run_with_retry(@client, 'select * from ctas1')
|
36
|
+
expect(columns.map(&:name)).to match_array(%w[c1 c2])
|
37
|
+
expect(rows).to eq(expected)
|
38
|
+
end
|
39
|
+
|
40
|
+
it 'next_uri' do
|
41
|
+
@client.query('show schemas') do |q|
|
42
|
+
expect(q.next_uri).to start_with('http://localhost:8080/v1/statement/')
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
it 'advance' do
|
47
|
+
@client.query('show schemas') do |q|
|
48
|
+
expect(q.advance).to be(true)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
|
52
|
+
it 'current query result' do
|
53
|
+
@client.query('show schemas') do |q|
|
54
|
+
expect(q.current_results.info_uri).to start_with('http://localhost:8080/ui/query.html')
|
55
|
+
end
|
56
|
+
end
|
57
|
+
|
58
|
+
it 'statement stats' do
|
59
|
+
@client.query('show schemas') do |q|
|
60
|
+
stats = q.current_results.stats
|
61
|
+
# Immediate subsequent request should get queued result
|
62
|
+
expect(stats.queued).to be(true)
|
63
|
+
expect(stats.scheduled).to be(false)
|
64
|
+
end
|
65
|
+
end
|
66
|
+
|
67
|
+
it 'partial cancel' do
|
68
|
+
@client.query('show schemas') do |q|
|
69
|
+
q.cancel
|
70
|
+
expect { q.query_info }.to raise_error(Presto::Client::PrestoHttpError, /Error 410 Gone/)
|
71
|
+
end
|
72
|
+
end
|
73
|
+
|
74
|
+
it 'row chunk' do
|
75
|
+
expected_schemas = %w[default information_schema]
|
76
|
+
@client.query('show schemas') do |q|
|
77
|
+
q.each_row do |r|
|
78
|
+
expect(expected_schemas).to include(r[0])
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
82
|
+
end
|
data/spec/spec_helper.rb
CHANGED
@@ -16,3 +16,27 @@ require 'webmock/rspec'
|
|
16
16
|
|
17
17
|
require 'presto-client'
|
18
18
|
include Presto::Client
|
19
|
+
|
20
|
+
require 'tiny-presto'
|
21
|
+
|
22
|
+
MAX_RETRY_COUNT = 5
|
23
|
+
RETRYABLE_ERRORS = [
|
24
|
+
/No nodes available to run query/
|
25
|
+
]
|
26
|
+
|
27
|
+
def run_with_retry(client, sql)
|
28
|
+
i = 0
|
29
|
+
while i < MAX_RETRY_COUNT
|
30
|
+
begin
|
31
|
+
columns, rows = @client.run(sql)
|
32
|
+
return columns, rows
|
33
|
+
rescue Presto::Client::PrestoQueryError => e
|
34
|
+
if RETRYABLE_ERRORS.any? { |error| e.message =~ error }
|
35
|
+
sleep(i)
|
36
|
+
i += 1
|
37
|
+
next
|
38
|
+
end
|
39
|
+
raise "Fail to run query: #{e}"
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
@@ -138,6 +138,27 @@ describe Presto::Client::StatementClient do
|
|
138
138
|
})
|
139
139
|
end.should raise_error(TypeError, /String to Hash/)
|
140
140
|
end
|
141
|
+
else
|
142
|
+
it "decodes DeleteTarget" do
|
143
|
+
dh = Models::DeleteTarget.decode({
|
144
|
+
"handle" => {
|
145
|
+
"catalogName" => "c1",
|
146
|
+
"connectorHandle" => {}
|
147
|
+
}
|
148
|
+
})
|
149
|
+
dh.handle.should be_a_kind_of Models::TableHandle
|
150
|
+
dh.handle.catalog_name.should == "c1"
|
151
|
+
dh.handle.connector_handle.should == {}
|
152
|
+
end
|
153
|
+
|
154
|
+
it "validates models" do
|
155
|
+
lambda do
|
156
|
+
Models::DeleteTarget.decode({
|
157
|
+
"catalogName" => "c1",
|
158
|
+
"handle" => "invalid"
|
159
|
+
})
|
160
|
+
end.should raise_error(TypeError, /String to Hash/)
|
161
|
+
end
|
141
162
|
end
|
142
163
|
|
143
164
|
it "receives headers of POST" do
|
@@ -588,11 +609,13 @@ describe Presto::Client::StatementClient do
|
|
588
609
|
to_return(body: early_running_response.to_json)
|
589
610
|
client.advance
|
590
611
|
|
591
|
-
sleep 1
|
592
612
|
stub_request(:get, "localhost/v1/next_uri").
|
593
613
|
with(headers: headers).
|
594
614
|
to_return(body: done_response.to_json)
|
595
|
-
client.advance
|
615
|
+
client.advance # set finished
|
616
|
+
|
617
|
+
sleep 1
|
618
|
+
client.advance # set finished
|
596
619
|
end
|
597
620
|
|
598
621
|
end
|
data/spec/tpch/q01.sql
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
SELECT
|
2
|
+
l.returnflag,
|
3
|
+
l.linestatus,
|
4
|
+
sum(l.quantity) AS sum_qty,
|
5
|
+
sum(l.extendedprice) AS sum_base_price,
|
6
|
+
sum(l.extendedprice * (1 - l.discount)) AS sum_disc_price,
|
7
|
+
sum(l.extendedprice * (1 - l.discount) * (1 + l.tax)) AS sum_charge,
|
8
|
+
avg(l.quantity) AS avg_qty,
|
9
|
+
avg(l.extendedprice) AS avg_price,
|
10
|
+
avg(l.discount) AS avg_disc,
|
11
|
+
count(*) AS count_order
|
12
|
+
FROM
|
13
|
+
"tpch"."tiny"."lineitem" AS l
|
14
|
+
WHERE
|
15
|
+
l.shipdate <= DATE '1998-12-01' - INTERVAL '90' DAY
|
16
|
+
GROUP BY
|
17
|
+
l.returnflag,
|
18
|
+
l.linestatus
|
19
|
+
ORDER BY
|
20
|
+
l.returnflag,
|
21
|
+
l.linestatus
|
data/spec/tpch/q02.sql
ADDED
@@ -0,0 +1,43 @@
|
|
1
|
+
SELECT
|
2
|
+
s.acctbal,
|
3
|
+
s.name,
|
4
|
+
n.name,
|
5
|
+
p.partkey,
|
6
|
+
p.mfgr,
|
7
|
+
s.address,
|
8
|
+
s.phone,
|
9
|
+
s.comment
|
10
|
+
FROM
|
11
|
+
"tpch"."tiny"."part" p,
|
12
|
+
"tpch"."tiny"."supplier" s,
|
13
|
+
"tpch"."tiny"."partsupp" ps,
|
14
|
+
"tpch"."tiny"."nation" n,
|
15
|
+
"tpch"."tiny"."region" r
|
16
|
+
WHERE
|
17
|
+
p.partkey = ps.partkey
|
18
|
+
AND s.suppkey = ps.suppkey
|
19
|
+
AND p.size = 15
|
20
|
+
AND p.type like '%BRASS'
|
21
|
+
AND s.nationkey = n.nationkey
|
22
|
+
AND n.regionkey = r.regionkey
|
23
|
+
AND r.name = 'EUROPE'
|
24
|
+
AND ps.supplycost = (
|
25
|
+
SELECT
|
26
|
+
min(ps.supplycost)
|
27
|
+
FROM
|
28
|
+
"tpch"."tiny"."partsupp" ps,
|
29
|
+
"tpch"."tiny"."supplier" s,
|
30
|
+
"tpch"."tiny"."nation" n,
|
31
|
+
"tpch"."tiny"."region" r
|
32
|
+
WHERE
|
33
|
+
p.partkey = ps.partkey
|
34
|
+
AND s.suppkey = ps.suppkey
|
35
|
+
AND s.nationkey = n.nationkey
|
36
|
+
AND n.regionkey = r.regionkey
|
37
|
+
AND r.name = 'EUROPE'
|
38
|
+
)
|
39
|
+
ORDER BY
|
40
|
+
s.acctbal desc,
|
41
|
+
n.name,
|
42
|
+
s.name,
|
43
|
+
p.partkey
|
@@ -0,0 +1,41 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe Presto::Client::Client do
|
4
|
+
before(:all) do
|
5
|
+
@spec_path = File.dirname(__FILE__)
|
6
|
+
WebMock.disable!
|
7
|
+
@cluster = TinyPresto::Cluster.new('316')
|
8
|
+
@container = @cluster.run
|
9
|
+
@client = Presto::Client.new(server: 'localhost:8080', catalog: 'tpch', user: 'test-user', schema: 'tiny')
|
10
|
+
loop do
|
11
|
+
begin
|
12
|
+
# Make sure to all workers are available.
|
13
|
+
@client.run('select 1234')
|
14
|
+
break
|
15
|
+
rescue StandardError => exception
|
16
|
+
puts "Waiting for cluster ready... #{exception}"
|
17
|
+
sleep(5)
|
18
|
+
end
|
19
|
+
end
|
20
|
+
puts 'Cluster is ready'
|
21
|
+
end
|
22
|
+
|
23
|
+
after(:all) do
|
24
|
+
@cluster.stop
|
25
|
+
WebMock.enable!
|
26
|
+
end
|
27
|
+
|
28
|
+
it 'q01' do
|
29
|
+
q = File.read("#{@spec_path}/tpch/q01.sql")
|
30
|
+
columns, rows = run_with_retry(@client, q)
|
31
|
+
expect(columns.length).to be(10)
|
32
|
+
expect(rows.length).to be(4)
|
33
|
+
end
|
34
|
+
|
35
|
+
it 'q02' do
|
36
|
+
q = File.read("#{@spec_path}/tpch/q02.sql")
|
37
|
+
columns, rows = run_with_retry(@client, q)
|
38
|
+
expect(columns.length).to be(8)
|
39
|
+
expect(rows.length).to be(4)
|
40
|
+
end
|
41
|
+
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: presto-client
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.6.
|
4
|
+
version: 0.6.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Sadayuki Furuhashi
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-
|
11
|
+
date: 2020-03-03 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: faraday
|
@@ -121,10 +121,12 @@ executables: []
|
|
121
121
|
extensions: []
|
122
122
|
extra_rdoc_files: []
|
123
123
|
files:
|
124
|
+
- ".github/PULL_REQUEST_TEMPLATE.md"
|
124
125
|
- ".gitignore"
|
125
126
|
- ".travis.yml"
|
126
127
|
- ChangeLog.md
|
127
128
|
- Gemfile
|
129
|
+
- LICENSE
|
128
130
|
- README.md
|
129
131
|
- Rakefile
|
130
132
|
- lib/presto-client.rb
|
@@ -149,10 +151,14 @@ files:
|
|
149
151
|
- modelgen/presto_models.rb
|
150
152
|
- presto-client.gemspec
|
151
153
|
- release.rb
|
154
|
+
- spec/basic_query_spec.rb
|
152
155
|
- spec/client_spec.rb
|
153
156
|
- spec/model_spec.rb
|
154
157
|
- spec/spec_helper.rb
|
155
158
|
- spec/statement_client_spec.rb
|
159
|
+
- spec/tpch/q01.sql
|
160
|
+
- spec/tpch/q02.sql
|
161
|
+
- spec/tpch_query_spec.rb
|
156
162
|
homepage: https://github.com/treasure-data/presto-client-ruby
|
157
163
|
licenses:
|
158
164
|
- Apache-2.0
|
@@ -172,13 +178,16 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
172
178
|
- !ruby/object:Gem::Version
|
173
179
|
version: '0'
|
174
180
|
requirements: []
|
175
|
-
|
176
|
-
rubygems_version: 2.7.6
|
181
|
+
rubygems_version: 3.0.3
|
177
182
|
signing_key:
|
178
183
|
specification_version: 4
|
179
184
|
summary: Presto client library
|
180
185
|
test_files:
|
186
|
+
- spec/basic_query_spec.rb
|
181
187
|
- spec/client_spec.rb
|
182
188
|
- spec/model_spec.rb
|
183
189
|
- spec/spec_helper.rb
|
184
190
|
- spec/statement_client_spec.rb
|
191
|
+
- spec/tpch/q01.sql
|
192
|
+
- spec/tpch/q02.sql
|
193
|
+
- spec/tpch_query_spec.rb
|