rb_snowflake_client 0.1.1 → 0.2.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: df4fb0a8172d346a1cf606a8f5ed5333725c1459dcaefc22ae149387c052e3aa
4
- data.tar.gz: b8d9ebe83e5553aca0be2983b8b47857b447afb304147bda40da2661bca791e4
3
+ metadata.gz: 2a93f0b1e503e2d219db8b1d1133395ca484b45aaa1f91ceef013a50a116b973
4
+ data.tar.gz: c292c5b0d25a921f8deba6882fe288ed084aa07919fa0d056899d20b921612fc
5
5
  SHA512:
6
- metadata.gz: 6f57ff1d40be759b042767cc8cb35c705f2228b638790659f0d9b238cdad9afcf8ebab389705e26bab87a89f64aba25d22feb5096f67312fd90e4712dd45142c
7
- data.tar.gz: 190ee97111afb77599e7d0e2bacdf4ab4c2d1b374bc7c0fb976e155809ab0f2f8a8e3e4d2f72df71619a468e6616de96a8a8c16db682610265ae405b05f43241
6
+ metadata.gz: 28428d4cf4bd70a11fefad327a2ab14deff17e95bf28b999a6b84d67756b35d3207c8224d6a028b9998ca85b62a7226f4df2beddea50f30e9a91350d7df73b4b
7
+ data.tar.gz: ad55068885a29f1cae0181e52d66764651cb4f7d541e46680bd15644ed8c17f10dda8cb9356ae4948cdf9b249e0c726abc540e26b03e1b76f9f2c538a8251cce
data/Gemfile.lock CHANGED
@@ -1,7 +1,13 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- rb_snowflake_client (0.1.1)
4
+ rb_snowflake_client (0.2.0)
5
+ concurrent-ruby (>= 1.2)
6
+ connection_pool (>= 2.4)
7
+ dotenv (>= 2.8)
8
+ jwt (>= 2.7)
9
+ oj (>= 3.16)
10
+ retryable (>= 3.0)
5
11
 
6
12
  GEM
7
13
  remote: https://rubygems.org/
data/README.md CHANGED
@@ -13,6 +13,8 @@ This library is implemented in ruby and while it leverages some libraries that h
13
13
 
14
14
  # Usage
15
15
 
16
+ ## Create a client
17
+
16
18
  Add to your Gemfile or use `gem install rb-snowflake-client`
17
19
  ```ruby
18
20
  gem "rb-snowflake-client"
@@ -27,11 +29,12 @@ require "rb_snowflake_client"
27
29
  # see: https://github.com/rinsed-org/pure-ruby-snowflake-client/blob/master/lib/ruby_snowflake/client.rb#L43
28
30
  client = RubySnowflake::Client.new(
29
31
  "https://yourinstance.region.snowflakecomputing.com", # insert your URL here
30
- File.read("secrets/my_key.pem"), # path to your private key
32
+ File.read("secrets/my_key.pem"), # your private key in PEM format (scroll down for instructions)
31
33
  "snowflake-organization", # your account name (doesn't match your URL)
32
34
  "snowflake-account", # typically your subdomain
33
35
  "snowflake-user", # Your snowflake user
34
36
  "some_warehouse", # The name of your warehouse to use by default
37
+ "some_database", # The name of the database in the context of which the queries will run
35
38
  max_connections: 12, # Config options can be passed in
36
39
  connection_timeout: 45, # See below for the full set of options
37
40
  )
@@ -40,20 +43,22 @@ client = RubySnowflake::Client.new(
40
43
  RubySnowflake::Client.from_env
41
44
  ```
42
45
  Available ENV variables (see below in the config section for details)
43
- `SNOWFLAKE_URI`
44
- `SNOWFLAKE_PRIVATE_KEY_PATH`
45
- or (use either the key, or the path, key takes precedence if both are provided)
46
- `SNOWFLAKE_PRIVATE_KEY`
47
- `SNOWFLAKE_ORGANIZATION`
48
- `SNOWFLAKE_ACCOUNT`
49
- `SNOWFLAKE_USER`
50
- `SNOWFLAKE_DEFAULT_WAREHOUSE`
51
- `SNOWFLAKE_JWT_TOKEN_TTL`
52
- `SNOWFLAKE_CONNECTION_TIMEOUT`
53
- `SNOWFLAKE_MAX_CONNECTIONS`
54
- `SNOWFLAKE_MAX_THREADS_PER_QUERY`
55
- `SNOWFLAKE_THREAD_SCALE_FACTOR`
56
- `SNOWFLAKE_HTTP_RETRIES`
46
+ - `SNOWFLAKE_URI`
47
+ - `SNOWFLAKE_PRIVATE_KEY_PATH` or `SNOWFLAKE_PRIVATE_KEY`
48
+ - Use either the key or the path. Key takes precedence if both are provided.
49
+ - `SNOWFLAKE_ORGANIZATION`
50
+ - `SNOWFLAKE_ACCOUNT`
51
+ - `SNOWFLAKE_USER`
52
+ - `SNOWFLAKE_DEFAULT_WAREHOUSE`
53
+ - `SNOWFLAKE_DEFAULT_DATABASE`
54
+ - `SNOWFLAKE_JWT_TOKEN_TTL`
55
+ - `SNOWFLAKE_CONNECTION_TIMEOUT`
56
+ - `SNOWFLAKE_MAX_CONNECTIONS`
57
+ - `SNOWFLAKE_MAX_THREADS_PER_QUERY`
58
+ - `SNOWFLAKE_THREAD_SCALE_FACTOR`
59
+ - `SNOWFLAKE_HTTP_RETRIES`
60
+
61
+ ## Make queries
57
62
 
58
63
  Once you have a client, make queries
59
64
  ```ruby
@@ -66,37 +71,60 @@ result.each do |row|
66
71
  puts row["name"] # or case insensitive strings
67
72
  puts row.to_h # and can produce a hash with keys/values
68
73
  end
74
+ ```
75
+
76
+ ## Stream results
77
+
78
+ You can also stream results and not hold them all in memory. The client will prefetch the next data partition only. If you have some IO in your processing there should usually be data available for you.
69
79
 
70
- # You can also stream results and not hold them all in memory.
71
- # The client will prefetch the next data partition only. If you
72
- # have some IO in your processing there should usually be data
73
- # available for you.
80
+ ```ruby
74
81
  result = client.query("SELECT * FROM HUGETABLE", streaming: true)
75
82
  result.each do |row|
76
83
  puts row
77
84
  end
78
85
  ```
79
86
 
87
+ ## Switching databases
88
+
89
+ You can also overwrite the database specified in the initializer, and run your query with a different context.
90
+
91
+ ```ruby
92
+ result = client.query("SELECT * FROM SECRET_TABLE", database: "OTHER_DB")
93
+ result.each do |row|
94
+ puts row
95
+ end
96
+ ```
97
+
98
+ ## Switching warehouses
99
+
100
+ Clients are not warehouse specific, you can override the default warehouse per query
101
+
102
+ ```ruby
103
+ client.query("SELECT * FROM BIGTABLE", warehouse: "FAST_WH")
104
+ ```
105
+
80
106
  # Configuration Options
81
107
 
82
- The client supports the following configuration options, each with their own getter/setter except connection pool options which must be set at construction. Additionally, all except logger can be configured with environment variables (see above, but the pattern is like: "SNOWFLAKE_HTTP_RETRIES".
108
+ The client supports the following configuration options, each with their own getter/setter except connection pool options which must be set at construction. Additionally, all except logger can be configured with environment variables (see above, but the pattern is like: "SNOWFLAKE_HTTP_RETRIES". Configuration options can only be set on initiialization through `new` or `from_env`.
83
109
 
84
- `logger` - takes any ruby logger (by default it's a std lib Logger.new(STDOUT), set at DEBUG level. Not available as an ENV variable config option
85
- `log_level` - takes a log level, type is dependent on logger, for the default ruby Logger, use a level like `Logger::WARN`. Not available as an ENV variable config option.
86
- `jwt_token_ttl` - The time to live set on JWT token in seconds, defaults to 3540 (59 minutes, the longest Snowflake supports is 60)
87
- `connection_timeout` - The amount of time in seconds that the client's connection pool will wait before erroring in handing out a valid connection, defaults to 60 seconds
88
- `max_connections` - The maximum number of http connections to hold open in the connection pool. If you use the client in a threaded context, you may need to increase this to be threads * client.max_threads_per_query, defaults to 16. Can only be set on initialization.
89
- `max_threads_per_query` - The maximum number of threads the client should use to retreive data, per query, defaults to 8. If you want the client to act in a single threaded way, set this to 1
90
- `thread_scale_factor` - When downloading a result set into memory, thread count is calculated by dividing a query's partition count by this number. For details on implementation see the code in `client.rb`.
91
- `http_retries` - By default the client will retry common typically transient errors (http responses) twice, you can change the number of retries with this.
110
+ - `logger` - takes any ruby logger (by default it's a std lib Logger.new(STDOUT), set at DEBUG level. Not available as an ENV variable config option
111
+ - `log_level` - takes a log level, type is dependent on logger, for the default ruby Logger, use a level like `Logger::WARN`. Not available as an ENV variable config option.
112
+ - `jwt_token_ttl` - The time to live set on JWT token in seconds, defaults to 3540 (59 minutes, the longest Snowflake supports is 60).
113
+ - `connection_timeout` - The amount of time in seconds that the client's connection pool will wait before erroring in handing out a valid connection, defaults to 60 seconds
114
+ - `max_connections` - The maximum number of http connections to hold open in the connection pool. If you use the client in a threaded context, you may need to increase this to be threads * client.max_threads_per_query, defaults to 16.
115
+ - `max_threads_per_query` - The maximum number of threads the client should use to retreive data, per query, defaults to 8. If you want the client to act in a single threaded way, set this to 1
116
+ - `thread_scale_factor` - When downloading a result set into memory, thread count is calculated by dividing a query's partition count by this number. For details on implementation see the code in `client.rb`.
117
+ - `http_retries` - By default the client will retry common typically transient errors (http responses) twice, you can change the number of retries with this.
92
118
 
93
119
  Example configuration:
94
120
  ```ruby
95
- client = RubySnowflake::Client.from_env
96
- client.logger = Rails.logger
97
- client.max_connections = 24
98
- client.http_retries = 1
121
+ client = RubySnowflake::Client.from_env(
122
+ logger: Rails.logger
123
+ max_connections: 24
124
+ http_retries 1
125
+ )
99
126
  end
127
+ ```
100
128
 
101
129
  # Gotchas
102
130
 
@@ -153,6 +181,7 @@ client = RubySnowflake::Client.new(
153
181
  "snowflake-account", # typically your subdomain
154
182
  "snowflake-user", # Your snowflake user
155
183
  "some_warehouse", # The name of your warehouse to use by default
184
+ "some_database", # The name of the database in the context of which the queries will run
156
185
  )
157
186
  ```
158
187
 
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module RubySnowflake
2
4
  class Client
3
5
  class HttpConnectionWrapper
@@ -0,0 +1,59 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "jwt"
4
+ require "openssl"
5
+ require "concurrent"
6
+
7
+ module RubySnowflake
8
+ class Client
9
+ class KeyPairJwtAuthManager
10
+ # requires text of a PEM formatted RSA private key
11
+ def initialize(organization, account, user, private_key, jwt_token_ttl)
12
+ @organization = organization
13
+ @account = account
14
+ @user = user
15
+ @private_key_pem = private_key
16
+ @jwt_token_ttl = jwt_token_ttl
17
+
18
+ # start with an expired value to force creation
19
+ @token_expires_at = Time.now.to_i - 1
20
+ @token_semaphore = Concurrent::Semaphore.new(1)
21
+ end
22
+
23
+ def jwt_token
24
+ return @token unless jwt_token_expired?
25
+
26
+ @token_semaphore.acquire do
27
+ now = Time.now.to_i
28
+ @token_expires_at = now + @jwt_token_ttl
29
+
30
+ private_key = OpenSSL::PKey.read(@private_key_pem)
31
+
32
+ payload = {
33
+ :iss => "#{@organization.upcase}-#{@account.upcase}.#{@user}.#{public_key_fingerprint}",
34
+ :sub => "#{@organization.upcase}-#{@account.upcase}.#{@user}",
35
+ :iat => now,
36
+ :exp => @token_expires_at
37
+ }
38
+
39
+ @token = JWT.encode payload, private_key, "RS256"
40
+ end
41
+ end
42
+
43
+ private
44
+ def jwt_token_expired?
45
+ Time.now.to_i > @token_expires_at
46
+ end
47
+
48
+ def public_key_fingerprint
49
+ reutrn @public_key_fingerprint unless @public_key_fingerprint.nil?
50
+
51
+ public_key_der = OpenSSL::PKey::RSA.new(@private_key_pem).public_key.to_der
52
+ digest = OpenSSL::Digest::SHA256.new.digest(public_key_der)
53
+ fingerprint = Base64.strict_encode64(digest)
54
+
55
+ @public_key_fingerprint = "SHA256:#{fingerprint}"
56
+ end
57
+ end
58
+ end
59
+ end
@@ -5,22 +5,21 @@ require "benchmark"
5
5
  require "concurrent"
6
6
  require "connection_pool"
7
7
  require "json"
8
- require "jwt"
9
8
  require "logger"
10
9
  require "net/http"
11
10
  require "oj"
12
- require "openssl"
13
11
  require "retryable"
14
12
  require "securerandom"
15
13
  require "uri"
16
14
 
17
15
 
18
- require_relative "result"
19
- require_relative "streaming_result"
20
16
  require_relative "client/http_connection_wrapper"
17
+ require_relative "client/key_pair_jwt_auth_manager"
21
18
  require_relative "client/single_thread_in_memory_strategy"
22
19
  require_relative "client/streaming_result_strategy"
23
20
  require_relative "client/threaded_in_memory_strategy"
21
+ require_relative "result"
22
+ require_relative "streaming_result"
24
23
 
25
24
  module RubySnowflake
26
25
  class Error < StandardError
@@ -55,11 +54,19 @@ module RubySnowflake
55
54
  # how many times to retry common retryable HTTP responses (i.e. 429, 504)
56
55
  DEFAULT_HTTP_RETRIES = 2
57
56
 
58
- # can't be set after initialization
59
- attr_reader :connection_timeout, :max_connections
60
- attr_accessor :logger, :jwt_token_ttl, :max_threads_per_query, :thread_scale_factor, :http_retries
57
+ OJ_OPTIONS = { :bigdecimal_load => :bigdecimal }.freeze
61
58
 
62
- def self.from_env
59
+ # can't be set after initialization
60
+ attr_reader :connection_timeout, :max_connections, :logger, :max_threads_per_query, :thread_scale_factor, :http_retries
61
+
62
+ def self.from_env(logger: DEFAULT_LOGGER,
63
+ log_level: DEFAULT_LOG_LEVEL,
64
+ jwt_token_ttl: env_option("SNOWFLAKE_JWT_TOKEN_TTL", DEFAULT_JWT_TOKEN_TTL),
65
+ connection_timeout: env_option("SNOWFLAKE_CONNECTION_TIMEOUT", DEFAULT_CONNECTION_TIMEOUT ),
66
+ max_connections: env_option("SNOWFLAKE_MAX_CONNECTIONS", DEFAULT_MAX_CONNECTIONS ),
67
+ max_threads_per_query: env_option("SNOWFLAKE_MAX_THREADS_PER_QUERY", DEFAULT_MAX_THREADS_PER_QUERY),
68
+ thread_scale_factor: env_option("SNOWFLAKE_THREAD_SCALE_FACTOR", DEFAULT_THREAD_SCALE_FACTOR),
69
+ http_retries: env_option("SNOWFLAKE_HTTP_RETRIES", DEFAULT_HTTP_RETRIES))
63
70
  private_key = ENV["SNOWFLAKE_PRIVATE_KEY"] || File.read(ENV["SNOWFLAKE_PRIVATE_KEY_PATH"])
64
71
 
65
72
  new(
@@ -69,53 +76,54 @@ module RubySnowflake
69
76
  ENV["SNOWFLAKE_ACCOUNT"],
70
77
  ENV["SNOWFLAKE_USER"],
71
78
  ENV["SNOWFLAKE_DEFAULT_WAREHOUSE"],
72
- jwt_token_ttl: env_option("SNOWFLAKE_JWT_TOKEN_TTL", DEFAULT_JWT_TOKEN_TTL),
73
- connection_timeout: env_option("SNOWFLAKE_CONNECTION_TIMEOUT", DEFAULT_CONNECTION_TIMEOUT ),
74
- max_connections: env_option("SNOWFLAKE_MAX_CONNECTIONS", DEFAULT_MAX_CONNECTIONS ),
75
- max_threads_per_query: env_option("SNOWFLAKE_MAX_THREADS_PER_QUERY", DEFAULT_MAX_THREADS_PER_QUERY),
76
- thread_scale_factor: env_option("SNOWFLAKE_THREAD_SCALE_FACTOR", DEFAULT_THREAD_SCALE_FACTOR),
77
- http_retries: env_option("SNOWFLAKE_HTTP_RETRIES", DEFAULT_HTTP_RETRIES),
79
+ ENV["SNOWFLAKE_DEFAULT_DATABASE"],
80
+ logger: logger,
81
+ log_level: log_level,
82
+ jwt_token_ttl: jwt_token_ttl,
83
+ connection_timeout: connection_timeout,
84
+ max_connections: max_connections,
85
+ max_threads_per_query: max_threads_per_query,
86
+ thread_scale_factor: thread_scale_factor,
87
+ http_retries: http_retries,
78
88
  )
79
89
  end
80
90
 
81
- def initialize(uri, private_key, organization, account, user, default_warehouse,
82
- logger: DEFAULT_LOGGER,
83
- log_level: DEFAULT_LOG_LEVEL,
84
- jwt_token_ttl: DEFAULT_JWT_TOKEN_TTL,
85
- connection_timeout: DEFAULT_CONNECTION_TIMEOUT,
86
- max_connections: DEFAULT_MAX_CONNECTIONS,
87
- max_threads_per_query: DEFAULT_MAX_THREADS_PER_QUERY,
88
- thread_scale_factor: DEFAULT_THREAD_SCALE_FACTOR,
89
- http_retries: DEFAULT_HTTP_RETRIES)
91
+ def initialize(
92
+ uri, private_key, organization, account, user, default_warehouse, default_database,
93
+ logger: DEFAULT_LOGGER,
94
+ log_level: DEFAULT_LOG_LEVEL,
95
+ jwt_token_ttl: DEFAULT_JWT_TOKEN_TTL,
96
+ connection_timeout: DEFAULT_CONNECTION_TIMEOUT,
97
+ max_connections: DEFAULT_MAX_CONNECTIONS,
98
+ max_threads_per_query: DEFAULT_MAX_THREADS_PER_QUERY,
99
+ thread_scale_factor: DEFAULT_THREAD_SCALE_FACTOR,
100
+ http_retries: DEFAULT_HTTP_RETRIES
101
+ )
90
102
  @base_uri = uri
91
- @private_key_pem = private_key
92
- @organization = organization
93
- @account = account
94
- @user = user
103
+ @key_pair_jwt_auth_manager =
104
+ KeyPairJwtAuthManager.new(organization, account, user, private_key, jwt_token_ttl)
95
105
  @default_warehouse = default_warehouse
96
- @public_key_fingerprint = public_key_fingerprint(@private_key_pem)
106
+ @default_database = default_database
97
107
 
98
108
  # set defaults for config settings
99
109
  @logger = logger
100
110
  @logger.level = log_level
101
- @jwt_token_ttl = jwt_token_ttl
102
111
  @connection_timeout = connection_timeout
103
112
  @max_connections = max_connections
104
113
  @max_threads_per_query = max_threads_per_query
105
114
  @thread_scale_factor = thread_scale_factor
106
115
  @http_retries = http_retries
107
-
108
- # start with an expired value to force creation
109
- @token_expires_at = Time.now.to_i - 1
110
- @token_semaphore = Concurrent::Semaphore.new(1)
111
116
  end
112
117
 
113
- def query(query, warehouse: nil, streaming: false)
118
+ def query(query, warehouse: nil, streaming: false, database: nil)
114
119
  warehouse ||= @default_warehouse
120
+ database ||= @default_database
115
121
 
116
122
  response = nil
117
123
  connection_pool.with do |connection|
118
- request_body = { "statement" => query, "warehouse" => warehouse }
124
+ request_body = {
125
+ "statement" => query, "warehouse" => warehouse, "database" => database
126
+ }
119
127
 
120
128
  response = request_with_auth_and_headers(
121
129
  connection,
@@ -150,36 +158,19 @@ module RubySnowflake
150
158
  @port ||= URI.parse(@base_uri).port
151
159
  end
152
160
 
153
- def jwt_token
154
- return @token unless jwt_token_expired?
155
-
156
- @token_semaphore.acquire do
157
- now = Time.now.to_i
158
- @token_expires_at = now + @jwt_token_ttl
159
-
160
- private_key = OpenSSL::PKey.read(@private_key_pem)
161
-
162
- payload = {
163
- :iss => "#{@organization.upcase}-#{@account.upcase}.#{@user}.#{@public_key_fingerprint}",
164
- :sub => "#{@organization.upcase}-#{@account.upcase}.#{@user}",
165
- :iat => now,
166
- :exp => @token_expires_at
167
- }
168
-
169
- @token = JWT.encode payload, private_key, "RS256"
161
+ def handle_errors(response)
162
+ if response.code != "200"
163
+ raise BadResponseError.new({}),
164
+ "Bad response! Got code: #{response.code}, w/ message #{response.body}"
170
165
  end
171
166
  end
172
167
 
173
- def jwt_token_expired?
174
- Time.now.to_i > @token_expires_at
175
- end
176
-
177
168
  def request_with_auth_and_headers(connection, request_class, path, body=nil)
178
169
  uri = URI.parse("#{@base_uri}#{path}")
179
170
  request = request_class.new(uri)
180
171
  request["Content-Type"] = "application/json"
181
172
  request["Accept"] = "application/json"
182
- request["Authorization"] = "Bearer #{jwt_token}"
173
+ request["Authorization"] = "Bearer #{@key_pair_jwt_auth_manager.jwt_token}"
183
174
  request["X-Snowflake-Authorization-Token-Type"] = "KEYPAIR_JWT"
184
175
  request.body = body unless body.nil?
185
176
 
@@ -223,7 +214,7 @@ module RubySnowflake
223
214
  end
224
215
 
225
216
  def retreive_result_set(response, streaming)
226
- json_body = Oj.load(response.body, oj_options)
217
+ json_body = Oj.load(response.body, OJ_OPTIONS)
227
218
  statement_handle = json_body["statementHandle"]
228
219
  num_threads = number_of_threads_to_use(json_body["resultSetMetaData"]["partitionInfo"].size)
229
220
  retreive_proc = ->(index) { retreive_partition_data(statement_handle, index) }
@@ -247,8 +238,8 @@ module RubySnowflake
247
238
  )
248
239
  end
249
240
 
250
- partition_json = nil
251
- bm = Benchmark.measure { partition_json = Oj.load(partition_response.body, oj_options) }
241
+ partition_json = {}
242
+ bm = Benchmark.measure { partition_json = Oj.load(partition_response.body, OJ_OPTIONS) }
252
243
  logger.debug { "JSON parsing took: #{bm.real}" }
253
244
  partition_data = partition_json["data"]
254
245
 
@@ -258,17 +249,5 @@ module RubySnowflake
258
249
  def number_of_threads_to_use(partition_count)
259
250
  [[1, (partition_count / @thread_scale_factor.to_f).ceil].max, @max_threads_per_query].min
260
251
  end
261
-
262
- def oj_options
263
- { :bigdecimal_load => :bigdecimal }
264
- end
265
-
266
- def public_key_fingerprint(private_key_pem_string)
267
- public_key_der = OpenSSL::PKey::RSA.new(private_key_pem_string).public_key.to_der
268
- digest = OpenSSL::Digest::SHA256.new.digest(public_key_der)
269
- fingerprint = Base64.strict_encode64(digest)
270
-
271
- "SHA256:#{fingerprint}"
272
- end
273
252
  end
274
253
  end
@@ -31,14 +31,19 @@ module RubySnowflake
31
31
  else
32
32
  BigDecimal(@data[index]).round(@row_types[index][:scale])
33
33
  end
34
+
35
+ # snowflake treats these all as 64 bit IEEE 754 floating point numbers, and will we too
34
36
  when :float, :double, :"double precision", :real
35
- # snowflake treats these all as 64 bit IEEE 754 floating point numbers, and will we too
36
37
  Float(@data[index])
37
- when :time, :datetime, :timestamp, :timestamp_ltz, :timestamp_ntz
38
+
39
+ # Despite snowflake indicating that it sends the offset in minutes, the actual time in UTC
40
+ # is always sent in the first half of the data. If an offset is sent it looks like:
41
+ # "1641008096.123000000 1980"
42
+ # If there isn't one, it's just like this:
43
+ # "1641065696.123000000"
44
+ # in all cases, the actual time, in UTC is the float value, and the offset is ignorable
45
+ when :time, :datetime, :timestamp, :timestamp_ntz, :timestamp_ltz, :timestamp_tz
38
46
  Time.strptime(@data[index], TIME_FORMAT).utc
39
- when :timestamp_tz
40
- timestamp, offset_minutes = @data[index].split(" ")
41
- Time.strptime(@data[index], TIME_FORMAT).utc - (Integer(offset_minutes) * 60)
42
47
  else
43
48
  @data[index]
44
49
  end
@@ -1,3 +1,3 @@
1
1
  module RubySnowflake
2
- VERSION = '0.1.1'
2
+ VERSION = '0.2.0'
3
3
  end
@@ -18,4 +18,10 @@ Gem::Specification.new do |s|
18
18
  end
19
19
 
20
20
  s.require_paths = ["lib"]
21
+ s.add_dependency "concurrent-ruby", ">= 1.2"
22
+ s.add_dependency "connection_pool", ">= 2.4"
23
+ s.add_dependency "dotenv", ">= 2.8"
24
+ s.add_dependency "jwt", ">= 2.7"
25
+ s.add_dependency "oj", ">= 3.16"
26
+ s.add_dependency "retryable", ">= 3.0"
21
27
  end
metadata CHANGED
@@ -1,15 +1,99 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: rb_snowflake_client
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.1
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Rinsed
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-12-01 00:00:00.000000000 Z
12
- dependencies: []
11
+ date: 2023-12-07 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: concurrent-ruby
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '1.2'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '1.2'
27
+ - !ruby/object:Gem::Dependency
28
+ name: connection_pool
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '2.4'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '2.4'
41
+ - !ruby/object:Gem::Dependency
42
+ name: dotenv
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '2.8'
48
+ type: :runtime
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '2.8'
55
+ - !ruby/object:Gem::Dependency
56
+ name: jwt
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '2.7'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '2.7'
69
+ - !ruby/object:Gem::Dependency
70
+ name: oj
71
+ requirement: !ruby/object:Gem::Requirement
72
+ requirements:
73
+ - - ">="
74
+ - !ruby/object:Gem::Version
75
+ version: '3.16'
76
+ type: :runtime
77
+ prerelease: false
78
+ version_requirements: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '3.16'
83
+ - !ruby/object:Gem::Dependency
84
+ name: retryable
85
+ requirement: !ruby/object:Gem::Requirement
86
+ requirements:
87
+ - - ">="
88
+ - !ruby/object:Gem::Version
89
+ version: '3.0'
90
+ type: :runtime
91
+ prerelease: false
92
+ version_requirements: !ruby/object:Gem::Requirement
93
+ requirements:
94
+ - - ">="
95
+ - !ruby/object:Gem::Version
96
+ version: '3.0'
13
97
  description: 'Using the HTTP V2 Api for Snowflake runs queries & creates native Ruby
14
98
  objects.
15
99
 
@@ -33,6 +117,7 @@ files:
33
117
  - lib/rb_snowflake_client.rb
34
118
  - lib/ruby_snowflake/client.rb
35
119
  - lib/ruby_snowflake/client/http_connection_wrapper.rb
120
+ - lib/ruby_snowflake/client/key_pair_jwt_auth_manager.rb
36
121
  - lib/ruby_snowflake/client/single_thread_in_memory_strategy.rb
37
122
  - lib/ruby_snowflake/client/streaming_result_strategy.rb
38
123
  - lib/ruby_snowflake/client/threaded_in_memory_strategy.rb