google-cloud-bigquery 0.24.0 → 0.25.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +1 -1
- data/lib/google/cloud/bigquery.rb +55 -57
- data/lib/google/cloud/bigquery/dataset.rb +35 -19
- data/lib/google/cloud/bigquery/dataset/access.rb +0 -1
- data/lib/google/cloud/bigquery/dataset/list.rb +6 -6
- data/lib/google/cloud/bigquery/job.rb +11 -2
- data/lib/google/cloud/bigquery/job/list.rb +6 -6
- data/lib/google/cloud/bigquery/project.rb +21 -20
- data/lib/google/cloud/bigquery/project/list.rb +6 -6
- data/lib/google/cloud/bigquery/query_job.rb +2 -2
- data/lib/google/cloud/bigquery/schema/field.rb +2 -2
- data/lib/google/cloud/bigquery/service.rb +9 -0
- data/lib/google/cloud/bigquery/table.rb +4 -4
- data/lib/google/cloud/bigquery/table/list.rb +6 -6
- data/lib/google/cloud/bigquery/time.rb +2 -2
- data/lib/google/cloud/bigquery/version.rb +1 -1
- data/lib/google/cloud/bigquery/view.rb +1 -1
- metadata +5 -5
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 97f13393ce0c9406234614341b3cea2d23c0f6c9
|
4
|
+
data.tar.gz: 47ea3b9d8fad528f8049e9c2bef6c71a3711cb72
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: da73e061c5e4192f36852a5d2c27debd5404d60f3d5da5bec250f45a045ca6ea16ac7f985ef9f0de68ba1d208a1e67e2b6e138fd88a467bec304e6b2d82c7f1c
|
7
|
+
data.tar.gz: 9bfc2e1f267473e8ca57656bedd76875d326dc9b588191566c87d586aba406fd790d9abfbbad2a6d28459499771605bee6ccc13684205acc9a868b1a5effbc0d
|
data/README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
[Google BigQuery](https://cloud.google.com/bigquery/) ([docs](https://cloud.google.com/bigquery/docs)) enables super-fast, SQL-like queries against append-only tables, using the processing power of Google's infrastructure. Simply move your data into BigQuery and let it handle the hard work. You can control access to both the project and your data based on your business needs, such as giving others the ability to view or query your data.
|
4
4
|
|
5
|
-
- [google-cloud-bigquery API documentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-bigquery/
|
5
|
+
- [google-cloud-bigquery API documentation](http://googlecloudplatform.github.io/google-cloud-ruby/#/docs/google-cloud-bigquery/latest)
|
6
6
|
- [google-cloud-bigquery on RubyGems](https://rubygems.org/gems/google-cloud-bigquery)
|
7
7
|
- [Google BigQuery documentation](https://cloud.google.com/bigquery/docs)
|
8
8
|
|
@@ -43,9 +43,10 @@ module Google
|
|
43
43
|
#
|
44
44
|
# ## Listing Datasets and Tables
|
45
45
|
#
|
46
|
-
# A BigQuery project
|
47
|
-
# that you have not yet created datasets or tables in your own
|
48
|
-
# let's connect to Google's `publicdata` project, and see what
|
46
|
+
# A BigQuery project contains datasets, which in turn contain tables.
|
47
|
+
# Assuming that you have not yet created datasets or tables in your own
|
48
|
+
# project, let's connect to Google's `publicdata` project, and see what we
|
49
|
+
# find.
|
49
50
|
#
|
50
51
|
# ```ruby
|
51
52
|
# require "google/cloud/bigquery"
|
@@ -62,10 +63,10 @@ module Google
|
|
62
63
|
# tables.map &:table_id #=> [..., "shakespeare", "trigrams", "wikipedia"]
|
63
64
|
# ```
|
64
65
|
#
|
65
|
-
# In addition listing all datasets and tables in the project, you can
|
66
|
-
# retrieve individual datasets and tables by ID. Let's look at the
|
67
|
-
# of the `shakespeare` table, which contains an entry for every
|
68
|
-
# every play written by Shakespeare.
|
66
|
+
# In addition to listing all datasets and tables in the project, you can
|
67
|
+
# also retrieve individual datasets and tables by ID. Let's look at the
|
68
|
+
# structure of the `shakespeare` table, which contains an entry for every
|
69
|
+
# word in every play written by Shakespeare.
|
69
70
|
#
|
70
71
|
# ```ruby
|
71
72
|
# require "google/cloud/bigquery"
|
@@ -79,15 +80,15 @@ module Google
|
|
79
80
|
# table.rows_count #=> 164656
|
80
81
|
# ```
|
81
82
|
#
|
82
|
-
# Now that you know the column names for the Shakespeare table,
|
83
|
-
# write and run a
|
83
|
+
# Now that you know the column names for the Shakespeare table, let's
|
84
|
+
# write and run a few queries against it.
|
84
85
|
#
|
85
86
|
# ## Running queries
|
86
87
|
#
|
87
88
|
# BigQuery supports two SQL dialects: [standard
|
88
89
|
# SQL](https://cloud.google.com/bigquery/docs/reference/standard-sql/)
|
89
|
-
# and the older [legacy
|
90
|
-
# SQL](https://cloud.google.com/bigquery/docs/reference/legacy-sql),
|
90
|
+
# and the older [legacy SQl (BigQuery
|
91
|
+
# SQL)](https://cloud.google.com/bigquery/docs/reference/legacy-sql),
|
91
92
|
# as discussed in the guide [Migrating from legacy
|
92
93
|
# SQL](https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql).
|
93
94
|
#
|
@@ -100,7 +101,7 @@ module Google
|
|
100
101
|
# Standard SQL is the preferred SQL dialect for querying data stored in
|
101
102
|
# BigQuery. It is compliant with the SQL 2011 standard, and has extensions
|
102
103
|
# that support querying nested and repeated data. This is the default
|
103
|
-
# syntax. It has several advantages over
|
104
|
+
# syntax. It has several advantages over legacy SQL, including:
|
104
105
|
#
|
105
106
|
# * Composability using `WITH` clauses and SQL functions
|
106
107
|
# * Subqueries in the `SELECT` list and `WHERE` clause
|
@@ -115,7 +116,7 @@ module Google
|
|
115
116
|
# For examples that demonstrate some of these features, see [Standard SQL
|
116
117
|
# highlights](https://cloud.google.com/bigquery/docs/reference/standard-sql/migrating-from-legacy-sql#standard_sql_highlights).
|
117
118
|
#
|
118
|
-
#
|
119
|
+
# As shown in this example, standard SQL is the library default:
|
119
120
|
#
|
120
121
|
# ```ruby
|
121
122
|
# require "google/cloud/bigquery"
|
@@ -136,10 +137,10 @@ module Google
|
|
136
137
|
# Before version 2.0, BigQuery executed queries using a non-standard SQL
|
137
138
|
# dialect known as BigQuery SQL. This variant is optional, and can be
|
138
139
|
# enabled by passing the flag `legacy_sql: true` with your query. (If you
|
139
|
-
# get an SQL syntax error with a query that may be written in
|
140
|
+
# get an SQL syntax error with a query that may be written in legacy SQL,
|
140
141
|
# be sure that you are passing this option.)
|
141
142
|
#
|
142
|
-
# To use legacy SQL, pass the option `legacy_sql: true` with your query
|
143
|
+
# To use legacy SQL, pass the option `legacy_sql: true` with your query:
|
143
144
|
#
|
144
145
|
# ```ruby
|
145
146
|
# require "google/cloud/bigquery"
|
@@ -201,35 +202,35 @@ module Google
|
|
201
202
|
# ### Synchronous queries
|
202
203
|
#
|
203
204
|
# Let's start with the simpler synchronous approach. Notice that this time
|
204
|
-
# you are connecting using your own default project.
|
205
|
-
# running a query, since queries need to
|
206
|
-
# results.
|
205
|
+
# you are connecting using your own default project. It is necessary to have
|
206
|
+
# write access to the project for running a query, since queries need to
|
207
|
+
# create tables to hold results.
|
207
208
|
#
|
208
209
|
# ```ruby
|
209
210
|
# require "google/cloud/bigquery"
|
210
211
|
#
|
211
212
|
# bigquery = Google::Cloud::Bigquery.new
|
212
213
|
#
|
213
|
-
# sql = "SELECT
|
214
|
-
# "
|
214
|
+
# sql = "SELECT APPROX_TOP_COUNT(corpus, 10) as title, " \
|
215
|
+
# "COUNT(*) as unique_words " \
|
216
|
+
# "FROM publicdata.samples.shakespeare"
|
215
217
|
# data = bigquery.query sql
|
216
218
|
#
|
217
|
-
# data.count #=> 50
|
218
219
|
# data.next? #=> false
|
219
|
-
# data.first #=> {
|
220
|
+
# data.first #=> {:title=>[{:value=>"hamlet", :count=>5318}, ...}
|
220
221
|
# ```
|
221
222
|
#
|
222
|
-
# The `
|
223
|
-
# offered by BigQuery. See the [Query
|
224
|
-
#
|
225
|
-
# listing.
|
223
|
+
# The `APPROX_TOP_COUNT` function shown above is just one of a variety of
|
224
|
+
# functions offered by BigQuery. See the [Query Reference (standard
|
225
|
+
# SQL)](https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators)
|
226
|
+
# for a full listing.
|
226
227
|
#
|
227
228
|
# ### Asynchronous queries
|
228
229
|
#
|
229
|
-
#
|
230
|
-
#
|
231
|
-
# BigQuery API
|
232
|
-
# asynchronous approach to running a query, an instance of
|
230
|
+
# It is usually best not to block for most BigQuery operations, including
|
231
|
+
# querying as well as importing, exporting, and copying data. Therefore, the
|
232
|
+
# BigQuery API provides facilities for managing longer-running jobs. With
|
233
|
+
# the asynchronous approach to running a query, an instance of
|
233
234
|
# {Google::Cloud::Bigquery::QueryJob} is returned, rather than an instance
|
234
235
|
# of {Google::Cloud::Bigquery::QueryData}.
|
235
236
|
#
|
@@ -238,24 +239,24 @@ module Google
|
|
238
239
|
#
|
239
240
|
# bigquery = Google::Cloud::Bigquery.new
|
240
241
|
#
|
241
|
-
# sql = "SELECT
|
242
|
-
# "
|
242
|
+
# sql = "SELECT APPROX_TOP_COUNT(corpus, 10) as title, " \
|
243
|
+
# "COUNT(*) as unique_words " \
|
244
|
+
# "FROM publicdata.samples.shakespeare"
|
243
245
|
# job = bigquery.query_job sql
|
244
246
|
#
|
245
247
|
# job.wait_until_done!
|
246
248
|
# if !job.failed?
|
247
|
-
# job.query_results.
|
248
|
-
#
|
249
|
-
# end
|
249
|
+
# job.query_results.first
|
250
|
+
# #=> {:title=>[{:value=>"hamlet", :count=>5318}, ...}
|
250
251
|
# end
|
251
252
|
# ```
|
252
253
|
#
|
253
254
|
# Once you have determined that the job is done and has not failed, you can
|
254
255
|
# obtain an instance of {Google::Cloud::Bigquery::QueryData} by calling
|
255
|
-
#
|
256
|
-
#
|
257
|
-
#
|
258
|
-
#
|
256
|
+
# `query_results` on the job instance. The query results for both of the
|
257
|
+
# above examples are stored in temporary tables with a lifetime of about 24
|
258
|
+
# hours. See the final example below for a demonstration of how to store
|
259
|
+
# query results in a permanent table.
|
259
260
|
#
|
260
261
|
# ## Creating Datasets and Tables
|
261
262
|
#
|
@@ -276,7 +277,7 @@ module Google
|
|
276
277
|
# example below shows a schema with a repeated record field named
|
277
278
|
# `cities_lived`. (For more information about nested and repeated fields,
|
278
279
|
# see [Preparing Data for
|
279
|
-
#
|
280
|
+
# Loading](https://cloud.google.com/bigquery/preparing-data-for-loading).)
|
280
281
|
#
|
281
282
|
# ```ruby
|
282
283
|
# require "google/cloud/bigquery"
|
@@ -298,14 +299,14 @@ module Google
|
|
298
299
|
#
|
299
300
|
# ## Loading records
|
300
301
|
#
|
302
|
+
# To follow along with these examples, you will need to set up billing on
|
303
|
+
# the [Google Developers Console](https://console.developers.google.com).
|
304
|
+
#
|
301
305
|
# In addition to CSV, data can be imported from files that are formatted as
|
302
306
|
# [Newline-delimited JSON](http://jsonlines.org/) or
|
303
307
|
# [Avro](http://avro.apache.org/), or from a Google Cloud Datastore backup.
|
304
308
|
# It can also be "streamed" into BigQuery.
|
305
309
|
#
|
306
|
-
# To follow along with these examples, you will need to set up billing on
|
307
|
-
# the [Google Developers Console](https://console.developers.google.com).
|
308
|
-
#
|
309
310
|
# ### Streaming records
|
310
311
|
#
|
311
312
|
# For situations in which you want new data to be available for querying as
|
@@ -355,8 +356,7 @@ module Google
|
|
355
356
|
# To follow along with this example, please download the
|
356
357
|
# [names.zip](http://www.ssa.gov/OACT/babynames/names.zip) archive from the
|
357
358
|
# U.S. Social Security Administration. Inside the archive you will find over
|
358
|
-
# 100 files containing baby name records since the year 1880.
|
359
|
-
# also contained in the archive specifies the schema used below.
|
359
|
+
# 100 files containing baby name records since the year 1880.
|
360
360
|
#
|
361
361
|
# ```ruby
|
362
362
|
# require "google/cloud/bigquery"
|
@@ -365,8 +365,8 @@ module Google
|
|
365
365
|
# dataset = bigquery.dataset "my_dataset"
|
366
366
|
# table = dataset.create_table "baby_names" do |schema|
|
367
367
|
# schema.string "name", mode: :required
|
368
|
-
# schema.string "
|
369
|
-
# schema.integer "
|
368
|
+
# schema.string "gender", mode: :required
|
369
|
+
# schema.integer "count", mode: :required
|
370
370
|
# end
|
371
371
|
#
|
372
372
|
# file = File.open "names/yob2014.txt"
|
@@ -395,37 +395,35 @@ module Google
|
|
395
395
|
# source_table = dataset.table "baby_names"
|
396
396
|
# result_table = dataset.create_table "baby_names_results"
|
397
397
|
#
|
398
|
-
# sql = "SELECT name,
|
398
|
+
# sql = "SELECT name, count " \
|
399
399
|
# "FROM baby_names " \
|
400
|
-
# "WHERE
|
401
|
-
# "ORDER BY count
|
400
|
+
# "WHERE gender = 'M' " \
|
401
|
+
# "ORDER BY count ASC LIMIT 5"
|
402
402
|
# query_job = dataset.query_job sql, table: result_table
|
403
403
|
#
|
404
404
|
# query_job.wait_until_done!
|
405
405
|
#
|
406
406
|
# if !query_job.failed?
|
407
|
-
#
|
408
|
-
# require "google/cloud/bigquery"
|
407
|
+
# require "google/cloud/storage"
|
409
408
|
#
|
410
409
|
# storage = Google::Cloud::Storage.new
|
411
410
|
# bucket_id = "bigquery-exports-#{SecureRandom.uuid}"
|
412
411
|
# bucket = storage.create_bucket bucket_id
|
413
|
-
# extract_url = "gs://#{bucket.id}/baby-names
|
412
|
+
# extract_url = "gs://#{bucket.id}/baby-names.csv"
|
414
413
|
#
|
415
414
|
# extract_job = result_table.extract extract_url
|
416
415
|
#
|
417
416
|
# extract_job.wait_until_done!
|
418
417
|
#
|
419
418
|
# # Download to local filesystem
|
420
|
-
# bucket.files.first.download "baby-names
|
421
|
-
#
|
419
|
+
# bucket.files.first.download "baby-names.csv"
|
422
420
|
# end
|
423
421
|
# ```
|
424
422
|
#
|
425
423
|
# If a table you wish to export contains a large amount of data, you can
|
426
424
|
# pass a wildcard URI to export to multiple files (for sharding), or an
|
427
|
-
# array of URIs (for partitioning), or both. See [Exporting
|
428
|
-
#
|
425
|
+
# array of URIs (for partitioning), or both. See [Exporting
|
426
|
+
# Data](https://cloud.google.com/bigquery/docs/exporting-data)
|
429
427
|
# for details.
|
430
428
|
#
|
431
429
|
# ## Configuring retries and timeout
|
@@ -279,8 +279,8 @@ module Google
|
|
279
279
|
# require "google/cloud/bigquery"
|
280
280
|
#
|
281
281
|
# bigquery = Google::Cloud::Bigquery.new
|
282
|
-
#
|
283
282
|
# dataset = bigquery.dataset "my_dataset"
|
283
|
+
#
|
284
284
|
# dataset.delete
|
285
285
|
#
|
286
286
|
# @!group Lifecycle
|
@@ -312,6 +312,7 @@ module Google
|
|
312
312
|
#
|
313
313
|
# bigquery = Google::Cloud::Bigquery.new
|
314
314
|
# dataset = bigquery.dataset "my_dataset"
|
315
|
+
#
|
315
316
|
# table = dataset.create_table "my_table"
|
316
317
|
#
|
317
318
|
# @example You can also pass name and description options.
|
@@ -319,6 +320,7 @@ module Google
|
|
319
320
|
#
|
320
321
|
# bigquery = Google::Cloud::Bigquery.new
|
321
322
|
# dataset = bigquery.dataset "my_dataset"
|
323
|
+
#
|
322
324
|
# table = dataset.create_table "my_table",
|
323
325
|
# name: "My Table",
|
324
326
|
# description: "A description of table."
|
@@ -342,6 +344,7 @@ module Google
|
|
342
344
|
#
|
343
345
|
# bigquery = Google::Cloud::Bigquery.new
|
344
346
|
# dataset = bigquery.dataset "my_dataset"
|
347
|
+
#
|
345
348
|
# table = dataset.create_table "my_table" do |t|
|
346
349
|
# t.name = "My Table",
|
347
350
|
# t.description = "A description of my table."
|
@@ -399,6 +402,7 @@ module Google
|
|
399
402
|
#
|
400
403
|
# bigquery = Google::Cloud::Bigquery.new
|
401
404
|
# dataset = bigquery.dataset "my_dataset"
|
405
|
+
#
|
402
406
|
# view = dataset.create_view "my_view",
|
403
407
|
# "SELECT name, age FROM proj.dataset.users"
|
404
408
|
#
|
@@ -407,6 +411,7 @@ module Google
|
|
407
411
|
#
|
408
412
|
# bigquery = Google::Cloud::Bigquery.new
|
409
413
|
# dataset = bigquery.dataset "my_dataset"
|
414
|
+
#
|
410
415
|
# view = dataset.create_view "my_view",
|
411
416
|
# "SELECT name, age FROM proj.dataset.users",
|
412
417
|
# name: "My View", description: "This is my view"
|
@@ -447,6 +452,7 @@ module Google
|
|
447
452
|
#
|
448
453
|
# bigquery = Google::Cloud::Bigquery.new
|
449
454
|
# dataset = bigquery.dataset "my_dataset"
|
455
|
+
#
|
450
456
|
# table = dataset.table "my_table"
|
451
457
|
# puts table.name
|
452
458
|
#
|
@@ -476,6 +482,7 @@ module Google
|
|
476
482
|
#
|
477
483
|
# bigquery = Google::Cloud::Bigquery.new
|
478
484
|
# dataset = bigquery.dataset "my_dataset"
|
485
|
+
#
|
479
486
|
# tables = dataset.tables
|
480
487
|
# tables.each do |table|
|
481
488
|
# puts table.name
|
@@ -486,6 +493,7 @@ module Google
|
|
486
493
|
#
|
487
494
|
# bigquery = Google::Cloud::Bigquery.new
|
488
495
|
# dataset = bigquery.dataset "my_dataset"
|
496
|
+
#
|
489
497
|
# tables = dataset.tables
|
490
498
|
# tables.all do |table|
|
491
499
|
# puts table.name
|
@@ -550,7 +558,7 @@ module Google
|
|
550
558
|
# should be stored. If not present, a new table will be created to
|
551
559
|
# store the results.
|
552
560
|
# @param [String] create Specifies whether the job is allowed to create
|
553
|
-
# new tables.
|
561
|
+
# new tables. The default value is `needed`.
|
554
562
|
#
|
555
563
|
# The following values are supported:
|
556
564
|
#
|
@@ -558,7 +566,7 @@ module Google
|
|
558
566
|
# * `never` - The table must already exist. A 'notFound' error is
|
559
567
|
# raised if the table does not exist.
|
560
568
|
# @param [String] write Specifies the action that occurs if the
|
561
|
-
# destination table already exists.
|
569
|
+
# destination table already exists. The default value is `empty`.
|
562
570
|
#
|
563
571
|
# The following values are supported:
|
564
572
|
#
|
@@ -591,12 +599,13 @@ module Google
|
|
591
599
|
#
|
592
600
|
# @return [Google::Cloud::Bigquery::QueryJob]
|
593
601
|
#
|
594
|
-
# @example
|
602
|
+
# @example Query using standard SQL:
|
595
603
|
# require "google/cloud/bigquery"
|
596
604
|
#
|
597
605
|
# bigquery = Google::Cloud::Bigquery.new
|
606
|
+
# dataset = bigquery.dataset "my_dataset"
|
598
607
|
#
|
599
|
-
# job =
|
608
|
+
# job = dataset.query_job "SELECT name FROM my_table"
|
600
609
|
#
|
601
610
|
# job.wait_until_done!
|
602
611
|
# if !job.failed?
|
@@ -609,9 +618,10 @@ module Google
|
|
609
618
|
# require "google/cloud/bigquery"
|
610
619
|
#
|
611
620
|
# bigquery = Google::Cloud::Bigquery.new
|
621
|
+
# dataset = bigquery.dataset "my_dataset"
|
612
622
|
#
|
613
|
-
# job =
|
614
|
-
#
|
623
|
+
# job = dataset.query_job "SELECT name FROM my_table",
|
624
|
+
# legacy_sql: true
|
615
625
|
#
|
616
626
|
# job.wait_until_done!
|
617
627
|
# if !job.failed?
|
@@ -624,9 +634,10 @@ module Google
|
|
624
634
|
# require "google/cloud/bigquery"
|
625
635
|
#
|
626
636
|
# bigquery = Google::Cloud::Bigquery.new
|
637
|
+
# dataset = bigquery.dataset "my_dataset"
|
627
638
|
#
|
628
|
-
# job =
|
629
|
-
#
|
639
|
+
# job = dataset.query_job "SELECT name FROM my_table WHERE id = ?",
|
640
|
+
# params: [1]
|
630
641
|
#
|
631
642
|
# job.wait_until_done!
|
632
643
|
# if !job.failed?
|
@@ -639,9 +650,10 @@ module Google
|
|
639
650
|
# require "google/cloud/bigquery"
|
640
651
|
#
|
641
652
|
# bigquery = Google::Cloud::Bigquery.new
|
653
|
+
# dataset = bigquery.dataset "my_dataset"
|
642
654
|
#
|
643
|
-
# job =
|
644
|
-
#
|
655
|
+
# job = dataset.query_job "SELECT name FROM my_table WHERE id = @id",
|
656
|
+
# params: { id: 1 }
|
645
657
|
#
|
646
658
|
# job.wait_until_done!
|
647
659
|
# if !job.failed?
|
@@ -747,12 +759,13 @@ module Google
|
|
747
759
|
#
|
748
760
|
# @return [Google::Cloud::Bigquery::QueryData]
|
749
761
|
#
|
750
|
-
# @example
|
762
|
+
# @example Query using standard SQL:
|
751
763
|
# require "google/cloud/bigquery"
|
752
764
|
#
|
753
765
|
# bigquery = Google::Cloud::Bigquery.new
|
766
|
+
# dataset = bigquery.dataset "my_dataset"
|
754
767
|
#
|
755
|
-
# data =
|
768
|
+
# data = dataset.query "SELECT name FROM my_table"
|
756
769
|
#
|
757
770
|
# data.each do |row|
|
758
771
|
# puts row["name"]
|
@@ -762,9 +775,10 @@ module Google
|
|
762
775
|
# require "google/cloud/bigquery"
|
763
776
|
#
|
764
777
|
# bigquery = Google::Cloud::Bigquery.new
|
778
|
+
# dataset = bigquery.dataset "my_dataset"
|
765
779
|
#
|
766
|
-
# data =
|
767
|
-
#
|
780
|
+
# data = dataset.query "SELECT name FROM my_table",
|
781
|
+
# legacy_sql: true
|
768
782
|
#
|
769
783
|
# data.each do |row|
|
770
784
|
# puts row["name"]
|
@@ -774,9 +788,10 @@ module Google
|
|
774
788
|
# require "google/cloud/bigquery"
|
775
789
|
#
|
776
790
|
# bigquery = Google::Cloud::Bigquery.new
|
791
|
+
# dataset = bigquery.dataset "my_dataset"
|
777
792
|
#
|
778
|
-
# data =
|
779
|
-
#
|
793
|
+
# data = dataset.query "SELECT name FROM my_table WHERE id = ?",
|
794
|
+
# params: [1]
|
780
795
|
#
|
781
796
|
# data.each do |row|
|
782
797
|
# puts row["name"]
|
@@ -786,9 +801,10 @@ module Google
|
|
786
801
|
# require "google/cloud/bigquery"
|
787
802
|
#
|
788
803
|
# bigquery = Google::Cloud::Bigquery.new
|
804
|
+
# dataset = bigquery.dataset "my_dataset"
|
789
805
|
#
|
790
|
-
# data =
|
791
|
-
#
|
806
|
+
# data = dataset.query "SELECT name FROM my_table WHERE id = @id",
|
807
|
+
# params: { id: 1 }
|
792
808
|
#
|
793
809
|
# data.each do |row|
|
794
810
|
# puts row["name"]
|
@@ -77,15 +77,15 @@ module Google
|
|
77
77
|
end
|
78
78
|
|
79
79
|
##
|
80
|
-
# Retrieves
|
81
|
-
# returns `false`. Calls the given block once for each
|
82
|
-
# is passed as the
|
80
|
+
# Retrieves remaining results by repeatedly invoking {#next} until
|
81
|
+
# {#next?} returns `false`. Calls the given block once for each
|
82
|
+
# result, which is passed as the argument to the block.
|
83
83
|
#
|
84
84
|
# An Enumerator is returned if no block is given.
|
85
85
|
#
|
86
|
-
# This method
|
87
|
-
# retrieved.
|
88
|
-
#
|
86
|
+
# This method will make repeated API calls until all remaining results
|
87
|
+
# are retrieved. (Unlike `#each`, for example, which merely iterates
|
88
|
+
# over the results returned by a single API call.) Use with caution.
|
89
89
|
#
|
90
90
|
# @param [Integer] request_limit The upper limit of API requests to
|
91
91
|
# make to load all datasets. Default is no limit.
|
@@ -44,8 +44,8 @@ module Google
|
|
44
44
|
#
|
45
45
|
# bigquery = Google::Cloud::Bigquery.new
|
46
46
|
#
|
47
|
-
#
|
48
|
-
#
|
47
|
+
# job = bigquery.query_job "SELECT COUNT(word) as count FROM " \
|
48
|
+
# "publicdata.samples.shakespeare"
|
49
49
|
#
|
50
50
|
# job.wait_until_done!
|
51
51
|
#
|
@@ -205,6 +205,15 @@ module Google
|
|
205
205
|
Array(@gapi.status.errors).map { |e| JSON.parse e.to_json }
|
206
206
|
end
|
207
207
|
|
208
|
+
##
|
209
|
+
# Cancels the job.
|
210
|
+
def cancel
|
211
|
+
ensure_service!
|
212
|
+
resp = service.cancel_job job_id
|
213
|
+
@gapi = resp.job
|
214
|
+
true
|
215
|
+
end
|
216
|
+
|
208
217
|
##
|
209
218
|
# Created a new job with the current configuration.
|
210
219
|
def rerun!
|
@@ -77,15 +77,15 @@ module Google
|
|
77
77
|
end
|
78
78
|
|
79
79
|
##
|
80
|
-
# Retrieves
|
81
|
-
# returns `false`. Calls the given block once for each
|
82
|
-
# passed as the
|
80
|
+
# Retrieves remaining results by repeatedly invoking {#next} until
|
81
|
+
# {#next?} returns `false`. Calls the given block once for each
|
82
|
+
# result, which is passed as the argument to the block.
|
83
83
|
#
|
84
84
|
# An Enumerator is returned if no block is given.
|
85
85
|
#
|
86
|
-
# This method
|
87
|
-
#
|
88
|
-
# with caution.
|
86
|
+
# This method will make repeated API calls until all remaining results
|
87
|
+
# are retrieved. (Unlike `#each`, for example, which merely iterates
|
88
|
+
# over the results returned by a single API call.) Use with caution.
|
89
89
|
#
|
90
90
|
# @param [Integer] request_limit The upper limit of API requests to
|
91
91
|
# make to load all jobs. Default is no limit.
|
@@ -13,7 +13,7 @@
|
|
13
13
|
# limitations under the License.
|
14
14
|
|
15
15
|
|
16
|
-
require "google/cloud/
|
16
|
+
require "google/cloud/env"
|
17
17
|
require "google/cloud/errors"
|
18
18
|
require "google/cloud/bigquery/service"
|
19
19
|
require "google/cloud/bigquery/credentials"
|
@@ -89,7 +89,7 @@ module Google
|
|
89
89
|
ENV["BIGQUERY_PROJECT"] ||
|
90
90
|
ENV["GOOGLE_CLOUD_PROJECT"] ||
|
91
91
|
ENV["GCLOUD_PROJECT"] ||
|
92
|
-
Google::Cloud
|
92
|
+
Google::Cloud.env.project_id
|
93
93
|
end
|
94
94
|
|
95
95
|
##
|
@@ -139,7 +139,7 @@ module Google
|
|
139
139
|
# should be stored. If not present, a new table will be created to
|
140
140
|
# store the results.
|
141
141
|
# @param [String] create Specifies whether the job is allowed to create
|
142
|
-
# new tables.
|
142
|
+
# new tables. The default value is `needed`.
|
143
143
|
#
|
144
144
|
# The following values are supported:
|
145
145
|
#
|
@@ -147,7 +147,7 @@ module Google
|
|
147
147
|
# * `never` - The table must already exist. A 'notFound' error is
|
148
148
|
# raised if the table does not exist.
|
149
149
|
# @param [String] write Specifies the action that occurs if the
|
150
|
-
# destination table already exists.
|
150
|
+
# destination table already exists. The default value is `empty`.
|
151
151
|
#
|
152
152
|
# The following values are supported:
|
153
153
|
#
|
@@ -183,13 +183,13 @@ module Google
|
|
183
183
|
#
|
184
184
|
# @return [Google::Cloud::Bigquery::QueryJob]
|
185
185
|
#
|
186
|
-
# @example
|
186
|
+
# @example Query using standard SQL:
|
187
187
|
# require "google/cloud/bigquery"
|
188
188
|
#
|
189
189
|
# bigquery = Google::Cloud::Bigquery.new
|
190
190
|
#
|
191
191
|
# job = bigquery.query_job "SELECT name FROM " \
|
192
|
-
# "
|
192
|
+
# "`my_project.my_dataset.my_table`"
|
193
193
|
#
|
194
194
|
# job.wait_until_done!
|
195
195
|
# if !job.failed?
|
@@ -204,7 +204,7 @@ module Google
|
|
204
204
|
# bigquery = Google::Cloud::Bigquery.new
|
205
205
|
#
|
206
206
|
# job = bigquery.query_job "SELECT name FROM " \
|
207
|
-
# "
|
207
|
+
# "[my_project:my_dataset.my_table]",
|
208
208
|
# legacy_sql: true
|
209
209
|
#
|
210
210
|
# job.wait_until_done!
|
@@ -220,7 +220,7 @@ module Google
|
|
220
220
|
# bigquery = Google::Cloud::Bigquery.new
|
221
221
|
#
|
222
222
|
# job = bigquery.query_job "SELECT name FROM " \
|
223
|
-
# "`
|
223
|
+
# "`my_dataset.my_table`" \
|
224
224
|
# " WHERE id = ?",
|
225
225
|
# params: [1]
|
226
226
|
#
|
@@ -237,7 +237,7 @@ module Google
|
|
237
237
|
# bigquery = Google::Cloud::Bigquery.new
|
238
238
|
#
|
239
239
|
# job = bigquery.query_job "SELECT name FROM " \
|
240
|
-
# "`
|
240
|
+
# "`my_dataset.my_table`" \
|
241
241
|
# " WHERE id = @id",
|
242
242
|
# params: { id: 1 }
|
243
243
|
#
|
@@ -347,12 +347,13 @@ module Google
|
|
347
347
|
#
|
348
348
|
# @return [Google::Cloud::Bigquery::QueryData]
|
349
349
|
#
|
350
|
-
# @example
|
350
|
+
# @example Query using standard SQL:
|
351
351
|
# require "google/cloud/bigquery"
|
352
352
|
#
|
353
353
|
# bigquery = Google::Cloud::Bigquery.new
|
354
354
|
#
|
355
|
-
#
|
355
|
+
# sql = "SELECT name FROM `my_project.my_dataset.my_table`"
|
356
|
+
# data = bigquery.query sql
|
356
357
|
#
|
357
358
|
# data.each do |row|
|
358
359
|
# puts row["name"]
|
@@ -363,8 +364,8 @@ module Google
|
|
363
364
|
#
|
364
365
|
# bigquery = Google::Cloud::Bigquery.new
|
365
366
|
#
|
366
|
-
#
|
367
|
-
#
|
367
|
+
# sql = "SELECT name FROM [my_project:my_dataset.my_table]"
|
368
|
+
# data = bigquery.query sql, legacy_sql: true
|
368
369
|
#
|
369
370
|
# data.each do |row|
|
370
371
|
# puts row["name"]
|
@@ -375,7 +376,7 @@ module Google
|
|
375
376
|
#
|
376
377
|
# bigquery = Google::Cloud::Bigquery.new
|
377
378
|
#
|
378
|
-
# data = bigquery.query "SELECT name FROM `
|
379
|
+
# data = bigquery.query "SELECT name FROM `my_dataset.my_table`"
|
379
380
|
#
|
380
381
|
# data.all do |row|
|
381
382
|
# puts row["name"]
|
@@ -387,7 +388,7 @@ module Google
|
|
387
388
|
# bigquery = Google::Cloud::Bigquery.new
|
388
389
|
#
|
389
390
|
# data = bigquery.query "SELECT name " \
|
390
|
-
# "FROM `
|
391
|
+
# "FROM `my_dataset.my_table`" \
|
391
392
|
# "WHERE id = ?",
|
392
393
|
# params: [1]
|
393
394
|
#
|
@@ -401,7 +402,7 @@ module Google
|
|
401
402
|
# bigquery = Google::Cloud::Bigquery.new
|
402
403
|
#
|
403
404
|
# data = bigquery.query "SELECT name " \
|
404
|
-
# "FROM `
|
405
|
+
# "FROM `my_dataset.my_table`" \
|
405
406
|
# "WHERE id = @id",
|
406
407
|
# params: { id: 1 }
|
407
408
|
#
|
@@ -487,8 +488,8 @@ module Google
|
|
487
488
|
#
|
488
489
|
# bigquery = Google::Cloud::Bigquery.new
|
489
490
|
#
|
490
|
-
# dataset = bigquery.create_dataset "my_dataset" do |
|
491
|
-
#
|
491
|
+
# dataset = bigquery.create_dataset "my_dataset" do |dataset|
|
492
|
+
# dataset.access.add_writer_user "writers@example.com"
|
492
493
|
# end
|
493
494
|
#
|
494
495
|
def create_dataset dataset_id, name: nil, description: nil,
|
@@ -708,7 +709,7 @@ module Google
|
|
708
709
|
#
|
709
710
|
# fourpm = bigquery.time 16, 0, 0
|
710
711
|
# data = bigquery.query "SELECT name " \
|
711
|
-
# "FROM `
|
712
|
+
# "FROM `my_dataset.my_table`" \
|
712
713
|
# "WHERE time_of_date = @time",
|
713
714
|
# params: { time: fourpm }
|
714
715
|
#
|
@@ -723,7 +724,7 @@ module Google
|
|
723
724
|
#
|
724
725
|
# precise_time = bigquery.time 16, 35, 15.376541
|
725
726
|
# data = bigquery.query "SELECT name " \
|
726
|
-
# "FROM `
|
727
|
+
# "FROM `my_dataset.my_table`" \
|
727
728
|
# "WHERE time_of_date >= @time",
|
728
729
|
# params: { time: precise_time }
|
729
730
|
#
|
@@ -78,15 +78,15 @@ module Google
|
|
78
78
|
end
|
79
79
|
|
80
80
|
##
|
81
|
-
# Retrieves
|
82
|
-
# returns `false`. Calls the given block once for each
|
83
|
-
# is passed as the
|
81
|
+
# Retrieves remaining results by repeatedly invoking {#next} until
|
82
|
+
# {#next?} returns `false`. Calls the given block once for each
|
83
|
+
# result, which is passed as the argument to the block.
|
84
84
|
#
|
85
85
|
# An Enumerator is returned if no block is given.
|
86
86
|
#
|
87
|
-
# This method
|
88
|
-
# retrieved.
|
89
|
-
#
|
87
|
+
# This method will make repeated API calls until all remaining results
|
88
|
+
# are retrieved. (Unlike `#each`, for example, which merely iterates
|
89
|
+
# over the results returned by a single API call.) Use with caution.
|
90
90
|
#
|
91
91
|
# @param [Integer] request_limit The upper limit of API requests to
|
92
92
|
# make to load all projects. Default is no limit.
|
@@ -130,8 +130,8 @@ module Google
|
|
130
130
|
#
|
131
131
|
# bigquery = Google::Cloud::Bigquery.new
|
132
132
|
#
|
133
|
-
#
|
134
|
-
# job = bigquery.query_job
|
133
|
+
# sql = "SELECT word FROM publicdata.samples.shakespeare"
|
134
|
+
# job = bigquery.query_job sql
|
135
135
|
#
|
136
136
|
# job.wait_until_done!
|
137
137
|
# data = job.query_results
|
@@ -32,8 +32,8 @@ module Google
|
|
32
32
|
# dataset = bigquery.dataset "my_dataset"
|
33
33
|
# table = dataset.table "my_table"
|
34
34
|
#
|
35
|
-
#
|
36
|
-
#
|
35
|
+
# field = table.schema.field "name"
|
36
|
+
# field.required? #=> true
|
37
37
|
class Field
|
38
38
|
# @private
|
39
39
|
MODES = %w( NULLABLE REQUIRED REPEATED )
|
@@ -61,6 +61,9 @@ module Google
|
|
61
61
|
service.request_options.retries = @retries || 3
|
62
62
|
service.request_options.timeout_sec = @timeout
|
63
63
|
service.request_options.open_timeout_sec = @timeout
|
64
|
+
service.request_options.header ||= {}
|
65
|
+
service.request_options.header["x-goog-api-client"] = \
|
66
|
+
"gl-ruby/#{RUBY_VERSION} gccl/#{Google::Cloud::Bigquery::VERSION}"
|
64
67
|
service.authorization = @credentials.client
|
65
68
|
service
|
66
69
|
end
|
@@ -200,6 +203,12 @@ module Google
|
|
200
203
|
end
|
201
204
|
end
|
202
205
|
|
206
|
+
##
|
207
|
+
# Cancel the job specified by jobId.
|
208
|
+
def cancel_job job_id
|
209
|
+
execute { service.cancel_job @project, job_id }
|
210
|
+
end
|
211
|
+
|
203
212
|
##
|
204
213
|
# Returns the job specified by jobID.
|
205
214
|
def get_job job_id
|
@@ -153,7 +153,7 @@ module Google
|
|
153
153
|
# dataset = bigquery.dataset "my_dataset"
|
154
154
|
# table = dataset.table "my_table"
|
155
155
|
#
|
156
|
-
# data = bigquery.query "SELECT
|
156
|
+
# data = bigquery.query "SELECT first_name FROM #{table.query_id}"
|
157
157
|
#
|
158
158
|
# @!group Attributes
|
159
159
|
#
|
@@ -450,7 +450,7 @@ module Google
|
|
450
450
|
# @param [Table, String] destination_table The destination for the
|
451
451
|
# copied data.
|
452
452
|
# @param [String] create Specifies whether the job is allowed to create
|
453
|
-
# new tables.
|
453
|
+
# new tables. The default value is `needed`.
|
454
454
|
#
|
455
455
|
# The following values are supported:
|
456
456
|
#
|
@@ -567,7 +567,7 @@ module Google
|
|
567
567
|
# * `avro` - [Avro](http://avro.apache.org/)
|
568
568
|
# * `datastore_backup` - Cloud Datastore backup
|
569
569
|
# @param [String] create Specifies whether the job is allowed to create
|
570
|
-
# new tables.
|
570
|
+
# new tables. The default value is `needed`.
|
571
571
|
#
|
572
572
|
# The following values are supported:
|
573
573
|
#
|
@@ -575,7 +575,7 @@ module Google
|
|
575
575
|
# * `never` - The table must already exist. A 'notFound' error is
|
576
576
|
# raised if the table does not exist.
|
577
577
|
# @param [String] write Specifies how to handle data already present in
|
578
|
-
# the table. The default value is `
|
578
|
+
# the table. The default value is `append`.
|
579
579
|
#
|
580
580
|
# The following values are supported:
|
581
581
|
#
|
@@ -84,15 +84,15 @@ module Google
|
|
84
84
|
end
|
85
85
|
|
86
86
|
##
|
87
|
-
# Retrieves
|
88
|
-
# returns `false`. Calls the given block once for each
|
89
|
-
# passed as the
|
87
|
+
# Retrieves remaining results by repeatedly invoking {#next} until
|
88
|
+
# {#next?} returns `false`. Calls the given block once for each
|
89
|
+
# result, which is passed as the argument to the block.
|
90
90
|
#
|
91
91
|
# An Enumerator is returned if no block is given.
|
92
92
|
#
|
93
|
-
# This method
|
94
|
-
# retrieved.
|
95
|
-
#
|
93
|
+
# This method will make repeated API calls until all remaining results
|
94
|
+
# are retrieved. (Unlike `#each`, for example, which merely iterates
|
95
|
+
# over the results returned by a single API call.) Use with caution.
|
96
96
|
#
|
97
97
|
# @param [Integer] request_limit The upper limit of API requests to
|
98
98
|
# make to load all tables. Default is no limit.
|
@@ -30,7 +30,7 @@ module Google
|
|
30
30
|
#
|
31
31
|
# fourpm = Google::Cloud::Bigquery::Time.new "16:00:00"
|
32
32
|
# data = bigquery.query "SELECT name " \
|
33
|
-
# "FROM `
|
33
|
+
# "FROM `my_project.my_dataset.my_table`" \
|
34
34
|
# "WHERE time_of_date = @time",
|
35
35
|
# params: { time: fourpm }
|
36
36
|
#
|
@@ -45,7 +45,7 @@ module Google
|
|
45
45
|
#
|
46
46
|
# precise_time = Google::Cloud::Bigquery::Time.new "16:35:15.376541"
|
47
47
|
# data = bigquery.query "SELECT name " \
|
48
|
-
# "FROM `
|
48
|
+
# "FROM `my_project.my_dataset.my_table`" \
|
49
49
|
# "WHERE time_of_date >= @time",
|
50
50
|
# params: { time: precise_time }
|
51
51
|
#
|
@@ -40,7 +40,7 @@ module Google
|
|
40
40
|
# bigquery = Google::Cloud::Bigquery.new
|
41
41
|
# dataset = bigquery.dataset "my_dataset"
|
42
42
|
# view = dataset.create_view "my_view",
|
43
|
-
# "SELECT name, age FROM
|
43
|
+
# "SELECT name, age FROM `my_project.my_dataset.my_table`"
|
44
44
|
#
|
45
45
|
class View
|
46
46
|
##
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: google-cloud-bigquery
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.25.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Mike Moore
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2017-
|
12
|
+
date: 2017-04-01 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: google-cloud-core
|
@@ -17,14 +17,14 @@ dependencies:
|
|
17
17
|
requirements:
|
18
18
|
- - "~>"
|
19
19
|
- !ruby/object:Gem::Version
|
20
|
-
version:
|
20
|
+
version: '1.0'
|
21
21
|
type: :runtime
|
22
22
|
prerelease: false
|
23
23
|
version_requirements: !ruby/object:Gem::Requirement
|
24
24
|
requirements:
|
25
25
|
- - "~>"
|
26
26
|
- !ruby/object:Gem::Version
|
27
|
-
version:
|
27
|
+
version: '1.0'
|
28
28
|
- !ruby/object:Gem::Dependency
|
29
29
|
name: google-api-client
|
30
30
|
requirement: !ruby/object:Gem::Requirement
|
@@ -222,7 +222,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
222
222
|
version: '0'
|
223
223
|
requirements: []
|
224
224
|
rubyforge_project:
|
225
|
-
rubygems_version: 2.6.
|
225
|
+
rubygems_version: 2.6.11
|
226
226
|
signing_key:
|
227
227
|
specification_version: 4
|
228
228
|
summary: API Client library for Google BigQuery
|