google-cloud-bigquery 1.21.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: f38543236358fc319ecfcc2058ffa499e24034027aa56644924d2cf496815550
+  data.tar.gz: 37196fa1c3db03e48df4cb0ae5ae5b9b1ee07b9d112063467ebc57d25b34551a
+SHA512:
+  metadata.gz: a234359fb8a04b42f22725f5540cf0378b7e51b3d847ddd4f24ba41f2a54b4de85068bb9aee10089995c29794c0a64694223ac6bf88202eda6a646ce02c85275
+  data.tar.gz: 99c6c915df70afde18de06907802c60d18df253238e6f3ab752827ea3cfc0aee0ab9bef8dc0816f5980db26334dd2b15ce0a2ecf7db3b026e2f0caa101dcafc8

data/.yardopts

@@ -0,0 +1,16 @@
+--no-private
+--title=Google Cloud BigQuery
+--markup markdown
+--markup-provider redcarpet
+--main OVERVIEW.md
+
+./lib/**/*.rb
+-
+OVERVIEW.md
+AUTHENTICATION.md
+LOGGING.md
+CONTRIBUTING.md
+TROUBLESHOOTING.md
+CHANGELOG.md
+CODE_OF_CONDUCT.md
+LICENSE

data/AUTHENTICATION.md

@@ -0,0 +1,158 @@
+# Authentication
+
+In general, the google-cloud-bigquery library uses [Service
+Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)
+credentials to connect to Google Cloud services. When running on Google Cloud
+Platform (GCP), including Google Compute Engine (GCE), Google Kubernetes Engine
+(GKE), Google App Engine (GAE), Google Cloud Functions (GCF) and Cloud Run,
+the credentials will be discovered automatically. When running on other
+environments, the Service Account credentials can be specified by providing the
+path to the [JSON
+keyfile](https://cloud.google.com/iam/docs/managing-service-account-keys) for
+the account (or the JSON itself) in environment variables. Additionally, Cloud
+SDK credentials can also be discovered automatically, but this is only
+recommended during development.
+
+## Project and Credential Lookup
+
+The google-cloud-bigquery library aims to make authentication as simple as
+possible, and provides several mechanisms to configure your system without
+providing **Project ID** and **Service Account Credentials** directly in code.
+
+**Project ID** is discovered in the following order:
+
+1. Specify project ID in method arguments
+2. Specify project ID in configuration
+3. Discover project ID in environment variables
+4. Discover GCE project ID
+
+**Credentials** are discovered in the following order:
+
+1. Specify credentials in method arguments
+2. Specify credentials in configuration
+3. Discover credentials path in environment variables
+4. Discover credentials JSON in environment variables
+5. Discover credentials file in the Cloud SDK's path
+6. Discover GCE credentials
+
+### Google Cloud Platform environments
+
+When running on Google Cloud Platform (GCP), including Google Compute Engine (GCE),
+Google Kubernetes Engine (GKE), Google App Engine (GAE), Google Cloud Functions
+(GCF) and Cloud Run, the **Project ID** and **Credentials** and are discovered
+automatically. Code should be written as if already authenticated.
+
+### Environment Variables
+
+The **Project ID** and **Credentials JSON** can be placed in environment
+variables instead of declaring them directly in code. Each service has its own
+environment variable, allowing for different service accounts to be used for
+different services. (See the READMEs for the individual service gems for
+details.) The path to the **Credentials JSON** file can be stored in the
+environment variable, or the **Credentials JSON** itself can be stored for
+environments such as Docker containers where writing files is difficult or not
+encouraged.
+
+The environment variables that BigQuery checks for project ID are:
+
+1. `BIGQUERY_PROJECT`
+2. `GOOGLE_CLOUD_PROJECT`
+
+The environment variables that BigQuery checks for credentials are configured on {Google::Cloud::Bigquery::Credentials}:
+
+1. `BIGQUERY_CREDENTIALS` - Path to JSON file, or JSON contents
+2. `BIGQUERY_KEYFILE` - Path to JSON file, or JSON contents
+3. `GOOGLE_CLOUD_CREDENTIALS` - Path to JSON file, or JSON contents
+4. `GOOGLE_CLOUD_KEYFILE` - Path to JSON file, or JSON contents
+5. `GOOGLE_APPLICATION_CREDENTIALS` - Path to JSON file
+
+```ruby
+require "google/cloud/bigquery"
+
+ENV["BIGQUERY_PROJECT"]     = "my-project-id"
+ENV["BIGQUERY_CREDENTIALS"] = "path/to/keyfile.json"
+
+bigquery = Google::Cloud::Bigquery.new
+```
+
+### Configuration
+
+The **Project ID** and **Credentials JSON** can be configured instead of placing them in environment variables or providing them as arguments.
+
+```ruby
+require "google/cloud/bigquery"
+
+Google::Cloud::Bigquery.configure do |config|
+  config.project_id  = "my-project-id"
+  config.credentials = "path/to/keyfile.json"
+end
+
+bigquery = Google::Cloud::Bigquery.new
+```
+
+### Cloud SDK
+
+This option allows for an easy way to authenticate during development. If
+credentials are not provided in code or in environment variables, then Cloud SDK
+credentials are discovered.
+
+To configure your system for this, simply:
+
+1. [Download and install the Cloud SDK](https://cloud.google.com/sdk)
+2. Authenticate using OAuth 2.0 `$ gcloud auth login`
+3. Write code as if already authenticated.
+
+**NOTE:** This is _not_ recommended for running in production. The Cloud SDK
+*should* only be used during development.
+
+[gce-how-to]: https://cloud.google.com/compute/docs/authentication#using
+[dev-console]: https://console.cloud.google.com/project
+
+[enable-apis]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/enable-apis.png
+
+[create-new-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account.png
+[create-new-service-account-existing-keys]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/create-new-service-account-existing-keys.png
+[reuse-service-account]: https://raw.githubusercontent.com/GoogleCloudPlatform/gcloud-common/master/authentication/reuse-service-account.png
+
+## Creating a Service Account
+
+Google Cloud requires a **Project ID** and **Service Account Credentials** to
+connect to the APIs. You will use the **Project ID** and **JSON key file** to
+connect to most services with google-cloud-bigquery.
+
+If you are not running this client on Google Compute Engine, you need a Google
+Developers service account.
+
+1. Visit the [Google Developers Console][dev-console].
+1. Create a new project or click on an existing project.
+1. Activate the slide-out navigation tray and select **API Manager**. From
+   here, you will enable the APIs that your application requires.
+
+   ![Enable the APIs that your application requires][enable-apis]
+
+   *Note: You may need to enable billing in order to use these services.*
+
+1. Select **Credentials** from the side navigation.
+
+   You should see a screen like one of the following.
+
+   ![Create a new service account][create-new-service-account]
+
+   ![Create a new service account With Existing Keys][create-new-service-account-existing-keys]
+
+   Find the "Add credentials" drop down and select "Service account" to be
+   guided through downloading a new JSON key file.
+
+   If you want to re-use an existing service account, you can easily generate a
+   new key file. Just select the account you wish to re-use, and click "Generate
+   new JSON key":
+
+   ![Re-use an existing service account][reuse-service-account]
+
+   The key file you download will be used by this library to authenticate API
+   requests and should be stored in a secure location.
+
+## Troubleshooting
+
+If you're having trouble authenticating you can ask for help by following the
+{file:TROUBLESHOOTING.md Troubleshooting Guide}.

data/CHANGELOG.md

@@ -0,0 +1,397 @@
+# Release History
+
+### 1.21.2 / 2020-07-21
+
+#### Documentation
+
+* Update Data#each samples
+
+### 1.21.1 / 2020-05-28
+
+#### Documentation
+
+* Fix a few broken links
+
+### 1.21.0 / 2020-03-31
+
+#### Features
+
+* Add Job#parent_job_id and Job#script_statistics
+  * Add parent_job to Project#jobs
+  * Add Job#num_child_jobs
+  * Add Job#parent_job_id
+  * Add Job#script_statistics
+
+### 1.20.0 / 2020-03-11
+
+#### Features
+
+* Add Range Partitioning
+  * Add range partitioning methods to Table and Table::Updater
+  * Add range partitioning methods to LoadJob
+  * Add range partitioning methods to QueryJob
+
+### 1.19.0 / 2020-02-11
+
+#### Features
+
+* Add Routine
+  * Add Dataset#create_routine
+  * Add Argument
+  * Update StandardSql classes to expose public initializer
+  * Add Data#ddl_target_routine and QueryJob#ddl_target_routine
+* Allow row inserts to skip insert_id generation
+  * Streaming inserts using an insert_id are not able to be inserted as fast as inserts without an insert_id
+  * Add the ability for users to skip insert_id generation in order to speed up the inserts
+  * The default behavior continues to generate insert_id values for each row inserted
+  * Add yield documentation for Dataset#insert
+
+### 1.18.1 / 2019-12-18
+
+#### Bug Fixes
+
+* Fix MonitorMixin usage on Ruby 2.7
+  * Ruby 2.7 will error if new_cond is called before super().
+  * Make the call to super() be the first call in initialize
+
+### 1.18.0 / 2019-11-06
+
+#### Features
+
+* Add optional query parameter types
+  * Allow query parameters to be nil/NULL when providing an optional
+  * Add types argument to the following methods:
+    * Project#query
+    * Project#query_job
+    * Dataset#query
+    * Dataset#query_job
+* Add param types helper methods
+  * Return the BigQuery field type code, using the same format as the
+  * Add Schema::Field#param_type
+  * Add Schema#param_types
+  * Add Data#param_types
+  * Add Table#param_types
+  * Add External::CvsSource#param_types
+  * Add External::JsonSource#param_types
+* Add support for all_users special role in Dataset access
+
+### 1.17.0 / 2019-10-29
+
+This release requires Ruby 2.4 or later.
+
+#### Documentation
+
+* Clarify which Google Cloud Platform environments support automatic authentication
+
+### 1.16.0 / 2019-10-03
+
+#### Features
+
+* Add Dataset default_encryption
+  * Add Dataset#default_encryption
+  * Add Dataset#default_encryption=
+
+### 1.15.0 / 2019-09-30
+
+#### Features
+
+* Add Model encryption
+  * Add Model#encryption
+  * Add Model#encryption=
+* Add range support for Google Sheets
+  * Add External::SheetsSource#range
+  * Add External::SheetsSource#range=
+* Support use_avro_logical_types on extract jobs
+  * Add ExtractJob#use_avro_logical_types?
+  * Add ExtractJob::Updater#use_avro_logical_types=
+
+### 1.14.1 / 2019-09-04
+
+#### Documentation
+
+* Add note about streaming insert issues
+  * Acknowledge tradeoffs when inserting rows soon after
+    table metadata has been changed.
+  * Add link to BigQuery Troubleshooting guide.
+
+### 1.14.0 / 2019-08-23
+
+#### Features
+
+* Support overriding of service endpoint
+
+#### Performance Improvements
+
+* Use MiniMime to detect content types
+
+#### Documentation
+
+* Update documentation
+
+### 1.13.0 / 2019-07-31
+
+* Add Table#require_partition_filter
+* List jobs using min and max created_at
+* Reduce thread usage at startup
+  * Allocate threads in pool as needed, not all up front
+* Update documentation links
+
+### 1.12.0 / 2019-07-10
+
+* Add BigQuery Model API
+  * Add Model
+  * Add StandardSql Field, DataType, StructType
+  * Add Dataset#model and Dataset#models
+* Correct Float value conversion
+  * Ensure that NaN, Infinity, and -Infinity are converted correctly.
+
+### 1.11.2 / 2019-06-11
+
+* Update "Loading data" link
+
+### 1.11.1 / 2019-05-21
+
+* Declare explicit dependency on mime-types
+
+### 1.11.0 / 2019-02-01
+
+* Make use of Credentials#project_id
+  * Use Credentials#project_id
+    If a project_id is not provided, use the value on the Credentials object.
+    This value was added in googleauth 0.7.0.
+  * Loosen googleauth dependency
+    Allow for new releases up to 0.10.
+    The googleauth devs have committed to maintaining the current API
+    and will not make backwards compatible changes before 0.10.
+
+### 1.10.0 / 2018-12-06
+
+* Add dryrun param to Project#query_job and Dataset#query_job
+* Add copy and extract methods to Project
+  * Add Project#extract and Project#extract_job
+  * Add Project#copy and Project#copy_job
+  * Deprecate dryrun param in Table#copy_job, Table#extract_job and
+    Table#load_job
+* Fix memoization in Dataset#exists? and Table#exists?
+  * Add force param to Dataset#exists? and Table#exists?
+
+### 1.9.0 / 2018-10-25
+
+* Add clustering fields to LoadJob, QueryJob and Table
+* Add DDL/DML support
+  * Update QueryJob#data to not return table rows for DDL/DML
+  * Add DDL/DML statistics attrs to QueryJob and Data
+* Add #numeric to Table::Updater and LoadJob::Updater (@leklund)
+
+### 1.8.2 / 2018-09-20
+
+* Update documentation.
+  * Change documentation URL to googleapis GitHub org.
+* Fix circular require warning.
+
+### 1.8.1 / 2018-09-12
+
+* Add missing documentation files to package.
+
+### 1.8.0 / 2018-09-10
+
+* Add support for OCR format.
+* Update documentation.
+
+### 1.7.1 / 2018-08-21
+
+* Update documentation.
+
+### 1.7.0 / 2018-06-29
+
+* Add #schema_update_options to LoadJob and #schema_update_options= to LoadJob::Updater.
+* Add time partitioning for the target table to LoadJob and QueryJob.
+* Add #statement_type, #ddl_operation_performed, #ddl_target_table to QueryJob.
+
+### 1.6.0 / 2018-06-22
+
+* Documentation updates.
+* Updated dependencies.
+
+### 1.5.0 / 2018-05-21
+
+* Add Schema.load and Schema.dump to read/write a table schema from/to a JSON file or other IO source. The JSON file schema is the same as for the bq CLI.
+* Add support for the NUMERIC data type.
+* Add documentation for enabling logging.
+
+### 1.4.0 / 2018-05-07
+
+* Add Parquet support to #load and #load_job.
+
+### 1.3.0 / 2018-04-05
+
+* Add insert_ids option to #insert in Dataset, Table, and AsyncInserter.
+* Add BigQuery Project#service_account_email.
+* Add support for setting Job location to nil in blocks for Job properties.
+
+### 1.2.0 / 2018-03-31
+
+* Add geo-regionalization (location) support to Jobs.
+* Add Project#encryption support to Jobs.
+* Rename Encryption to EncryptionConfiguration.
+* Add blocks for setting Job properties to all Job creation methods.
+* Add support for lists of URLs to #load and #load_job. (jeremywadsack)
+* Fix Schema::Field type helpers.
+* Fix Table#load example in README.
+
+### 1.1.0 / 2018-02-27
+
+* Support table partitioning by field.
+* Support Shared Configuration.
+* Improve AsyncInserter performance.
+
+### 1.0.0 / 2018-01-10
+
+* Release 1.0.0
+* Update authentication documentation
+  * Update Data documentation and code examples
+  * Remove reference to sync and async queries
+* Allow use of URI objects for Dataset#load, Table#load, and Table#load_job
+
+### 0.30.0 / 2017-11-14
+
+* Add `Google::Cloud::Bigquery::Credentials` class.
+* Rename constructor arguments to `project_id` and `credentials`.
+  (The previous arguments `project` and `keyfile` are still supported.)
+* Support creating `Dataset` and `Table` objects without making API calls using
+  `skip_lookup` argument.
+  * Add `Dataset#reference?` and `Dataset#resource?` helper method.
+  * Add `Table#reference?` and `Table#resource?` and `Table#resource_partial?`
+    and `Table#resource_full?` helper methods.
+* `Dataset#insert_async` and `Dataset#insert_async` now yields a
+  `Table::AsyncInserter::Result` object.
+* `View` is removed, now uses `Table` class.
+  * Needed to support `skip_lookup` argument.
+  * Calling `Table#data` on a view now raises (breaking change).
+* Performance improvements for queries.
+* Updated `google-api-client`, `googleauth` dependencies.
+
+### 0.29.0 / 2017-10-09
+
+This is a major release with many new features and several breaking changes.
+
+#### Major Changes
+
+* All queries now use a new implementation, using a job and polling for results.
+* The copy, load, extract methods now all have high-level and low-level versions, similar to `query` and `query_job`.
+* Added asynchronous row insertion, allowing data to be collected and inserted in batches.
+* Support external data sources for both queries and table views.
+* Added create-on-insert support for tables.
+* Allow for customizing job IDs to aid in organizing jobs.
+
+#### Change Details
+
+* Update high-level queries as follows:
+  * Update `QueryJob#wait_until_done!` to use `getQueryResults`.
+  * Update `Project#query` and `Dataset#query` with breaking changes:
+    * Remove `timeout` and `dryrun` parameters.
+    * Change return type from `QueryData` to `Data`.
+  * Add `QueryJob#data`
+  * Alias `QueryJob#query_results` to `QueryJob#data` with breaking changes:
+    * Remove the `timeout` parameter.
+    * Change the return type from `QueryData` to `Data`.
+  * Update `View#data` with breaking changes:
+    * Remove the `timeout` and `dryrun` parameters.
+    * Change the return type from `QueryData` to `Data`.
+  * Remove `QueryData`.
+  * Update `Project#query` and `Dataset#query` with improved errors, replacing the previous simple error with one that contains all available information for why the job failed.
+* Rename `Dataset#load` to `Dataset#load_job`; add high-level, synchronous version as `Dataset#load`.
+* Rename `Table#copy` to `Table#copy_job`; add high-level, synchronous version as `Table#copy`.
+* Rename `Table#extract` to `Table#extract_job`; add high-level, synchronous version as `Table#extract`.
+* Rename `Table#load` to `Table#load_job`; add high-level, synchronous version as `Table#load`.
+* Add support for querying external data sources with `External`.
+* Add `Table::AsyncInserter`, `Dataset#insert_async` and `Table#insert_async` to collect and insert rows in batches.
+* Add `Dataset#insert` to support creating a table while inserting rows if the table does not exist.
+* Update retry logic to conform to the [BigQuery SLA](https://cloud.google.com/bigquery/sla).
+  * Use a minimum back-off interval of 1 second; for each consecutive error, increase the back-off interval exponentially up to 32 seconds.
+  * Retry if all error reasons are retriable, not if any of the error reasons are retriable.
+* Add support for labels to `Dataset`, `Table`, `View` and `Job`.
+  * Add `filter` option to `Project#datasets` and `Project#jobs`.
+* Add support for user-defined functions to `Project#query_job`, `Dataset#query_job`, `QueryJob` and `View`.
+* In `Dataset`, `Table`, and `View` updates, add the use of ETags for optimistic concurrency control.
+* Update `Dataset#load` and `Table#load`:
+  * Add `null_marker` option and `LoadJob#null_marker`.
+  * Add `autodetect` option and `LoadJob#autodetect?`.
+* Fix the default value for `LoadJob#quoted_newlines?`.
+* Add `job_id` and `prefix` options for controlling client-side job ID generation to `Project#query_job`, `Dataset#load`, `Dataset#query_job`, `Table#copy`, `Table#extract`, and `Table#load`.
+* Add `Job#user_email`.
+* Set the maximum delay of `Job#wait_until_done!` polling to 60 seconds.
+* Automatically retry `Job#cancel`.
+* Allow users to specify if a `View` query is using Standard vs. Legacy SQL.
+* Add `project` option to `Project#query_job`.
+* Add `QueryJob#query_plan`, `QueryJob::Stage` and `QueryJob::Step` to expose query plan information.
+* Add `Table#buffer_bytes`, `Table#buffer_rows` and `Table#buffer_oldest_at` to expose streaming buffer information.
+* Update `Dataset#insert` and `Table#insert` to raise an error if `rows` is empty.
+* Update `Error` with a mapping from code 412 to `FailedPreconditionError`.
+* Update `Data#schema` to freeze the returned `Schema` object (as in `View` and `LoadJob`.)
+
+### 0.28.0 / 2017-09-28
+
+* Update Google API Client dependency to 0.14.x.
+
+### 0.27.1 / 2017-07-11
+
+* Add `InsertResponse::InsertError#index` (zedalaye)
+
+### 0.27.0 / 2017-06-28
+
+* Add `maximum_billing_tier` and `maximum_bytes_billed` to `QueryJob`, `Project#query_job` and `Dataset#query_job`.
+* Add `Dataset#load` to support creating, configuring and loading a table in one API call.
+* Add `Project#schema`.
+* Upgrade dependency on Google API Client.
+* Update gem spec homepage links.
+* Update examples of field access to use symbols instead of strings in the documentation.
+
+### 0.26.0 / 2017-04-05
+
+* Upgrade dependency on Google API Client
+
+### 0.25.0 / 2017-03-31
+
+* Add `#cancel` to `Job`
+* Updated documentation
+
+### 0.24.0 / 2017-03-03
+
+Major release, several new features, some breaking changes.
+
+* Standard SQL is now the default syntax.
+* Legacy SQL syntax can be enabled by providing `legacy_sql: true`.
+* Several fixes to how data values are formatted when returned from BigQuery.
+* Returned data rows are now hashes with Symbol keys instead of String keys.
+* Several fixes to how data values are formatted when importing to BigQuery.
+* Several improvements to manipulating table schema fields.
+* Removal of `Schema#fields=` and `Data#raw` methods.
+* Removal of `fields` argument from `Dataset#create_table` method.
+* Dependency on Google API Client has been updated to 0.10.x.
+
+### 0.23.0 / 2016-12-8
+
+* Support Query Parameters using `params` method arguments to `query` and `query_job`
+* Add `standard_sql`/`legacy_sql` method arguments to to `query` and `query_job`
+* Add `standard_sql?`/`legacy_sql?` attributes to `QueryJob`
+* Many documentation improvements
+
+### 0.21.0 / 2016-10-20
+
+* New service constructor Google::Cloud::Bigquery.new
+
+### 0.20.2 / 2016-09-30
+
+* Add list of projects that the current credentials can access. (remi)
+
+### 0.20.1 / 2016-09-02
+
+* Fix for timeout on uploads.
+
+### 0.20.0 / 2016-08-26
+
+This gem contains the Google BigQuery service implementation for the `google-cloud` gem. The `google-cloud` gem replaces the old `gcloud` gem. Legacy code can continue to use the `gcloud` gem.
+
+* Namespace is now `Google::Cloud`
+* The `google-cloud` gem is now an umbrella package for individual gems