google-cloud-storage 2.18.2__tar.gz → 3.0.0__tar.gz

{google_cloud_storage-2.18.2/google_cloud_storage.egg-info → google_cloud_storage-3.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: google-cloud-storage
-Version: 2.18.2
+Version: 3.0.0
 Summary: Google Cloud Storage API client library
 Home-page: https://github.com/googleapis/python-storage
 Author: Google LLC
@@ -72,6 +72,76 @@ Google APIs Client Libraries, in `Client Libraries Explained`_.
 .. _Storage Control API: https://cloud.google.com/storage/docs/reference/rpc/google.storage.control.v2
 .. _Client Libraries Explained: https://cloud.google.com/apis/docs/client-libraries-explained
 
+3.0 Major Version Notes
+-----------------------
+
+Feedback Welcome
+~~~~~~~~~~~~~~~~
+
+If you experience that backwards compatibility for your application is broken
+with this major version release, please let us know through the Github issues
+system. While some breaks of backwards compatibility may be unavoidable due to
+new features in the major version release, we will do our best to minimize
+them. Thank you.
+
+Exception Handling
+~~~~~~~~~~~~~~~~~~
+
+In Python Storage 3.0, the dependency ``google-resumable-media`` was integrated.
+The ``google-resumable-media`` dependency included exceptions
+``google.resumable_media.common.InvalidResponse`` and
+``google.resumable_media.common.DataCorruption``, which were often imported
+directly in user application code. The replacements for these exceptions are
+``google.cloud.storage.exceptions.InvalidResponse`` and
+``google.cloud.storage.exceptions.DataCorruption``. Please update application code
+to import and use these exceptions instead.
+
+For backwards compatibility, if ``google-resumable-media`` is installed, the new
+exceptions will be defined as subclasses of the old exceptions, so applications
+should continue to work without modification. This backwards compatibility
+feature may be removed in a future major version update.
+
+Some users may be using the original exception classes from the
+``google-resumable-media`` library without explicitly installing that library. So
+as not to break user applications following this pattern,
+``google-resumable-media`` is still in the list of dependencies in this package's
+setup.py file. Applications which do not import directly from
+``google-resumable-media`` can safely disregard this dependency.
+This backwards compatibility feature **will be removed** in a future major
+version update. Please migrate to using the ``google.cloud.storage.exceptions``
+classes as above.
+
+Checksum Defaults
+~~~~~~~~~~~~~~~~~
+
+In Python Storage 3.0, uploads and downloads now have a default of "auto" where
+applicable. "Auto" will use crc32c checksums, except for unusual cases where the
+fast (C extension) crc32c implementation is not available, in which case it will
+use md5 instead. Before Python Storage 3.0, the default was md5 for most
+downloads and None for most uploads. Note that ranged downloads ("start" or
+"end" set) still do not support any checksumming, and some features in
+``transfer_manager.py`` still support crc32c only.
+
+Note: The method ``Blob.upload_from_file()`` requires a file in bytes mode, but
+when checksum is set to None, as was the previous default, would not throw an
+error if passed a file in string mode under some circumstances. With the new
+defaults, it will now raise a TypeError. Please use a file opened in bytes
+reading mode as required.
+
+Miscellaneous
+~~~~~~~~~~~~~
+
+- The ``BlobWriter`` class now attempts to terminate an ongoing resumable upload if
+  the writer exits with an exception.
+- Retry behavior is now identical between media operations (uploads and
+  downloads) and other operations, and custom predicates are now supported for
+  media operations as well.
+- ``Blob.download_as_filename()`` will now delete the empty file if it results in a
+  google.cloud.exceptions.NotFound exception (HTTP 404).
+- Previously, object upload, metadata update, and delete methods had retries
+  disabled by default unless the generation or metageneration was specified in
+  the request. This has now changed so that retries are enabled by default.
+
 Quick Start
 -----------
 

{google_cloud_storage-2.18.2 → google_cloud_storage-3.0.0}/README.rst

@@ -37,6 +37,76 @@ Google APIs Client Libraries, in `Client Libraries Explained`_.
 .. _Storage Control API: https://cloud.google.com/storage/docs/reference/rpc/google.storage.control.v2
 .. _Client Libraries Explained: https://cloud.google.com/apis/docs/client-libraries-explained
 
+3.0 Major Version Notes
+-----------------------
+
+Feedback Welcome
+~~~~~~~~~~~~~~~~
+
+If you experience that backwards compatibility for your application is broken
+with this major version release, please let us know through the Github issues
+system. While some breaks of backwards compatibility may be unavoidable due to
+new features in the major version release, we will do our best to minimize
+them. Thank you.
+
+Exception Handling
+~~~~~~~~~~~~~~~~~~
+
+In Python Storage 3.0, the dependency ``google-resumable-media`` was integrated.
+The ``google-resumable-media`` dependency included exceptions
+``google.resumable_media.common.InvalidResponse`` and
+``google.resumable_media.common.DataCorruption``, which were often imported
+directly in user application code. The replacements for these exceptions are
+``google.cloud.storage.exceptions.InvalidResponse`` and
+``google.cloud.storage.exceptions.DataCorruption``. Please update application code
+to import and use these exceptions instead.
+
+For backwards compatibility, if ``google-resumable-media`` is installed, the new
+exceptions will be defined as subclasses of the old exceptions, so applications
+should continue to work without modification. This backwards compatibility
+feature may be removed in a future major version update.
+
+Some users may be using the original exception classes from the
+``google-resumable-media`` library without explicitly installing that library. So
+as not to break user applications following this pattern,
+``google-resumable-media`` is still in the list of dependencies in this package's
+setup.py file. Applications which do not import directly from
+``google-resumable-media`` can safely disregard this dependency.
+This backwards compatibility feature **will be removed** in a future major
+version update. Please migrate to using the ``google.cloud.storage.exceptions``
+classes as above.
+
+Checksum Defaults
+~~~~~~~~~~~~~~~~~
+
+In Python Storage 3.0, uploads and downloads now have a default of "auto" where
+applicable. "Auto" will use crc32c checksums, except for unusual cases where the
+fast (C extension) crc32c implementation is not available, in which case it will
+use md5 instead. Before Python Storage 3.0, the default was md5 for most
+downloads and None for most uploads. Note that ranged downloads ("start" or
+"end" set) still do not support any checksumming, and some features in
+``transfer_manager.py`` still support crc32c only.
+
+Note: The method ``Blob.upload_from_file()`` requires a file in bytes mode, but
+when checksum is set to None, as was the previous default, would not throw an
+error if passed a file in string mode under some circumstances. With the new
+defaults, it will now raise a TypeError. Please use a file opened in bytes
+reading mode as required.
+
+Miscellaneous
+~~~~~~~~~~~~~
+
+- The ``BlobWriter`` class now attempts to terminate an ongoing resumable upload if
+  the writer exits with an exception.
+- Retry behavior is now identical between media operations (uploads and
+  downloads) and other operations, and custom predicates are now supported for
+  media operations as well.
+- ``Blob.download_as_filename()`` will now delete the empty file if it results in a
+  google.cloud.exceptions.NotFound exception (HTTP 404).
+- Previously, object upload, metadata update, and delete methods had retries
+  disabled by default unless the generation or metageneration was specified in
+  the request. This has now changed so that retries are enabled by default.
+
 Quick Start
 -----------
 

{google_cloud_storage-2.18.2 → google_cloud_storage-3.0.0}/google/cloud/storage/_helpers.py

@@ -25,7 +25,6 @@ from urllib.parse import urlsplit
 from urllib.parse import urlunsplit
 from uuid import uuid4
 
-from google import resumable_media
 from google.auth import environment_vars
 from google.cloud.storage.constants import _DEFAULT_TIMEOUT
 from google.cloud.storage.retry import DEFAULT_RETRY
@@ -72,12 +71,6 @@ _GENERATION_MATCH_PARAMETERS = (
     ("if_source_metageneration_not_match", "ifSourceMetagenerationNotMatch"),
 )
 
-_NUM_RETRIES_MESSAGE = (
-    "`num_retries` has been deprecated and will be removed in a future "
-    "release. Use the `retry` argument with a Retry or ConditionalRetryPolicy "
-    "object, or None, instead."
-)
-
 # _NOW() returns the current local date and time.
 # It is preferred to use timezone-aware datetimes _NOW(_UTC),
 # which returns the current UTC date and time.
@@ -293,6 +286,9 @@ class _PropertyMixin(object):
         )
         if soft_deleted is not None:
             query_params["softDeleted"] = soft_deleted
+            # Soft delete reload requires a generation, even for targets
+            # that don't include them in default query params (buckets).
+            query_params["generation"] = self.generation
         headers = self._encryption_headers()
         _add_etag_match_headers(
             headers, if_etag_match=if_etag_match, if_etag_not_match=if_etag_not_match
@@ -343,7 +339,7 @@ class _PropertyMixin(object):
         if_metageneration_match=None,
         if_metageneration_not_match=None,
         timeout=_DEFAULT_TIMEOUT,
-        retry=DEFAULT_RETRY_IF_METAGENERATION_SPECIFIED,
+        retry=DEFAULT_RETRY,
         override_unlocked_retention=False,
     ):
         """Sends all changed properties in a PATCH request.
@@ -631,40 +627,6 @@ def _bucket_bound_hostname_url(host, scheme=None):
     return f"{scheme}://{host}"
 
 
-def _api_core_retry_to_resumable_media_retry(retry, num_retries=None):
-    """Convert google.api.core.Retry to google.resumable_media.RetryStrategy.
-
-    Custom predicates are not translated.
-
-    :type retry: google.api_core.Retry
-    :param retry: (Optional) The google.api_core.Retry object to translate.
-
-    :type num_retries: int
-    :param num_retries: (Optional) The number of retries desired. This is
-        supported for backwards compatibility and is mutually exclusive with
-        `retry`.
-
-    :rtype: google.resumable_media.RetryStrategy
-    :returns: A RetryStrategy with all applicable attributes copied from input,
-              or a RetryStrategy with max_retries set to 0 if None was input.
-    """
-
-    if retry is not None and num_retries is not None:
-        raise ValueError("num_retries and retry arguments are mutually exclusive")
-
-    elif retry is not None:
-        return resumable_media.RetryStrategy(
-            max_sleep=retry._maximum,
-            max_cumulative_retry=retry._deadline,
-            initial_delay=retry._initial,
-            multiplier=retry._multiplier,
-        )
-    elif num_retries is not None:
-        return resumable_media.RetryStrategy(max_retries=num_retries)
-    else:
-        return resumable_media.RetryStrategy(max_retries=0)
-
-
 def _get_invocation_id():
     return "gccl-invocation-id/" + str(uuid4())
 

google_cloud_storage-3.0.0/google/cloud/storage/_media/__init__.py

@@ -0,0 +1,34 @@
+# Copyright 2017 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Utilities for Google Media Downloads and Resumable Uploads.
+
+===========
+Subpackages
+===========
+
+Each subpackage is tailored to a specific transport library:
+
+* the :mod:`~google.cloud.storage._media.requests` subpackage uses the ``requests``
+  transport library.
+
+.. _requests: http://docs.python-requests.org/
+"""
+
+from google.cloud.storage._media.common import UPLOAD_CHUNK_SIZE
+
+
+__all__ = [
+    "UPLOAD_CHUNK_SIZE",
+]