@google-cloud/pubsub 4.7.2 → 4.9.0

package/CHANGELOG.md

@@ -4,6 +4,26 @@
 
 [1]: https://www.npmjs.com/package/@google-cloud/pubsub?activeTab=versions
 
+## [4.9.0](https://github.com/googleapis/nodejs-pubsub/compare/v4.8.0...v4.9.0) (2024-11-12)
+
+
+### Features
+
+* Add IngestionFailureEvent to the external proto ([#1984](https://github.com/googleapis/nodejs-pubsub/issues/1984)) ([7075430](https://github.com/googleapis/nodejs-pubsub/commit/70754309fb600c54d0a573f2d49ad4c419577550))
+
+
+### Bug Fixes
+
+* KiB, not MiB for ack size limits ([#1999](https://github.com/googleapis/nodejs-pubsub/issues/1999)) ([798270d](https://github.com/googleapis/nodejs-pubsub/commit/798270db9c5ef71f75c3e24e70d9592bbd068212))
+
+## [4.8.0](https://github.com/googleapis/nodejs-pubsub/compare/v4.7.2...v4.8.0) (2024-10-15)
+
+
+### Features
+
+* Add ingestion Cloud Storage fields and Platform Logging fields to Topic ([#1974](https://github.com/googleapis/nodejs-pubsub/issues/1974)) ([afec9a1](https://github.com/googleapis/nodejs-pubsub/commit/afec9a1ad3f665a71f08e748623f0fdaa332d17b))
+* Return listing information for subscriptions created via Analytics Hub ([afec9a1](https://github.com/googleapis/nodejs-pubsub/commit/afec9a1ad3f665a71f08e748623f0fdaa332d17b))
+
 ## [4.7.2](https://github.com/googleapis/nodejs-pubsub/compare/v4.7.1...v4.7.2) (2024-09-13)
 
 

package/README.md

@@ -52,7 +52,7 @@ Google APIs Client Libraries, in [Client Libraries Explained][explained].
 1.  [Select or create a Cloud Platform project][projects].
 1.  [Enable billing for your project][billing].
 1.  [Enable the Google Cloud Pub/Sub API][enable_api].
-1.  [Set up authentication with a service account][auth] so you can access the
+1.  [Set up authentication][auth] so you can access the
     API from your local workstation.
 
 ### Installing the client library
@@ -138,6 +138,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-pubsub/tree
 | Create Subscription with ordering enabled | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithOrdering.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithOrdering.js,samples/README.md) |
 | Create Subscription With Retry Policy | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithRetryPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithRetryPolicy.js,samples/README.md) |
 | Create Topic | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createTopic.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createTopic.js,samples/README.md) |
+| Create Topic With Cloud Storage Ingestion | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createTopicWithCloudStorageIngestion.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createTopicWithCloudStorageIngestion.js,samples/README.md) |
 | Create Topic With Kinesis Ingestion | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createTopicWithKinesisIngestion.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createTopicWithKinesisIngestion.js,samples/README.md) |
 | Create Topic With Schema | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createTopicWithSchema.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createTopicWithSchema.js,samples/README.md) |
 | Create Topic With Schema Revisions | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createTopicWithSchemaRevisions.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createTopicWithSchemaRevisions.js,samples/README.md) |
@@ -165,6 +166,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-pubsub/tree
 | Listen For Messages With Custom Attributes | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/listenWithCustomAttributes.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/listenWithCustomAttributes.js,samples/README.md) |
 | Subscribe with OpenTelemetry Tracing | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/listenWithOpenTelemetryTracing.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/listenWithOpenTelemetryTracing.js,samples/README.md) |
 | Modify Push Configuration | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/modifyPushConfig.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/modifyPushConfig.js,samples/README.md) |
+| Optimistic Subscribe | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/optimisticSubscribe.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/optimisticSubscribe.js,samples/README.md) |
 | Publish Avro Records to a Topic | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/publishAvroRecords.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/publishAvroRecords.js,samples/README.md) |
 | Publish Batched Messages | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/publishBatchedMessages.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/publishBatchedMessages.js,samples/README.md) |
 | Publish Message | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/publishMessage.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/publishMessage.js,samples/README.md) |
@@ -259,4 +261,4 @@ See [LICENSE](https://github.com/googleapis/nodejs-pubsub/blob/main/LICENSE)
 [projects]: https://console.cloud.google.com/project
 [billing]: https://support.google.com/cloud/answer/6293499#enable-billing
 [enable_api]: https://console.cloud.google.com/flows/enableapi?apiid=pubsub.googleapis.com
-[auth]: https://cloud.google.com/docs/authentication/getting-started
+[auth]: https://cloud.google.com/docs/authentication/external/set-up-adc-local

package/build/protos/google/pubsub/v1/pubsub.proto

@@ -237,10 +237,188 @@ message IngestionDataSourceSettings {
     string gcp_service_account = 5 [(google.api.field_behavior) = REQUIRED];
   }
 
+  // Ingestion settings for Cloud Storage.
+  message CloudStorage {
+    // Possible states for ingestion from Cloud Storage.
+    enum State {
+      // Default value. This value is unused.
+      STATE_UNSPECIFIED = 0;
+
+      // Ingestion is active.
+      ACTIVE = 1;
+
+      // Permission denied encountered while calling the Cloud Storage API. This
+      // can happen if the Pub/Sub SA has not been granted the
+      // [appropriate
+      // permissions](https://cloud.google.com/storage/docs/access-control/iam-permissions):
+      // - storage.objects.list: to list the objects in a bucket.
+      // - storage.objects.get: to read the objects in a bucket.
+      // - storage.buckets.get: to verify the bucket exists.
+      CLOUD_STORAGE_PERMISSION_DENIED = 2;
+
+      // Permission denied encountered while publishing to the topic. This can
+      // happen if the Pub/Sub SA has not been granted the [appropriate publish
+      // permissions](https://cloud.google.com/pubsub/docs/access-control#pubsub.publisher)
+      PUBLISH_PERMISSION_DENIED = 3;
+
+      // The provided Cloud Storage bucket doesn't exist.
+      BUCKET_NOT_FOUND = 4;
+
+      // The Cloud Storage bucket has too many objects, ingestion will be
+      // paused.
+      TOO_MANY_OBJECTS = 5;
+    }
+
+    // Configuration for reading Cloud Storage data in text format. Each line of
+    // text as specified by the delimiter will be set to the `data` field of a
+    // Pub/Sub message.
+    message TextFormat {
+      // Optional. When unset, '\n' is used.
+      optional string delimiter = 1 [(google.api.field_behavior) = OPTIONAL];
+    }
+
+    // Configuration for reading Cloud Storage data in Avro binary format. The
+    // bytes of each object will be set to the `data` field of a Pub/Sub
+    // message.
+    message AvroFormat {}
+
+    // Configuration for reading Cloud Storage data written via [Cloud Storage
+    // subscriptions](https://cloud.google.com/pubsub/docs/cloudstorage). The
+    // data and attributes fields of the originally exported Pub/Sub message
+    // will be restored when publishing.
+    message PubSubAvroFormat {}
+
+    // Output only. An output-only field that indicates the state of the Cloud
+    // Storage ingestion source.
+    State state = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+    // Optional. Cloud Storage bucket. The bucket name must be without any
+    // prefix like "gs://". See the [bucket naming requirements]
+    // (https://cloud.google.com/storage/docs/buckets#naming).
+    string bucket = 2 [(google.api.field_behavior) = OPTIONAL];
+
+    // Defaults to text format.
+    oneof input_format {
+      // Optional. Data from Cloud Storage will be interpreted as text.
+      TextFormat text_format = 3 [(google.api.field_behavior) = OPTIONAL];
+
+      // Optional. Data from Cloud Storage will be interpreted in Avro format.
+      AvroFormat avro_format = 4 [(google.api.field_behavior) = OPTIONAL];
+
+      // Optional. It will be assumed data from Cloud Storage was written via
+      // [Cloud Storage
+      // subscriptions](https://cloud.google.com/pubsub/docs/cloudstorage).
+      PubSubAvroFormat pubsub_avro_format = 5
+          [(google.api.field_behavior) = OPTIONAL];
+    }
+
+    // Optional. Only objects with a larger or equal creation timestamp will be
+    // ingested.
+    google.protobuf.Timestamp minimum_object_create_time = 6
+        [(google.api.field_behavior) = OPTIONAL];
+
+    // Optional. Glob pattern used to match objects that will be ingested. If
+    // unset, all objects will be ingested. See the [supported
+    // patterns](https://cloud.google.com/storage/docs/json_api/v1/objects/list#list-objects-and-prefixes-using-glob).
+    string match_glob = 9 [(google.api.field_behavior) = OPTIONAL];
+  }
+
   // Only one source type can have settings set.
   oneof source {
     // Optional. Amazon Kinesis Data Streams.
     AwsKinesis aws_kinesis = 1 [(google.api.field_behavior) = OPTIONAL];
+
+    // Optional. Cloud Storage.
+    CloudStorage cloud_storage = 2 [(google.api.field_behavior) = OPTIONAL];
+  }
+
+  // Optional. Platform Logs settings. If unset, no Platform Logs will be
+  // generated.
+  PlatformLogsSettings platform_logs_settings = 4
+      [(google.api.field_behavior) = OPTIONAL];
+}
+
+// Settings for Platform Logs produced by Pub/Sub.
+message PlatformLogsSettings {
+  // Severity levels of Platform Logs.
+  enum Severity {
+    // Default value. Logs level is unspecified. Logs will be disabled.
+    SEVERITY_UNSPECIFIED = 0;
+
+    // Logs will be disabled.
+    DISABLED = 1;
+
+    // Debug logs and higher-severity logs will be written.
+    DEBUG = 2;
+
+    // Info logs and higher-severity logs will be written.
+    INFO = 3;
+
+    // Warning logs and higher-severity logs will be written.
+    WARNING = 4;
+
+    // Only error logs will be written.
+    ERROR = 5;
+  }
+
+  // Optional. The minimum severity level of Platform Logs that will be written.
+  Severity severity = 1 [(google.api.field_behavior) = OPTIONAL];
+}
+
+// Payload of the Platform Log entry sent when a failure is encountered while
+// ingesting.
+message IngestionFailureEvent {
+  // Specifies the reason why some data may have been left out of
+  // the desired Pub/Sub message due to the API message limits
+  // (https://cloud.google.com/pubsub/quotas#resource_limits). For example,
+  // when the number of attributes is larger than 100, the number of
+  // attributes is truncated to 100 to respect the limit on the attribute count.
+  // Other attribute limits are treated similarly. When the size of the desired
+  // message would've been larger than 10MB, the message won't be published at
+  // all, and ingestion of the subsequent messages will proceed as normal.
+  message ApiViolationReason {}
+
+  // Set when an Avro file is unsupported or its format is not valid. When this
+  // occurs, one or more Avro objects won't be ingested.
+  message AvroFailureReason {}
+
+  // Failure when ingesting from a Cloud Storage source.
+  message CloudStorageFailure {
+    // Optional. Name of the Cloud Storage bucket used for ingestion.
+    string bucket = 1 [(google.api.field_behavior) = OPTIONAL];
+
+    // Optional. Name of the Cloud Storage object which contained the section
+    // that couldn't be ingested.
+    string object_name = 2 [(google.api.field_behavior) = OPTIONAL];
+
+    // Optional. Generation of the Cloud Storage object which contained the
+    // section that couldn't be ingested.
+    int64 object_generation = 3 [(google.api.field_behavior) = OPTIONAL];
+
+    // Reason why ingestion failed for the specified object.
+    oneof reason {
+      // Optional. Failure encountered when parsing an Avro file.
+      AvroFailureReason avro_failure_reason = 5
+          [(google.api.field_behavior) = OPTIONAL];
+
+      // Optional. The Pub/Sub API limits prevented the desired message from
+      // being published.
+      ApiViolationReason api_violation_reason = 6
+          [(google.api.field_behavior) = OPTIONAL];
+    }
+  }
+
+  // Required. Name of the import topic. Format is:
+  // projects/{project_name}/topics/{topic_name}.
+  string topic = 1 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. Error details explaining why ingestion to Pub/Sub has failed.
+  string error_message = 2 [(google.api.field_behavior) = REQUIRED];
+
+  oneof failure {
+    // Optional. Failure when ingesting from Cloud Storage.
+    CloudStorageFailure cloud_storage_failure = 3
+        [(google.api.field_behavior) = OPTIONAL];
   }
 }
 
@@ -774,6 +952,20 @@ message Subscription {
     RESOURCE_ERROR = 2;
   }
 
+  // Information about an associated Analytics Hub subscription
+  // (https://cloud.google.com/bigquery/docs/analytics-hub-manage-subscriptions).
+  message AnalyticsHubSubscriptionInfo {
+    // Optional. The name of the associated Analytics Hub listing resource.
+    // Pattern:
+    // "projects/{project}/locations/{location}/dataExchanges/{data_exchange}/listings/{listing}"
+    string listing = 1 [(google.api.field_behavior) = OPTIONAL];
+
+    // Optional. The name of the associated Analytics Hub subscription resource.
+    // Pattern:
+    // "projects/{project}/locations/{location}/subscriptions/{subscription}"
+    string subscription = 2 [(google.api.field_behavior) = OPTIONAL];
+  }
+
   // Required. The name of the subscription. It must have the format
   // `"projects/{project}/subscriptions/{subscription}"`. `{subscription}` must
   // start with a letter, and contain only letters (`[A-Za-z]`), numbers
@@ -837,7 +1029,7 @@ message Subscription {
   // backlog, from the moment a message is published. If `retain_acked_messages`
   // is true, then this also configures the retention of acknowledged messages,
   // and thus configures how far back in time a `Seek` can be done. Defaults to
-  // 7 days. Cannot be more than 7 days or less than 10 minutes.
+  // 7 days. Cannot be more than 31 days or less than 10 minutes.
   google.protobuf.Duration message_retention_duration = 8
       [(google.api.field_behavior) = OPTIONAL];
 
@@ -922,6 +1114,11 @@ message Subscription {
   // Output only. An output-only field indicating whether or not the
   // subscription can receive messages.
   State state = 19 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Information about the associated Analytics Hub subscription.
+  // Only set if the subscritpion is created by Analytics Hub.
+  AnalyticsHubSubscriptionInfo analytics_hub_subscription_info = 23
+      [(google.api.field_behavior) = OUTPUT_ONLY];
 }
 
 // A policy that specifies how Pub/Sub retries message delivery.