@google-cloud/discoveryengine 1.4.1 → 1.5.0

package/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## [1.5.0](https://github.com/googleapis/google-cloud-node/compare/discoveryengine-v1.4.1...discoveryengine-v1.5.0) (2024-04-05)
+
+
+### Features
+
+* [discoveryengine] support import data from Cloud Spanner, BigTable, SQL and Firestore ([#5218](https://github.com/googleapis/google-cloud-node/issues/5218)) ([cc25e93](https://github.com/googleapis/google-cloud-node/commit/cc25e935f087526738303cd94c406bd352a7b595))
+
 ## [1.4.1](https://github.com/googleapis/google-cloud-node/compare/discoveryengine-v1.4.0...discoveryengine-v1.4.1) (2024-03-12)
 
 

package/README.md

@@ -195,6 +195,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/google-cloud-node/
 | Document_service.create_document | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.create_document.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.create_document.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Document_service.delete_document | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.delete_document.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.delete_document.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Document_service.get_document | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.get_document.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.get_document.js,packages/google-cloud-discoveryengine/samples/README.md) |
+| Document_service.get_processed_document | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.get_processed_document.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.get_processed_document.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Document_service.import_documents | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.import_documents.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.import_documents.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Document_service.list_documents | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.list_documents.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.list_documents.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Document_service.purge_documents | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.purge_documents.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/document_service.purge_documents.js,packages/google-cloud-discoveryengine/samples/README.md) |
@@ -208,6 +209,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/google-cloud-node/
 | Engine_service.tune_engine | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/engine_service.tune_engine.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/engine_service.tune_engine.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Engine_service.update_engine | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/engine_service.update_engine.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/engine_service.update_engine.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Estimate_billing_service.estimate_data_size | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/estimate_billing_service.estimate_data_size.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/estimate_billing_service.estimate_data_size.js,packages/google-cloud-discoveryengine/samples/README.md) |
+| Rank_service.rank | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/rank_service.rank.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/rank_service.rank.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Recommendation_service.recommend | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/recommendation_service.recommend.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/recommendation_service.recommend.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Schema_service.create_schema | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/schema_service.create_schema.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/schema_service.create_schema.js,packages/google-cloud-discoveryengine/samples/README.md) |
 | Schema_service.delete_schema | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-discoveryengine/samples/generated/v1alpha/schema_service.delete_schema.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-discoveryengine/samples/generated/v1alpha/schema_service.delete_schema.js,packages/google-cloud-discoveryengine/samples/README.md) |

package/build/protos/google/cloud/discoveryengine/v1alpha/chunk.proto

@@ -48,6 +48,34 @@ message Chunk {
     string title = 2;
   }
 
+  // Page span of the chunk.
+  message PageSpan {
+    // The start page of the chunk.
+    int32 page_start = 1;
+
+    // The end page of the chunk.
+    int32 page_end = 2;
+  }
+
+  // Metadata of the current chunk. This field is only populated on
+  // [SearchService.Search][google.cloud.discoveryengine.v1alpha.SearchService.Search]
+  // API.
+  message ChunkMetadata {
+    // The previous chunks of the current chunk. The number is controlled by
+    // [SearchRequest.ContentSearchSpec.ChunkSpec.num_previous_chunks][google.cloud.discoveryengine.v1alpha.SearchRequest.ContentSearchSpec.ChunkSpec.num_previous_chunks].
+    // This field is only populated on
+    // [SearchService.Search][google.cloud.discoveryengine.v1alpha.SearchService.Search]
+    // API.
+    repeated Chunk previous_chunks = 1;
+
+    // The next chunks of the current chunk. The number is controlled by
+    // [SearchRequest.ContentSearchSpec.ChunkSpec.num_next_chunks][google.cloud.discoveryengine.v1alpha.SearchRequest.ContentSearchSpec.ChunkSpec.num_next_chunks].
+    // This field is only populated on
+    // [SearchService.Search][google.cloud.discoveryengine.v1alpha.SearchService.Search]
+    // API.
+    repeated Chunk next_chunks = 2;
+  }
+
   // The full resource name of the chunk.
   // Format:
   // `projects/{project}/locations/{location}/collections/{collection}/dataStores/{data_store}/branches/{branch}/documents/{document_id}/chunks/{chunk_id}`.
@@ -56,7 +84,7 @@ message Chunk {
   // characters.
   string name = 1;
 
-  // Unique chunk id of the current chunk.
+  // Unique chunk ID of the current chunk.
   string id = 2;
 
   // Content is a string from a document (parsed content).
@@ -69,4 +97,10 @@ message Chunk {
   // It contains derived data that are not in the original input document.
   google.protobuf.Struct derived_struct_data = 4
       [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Page span of the chunk.
+  PageSpan page_span = 6;
+
+  // Output only. Metadata of the current chunk.
+  ChunkMetadata chunk_metadata = 7 [(google.api.field_behavior) = OUTPUT_ONLY];
 }

package/build/protos/google/cloud/discoveryengine/v1alpha/common.proto

@@ -39,6 +39,18 @@ option (google.api.resource_definition) = {
   type: "discoveryengine.googleapis.com/Location"
   pattern: "projects/{project}/locations/{location}"
 };
+option (google.api.resource_definition) = {
+  type: "discoveryengine.googleapis.com/GroundingConfig"
+  pattern: "projects/{project}/locations/{location}/groundingConfigs/{grounding_config}"
+};
+option (google.api.resource_definition) = {
+  type: "discoveryengine.googleapis.com/RankingConfig"
+  pattern: "projects/{project}/locations/{location}/rankingConfigs/{ranking_config}"
+};
+option (google.api.resource_definition) = {
+  type: "healthcare.googleapis.com/FhirStore"
+  pattern: "projects/{project}/locations/{location}/datasets/{dataset}/fhirStores/{fhir_store}"
+};
 
 // The industry vertical associated with the
 // [DataStore][google.cloud.discoveryengine.v1alpha.DataStore].
@@ -52,6 +64,9 @@ enum IndustryVertical {
 
   // The media industry vertical.
   MEDIA = 2;
+
+  // The healthcare FHIR vertical.
+  HEALTHCARE_FHIR = 7;
 }
 
 // The type of solution.
@@ -67,6 +82,11 @@ enum SolutionType {
 
   // Used for use cases related to the Generative AI agent.
   SOLUTION_TYPE_CHAT = 3;
+
+  // Used for use cases related to the Generative Chat agent.
+  // It's used for Generative chat engine only, the associated data stores
+  // must enrolled with `SOLUTION_TYPE_CHAT` solution.
+  SOLUTION_TYPE_GENERATIVE_CHAT = 4;
 }
 
 // Tiers of search features. Different tiers might have different

package/build/protos/google/cloud/discoveryengine/v1alpha/document.proto

@@ -202,3 +202,22 @@ message Document {
   google.protobuf.Timestamp index_time = 13
       [(google.api.field_behavior) = OUTPUT_ONLY];
 }
+
+// Document captures all raw metadata information of items to be recommended or
+// searched.
+message ProcessedDocument {
+  // Output format of the processed document.
+  oneof processed_data_format {
+    // The JSON string representation of the processed document.
+    string json_data = 2;
+  }
+
+  // Required. Full resource name of the referenced document, in the format
+  // `projects/*/locations/*/collections/*/dataStores/*/branches/*/documents/*`.
+  string document = 1 [
+    (google.api.field_behavior) = REQUIRED,
+    (google.api.resource_reference) = {
+      type: "discoveryengine.googleapis.com/Document"
+    }
+  ];
+}

package/build/protos/google/cloud/discoveryengine/v1alpha/document_service.proto

@@ -158,6 +158,19 @@ service DocumentService {
       metadata_type: "google.cloud.discoveryengine.v1alpha.PurgeDocumentsMetadata"
     };
   }
+
+  // Gets the parsed layout information for a
+  // [Document][google.cloud.discoveryengine.v1alpha.Document].
+  rpc GetProcessedDocument(GetProcessedDocumentRequest)
+      returns (ProcessedDocument) {
+    option (google.api.http) = {
+      get: "/v1alpha/{name=projects/*/locations/*/dataStores/*/branches/*/documents/*}:getProcessedDocument"
+      additional_bindings {
+        get: "/v1alpha/{name=projects/*/locations/*/collections/*/dataStores/*/branches/*/documents/*}:getProcessedDocument"
+      }
+    };
+    option (google.api.method_signature) = "name";
+  }
 }
 
 // Request message for
@@ -322,3 +335,54 @@ message DeleteDocumentRequest {
     }
   ];
 }
+
+// Request message for
+// [DocumentService.GetDocument][google.cloud.discoveryengine.v1alpha.DocumentService.GetDocument]
+// method.
+message GetProcessedDocumentRequest {
+  // The type of processing to return in the response.
+  enum ProcessedDocumentType {
+    // Default value.
+    PROCESSED_DOCUMENT_TYPE_UNSPECIFIED = 0;
+
+    // Available for all data store parsing configs.
+    PARSED_DOCUMENT = 1;
+
+    // Only available if ChunkingConfig is enabeld on the data store.
+    CHUNKED_DOCUMENT = 2;
+  }
+
+  // The format of the returned processed document. If unspecified, defaults to
+  // JSON.
+  enum ProcessedDocumentFormat {
+    // Default value.
+    PROCESSED_DOCUMENT_FORMAT_UNSPECIFIED = 0;
+
+    // output format will be a JSON string representation of processed document.
+    JSON = 1;
+  }
+
+  // Required. Full resource name of
+  // [Document][google.cloud.discoveryengine.v1alpha.Document], such as
+  // `projects/{project}/locations/{location}/collections/{collection}/dataStores/{data_store}/branches/{branch}/documents/{document}`.
+  //
+  // If the caller does not have permission to access the
+  // [Document][google.cloud.discoveryengine.v1alpha.Document], regardless of
+  // whether or not it exists, a `PERMISSION_DENIED` error is returned.
+  //
+  // If the requested [Document][google.cloud.discoveryengine.v1alpha.Document]
+  // does not exist, a `NOT_FOUND` error is returned.
+  string name = 1 [
+    (google.api.field_behavior) = REQUIRED,
+    (google.api.resource_reference) = {
+      type: "discoveryengine.googleapis.com/Document"
+    }
+  ];
+
+  // Required. What type of processing to return.
+  ProcessedDocumentType processed_document_type = 2
+      [(google.api.field_behavior) = REQUIRED];
+
+  // What format output should be.  If unspecified, defaults to JSON.
+  ProcessedDocumentFormat processed_document_format = 3;
+}

package/build/protos/google/cloud/discoveryengine/v1alpha/import_config.proto

@@ -21,6 +21,7 @@ import "google/api/resource.proto";
 import "google/cloud/discoveryengine/v1alpha/completion.proto";
 import "google/cloud/discoveryengine/v1alpha/document.proto";
 import "google/cloud/discoveryengine/v1alpha/user_event.proto";
+import "google/protobuf/field_mask.proto";
 import "google/protobuf/timestamp.proto";
 import "google/rpc/status.proto";
 import "google/type/date.proto";
@@ -123,6 +124,233 @@ message BigQuerySource {
   string data_schema = 6;
 }
 
+// The Spanner source for importing data
+message SpannerSource {
+  // The project ID that the Spanner source is in with a length limit of 128
+  // characters. If not specified, inherits the project ID from the parent
+  // request.
+  string project_id = 1;
+
+  // Required. The instance ID of the source Spanner table.
+  string instance_id = 2 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The database ID of the source Spanner table.
+  string database_id = 3 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The table name of the Spanner database that needs to be imported.
+  string table_id = 4 [(google.api.field_behavior) = REQUIRED];
+
+  // Whether to apply data boost on Spanner export. Enabling this option will
+  // incur additional cost. More info can be found
+  // [here](https://cloud.google.com/spanner/docs/databoost/databoost-overview#billing_and_quotas).
+  bool enable_data_boost = 5;
+}
+
+// The Bigtable Options object that contains information to support
+// the import.
+message BigtableOptions {
+  // The column family of the Bigtable.
+  message BigtableColumnFamily {
+    // The field name to use for this column family in the document. The
+    // name has to match the pattern `[a-zA-Z0-9][a-zA-Z0-9-_]*`. If not set,
+    // it is parsed from the family name with best effort. However, due to
+    // different naming patterns, field name collisions could happen, where
+    // parsing behavior is undefined.
+    string field_name = 1;
+
+    // The encoding mode of the values when the type is not STRING.
+    // Acceptable encoding values are:
+    //
+    // * `TEXT`: indicates values are alphanumeric text strings.
+    // * `BINARY`: indicates values are encoded using `HBase Bytes.toBytes`
+    // family of functions. This can be overridden for a specific column
+    // by listing that column in `columns` and specifying an encoding for it.
+    Encoding encoding = 2;
+
+    // The type of values in this column family.
+    // The values are expected to be encoded using `HBase Bytes.toBytes`
+    // function when the encoding value is set to `BINARY`.
+    Type type = 3;
+
+    // The list of objects that contains column level information for each
+    // column. If a column is not present in this list it will be ignored.
+    repeated BigtableColumn columns = 4;
+  }
+
+  // The column of the Bigtable.
+  message BigtableColumn {
+    // Required. Qualifier of the column. If it cannot be decoded with utf-8,
+    // use a base-64 encoded string instead.
+    bytes qualifier = 1 [(google.api.field_behavior) = REQUIRED];
+
+    // The field name to use for this column in the document. The name has to
+    // match the pattern `[a-zA-Z0-9][a-zA-Z0-9-_]*`.
+    // If not set, it is parsed from the qualifier bytes with best effort.
+    // However, due to different naming patterns, field name collisions could
+    // happen, where parsing behavior is undefined.
+    string field_name = 2;
+
+    // The encoding mode of the values when the type is not `STRING`.
+    // Acceptable encoding values are:
+    //
+    // * `TEXT`: indicates values are alphanumeric text strings.
+    // * `BINARY`: indicates values are encoded using `HBase Bytes.toBytes`
+    // family of functions. This can be overridden for a specific column
+    // by listing that column in `columns` and specifying an encoding for it.
+    Encoding encoding = 3;
+
+    // The type of values in this column family.
+    // The values are expected to be encoded using `HBase Bytes.toBytes`
+    // function when the encoding value is set to `BINARY`.
+    Type type = 4;
+  }
+
+  // The type of values in a Bigtable column or column family.
+  // The values are expected to be encoded using
+  // [HBase
+  // Bytes.toBytes](https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/util/Bytes.html)
+  // function when the encoding value is set to `BINARY`.
+  enum Type {
+    // The type is unspecified.
+    TYPE_UNSPECIFIED = 0;
+
+    // String type.
+    STRING = 1;
+
+    // Numerical type.
+    NUMBER = 2;
+
+    // Integer type.
+    INTEGER = 3;
+
+    // Variable length integer type.
+    VAR_INTEGER = 4;
+
+    // BigDecimal type.
+    BIG_NUMERIC = 5;
+
+    // Boolean type.
+    BOOLEAN = 6;
+
+    // JSON type.
+    JSON = 7;
+  }
+
+  // The encoding mode of a Bigtable column or column family.
+  enum Encoding {
+    // The encoding is unspecified.
+    ENCODING_UNSPECIFIED = 0;
+
+    // Text encoding.
+    TEXT = 1;
+
+    // Binary encoding.
+    BINARY = 2;
+  }
+
+  // The field name used for saving row key value in the document. The name has
+  // to match the pattern `[a-zA-Z0-9][a-zA-Z0-9-_]*`.
+  string key_field_name = 1;
+
+  // The mapping from family names to an object that contains column families
+  // level information for the given column family. If a family is not present
+  // in this map it will be ignored.
+  map<string, BigtableColumnFamily> families = 2;
+}
+
+// The Cloud Bigtable source for importing data.
+message BigtableSource {
+  // The project ID that the Bigtable source is in with a length limit of 128
+  // characters. If not specified, inherits the project ID from the parent
+  // request.
+  string project_id = 1;
+
+  // Required. The instance ID of the Cloud Bigtable that needs to be imported.
+  string instance_id = 2 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The table ID of the Cloud Bigtable that needs to be imported.
+  string table_id = 3 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. Bigtable options that contains information needed when parsing
+  // data into typed structures. For example, column type annotations.
+  BigtableOptions bigtable_options = 4 [(google.api.field_behavior) = REQUIRED];
+}
+
+// Cloud FhirStore source import data from.
+message FhirStoreSource {
+  // Required. The full resource name of the FHIR store to import data from, in
+  // the format of
+  // `projects/{project}/locations/{location}/datasets/{dataset}/fhirStores/{fhir_store}`.
+  string fhir_store = 1 [
+    (google.api.field_behavior) = REQUIRED,
+    (google.api.resource_reference) = {
+      type: "healthcare.googleapis.com/FhirStore"
+    }
+  ];
+
+  // Intermediate Cloud Storage directory used for the import with a length
+  // limit of 2,000 characters. Can be specified if one wants to have the
+  // FhirStore export to a specific Cloud Storage directory.
+  string gcs_staging_dir = 2;
+}
+
+// Cloud SQL source import data from.
+message CloudSqlSource {
+  // The project ID that the Cloud SQL source is in with a length limit of 128
+  // characters. If not specified, inherits the project ID from the parent
+  // request.
+  string project_id = 1;
+
+  // Required. The Cloud SQL instance to copy the data from with a length limit
+  // of 256 characters.
+  string instance_id = 2 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The Cloud SQL database to copy the data from with a length limit
+  // of 256 characters.
+  string database_id = 3 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The Cloud SQL table to copy the data from with a length limit of
+  // 256 characters.
+  string table_id = 4 [(google.api.field_behavior) = REQUIRED];
+
+  // Intermediate Cloud Storage directory used for the import with a length
+  // limit of 2,000 characters. Can be specified if one wants to have the
+  // Cloud SQL export to a specific Cloud Storage directory.
+  //
+  // Please ensure that the Cloud SQL service account has the necessary Cloud
+  // Storage Admin permissions to access the specified Cloud Storage directory.
+  string gcs_staging_dir = 5;
+
+  // Option for serverless export. Enabling this option will incur
+  // additional cost. More info can be found
+  // [here](https://cloud.google.com/sql/pricing#serverless).
+  bool offload = 6;
+}
+
+// Firestore source import data from.
+message FirestoreSource {
+  // The project ID that the Cloud SQL source is in with a length limit of 128
+  // characters. If not specified, inherits the project ID from the parent
+  // request.
+  string project_id = 1;
+
+  // Required. The Firestore database to copy the data from with a length limit
+  // of 256 characters.
+  string database_id = 2 [(google.api.field_behavior) = REQUIRED];
+
+  // Required. The Firestore collection to copy the data from with a length
+  // limit of 1,500 characters.
+  string collection_id = 3 [(google.api.field_behavior) = REQUIRED];
+
+  // Intermediate Cloud Storage directory used for the import with a length
+  // limit of 2,000 characters. Can be specified if one wants to have the
+  // Firestore export to a specific Cloud Storage directory.
+  //
+  // Please ensure that the Firestore service account has the necessary Cloud
+  // Storage Admin permissions to access the specified Cloud Storage directory.
+  string gcs_staging_dir = 4;
+}
+
 // Configuration of destination for Import related errors.
 message ImportErrorConfig {
   // Required. Errors destination.
@@ -220,6 +448,9 @@ message ImportDocumentsMetadata {
 
   // Count of entries that encountered errors while processing.
   int64 failure_count = 4;
+
+  // Total count of entries that were processed.
+  int64 total_count = 5;
 }
 
 // Request message for Import methods.
@@ -256,6 +487,21 @@ message ImportDocumentsRequest {
 
     // BigQuery input source.
     BigQuerySource bigquery_source = 4;
+
+    // FhirStore input source.
+    FhirStoreSource fhir_store_source = 10;
+
+    // Spanner input source.
+    SpannerSource spanner_source = 11;
+
+    // Cloud SQL input source.
+    CloudSqlSource cloud_sql_source = 12;
+
+    // Firestore input source.
+    FirestoreSource firestore_source = 13;
+
+    // Cloud Bigtable input source.
+    BigtableSource bigtable_source = 15;
   }
 
   // Required. The parent branch resource name, such as
@@ -276,6 +522,10 @@ message ImportDocumentsRequest {
   // [ReconciliationMode.INCREMENTAL][google.cloud.discoveryengine.v1alpha.ImportDocumentsRequest.ReconciliationMode.INCREMENTAL].
   ReconciliationMode reconciliation_mode = 6;
 
+  // Indicates which fields in the provided imported documents to update. If
+  // not set, the default is to update all fields.
+  google.protobuf.FieldMask update_mask = 7;
+
   // Whether to automatically generate IDs for the documents if absent.
   //
   // If set to `true`,
@@ -289,45 +539,54 @@ message ImportDocumentsRequest {
   // [id_field][google.cloud.discoveryengine.v1alpha.ImportDocumentsRequest.id_field],
   // otherwise, documents without IDs fail to be imported.
   //
-  // Only set this field when using
-  // [GcsSource][google.cloud.discoveryengine.v1alpha.GcsSource] or
-  // [BigQuerySource][google.cloud.discoveryengine.v1alpha.BigQuerySource], and
-  // when
+  // Supported data sources:
+  //
+  // * [GcsSource][google.cloud.discoveryengine.v1alpha.GcsSource].
   // [GcsSource.data_schema][google.cloud.discoveryengine.v1alpha.GcsSource.data_schema]
-  // or
+  // must be `custom` or `csv`. Otherwise, an INVALID_ARGUMENT error is thrown.
+  // * [BigQuerySource][google.cloud.discoveryengine.v1alpha.BigQuerySource].
   // [BigQuerySource.data_schema][google.cloud.discoveryengine.v1alpha.BigQuerySource.data_schema]
-  // is `custom` or `csv`. Otherwise, an INVALID_ARGUMENT error is thrown.
+  // must be `custom` or `csv`. Otherwise, an INVALID_ARGUMENT error is thrown.
+  // * [SpannerSource][google.cloud.discoveryengine.v1alpha.SpannerSource].
+  // * [CloudSqlSource][google.cloud.discoveryengine.v1alpha.CloudSqlSource].
+  // * [FirestoreSource][google.cloud.discoveryengine.v1alpha.FirestoreSource].
+  // * [BigtableSource][google.cloud.discoveryengine.v1alpha.BigtableSource].
   bool auto_generate_ids = 8;
 
-  // The field in the Cloud Storage and BigQuery sources that indicates the
-  // unique IDs of the documents.
+  // The field indicates the ID field or column to be used as unique IDs of
+  // the documents.
   //
   // For [GcsSource][google.cloud.discoveryengine.v1alpha.GcsSource] it is the
   // key of the JSON field. For instance, `my_id` for JSON `{"my_id":
-  // "some_uuid"}`. For
-  // [BigQuerySource][google.cloud.discoveryengine.v1alpha.BigQuerySource] it is
-  // the column name of the BigQuery table where the unique ids are stored.
+  // "some_uuid"}`. For others, it may be the column name of the table where the
+  // unique ids are stored.
   //
-  // The values of the JSON field or the BigQuery column are used as the
+  // The values of the JSON field or the table column are used as the
   // [Document.id][google.cloud.discoveryengine.v1alpha.Document.id]s. The JSON
-  // field or the BigQuery column must be of string type, and the values must be
+  // field or the table column must be of string type, and the values must be
   // set as valid strings conform to
   // [RFC-1034](https://tools.ietf.org/html/rfc1034) with 1-63 characters.
   // Otherwise, documents without valid IDs fail to be imported.
   //
-  // Only set this field when using
-  // [GcsSource][google.cloud.discoveryengine.v1alpha.GcsSource] or
-  // [BigQuerySource][google.cloud.discoveryengine.v1alpha.BigQuerySource], and
-  // when
-  // [GcsSource.data_schema][google.cloud.discoveryengine.v1alpha.GcsSource.data_schema]
-  // or
-  // [BigQuerySource.data_schema][google.cloud.discoveryengine.v1alpha.BigQuerySource.data_schema]
-  // is `custom`. And only set this field when
+  // Only set this field when
   // [auto_generate_ids][google.cloud.discoveryengine.v1alpha.ImportDocumentsRequest.auto_generate_ids]
   // is unset or set as `false`. Otherwise, an INVALID_ARGUMENT error is thrown.
   //
   // If it is unset, a default value `_id` is used when importing from the
   // allowed data sources.
+  //
+  // Supported data sources:
+  //
+  // * [GcsSource][google.cloud.discoveryengine.v1alpha.GcsSource].
+  // [GcsSource.data_schema][google.cloud.discoveryengine.v1alpha.GcsSource.data_schema]
+  // must be `custom` or `csv`. Otherwise, an INVALID_ARGUMENT error is thrown.
+  // * [BigQuerySource][google.cloud.discoveryengine.v1alpha.BigQuerySource].
+  // [BigQuerySource.data_schema][google.cloud.discoveryengine.v1alpha.BigQuerySource.data_schema]
+  // must be `custom` or `csv`. Otherwise, an INVALID_ARGUMENT error is thrown.
+  // * [SpannerSource][google.cloud.discoveryengine.v1alpha.SpannerSource].
+  // * [CloudSqlSource][google.cloud.discoveryengine.v1alpha.CloudSqlSource].
+  // * [FirestoreSource][google.cloud.discoveryengine.v1alpha.FirestoreSource].
+  // * [BigtableSource][google.cloud.discoveryengine.v1alpha.BigtableSource].
   string id_field = 9;
 }
 
@@ -405,3 +664,35 @@ message ImportSuggestionDenyListEntriesMetadata {
   // finish time.
   google.protobuf.Timestamp update_time = 2;
 }
+
+// Response of the
+// [CompletionService.ImportCompletionSuggestions][google.cloud.discoveryengine.v1alpha.CompletionService.ImportCompletionSuggestions]
+// method. If the long running operation is done, this message is returned by
+// the google.longrunning.Operations.response field if the operation is
+// successful.
+message ImportCompletionSuggestionsResponse {
+  // A sample of errors encountered while processing the request.
+  repeated google.rpc.Status error_samples = 1;
+
+  // Count of
+  // [CompletionSuggestion][google.cloud.discoveryengine.v1alpha.CompletionSuggestion]s
+  // successfully imported.
+  int64 success_count = 2;
+
+  // Count of
+  // [CompletionSuggestion][google.cloud.discoveryengine.v1alpha.CompletionSuggestion]s
+  // that failed to be imported.
+  int64 failure_count = 3;
+}
+
+// Metadata related to the progress of the ImportCompletionSuggestions
+// operation. This will be returned by the google.longrunning.Operation.metadata
+// field.
+message ImportCompletionSuggestionsMetadata {
+  // Operation create time.
+  google.protobuf.Timestamp create_time = 1;
+
+  // Operation last update time. If the operation is done, this is also the
+  // finish time.
+  google.protobuf.Timestamp update_time = 2;
+}