dataproc-spark-connect 0.2.1__py2.py3-none-any.whl → 0.7.0__py2.py3-none-any.whl

dataproc_spark_connect-0.2.1.dist-info/METADATA

@@ -1,119 +0,0 @@
-Metadata-Version: 2.1
-Name: dataproc-spark-connect
-Version: 0.2.1
-Summary: Dataproc client library for Spark Connect
-Home-page: https://github.com/GoogleCloudDataproc/dataproc-spark-connect-python
-Author: Google LLC
-License: Apache 2.0
-License-File: LICENSE
-Requires-Dist: google-api-core>=2.19.1
-Requires-Dist: google-cloud-dataproc>=5.15.1
-Requires-Dist: wheel
-Requires-Dist: websockets
-Requires-Dist: pyspark>=3.5
-Requires-Dist: pandas
-Requires-Dist: pyarrow
-
-# Dataproc Spark Connect Client
-
-> ⚠️ **Warning:**  
-The package `dataproc-spark-connect` has been renamed to `google-spark-connect`. `dataproc-spark-connect` will no longer be updated.
-For help using `google-spark-connect`, please see [guide](https://github.com/GoogleCloudDataproc/dataproc-spark-connect-python/blob/main/README.md). 
-
-
-A wrapper of the Apache [Spark Connect](https://spark.apache.org/spark-connect/) client with
-additional functionalities that allow applications to communicate with a remote Dataproc
-Spark cluster using the Spark Connect protocol without requiring additional steps.
-
-## Install
-
-.. code-block:: console
-
-      pip install dataproc_spark_connect
-
-## Uninstall
-
-.. code-block:: console
-
-      pip uninstall dataproc_spark_connect
-
-
-## Setup
-This client requires permissions to manage [Dataproc sessions and session templates](https://cloud.google.com/dataproc-serverless/docs/concepts/iam).
-If you are running the client outside of Google Cloud, you must set following environment variables:
-
-* GOOGLE_CLOUD_PROJECT - The Google Cloud project you use to run Spark workloads
-* GOOGLE_CLOUD_REGION - The Compute Engine [region](https://cloud.google.com/compute/docs/regions-zones#available) where you run the Spark workload.
-* GOOGLE_APPLICATION_CREDENTIALS - Your [Application Credentials](https://cloud.google.com/docs/authentication/provide-credentials-adc)
-* DATAPROC_SPARK_CONNECT_SESSION_DEFAULT_CONFIG (Optional) - The config location, such as `tests/integration/resources/session.textproto`
-
-## Usage
-
-1. Install the latest version of Dataproc Python client and Dataproc Spark Connect modules:
-
-      .. code-block:: console
-
-            pip install google_cloud_dataproc --force-reinstall
-            pip install dataproc_spark_connect --force-reinstall
-
-2. Add the required import into your PySpark application or notebook:
-
-      .. code-block:: python
-
-            from google.cloud.dataproc_spark_connect import DataprocSparkSession
-
-3. There are two ways to create a spark session,
-
-   1. Start a Spark session using properties defined in `DATAPROC_SPARK_CONNECT_SESSION_DEFAULT_CONFIG`:
-
-      .. code-block:: python
-
-            spark = DataprocSparkSession.builder.getOrCreate()
-
-   2. Start a Spark session with the following code instead of using a config file:
-
-      .. code-block:: python
-
-            from google.cloud.dataproc_v1 import SparkConnectConfig
-            from google.cloud.dataproc_v1 import Session
-            dataproc_config = Session()
-            dataproc_config.spark_connect_session = SparkConnectConfig()
-            dataproc_config.environment_config.execution_config.subnetwork_uri = "<subnet>"
-            dataproc_config.runtime_config.version = '3.0'
-            spark = DataprocSparkSession.builder.dataprocConfig(dataproc_config).getOrCreate()
-
-## Billing
-As this client runs the spark workload on Dataproc, your project will be billed as per [Dataproc Serverless Pricing](https://cloud.google.com/dataproc-serverless/pricing).
-This will happen even if you are running the client from a non-GCE instance.
-
-## Contributing
-### Building and Deploying SDK
-
-1. Install the requirements in virtual environment.
-
-      .. code-block:: console
-
-            pip install -r requirements.txt
-
-2. Build the code.
-
-      .. code-block:: console
-
-            python setup.py sdist bdist_wheel
-
-
-3. Copy the generated `.whl` file to Cloud Storage. Use the version specified in the `setup.py` file.
-
-      .. code-block:: console
-
-            VERSION=<version> gsutil cp dist/dataproc_spark_connect-${VERSION}-py2.py3-none-any.whl gs://<your_bucket_name>
-
-4. Download the new SDK on Vertex, then uninstall the old version and install the new one.
-
-      .. code-block:: console
-
-            %%bash
-            export VERSION=<version>
-            gsutil cp gs://<your_bucket_name>/dataproc_spark_connect-${VERSION}-py2.py3-none-any.whl .
-            yes | pip uninstall dataproc_spark_connect
-            pip install dataproc_spark_connect-${VERSION}-py2.py3-none-any.whl

dataproc_spark_connect-0.2.1.dist-info/RECORD

@@ -1,10 +0,0 @@
-google/cloud/dataproc_spark_connect/__init__.py,sha256=G5Gy26z7Z2u_7EUSYGfcBArKLNTWlSG5OmzuCYzcwjA,969
-google/cloud/dataproc_spark_connect/session.py,sha256=A42Wo87VSunG0D3sB-biWyNvU33WhI92mmrJbXI1oNo,23017
-google/cloud/dataproc_spark_connect/client/__init__.py,sha256=6hCNSsgYlie6GuVpc5gjFsPnyeMTScTpXSPYqp1fplY,615
-google/cloud/dataproc_spark_connect/client/core.py,sha256=7Wy6QwkcWxlHBdo4NsktJEknggPpGkx9F5CS5IpQ7iM,3630
-google/cloud/dataproc_spark_connect/client/proxy.py,sha256=ScrbaGsEvqi8wp4ngfD-T9K9mFHXBkVMZkTSr7mdNBs,8926
-dataproc_spark_connect-0.2.1.dist-info/LICENSE,sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ,11357
-dataproc_spark_connect-0.2.1.dist-info/METADATA,sha256=S4BrILUItFH_SFQ7fciFargIF5HQsSck4VF8zqFOGN4,4509
-dataproc_spark_connect-0.2.1.dist-info/WHEEL,sha256=OpXWERl2xLPRHTvd2ZXo_iluPEQd8uSbYkJ53NAER_Y,109
-dataproc_spark_connect-0.2.1.dist-info/top_level.txt,sha256=_1QvSJIhFAGfxb79D6DhB7SUw2X6T4rwnz_LLrbcD3c,7
-dataproc_spark_connect-0.2.1.dist-info/RECORD,,