kyuubi-dlc-client 0.1.0__tar.gz

kyuubi_dlc_client-0.1.0/PKG-INFO

@@ -0,0 +1,185 @@
+Metadata-Version: 2.1
+Name: kyuubi_dlc_client
+Version: 0.1.0
+Summary: Python client for Apache Kyuubi (DLC fork): DB-API 2.0 over Thrift + REST API
+Home-page: https://git.example.com/dlc/kyuubi-dlc-client
+Author: DLC Kyuubi Team
+License: Apache-2.0
+Project-URL: Source, https://git.example.com/dlc/kyuubi-dlc-client
+Project-URL: Documentation, https://kyuubi-dlc-client.readthedocs.io/
+Project-URL: Homepage, https://git.example.com/dlc/kyuubi-dlc-client
+Description: # kyuubi-dlc-client
+        
+        Python client for the Kyuubi SQL gateway (DLC fork). Provides two independent
+        layers over the two protocol entrances of a Kyuubi server:
+        
+        | Layer | Protocol | Port | Use case |
+        |---|---|---|---|
+        | `kyuubi_dlc_client.dbapi` | Thrift (HiveServer2) | 10009 | Interactive SQL, low latency, session state, streamed columnar results |
+        | `kyuubi_dlc_client.rest` | REST API v1 | 10099 | Spark batch jobs (`/batches`), interactive SQL over HTTP |
+        
+        Both layers share the same authentication modes.
+        
+        ## Highlights
+        
+        - **DB-API 2.0** (PEP 249) compliant Thrift interface: `connect()` / `cursor()` /
+          `execute()` / `fetch*()`, plus non-standard async execution (`async_=True`),
+          polling, progress logs and cancellation.
+        - **REST batch client** with UUID-based idempotent submission, exponential
+          backoff polling, local log retrieval and cancellation.
+        - **Authentication**: `NONE`, `CUSTOM`, and DLC `AK/SK` (access key) modes.
+        - Robustness patches validated against Kyuubi: empty column payloads,
+          unknown type codes, custom SSL contexts.
+        
+        ## Installation
+        
+        ```bash
+        pip install .            # runtime only
+        pip install -e '.[dev]'  # development (tests, lint)
+        ```
+        
+        Requires Python 3.9+.
+        
+        ## Quickstart: interactive SQL (Thrift)
+        
+        ```python
+        import kyuubi_dlc_client
+        
+        conn = kyuubi_dlc_client.connect(
+            host="127.0.0.1",
+            port=10009,
+            configuration={                      # session-level settings
+                "kyuubi.engine.share.level": "USER",
+            },
+        )
+        try:
+            cur = conn.cursor()
+            cur.execute("SELECT 1 AS id, 'hello' AS msg")
+            print(cur.fetchall())                # [(1, 'hello')]
+        finally:
+            conn.close()
+        ```
+        
+        Async execution with timeout and cancellation:
+        
+        ```python
+        cur.execute("SELECT count(*) FROM range(1000000)", async_=True)
+        cur.wait(timeout_s=300)                  # raises ExecutionTimeoutError + cancels
+        print(cur.fetchall())
+        ```
+        
+        ## Quickstart: Spark batch (REST)
+        
+        ```python
+        import uuid
+        from kyuubi_dlc_client.rest import KyuubiRestClient
+        
+        with KyuubiRestClient("http://127.0.0.1:10099/api/v1") as client:
+            batch = client.submit_batch(
+                resource="cos://bucket/spark-examples.jar",
+                class_name="org.apache.spark.examples.SparkPi",
+                args=["10"],
+                batch_id=str(uuid.uuid4()),       # UUID is mandatory; enables idempotent retry
+            )
+            finished = client.wait_batch(batch.id, timeout_s=600)
+            print(finished.state, finished.appId)
+            logs = client.get_batch_local_log(batch.id, size=50)
+        ```
+        
+        ## Authentication
+        
+        ```python
+        # Anonymous (Kyuubi NONE auth)
+        conn = kyuubi_dlc_client.connect(host, port)
+        
+        # Username/password against a custom authenticator
+        conn = kyuubi_dlc_client.connect(host, port, auth="CUSTOM",
+                                         username="user", password="pass")
+        
+        # DLC access key
+        conn = kyuubi_dlc_client.connect(
+            host, port,
+            secret_id=os.environ["SECRET_ID"],
+            secret_key=os.environ["SECRET_KEY"],
+        )
+        ```
+        
+        > AK/SK wire format: the default presents `secret_id`/`secret_key` over
+        > SASL/PLAIN. Align `auth.AkSkAuth.credentials()` with the Kyuubi DLC
+        > authenticator plugin once its contract is frozen.
+        
+        ## Interactive SQL over HTTP (REST sessions)
+        
+        Use when only the HTTP port is reachable. Note that result sets are JSON
+        rows (slower for large data) and multi-instance deployments require routing
+        requests to the owning `kyuubiInstance`.
+        
+        ```python
+        session = client.open_session()
+        operation = client.execute_statement(session, "SELECT 1", run_async=True)
+        # poll GET /operations/{id}/event, then:
+        rowset = client.fetch_rowset(operation)
+        client.close_session(session)
+        ```
+        
+        ## Examples & smoke test
+        
+        ```bash
+        # Thrift SQL quickstart
+        KYUUBI_HOST=127.0.0.1 python examples/sql_quickstart.py
+        
+        # Batch quickstart (needs a spark-examples jar reachable by the server)
+        KYUUBI_HOST=127.0.0.1 \
+        BATCH_RESOURCE=local:///abs/path/spark-examples_2.12-3.5.1.jar \
+        python examples/batch_submit.py
+        
+        # End-to-end smoke (SQL + batch)
+        KYUUBI_HOST=127.0.0.1 bash scripts/e2e_smoke.sh
+        ```
+        
+        ## Development
+        
+        ```bash
+        pip install -e '.[dev]'
+        ruff check src tests
+        pytest -m "not integration"
+        ```
+        
+        ## Security notes
+        
+        - Never hard-code credentials; read them from environment variables or a
+          secret store.
+        - The Thrift/REST ports of a Kyuubi server are internal infrastructure;
+          production access must go through the managed network entry point.
+        - Batch `resource` paths are validated by the gateway (e.g. `cos://`
+          allow-list); clients must not accept arbitrary resource URIs from
+          untrusted input (SSRF).
+        
+        ## Credits & license
+        
+        The TCLIService thrift bindings derive from PyHive (Apache-2.0), accepted into
+        Apache Kyuubi via Apache IP Clearance. The pure-sasl compatibility layer
+        originates from Impyla (Apache-2.0). See `NOTICE`.
+        
+        Licensed under the Apache License, Version 2.0 (see `LICENSE`).
+        
+Keywords: kyuubi,dlc,spark,sql,hiveserver2,dbapi,thrift
+Platform: UNKNOWN
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Topic :: Database :: Front-Ends
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Provides-Extra: yaml
+Provides-Extra: sqlalchemy
+Provides-Extra: arrow
+Provides-Extra: dev
+Provides-Extra: thrift

kyuubi_dlc_client-0.1.0/README.md

@@ -0,0 +1,154 @@
+# kyuubi-dlc-client
+
+Python client for the Kyuubi SQL gateway (DLC fork). Provides two independent
+layers over the two protocol entrances of a Kyuubi server:
+
+| Layer | Protocol | Port | Use case |
+|---|---|---|---|
+| `kyuubi_dlc_client.dbapi` | Thrift (HiveServer2) | 10009 | Interactive SQL, low latency, session state, streamed columnar results |
+| `kyuubi_dlc_client.rest` | REST API v1 | 10099 | Spark batch jobs (`/batches`), interactive SQL over HTTP |
+
+Both layers share the same authentication modes.
+
+## Highlights
+
+- **DB-API 2.0** (PEP 249) compliant Thrift interface: `connect()` / `cursor()` /
+  `execute()` / `fetch*()`, plus non-standard async execution (`async_=True`),
+  polling, progress logs and cancellation.
+- **REST batch client** with UUID-based idempotent submission, exponential
+  backoff polling, local log retrieval and cancellation.
+- **Authentication**: `NONE`, `CUSTOM`, and DLC `AK/SK` (access key) modes.
+- Robustness patches validated against Kyuubi: empty column payloads,
+  unknown type codes, custom SSL contexts.
+
+## Installation
+
+```bash
+pip install .            # runtime only
+pip install -e '.[dev]'  # development (tests, lint)
+```
+
+Requires Python 3.9+.
+
+## Quickstart: interactive SQL (Thrift)
+
+```python
+import kyuubi_dlc_client
+
+conn = kyuubi_dlc_client.connect(
+    host="127.0.0.1",
+    port=10009,
+    configuration={                      # session-level settings
+        "kyuubi.engine.share.level": "USER",
+    },
+)
+try:
+    cur = conn.cursor()
+    cur.execute("SELECT 1 AS id, 'hello' AS msg")
+    print(cur.fetchall())                # [(1, 'hello')]
+finally:
+    conn.close()
+```
+
+Async execution with timeout and cancellation:
+
+```python
+cur.execute("SELECT count(*) FROM range(1000000)", async_=True)
+cur.wait(timeout_s=300)                  # raises ExecutionTimeoutError + cancels
+print(cur.fetchall())
+```
+
+## Quickstart: Spark batch (REST)
+
+```python
+import uuid
+from kyuubi_dlc_client.rest import KyuubiRestClient
+
+with KyuubiRestClient("http://127.0.0.1:10099/api/v1") as client:
+    batch = client.submit_batch(
+        resource="cos://bucket/spark-examples.jar",
+        class_name="org.apache.spark.examples.SparkPi",
+        args=["10"],
+        batch_id=str(uuid.uuid4()),       # UUID is mandatory; enables idempotent retry
+    )
+    finished = client.wait_batch(batch.id, timeout_s=600)
+    print(finished.state, finished.appId)
+    logs = client.get_batch_local_log(batch.id, size=50)
+```
+
+## Authentication
+
+```python
+# Anonymous (Kyuubi NONE auth)
+conn = kyuubi_dlc_client.connect(host, port)
+
+# Username/password against a custom authenticator
+conn = kyuubi_dlc_client.connect(host, port, auth="CUSTOM",
+                                 username="user", password="pass")
+
+# DLC access key
+conn = kyuubi_dlc_client.connect(
+    host, port,
+    secret_id=os.environ["SECRET_ID"],
+    secret_key=os.environ["SECRET_KEY"],
+)
+```
+
+> AK/SK wire format: the default presents `secret_id`/`secret_key` over
+> SASL/PLAIN. Align `auth.AkSkAuth.credentials()` with the Kyuubi DLC
+> authenticator plugin once its contract is frozen.
+
+## Interactive SQL over HTTP (REST sessions)
+
+Use when only the HTTP port is reachable. Note that result sets are JSON
+rows (slower for large data) and multi-instance deployments require routing
+requests to the owning `kyuubiInstance`.
+
+```python
+session = client.open_session()
+operation = client.execute_statement(session, "SELECT 1", run_async=True)
+# poll GET /operations/{id}/event, then:
+rowset = client.fetch_rowset(operation)
+client.close_session(session)
+```
+
+## Examples & smoke test
+
+```bash
+# Thrift SQL quickstart
+KYUUBI_HOST=127.0.0.1 python examples/sql_quickstart.py
+
+# Batch quickstart (needs a spark-examples jar reachable by the server)
+KYUUBI_HOST=127.0.0.1 \
+BATCH_RESOURCE=local:///abs/path/spark-examples_2.12-3.5.1.jar \
+python examples/batch_submit.py
+
+# End-to-end smoke (SQL + batch)
+KYUUBI_HOST=127.0.0.1 bash scripts/e2e_smoke.sh
+```
+
+## Development
+
+```bash
+pip install -e '.[dev]'
+ruff check src tests
+pytest -m "not integration"
+```
+
+## Security notes
+
+- Never hard-code credentials; read them from environment variables or a
+  secret store.
+- The Thrift/REST ports of a Kyuubi server are internal infrastructure;
+  production access must go through the managed network entry point.
+- Batch `resource` paths are validated by the gateway (e.g. `cos://`
+  allow-list); clients must not accept arbitrary resource URIs from
+  untrusted input (SSRF).
+
+## Credits & license
+
+The TCLIService thrift bindings derive from PyHive (Apache-2.0), accepted into
+Apache Kyuubi via Apache IP Clearance. The pure-sasl compatibility layer
+originates from Impyla (Apache-2.0). See `NOTICE`.
+
+Licensed under the Apache License, Version 2.0 (see `LICENSE`).

kyuubi_dlc_client-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

kyuubi_dlc_client-0.1.0/setup.py

@@ -0,0 +1,77 @@
+import io
+import os
+import re
+
+from setuptools import find_packages, setup
+
+ROOT = os.path.dirname(os.path.abspath(__file__))
+
+
+def read(relative_path):
+    with io.open(os.path.join(ROOT, relative_path), encoding="utf-8") as source:  # noqa: UP020
+        return source.read()
+
+
+version_source = read("src/kyuubi_dlc_client/__version__.py")
+version_match = re.search(r"^__version__\s*=\s*['\"]([^'\"]+)['\"]", version_source, re.M)
+if version_match is None:
+    raise RuntimeError("Unable to find version string.")
+
+
+if __name__ == "__main__":
+    setup(
+        name="kyuubi_dlc_client",
+        version=version_match.group(1),
+        description="Python client for Apache Kyuubi (DLC fork): DB-API 2.0 over Thrift + REST API",
+        long_description=read("README.md"),
+        long_description_content_type="text/markdown",
+        python_requires=">=3.9",
+        license="Apache-2.0",
+        license_files=["LICENSE", "NOTICE"],
+        author="DLC Kyuubi Team",
+        url="https://git.example.com/dlc/kyuubi-dlc-client",
+        project_urls={
+            "Homepage": "https://git.example.com/dlc/kyuubi-dlc-client",
+            "Documentation": "https://kyuubi-dlc-client.readthedocs.io/",
+            "Source": "https://git.example.com/dlc/kyuubi-dlc-client",
+        },
+        keywords=["kyuubi", "dlc", "spark", "sql", "hiveserver2", "dbapi", "thrift"],
+        classifiers=[
+            "Development Status :: 3 - Alpha",
+            "Intended Audience :: Developers",
+            "License :: OSI Approved :: Apache Software License",
+            "Operating System :: OS Independent",
+            "Programming Language :: Python :: 3",
+            "Programming Language :: Python :: 3.9",
+            "Programming Language :: Python :: 3.10",
+            "Programming Language :: Python :: 3.11",
+            "Programming Language :: Python :: 3.12",
+            "Topic :: Database",
+            "Topic :: Database :: Front-Ends",
+        ],
+        package_dir={"": "src"},
+        packages=find_packages(where="src"),
+        install_requires=["requests>=2.28.0"],
+        extras_require={
+            "thrift": [
+                "thrift>=0.16.0,<0.19.0",
+                "thrift-sasl>=0.4.3",
+                "pure-sasl>=0.6.2",
+            ],
+            "sqlalchemy": ["sqlalchemy>=1.4"],
+            "arrow": ["pyarrow>=12.0", "pytz>=2024.1"],
+            "yaml": ["pyyaml>=6.0"],
+            "dev": [
+                "pytest>=7.0",
+                "pytest-cov>=4.0",
+                "ruff>=0.1",
+                "build>=1.0",
+                "responses>=0.23",
+                "thrift>=0.16.0,<0.19.0",
+                "thrift-sasl>=0.4.3",
+                "pure-sasl>=0.6.2",
+                "pyarrow>=12.0",
+                "pyyaml>=6.0",
+            ],
+        },
+    )