PyPI - iomete-sqlalchemy - Versions diffs - 1.0.20rc3__tar.gz - Mend

iomete-sqlalchemy 1.0.20rc3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (15) hide show

iomete_sqlalchemy-1.0.20rc3/LICENSE ADDED Viewed

@@ -0,0 +1,202 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+   1. Definitions.
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+   END OF TERMS AND CONDITIONS
+   APPENDIX: How to apply the Apache License to your work.
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+   Copyright 2026 IOMETE Inc.
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+       http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

iomete_sqlalchemy-1.0.20rc3/PKG-INFO ADDED Viewed

@@ -0,0 +1,120 @@
+Metadata-Version: 2.4
+Name: iomete-sqlalchemy
+Version: 1.0.20rc3
+Summary: SQLAlchemy dialect for IOMETE via Arrow Flight SQL
+License: Apache-2.0
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: sqlalchemy>=2.0.30
+Requires-Dist: adbc-driver-flightsql>=1.1.0
+Requires-Dist: pyarrow>=16.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: python-dotenv; extra == "dev"
+Dynamic: license-file
+# iomete-sqlalchemy
+A SQLAlchemy dialect for [IOMETE](https://iomete.com) using Arrow Flight SQL (`adbc-driver-flightsql`).
+All reflection methods are implemented and relational stubs (PK, FK, indexes) return graceful empty results instead of raising `NotImplementedError`, making it compatible with schema inspection tools and data catalog integrations.
+---
+## Requirements
+- `Python 3.9+`
+- `sqlalchemy >= 2.0.30`
+- `adbc-driver-flightsql >= 1.1.0`
+- `pyarrow >= 16.0`
+---
+## Installation
+```bash
+pip install iomete-sqlalchemy
+```
+---
+## Connection URL
+```
+iomete+flightsql://<user>:<password>@<host>:<port>/<catalog>/<schema>
+    ?cluster=<cluster>
+    &data_plane=<data_plane>
+    [&tls=true]
+    [&max_msg_size=134217728]
+```
+| Parameter | Description | Default |
+|---|---|---|
+| `host` | IOMETE host | — |
+| `port` | gRPC port | `443` |
+| `catalog` | Top-level catalog (e.g. `spark_catalog`) | — |
+| `schema` | Schema / database inside the catalog | — |
+| `cluster` | IOMETE compute cluster name | — |
+| `data_plane` | IOMETE data plane name | — |
+| `tls` | Use `grpc+tls` transport | `true` |
+| `max_msg_size` | Max gRPC message size in bytes | `134217728` (128 MB) |
+### Example
+```python
+from sqlalchemy import create_engine, inspect, text
+HOST       = "dev.iomete.cloud"
+PORT       = 443
+USERNAME   = "your-username"
+PASSWORD   = "your-password"
+CATALOG    = "spark_catalog"
+SCHEMA     = "default"
+CLUSTER    = "your-cluster"
+DATA_PLANE = "your-data-plane"
+engine = create_engine(
+    f"iomete+flightsql://{USERNAME}:{PASSWORD}@{HOST}:{PORT}"
+    f"/{CATALOG}/{SCHEMA}"
+    f"?cluster={CLUSTER}&data_plane={DATA_PLANE}"
+)
+# Check connection
+with engine.connect() as conn:
+    print(conn.execute(text("SELECT 1")).scalar())  # → 1
+# Reflect schema
+inspector = inspect(engine)
+print(inspector.get_schema_names())
+print(inspector.get_table_names(schema=f"{CATALOG}.{SCHEMA}"))
+```
+---
+## Supported Type Mappings
+| Spark / Iceberg type | SQLAlchemy type |
+|---|---|
+| `boolean` | `Boolean` |
+| `tinyint`, `smallint` | `SmallInteger` |
+| `int`, `integer` | `Integer` |
+| `bigint` | `BigInteger` |
+| `float`, `real` | `Float(24)` |
+| `double` | `Float(53)` |
+| `decimal(p, s)` | `Numeric(p, s)` |
+| `char(n)` | `CHAR(n)` |
+| `varchar(n)` | `VARCHAR(n)` |
+| `string`, `text` | `Text` |
+| `binary` | `LargeBinary` |
+| `date` | `Date` |
+| `timestamp`, `timestamp_ntz` | `DateTime` |
+| `array<…>`, `map<…>`, `struct<…>` | `JSON` |
+---
+## Contributing
+See [CONTRIBUTING.md](CONTRIBUTING.md) for architecture overview, codebase layout, and how to run tests.

iomete_sqlalchemy-1.0.20rc3/README.md ADDED Viewed

@@ -0,0 +1,102 @@
+# iomete-sqlalchemy
+A SQLAlchemy dialect for [IOMETE](https://iomete.com) using Arrow Flight SQL (`adbc-driver-flightsql`).
+All reflection methods are implemented and relational stubs (PK, FK, indexes) return graceful empty results instead of raising `NotImplementedError`, making it compatible with schema inspection tools and data catalog integrations.
+---
+## Requirements
+- `Python 3.9+`
+- `sqlalchemy >= 2.0.30`
+- `adbc-driver-flightsql >= 1.1.0`
+- `pyarrow >= 16.0`
+---
+## Installation
+```bash
+pip install iomete-sqlalchemy
+```
+---
+## Connection URL
+```
+iomete+flightsql://<user>:<password>@<host>:<port>/<catalog>/<schema>
+    ?cluster=<cluster>
+    &data_plane=<data_plane>
+    [&tls=true]
+    [&max_msg_size=134217728]
+```
+| Parameter | Description | Default |
+|---|---|---|
+| `host` | IOMETE host | — |
+| `port` | gRPC port | `443` |
+| `catalog` | Top-level catalog (e.g. `spark_catalog`) | — |
+| `schema` | Schema / database inside the catalog | — |
+| `cluster` | IOMETE compute cluster name | — |
+| `data_plane` | IOMETE data plane name | — |
+| `tls` | Use `grpc+tls` transport | `true` |
+| `max_msg_size` | Max gRPC message size in bytes | `134217728` (128 MB) |
+### Example
+```python
+from sqlalchemy import create_engine, inspect, text
+HOST       = "dev.iomete.cloud"
+PORT       = 443
+USERNAME   = "your-username"
+PASSWORD   = "your-password"
+CATALOG    = "spark_catalog"
+SCHEMA     = "default"
+CLUSTER    = "your-cluster"
+DATA_PLANE = "your-data-plane"
+engine = create_engine(
+    f"iomete+flightsql://{USERNAME}:{PASSWORD}@{HOST}:{PORT}"
+    f"/{CATALOG}/{SCHEMA}"
+    f"?cluster={CLUSTER}&data_plane={DATA_PLANE}"
+)
+# Check connection
+with engine.connect() as conn:
+    print(conn.execute(text("SELECT 1")).scalar())  # → 1
+# Reflect schema
+inspector = inspect(engine)
+print(inspector.get_schema_names())
+print(inspector.get_table_names(schema=f"{CATALOG}.{SCHEMA}"))
+```
+---
+## Supported Type Mappings
+| Spark / Iceberg type | SQLAlchemy type |
+|---|---|
+| `boolean` | `Boolean` |
+| `tinyint`, `smallint` | `SmallInteger` |
+| `int`, `integer` | `Integer` |
+| `bigint` | `BigInteger` |
+| `float`, `real` | `Float(24)` |
+| `double` | `Float(53)` |
+| `decimal(p, s)` | `Numeric(p, s)` |
+| `char(n)` | `CHAR(n)` |
+| `varchar(n)` | `VARCHAR(n)` |
+| `string`, `text` | `Text` |
+| `binary` | `LargeBinary` |
+| `date` | `Date` |
+| `timestamp`, `timestamp_ntz` | `DateTime` |
+| `array<…>`, `map<…>`, `struct<…>` | `JSON` |
+---
+## Contributing
+See [CONTRIBUTING.md](CONTRIBUTING.md) for architecture overview, codebase layout, and how to run tests.

iomete_sqlalchemy-1.0.20rc3/pyproject.toml ADDED Viewed

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+[project]
+name = "iomete-sqlalchemy"
+version = "1.0.20rc3"
+description = "SQLAlchemy dialect for IOMETE via Arrow Flight SQL"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.9"
+dependencies = [
+    "sqlalchemy>=2.0.30",
+    "adbc-driver-flightsql>=1.1.0",
+    "pyarrow>=16.0",
+]
+[project.optional-dependencies]
+dev = [
+    "pytest",
+    "pytest-cov",
+    "ruff",
+    "python-dotenv",
+]
+[project.entry-points."sqlalchemy.dialects"]
+"iomete.flightsql" = "iomete_sqlalchemy.dialect:IOMETEDialect"
+[tool.setuptools.packages.find]
+where = ["src"]
+[tool.pytest.ini_options]
+filterwarnings = [
+    "ignore:Cannot disable autocommit:Warning",
+]
+addopts = ""
+[tool.coverage.report]
+fail_under = 80

iomete_sqlalchemy-1.0.20rc3/setup.cfg ADDED Viewed

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build =
+tag_date = 0

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy/__init__.py ADDED Viewed

@@ -0,0 +1,4 @@
+from iomete_sqlalchemy.dialect import IOMETEDialect
+__version__ = "0.1.0"
+__all__ = ["IOMETEDialect"]

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy/dbapi.py ADDED Viewed

@@ -0,0 +1,38 @@
+"""
+Thin DBAPI-2 wrapper around adbc_driver_flightsql.dbapi.
+SQLAlchemy's DefaultDialect calls ``dialect.dbapi()`` (a classmethod) to get
+the DBAPI module, then uses ``module.connect(**kwargs)`` to open connections.
+We simply re-export ``adbc_driver_flightsql.dbapi`` so that all PEP-249
+symbols (connect, Error, Warning, …) are available at the module level.
+"""
+from adbc_driver_flightsql.dbapi import (  # noqa: F401
+    BINARY,
+    DATETIME,
+    NUMBER,
+    ROWID,
+    STRING,
+    Connection,
+    Cursor,
+    DataError,
+    DatabaseError,
+    Date,
+    DateFromTicks,
+    Error,
+    IntegrityError,
+    InterfaceError,
+    InternalError,
+    NotSupportedError,
+    OperationalError,
+    ProgrammingError,
+    Time,
+    TimeFromTicks,
+    Timestamp,
+    TimestampFromTicks,
+    Warning,
+    connect,
+)
+apilevel = "2.0"
+threadsafety = 1
+paramstyle = "qmark"

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy/dialect.py ADDED Viewed

@@ -0,0 +1,439 @@
+"""
+SQLAlchemy dialect for IOMETE via Arrow Flight SQL.
+Connection URL format
+---------------------
+  iomete+flightsql://<user>:<password>@<host>:<port>/<catalog>/<database>
+    ?cluster=<cluster_name>
+    &data_plane=<data_plane_name>
+    [&tls=true]
+    [&max_msg_size=134217728]
+IOMETE uses a three-level namespace: catalog → database → table.
+SQLAlchemy maps the second level to its ``schema`` parameter throughout
+the reflection API — so wherever SQLAlchemy says "schema", IOMETE means
+"database" (e.g. ``spark_catalog.default``).
+Example
+-------
+  iomete+flightsql://alice:secret@dev.iomete.cloud:443/spark_catalog/default
+      ?cluster=dwh&data_plane=spark-resources
+"""
+from __future__ import annotations
+import logging
+import re
+from typing import List, Optional
+from urllib.parse import unquote_plus
+from sqlalchemy import pool, text
+from sqlalchemy.engine import URL
+from sqlalchemy.engine.default import DefaultDialect
+from sqlalchemy.engine.interfaces import ReflectedColumn
+from sqlalchemy.exc import DatabaseError
+from sqlalchemy.sql.compiler import GenericTypeCompiler, IdentifierPreparer, SQLCompiler
+from iomete_sqlalchemy import dbapi as iomete_dbapi
+from iomete_sqlalchemy.types import spark_type_to_sqla
+logger = logging.getLogger(__name__)
+_NOT_FOUND_PATTERNS = (
+    "TABLE_OR_VIEW_NOT_FOUND",
+    "SCHEMA_NOT_FOUND",
+    "NoSuchNamespaceException",
+    "NoSuchDatabaseException",
+)
+def _is_not_found(exc: Exception) -> bool:
+    """Return True if the exception message indicates a missing object.
+    Used to distinguish expected "object does not exist" errors (which should
+    result in graceful empty/None returns) from unexpected failures (which
+    should be logged as warnings).
+    """
+    msg = str(exc)
+    return any(pattern in msg for pattern in _NOT_FOUND_PATTERNS)
+class SparkIdentifierPreparer(IdentifierPreparer):
+    """Use backtick quoting for identifiers.
+    Spark SQL does not accept ANSI double-quote identifiers (``"name"``);
+    backticks (`` `name` ``) are the only supported quoting style.
+    """
+    def __init__(self, dialect):
+        super().__init__(dialect, initial_quote="`", final_quote="`")
+    def quote_schema(self, schema, force=None):
+        """Quote each dot-separated part of the schema individually.
+        SQLAlchemy encodes IOMETE's two-level schema as ``"catalog.database"``.
+        The base implementation would quote this as a single token
+        (`` `catalog.database` ``), which Spark rejects.  Splitting on ``.``
+        and quoting each part produces `` `catalog`.`database` `` instead.
+        """
+        return ".".join(self.quote(part, force) for part in schema.split("."))
+class SparkSQLCompiler(SQLCompiler):
+    """Compiler overrides for Spark SQL dialect quirks.
+    Spark requires integer literals for LIMIT/OFFSET (e.g. ``LIMIT 10``),
+    not bind parameters (``LIMIT ?``). The base SQLCompiler emits bind params,
+    so we override ``limit_clause`` to force ``literal_binds=True``.
+    """
+    def limit_clause(self, select, **kw):
+        limit = select._limit_clause
+        offset = select._offset_clause
+        clause = ""
+        if limit is not None:
+            clause += "\n LIMIT " + self.process(limit, literal_binds=True)
+        if offset is not None:
+            clause += "\n OFFSET " + self.process(offset, literal_binds=True)
+        return clause
+    def fetch_clause(self, select, **kw):
+        # Spark doesn't support FETCH FIRST n ROWS either
+        return self.limit_clause(select, **kw)
+class SparkTypeCompiler(GenericTypeCompiler):
+    """DDL type compiler that maps SQLAlchemy types to Spark SQL DDL equivalents.
+    Spark SQL quirks handled here:
+    - ``DATETIME`` is unsupported; ``TIMESTAMP`` is the correct keyword.
+    - ``VARCHAR(n)`` is valid in Spark 3.x, but bare ``VARCHAR`` without a length
+      is never accepted — Spark has no unbounded VARCHAR type.  SQLAlchemy's
+      ``String()`` (no length) must therefore map to ``STRING``, which is Spark's
+      native unbounded string type.
+    """
+    def visit_DATETIME(self, type_, **kw):
+        return "TIMESTAMP"
+    def visit_datetime(self, type_, **kw):
+        return "TIMESTAMP"
+    def visit_VARCHAR(self, type_, **kw):
+        if type_.length:
+            return f"VARCHAR({type_.length})"
+        return "STRING"
+    def visit_NVARCHAR(self, type_, **kw):
+        if type_.length:
+            return f"VARCHAR({type_.length})"
+        return "STRING"
+    def visit_TEXT(self, type_, **kw):
+        return "STRING"
+class IOMETEDialect(DefaultDialect):
+    # ── dialect identity ────────────────────────────────────────────────────
+    name = "iomete"
+    driver = "flightsql"
+    preparer = SparkIdentifierPreparer
+    statement_compiler = SparkSQLCompiler
+    type_compiler_cls = SparkTypeCompiler
+    # ── capabilities ────────────────────────────────────────────────────────
+    supports_alter = False
+    supports_empty_insert = False
+    supports_native_boolean = True
+    supports_multivalues_insert = True
+    supports_statement_cache = True  # must be explicit — SQLAlchemy checks __dict__, not inheritance
+    default_schema_name = "default"
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self._catalog: str | None = None
+    # ── pooling ─────────────────────────────────────────────────────────────
+    @classmethod
+    def import_dbapi(cls):
+        return iomete_dbapi
+    # backwards-compat alias for SQLAlchemy < 2.0
+    @classmethod
+    def dbapi(cls):
+        return cls.import_dbapi()
+    # Use NullPool by default: Flight SQL connections are stateful gRPC
+    # streams; connection pooling doesn't add value here.
+    @classmethod
+    def get_pool_class(cls, url):
+        return pool.NullPool
+    # ── URL → connect args ──────────────────────────────────────────────────
+    def create_connect_args(self, url: URL):
+        # url.database is "catalog/schema"; extract only the catalog part.
+        database = url.database or ""
+        catalog_part, _, _ = database.partition("/")
+        self._catalog = catalog_part or None
+        query_params = url.query
+        def get_query_param(key: str, default: str | None = None) -> str | None:
+            value = query_params.get(key, default)
+            if isinstance(value, (list, tuple)):
+                *_, value = value
+            return unquote_plus(value) if value is not None else None
+        tls = get_query_param("tls", "true").lower() not in ("false", "0", "no")
+        scheme = "grpc+tls" if tls else "grpc"
+        host = url.host or "localhost"
+        port = url.port or 443
+        uri = f"{scheme}://{host}:{port}"
+        db_kwargs: dict[str, str] = {}
+        if url.username:
+            db_kwargs["username"] = unquote_plus(url.username)
+        if url.password:
+            db_kwargs["password"] = unquote_plus(str(url.password))
+        cluster = get_query_param("cluster")
+        if cluster:
+            db_kwargs["adbc.flight.sql.rpc.call_header.cluster"] = cluster
+        data_plane = get_query_param("data_plane")
+        if data_plane:
+            db_kwargs["adbc.flight.sql.rpc.call_header.data-plane"] = data_plane
+        max_msg_size = get_query_param("max_msg_size", "134217728")
+        db_kwargs["adbc.flight.sql.client_option.with_max_msg_size"] = max_msg_size
+        return [], {"uri": uri, "db_kwargs": db_kwargs}
+    # ── schema helpers ──────────────────────────────────────────────────────
+    @staticmethod
+    def _catalog_schema(schema: str | None) -> tuple[str | None, str | None]:
+        """
+        IOMETE uses a three-level namespace: catalog.database.table.
+        SQLAlchemy's ``schema`` parameter encodes both levels as
+        ``"catalog.database"`` (dot-separated), e.g. ``"spark_catalog.default"``.
+        If only one token is given it is treated as the database name within
+        the default catalog.
+        """
+        if not schema:
+            return None, None
+        catalog, sep, db = schema.partition(".")
+        if sep:
+            return catalog, db
+        return None, catalog
+    def _schema_qualifier(self, schema: str | None) -> str:
+        """Return a backtick-quoted ``catalog.database`` qualifier, or empty string."""
+        catalog, db = self._catalog_schema(schema)
+        if not db:
+            return ""
+        return f"`{catalog}`.`{db}`" if catalog else f"`{db}`"
+    def _full_name(self, schema: str | None, object_name: str) -> str:
+        """Build a fully-qualified backtick-quoted name from schema + object."""
+        catalog, db = self._catalog_schema(schema)
+        parts = []
+        if catalog:
+            parts.append(f"`{catalog}`")
+        if db:
+            parts.append(f"`{db}`")
+        parts.append(f"`{object_name}`")
+        return ".".join(parts)
+    # ── dialect initialisation ──────────────────────────────────────────────
+    def on_connect(self):
+        def connect(dbapi_connection):
+            catalog = self._catalog
+            if catalog:
+                cursor = dbapi_connection.cursor()
+                cursor.execute(f"USE {catalog}")
+                cursor.close()
+        return connect
+    def initialize(self, connection):
+        pass  # nothing to probe at startup
+    def do_executemany(self, cursor, statement, parameters, context=None):
+        """Execute a DML statement one row at a time.
+        IOMETE's Arrow Flight SQL server rejects multi-row batches, so each row is
+        sent individually via ``cursor.executemany(statement, [row])``.
+        https://github.com/iomete/spark/blob/18a173f3b11cdbf21cb20b2130572a57dbdb1941/iomete/arrow-server/src/main/java/com/iomete/spark/arrow/flight/sql/handlers/PreparedStatementHandler.java#L296
+        TODO: remove this workaround once the server supports batch parameter binding.
+        """
+        for params in parameters:
+            cursor.executemany(statement, [params])
+    def do_execute(self, cursor, statement, parameters, context=None):
+        is_write = context is not None and (context.isddl or context.is_crud)
+        if is_write and parameters:
+            # Parameterized DML — DoPut write path, synchronous.
+            cursor.executemany(statement, [parameters])
+        elif is_write:
+            # TODO: investigate DML/DDL failure on dev cluster for parameterless DDL/DML statements.
+            # Works fine against my locally running Flight SQL server 3.5.5, but fails on the dev cluster.
+            # Not investigated deeply — may be caused by an unknown deployment difference from
+            # another team member. Draining the cursor is a workaround for now.
+            cursor.execute(statement, parameters)
+            try:
+                cursor.fetchall()
+            except Exception as e:
+                logger.debug("Ignoring error while draining cursor: %s", e)
+        else:
+            cursor.execute(statement, parameters)
+    def do_rollback(self, dbapi_connection):
+        pass
+    # ── server version ──────────────────────────────────────────────────────
+    def _get_server_version_info(self, connection):
+        try:
+            row = connection.execute(text("SELECT version()")).fetchone()
+            if row:
+                version_match = re.search(r"(\d+)\.(\d+)\.(\d+)", row[0])
+                if version_match:
+                    return tuple(int(x) for x in version_match.groups())
+        except DatabaseError as e:
+            logger.debug("Could not retrieve server version: %s", e)
+        return 0, 0, 0
+    # ── schema / table listing ──────────────────────────────────────────────
+    def get_schema_names(self, connection, **kw) -> List[str]:
+        """Return all schemas visible in the current catalog."""
+        rows = connection.execute(text("SHOW SCHEMAS")).fetchall()
+        return [row[0] for row in rows]
+    def get_table_names(self, connection, schema: str | None = None, **kw) -> List[str]:
+        qualifier = self._schema_qualifier(schema)
+        sql = f"SHOW TABLES{' IN ' + qualifier if qualifier else ''}"
+        rows = connection.execute(text(sql)).fetchall()
+        # Iomete SHOW TABLES returns (namespace, tableName, isTemporary)
+        return [row[1] if len(row) > 1 else row[0] for row in rows]
+    def get_view_definition(
+        self,
+        connection,
+        view_name: str,
+        schema: str | None = None,
+        **kw,
+    ) -> Optional[str]:
+        full_name = self._full_name(schema, view_name)
+        try:
+            rows = connection.execute(text(f"SHOW CREATE TABLE {full_name}")).fetchall()
+            if rows:
+                return "\n".join(row[0] for row in rows)
+            return None
+        except DatabaseError as e:
+            if _is_not_found(e):
+                return None
+            logger.warning(
+                "get_view_definition(%r) failed unexpectedly: %s", view_name, e
+            )
+            return None
+    def get_view_names(self, connection, schema: str | None = None, **kw) -> List[str]:
+        qualifier = self._schema_qualifier(schema)
+        sql = f"SHOW VIEWS{' IN ' + qualifier if qualifier else ''}"
+        try:
+            rows = connection.execute(text(sql)).fetchall()
+            return [row[1] if len(row) > 1 else row[0] for row in rows]
+        except DatabaseError as e:
+            logger.warning("get_view_names(schema=%r) failed: %s", schema, e)
+            return []
+    # ── column reflection ───────────────────────────────────────────────────
+    def get_columns(
+        self,
+        connection,
+        table_name: str,
+        schema: str | None = None,
+        **kw,
+    ) -> List[ReflectedColumn]:
+        full_name = self._full_name(schema, table_name)
+        rows = connection.execute(text(f"DESCRIBE TABLE {full_name}")).fetchall()
+        columns: List[ReflectedColumn] = []
+        for row in rows:
+            col_name = row[0]
+            type_str = row[1] if len(row) > 1 else "string"
+            comment = row[2] if len(row) > 2 else None
+            # DESCRIBE sometimes includes partition/metadata separators
+            if not col_name or col_name.startswith("#"):
+                break
+            col: ReflectedColumn = {
+                "name": col_name,
+                "type": spark_type_to_sqla(type_str),
+                "nullable": True,  # DESCRIBE TABLE doesn't expose nullability; default to True
+                "default": None,
+                "autoincrement": False,
+            }
+            if comment:
+                col["comment"] = comment
+            columns.append(col)
+        return columns
+    # ── graceful stubs (Lakehouse has no PK / FK / index concepts) ──────────
+    def get_pk_constraint(self, connection, table_name, schema=None, **kw):
+        return {"constrained_columns": [], "name": None}
+    def get_foreign_keys(self, connection, table_name, schema=None, **kw):
+        return []
+    def get_indexes(self, connection, table_name, schema=None, **kw):
+        return []
+    def get_unique_constraints(self, connection, table_name, schema=None, **kw):
+        return []
+    def get_check_constraints(self, connection, table_name, schema=None, **kw):
+        return []
+    def get_table_comment(self, connection, table_name, schema=None, **kw):
+        full_name = self._full_name(schema, table_name)
+        try:
+            rows = connection.execute(
+                text(f"DESCRIBE TABLE EXTENDED {full_name}")
+            ).fetchall()
+            for row in rows:
+                if row[0].strip().lower() == "comment":
+                    comment = row[1].strip() if row[1] else None
+                    return {"text": comment or None}
+        except DatabaseError as e:
+            if not _is_not_found(e):
+                logger.warning(
+                    "get_table_comment(%r) failed unexpectedly: %s", table_name, e
+                )
+        return {"text": None}
+    # ── existence checks ────────────────────────────────────────────────────
+    def has_table(
+        self, connection, table_name: str, schema: str | None = None, **kw
+    ) -> bool:
+        try:
+            return table_name in self.get_table_names(connection, schema=schema)
+        except DatabaseError as e:
+            logger.warning("has_table(%r) check failed: %s", table_name, e)
+            return False
+    def has_schema(self, connection, schema_name: str, **kw) -> bool:
+        try:
+            return schema_name in self.get_schema_names(connection)
+        except DatabaseError as e:
+            logger.warning("has_schema(%r) check failed: %s", schema_name, e)
+            return False
+    # ── default schema detection ─────────────────────────────────────────────
+    def _get_default_schema_name(self, connection):
+        try:
+            row = connection.execute(text("SELECT current_database()")).fetchone()
+            if row:
+                return row[0]
+        except DatabaseError as e:
+            logger.debug("Could not retrieve default schema name: %s", e)
+        return self.default_schema_name

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy/types.py ADDED Viewed

@@ -0,0 +1,117 @@
+"""
+Mapping from Spark/Iceberg type strings to SQLAlchemy types.
+"""
+import re
+from sqlalchemy import types as sqltypes
+# Spark/Iceberg types that map directly to a SQLAlchemy type with no arguments.
+# Spark often has multiple aliases for the same type (e.g. "int" / "integer",
+# "bigint" / "long"), so all aliases are listed here.
+_KEYWORD_TYPE_MAP: dict[str, sqltypes.TypeEngine] = {
+    # booleans
+    "boolean": sqltypes.Boolean(),
+    "bool": sqltypes.Boolean(),
+    # integers
+    "tinyint": sqltypes.SmallInteger(),
+    "byte": sqltypes.SmallInteger(),
+    "smallint": sqltypes.SmallInteger(),
+    "short": sqltypes.SmallInteger(),
+    "int": sqltypes.Integer(),
+    "integer": sqltypes.Integer(),
+    "bigint": sqltypes.BigInteger(),
+    "long": sqltypes.BigInteger(),
+    # floating-point
+    "float": sqltypes.Float(precision=24),
+    "real": sqltypes.Float(precision=24),
+    "double": sqltypes.Float(precision=53),
+    "double precision": sqltypes.Float(precision=53),
+    # strings / binary
+    "string": sqltypes.Text(),
+    "text": sqltypes.Text(),
+    "binary": sqltypes.LargeBinary(),
+    "bytes": sqltypes.LargeBinary(),
+    # date / time
+    "date": sqltypes.Date(),
+    "timestamp": sqltypes.DateTime(),
+    "timestamp_ntz": sqltypes.DateTime(),
+    "interval": sqltypes.Interval(),
+    # misc
+    "void": sqltypes.NullType(),
+    "null": sqltypes.NullType(),
+}
+# Matches decimal(precision, scale)  e.g. "decimal(10, 2)"
+_DECIMAL_WITH_SCALE    = re.compile(r"decimal\s*\(\s*(\d+)\s*,\s*(\d+)\s*\)")
+# Matches decimal(precision) only    e.g. "decimal(10)"
+_DECIMAL_PRECISION_ONLY = re.compile(r"decimal\s*\(\s*(\d+)\s*\)")
+# Matches char(length)    e.g. "char(64)"
+_CHAR_WITH_LENGTH    = re.compile(r"char\s*\(\s*(\d+)\s*\)")
+# Matches varchar(length) e.g. "varchar(255)"
+_VARCHAR_WITH_LENGTH = re.compile(r"varchar\s*\(\s*(\d+)\s*\)")
+# Spark complex type prefixes — no SQLAlchemy equivalent, mapped to JSON
+_COMPLEX_TYPE_PREFIXES = ("array<", "map<", "struct<")
+def spark_type_to_sqla(type_str: str) -> sqltypes.TypeEngine:
+    """
+    Convert a Spark/Iceberg type string (Spark → SQLAlchemy direction).
+    Called during reflection: ``DESCRIBE TABLE`` returns a type string such as
+    ``"decimal(10, 2)"`` or ``"array<string>"``; this function maps it to the
+    corresponding SQLAlchemy ``TypeEngine`` instance.
+    Handles parameterised forms such as:
+      - decimal(10, 2)
+      - char(64)
+      - varchar(255)
+      - array<string>
+      - map<string, int>
+      - struct<field:type, …>
+    """
+    if not type_str:
+        return sqltypes.NullType()
+    normalized = type_str.strip().lower()
+    # ── decimal(precision, scale) ──────────────────────────────────────────
+    match = _DECIMAL_WITH_SCALE.fullmatch(normalized)
+    if match:
+        precision, scale = int(match.group(1)), int(match.group(2))
+        return sqltypes.Numeric(precision=precision, scale=scale)
+    match = _DECIMAL_PRECISION_ONLY.fullmatch(normalized)
+    if match:
+        return sqltypes.Numeric(precision=int(match.group(1)))
+    if normalized in ("decimal", "numeric"):
+        return sqltypes.Numeric()
+    # ── char(length) / varchar(length) ────────────────────────────────────
+    match = _CHAR_WITH_LENGTH.fullmatch(normalized)
+    if match:
+        return sqltypes.CHAR(length=int(match.group(1)))
+    match = _VARCHAR_WITH_LENGTH.fullmatch(normalized)
+    if match:
+        return sqltypes.VARCHAR(length=int(match.group(1)))
+    if normalized in ("char", "varchar", "character varying"):
+        return sqltypes.Text()
+    # ── complex types: array<…>, map<…>, struct<…> ────────────────────────
+    # SQLAlchemy has no native equivalent; JSON is the closest approximation
+    # for schema reflection and metadata tooling.
+    if normalized.startswith(_COMPLEX_TYPE_PREFIXES):
+        return sqltypes.JSON()
+    # ── plain keyword lookup ───────────────────────────────────────────────
+    if normalized in _KEYWORD_TYPE_MAP:
+        return _KEYWORD_TYPE_MAP[normalized]
+    # ── unknown type → Text so reflection never crashes ───────────────────
+    return sqltypes.Text()

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/PKG-INFO ADDED Viewed

@@ -0,0 +1,120 @@
+Metadata-Version: 2.4
+Name: iomete-sqlalchemy
+Version: 1.0.20rc3
+Summary: SQLAlchemy dialect for IOMETE via Arrow Flight SQL
+License: Apache-2.0
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: sqlalchemy>=2.0.30
+Requires-Dist: adbc-driver-flightsql>=1.1.0
+Requires-Dist: pyarrow>=16.0
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: python-dotenv; extra == "dev"
+Dynamic: license-file
+# iomete-sqlalchemy
+A SQLAlchemy dialect for [IOMETE](https://iomete.com) using Arrow Flight SQL (`adbc-driver-flightsql`).
+All reflection methods are implemented and relational stubs (PK, FK, indexes) return graceful empty results instead of raising `NotImplementedError`, making it compatible with schema inspection tools and data catalog integrations.
+---
+## Requirements
+- `Python 3.9+`
+- `sqlalchemy >= 2.0.30`
+- `adbc-driver-flightsql >= 1.1.0`
+- `pyarrow >= 16.0`
+---
+## Installation
+```bash
+pip install iomete-sqlalchemy
+```
+---
+## Connection URL
+```
+iomete+flightsql://<user>:<password>@<host>:<port>/<catalog>/<schema>
+    ?cluster=<cluster>
+    &data_plane=<data_plane>
+    [&tls=true]
+    [&max_msg_size=134217728]
+```
+| Parameter | Description | Default |
+|---|---|---|
+| `host` | IOMETE host | — |
+| `port` | gRPC port | `443` |
+| `catalog` | Top-level catalog (e.g. `spark_catalog`) | — |
+| `schema` | Schema / database inside the catalog | — |
+| `cluster` | IOMETE compute cluster name | — |
+| `data_plane` | IOMETE data plane name | — |
+| `tls` | Use `grpc+tls` transport | `true` |
+| `max_msg_size` | Max gRPC message size in bytes | `134217728` (128 MB) |
+### Example
+```python
+from sqlalchemy import create_engine, inspect, text
+HOST       = "dev.iomete.cloud"
+PORT       = 443
+USERNAME   = "your-username"
+PASSWORD   = "your-password"
+CATALOG    = "spark_catalog"
+SCHEMA     = "default"
+CLUSTER    = "your-cluster"
+DATA_PLANE = "your-data-plane"
+engine = create_engine(
+    f"iomete+flightsql://{USERNAME}:{PASSWORD}@{HOST}:{PORT}"
+    f"/{CATALOG}/{SCHEMA}"
+    f"?cluster={CLUSTER}&data_plane={DATA_PLANE}"
+)
+# Check connection
+with engine.connect() as conn:
+    print(conn.execute(text("SELECT 1")).scalar())  # → 1
+# Reflect schema
+inspector = inspect(engine)
+print(inspector.get_schema_names())
+print(inspector.get_table_names(schema=f"{CATALOG}.{SCHEMA}"))
+```
+---
+## Supported Type Mappings
+| Spark / Iceberg type | SQLAlchemy type |
+|---|---|
+| `boolean` | `Boolean` |
+| `tinyint`, `smallint` | `SmallInteger` |
+| `int`, `integer` | `Integer` |
+| `bigint` | `BigInteger` |
+| `float`, `real` | `Float(24)` |
+| `double` | `Float(53)` |
+| `decimal(p, s)` | `Numeric(p, s)` |
+| `char(n)` | `CHAR(n)` |
+| `varchar(n)` | `VARCHAR(n)` |
+| `string`, `text` | `Text` |
+| `binary` | `LargeBinary` |
+| `date` | `Date` |
+| `timestamp`, `timestamp_ntz` | `DateTime` |
+| `array<…>`, `map<…>`, `struct<…>` | `JSON` |
+---
+## Contributing
+See [CONTRIBUTING.md](CONTRIBUTING.md) for architecture overview, codebase layout, and how to run tests.

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/SOURCES.txt ADDED Viewed

@@ -0,0 +1,13 @@
+LICENSE
+README.md
+pyproject.toml
+src/iomete_sqlalchemy/__init__.py
+src/iomete_sqlalchemy/dbapi.py
+src/iomete_sqlalchemy/dialect.py
+src/iomete_sqlalchemy/types.py
+src/iomete_sqlalchemy.egg-info/PKG-INFO
+src/iomete_sqlalchemy.egg-info/SOURCES.txt
+src/iomete_sqlalchemy.egg-info/dependency_links.txt
+src/iomete_sqlalchemy.egg-info/entry_points.txt
+src/iomete_sqlalchemy.egg-info/requires.txt
+src/iomete_sqlalchemy.egg-info/top_level.txt

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/dependency_links.txt ADDED Viewed

	@@ -0,0 +1 @@
1	+

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/entry_points.txt ADDED Viewed

	@@ -0,0 +1,2 @@
1	+ [sqlalchemy.dialects]
2	+ iomete.flightsql = iomete_sqlalchemy.dialect:IOMETEDialect

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/requires.txt ADDED Viewed

@@ -0,0 +1,9 @@
+sqlalchemy>=2.0.30
+adbc-driver-flightsql>=1.1.0
+pyarrow>=16.0
+[dev]
+pytest
+pytest-cov
+ruff
+python-dotenv

iomete_sqlalchemy-1.0.20rc3/src/iomete_sqlalchemy.egg-info/top_level.txt ADDED Viewed

	@@ -0,0 +1 @@
1	+ iomete_sqlalchemy