sqlmesh-gizmosql 0.1.0__tar.gz

sqlmesh_gizmosql-0.1.0/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 2.4
+Name: sqlmesh-gizmosql
+Version: 0.1.0
+Summary: GizmoSQL engine adapter for SQLMesh
+Author: Philip
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/gizmodata/sqlmesh-gizmosql
+Project-URL: Repository, https://github.com/gizmodata/sqlmesh-gizmosql
+Project-URL: Documentation, https://github.com/gizmodata/sqlmesh-gizmosql#readme
+Keywords: sqlmesh,gizmosql,duckdb,arrow,flight-sql,data-engineering
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: sqlmesh>=0.100.0
+Requires-Dist: adbc-driver-flightsql>=1.10.0
+Requires-Dist: pyarrow>=22.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+
+# SQLMesh GizmoSQL Adapter
+
+A [SQLMesh](https://sqlmesh.com) engine adapter for [GizmoSQL](https://github.com/gizmodata/gizmosql) - a database server that uses DuckDB as its execution engine and exposes an Arrow Flight SQL interface for remote connections.
+
+## Installation
+
+```bash
+pip install sqlmesh-gizmosql
+```
+
+This will install `sqlmesh`, `adbc-driver-flightsql`, and `pyarrow` as dependencies.
+
+## Usage
+
+### 1. Import the adapter
+
+Simply import the package before using SQLMesh. The adapter registers itself automatically:
+
+```python
+import sqlmesh_gizmosql  # Registers GizmoSQL adapter
+from sqlmesh import Context
+
+context = Context(paths="path/to/project")
+```
+
+### 2. Configure your connection
+
+Add a GizmoSQL connection to your `config.yaml`:
+
+```yaml
+gateways:
+  my_gizmosql:
+    connection:
+      type: gizmosql
+      host: localhost
+      port: 31337
+      username: your_username
+      password: your_password
+      database: my_database  # optional, default catalog
+      use_encryption: true   # default: true (uses TLS)
+      disable_certificate_verification: false  # for self-signed certs
+```
+
+### 3. Use SQLMesh as normal
+
+```bash
+sqlmesh plan
+sqlmesh run
+```
+
+## Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `host` | str | `localhost` | GizmoSQL server hostname |
+| `port` | int | `31337` | GizmoSQL server port |
+| `username` | str | *required* | Authentication username |
+| `password` | str | *required* | Authentication password |
+| `database` | str | `None` | Default database/catalog |
+| `use_encryption` | bool | `True` | Use TLS encryption |
+| `disable_certificate_verification` | bool | `False` | Skip TLS cert verification |
+| `concurrent_tasks` | int | `4` | Max concurrent tasks |
+| `register_comments` | bool | `True` | Register model comments |
+| `pre_ping` | bool | `False` | Pre-ping connections |
+
+## Features
+
+- **Arrow Flight SQL**: Efficient data transfer using Arrow's columnar format
+- **Full Catalog Support**: Create, drop, and switch between databases
+- **Transaction Support**: Full transaction control via SQL statements
+- **ADBC Bulk Ingestion**: Fast data loading using Arrow-native bulk operations
+- **DuckDB Compatibility**: Uses DuckDB SQL dialect for query generation
+
+## Requirements
+
+- Python >= 3.10
+- SQLMesh >= 0.100.0
+- A running GizmoSQL server with DuckDB backend
+
+## Development
+
+```bash
+# Clone the repository
+git clone https://github.com/philip/sqlmesh-gizmosql.git
+cd sqlmesh-gizmosql
+
+# Install dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run linting
+ruff check .
+mypy sqlmesh_gizmosql
+```
+
+## License
+
+Apache 2.0

sqlmesh_gizmosql-0.1.0/README.md

@@ -0,0 +1,100 @@
+# SQLMesh GizmoSQL Adapter
+
+A [SQLMesh](https://sqlmesh.com) engine adapter for [GizmoSQL](https://github.com/gizmodata/gizmosql) - a database server that uses DuckDB as its execution engine and exposes an Arrow Flight SQL interface for remote connections.
+
+## Installation
+
+```bash
+pip install sqlmesh-gizmosql
+```
+
+This will install `sqlmesh`, `adbc-driver-flightsql`, and `pyarrow` as dependencies.
+
+## Usage
+
+### 1. Import the adapter
+
+Simply import the package before using SQLMesh. The adapter registers itself automatically:
+
+```python
+import sqlmesh_gizmosql  # Registers GizmoSQL adapter
+from sqlmesh import Context
+
+context = Context(paths="path/to/project")
+```
+
+### 2. Configure your connection
+
+Add a GizmoSQL connection to your `config.yaml`:
+
+```yaml
+gateways:
+  my_gizmosql:
+    connection:
+      type: gizmosql
+      host: localhost
+      port: 31337
+      username: your_username
+      password: your_password
+      database: my_database  # optional, default catalog
+      use_encryption: true   # default: true (uses TLS)
+      disable_certificate_verification: false  # for self-signed certs
+```
+
+### 3. Use SQLMesh as normal
+
+```bash
+sqlmesh plan
+sqlmesh run
+```
+
+## Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `host` | str | `localhost` | GizmoSQL server hostname |
+| `port` | int | `31337` | GizmoSQL server port |
+| `username` | str | *required* | Authentication username |
+| `password` | str | *required* | Authentication password |
+| `database` | str | `None` | Default database/catalog |
+| `use_encryption` | bool | `True` | Use TLS encryption |
+| `disable_certificate_verification` | bool | `False` | Skip TLS cert verification |
+| `concurrent_tasks` | int | `4` | Max concurrent tasks |
+| `register_comments` | bool | `True` | Register model comments |
+| `pre_ping` | bool | `False` | Pre-ping connections |
+
+## Features
+
+- **Arrow Flight SQL**: Efficient data transfer using Arrow's columnar format
+- **Full Catalog Support**: Create, drop, and switch between databases
+- **Transaction Support**: Full transaction control via SQL statements
+- **ADBC Bulk Ingestion**: Fast data loading using Arrow-native bulk operations
+- **DuckDB Compatibility**: Uses DuckDB SQL dialect for query generation
+
+## Requirements
+
+- Python >= 3.10
+- SQLMesh >= 0.100.0
+- A running GizmoSQL server with DuckDB backend
+
+## Development
+
+```bash
+# Clone the repository
+git clone https://github.com/philip/sqlmesh-gizmosql.git
+cd sqlmesh-gizmosql
+
+# Install dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run linting
+ruff check .
+mypy sqlmesh_gizmosql
+```
+
+## License
+
+Apache 2.0

sqlmesh_gizmosql-0.1.0/pyproject.toml

@@ -0,0 +1,76 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "sqlmesh-gizmosql"
+version = "0.1.0"
+description = "GizmoSQL engine adapter for SQLMesh"
+readme = "README.md"
+license = {text = "Apache-2.0"}
+requires-python = ">=3.10"
+authors = [
+    {name = "Philip"}
+]
+keywords = ["sqlmesh", "gizmosql", "duckdb", "arrow", "flight-sql", "data-engineering"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: Apache Software License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Topic :: Database",
+    "Topic :: Scientific/Engineering :: Information Analysis",
+]
+
+dependencies = [
+    "sqlmesh>=0.100.0",
+    "adbc-driver-flightsql>=1.10.0",
+    "pyarrow>=22.0.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0.0",
+    "pytest-cov>=4.0.0",
+    "mypy>=1.0.0",
+    "ruff>=0.1.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/gizmodata/sqlmesh-gizmosql"
+Repository = "https://github.com/gizmodata/sqlmesh-gizmosql"
+Documentation = "https://github.com/gizmodata/sqlmesh-gizmosql#readme"
+
+[tool.setuptools.packages.find]
+include = ["sqlmesh_gizmosql*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_*.py"]
+python_functions = ["test_*"]
+addopts = "-v --tb=short -m 'not integration'"
+markers = [
+    "integration: marks tests as integration tests (require Docker)",
+    "docker: marks tests that require Docker",
+]
+
+[tool.mypy]
+python_version = "3.10"
+warn_return_any = true
+warn_unused_configs = true
+ignore_missing_imports = true
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "W"]
+ignore = ["E501"]

sqlmesh_gizmosql-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql/__init__.py

@@ -0,0 +1,65 @@
+"""
+SQLMesh GizmoSQL Adapter.
+
+This package provides GizmoSQL support for SQLMesh. When imported, it automatically
+registers the GizmoSQL engine adapter and connection configuration with SQLMesh.
+
+Usage:
+    # Option 1: Explicit import (recommended for clarity)
+    import sqlmesh_gizmosql
+    from sqlmesh import Context
+
+    # Option 2: Just import - registration happens automatically
+    import sqlmesh_gizmosql  # noqa: F401
+
+    # Then use SQLMesh normally with type: gizmosql in your config
+
+Example config.yaml:
+    gateways:
+      my_gizmosql:
+        connection:
+          type: gizmosql
+          host: localhost
+          port: 31337
+          username: user
+          password: pass
+          database: my_database
+"""
+from sqlmesh_gizmosql.adapter import GizmoSQLEngineAdapter
+from sqlmesh_gizmosql.connection import GizmoSQLConnectionConfig
+
+__version__ = "0.1.0"
+__all__ = ["GizmoSQLEngineAdapter", "GizmoSQLConnectionConfig", "register", "__version__"]
+
+_registered = False
+
+
+def register() -> None:
+    """
+    Register GizmoSQL adapter and connection config with SQLMesh.
+
+    This function is called automatically when the module is imported,
+    but can also be called explicitly if needed.
+    """
+    global _registered
+    if _registered:
+        return
+
+    # Register the engine adapter
+    from sqlmesh.core import engine_adapter
+    if "gizmosql" not in engine_adapter.DIALECT_TO_ENGINE_ADAPTER:
+        engine_adapter.DIALECT_TO_ENGINE_ADAPTER["gizmosql"] = GizmoSQLEngineAdapter
+
+    # Register the connection config
+    from sqlmesh.core.config import connection as conn_module
+    if "gizmosql" not in conn_module.CONNECTION_CONFIG_TO_TYPE:
+        conn_module.CONNECTION_CONFIG_TO_TYPE["gizmosql"] = GizmoSQLConnectionConfig
+
+    if "gizmosql" not in conn_module.DIALECT_TO_TYPE:
+        conn_module.DIALECT_TO_TYPE["gizmosql"] = GizmoSQLConnectionConfig.DIALECT
+
+    _registered = True
+
+
+# Auto-register on import
+register()

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql/adapter.py

@@ -0,0 +1,289 @@
+"""
+GizmoSQL Engine Adapter for SQLMesh.
+
+GizmoSQL is a database server that uses DuckDB as its execution engine and
+exposes an Arrow Flight SQL interface for remote connections.
+"""
+from __future__ import annotations
+
+import contextlib
+import typing as t
+
+from sqlglot import exp
+from sqlmesh.core.engine_adapter.mixins import (
+    GetCurrentCatalogFromFunctionMixin,
+    LogicalMergeMixin,
+    PandasNativeFetchDFSupportMixin,
+    RowDiffMixin,
+)
+from sqlmesh.core.engine_adapter.shared import (
+    CatalogSupport,
+    CommentCreationTable,
+    CommentCreationView,
+    DataObject,
+    DataObjectType,
+    SourceQuery,
+    set_catalog,
+)
+
+if t.TYPE_CHECKING:
+    from sqlmesh.core._typing import SchemaName, TableName
+    from sqlmesh.core.engine_adapter._typing import DF
+
+
+@set_catalog(override_mapping={"_get_data_objects": CatalogSupport.REQUIRES_SET_CATALOG})
+class GizmoSQLEngineAdapter(
+    LogicalMergeMixin,
+    GetCurrentCatalogFromFunctionMixin,
+    PandasNativeFetchDFSupportMixin,
+    RowDiffMixin,
+):
+    """
+    GizmoSQL Engine Adapter.
+
+    GizmoSQL is a database server that uses DuckDB as its execution engine and
+    exposes an Arrow Flight SQL interface for remote connections. This adapter
+    uses ADBC (Arrow Database Connectivity) with the Flight SQL driver to
+    communicate with GizmoSQL servers.
+
+    Key characteristics:
+    - Uses DuckDB SQL dialect for query generation
+    - Connects remotely via Arrow Flight SQL (ADBC)
+    - Supports full catalog operations
+    - Returns data as Arrow tables for efficient transfer
+    """
+
+    DIALECT = "duckdb"
+    SUPPORTS_TRANSACTIONS = True
+    SCHEMA_DIFFER_KWARGS = {
+        "parameterized_type_defaults": {
+            exp.DataType.build("DECIMAL", dialect="duckdb").this: [(18, 3), (0,)],
+        },
+    }
+    COMMENT_CREATION_TABLE = CommentCreationTable.COMMENT_COMMAND_ONLY
+    COMMENT_CREATION_VIEW = CommentCreationView.COMMENT_COMMAND_ONLY
+    SUPPORTS_CREATE_DROP_CATALOG = True
+    SUPPORTED_DROP_CASCADE_OBJECT_KINDS = ["SCHEMA", "TABLE", "VIEW"]
+
+    @property
+    def catalog_support(self) -> CatalogSupport:
+        return CatalogSupport.FULL_SUPPORT
+
+    # DDL/DML keywords that need fetch to trigger GizmoSQL's lazy execution
+    _DDL_DML_KEYWORDS = frozenset({
+        "CREATE", "DROP", "ALTER", "TRUNCATE", "RENAME", "COMMENT", "USE", "SET",
+        "INSERT", "UPDATE", "DELETE", "MERGE", "COPY", "ATTACH", "DETACH",
+    })
+
+    def _execute(self, sql: str, track_rows_processed: bool = False, **kwargs: t.Any) -> None:
+        """
+        Execute a SQL statement.
+
+        GizmoSQL uses lazy execution - statements are not actually executed
+        until results are fetched. For DDL/DML statements, we call fetchall()
+        to ensure immediate execution. For SELECT queries, we let the caller
+        fetch the results.
+        """
+        self.cursor.execute(sql, **kwargs)
+
+        # For DDL/DML, fetch to trigger GizmoSQL's lazy execution
+        sql_upper = sql.strip().upper()
+        first_word = sql_upper.split()[0] if sql_upper else ""
+        if first_word in self._DDL_DML_KEYWORDS:
+            self.cursor.fetchall()
+
+    @contextlib.contextmanager
+    def transaction(
+        self,
+        condition: t.Optional[bool] = None,
+    ) -> t.Iterator[None]:
+        """
+        A transaction context manager using SQL statements.
+
+        GizmoSQL's ADBC connection doesn't support the standard begin/commit/rollback
+        methods, so we use explicit SQL statements (BEGIN TRANSACTION, COMMIT, ROLLBACK)
+        for transaction control.
+        """
+        if (
+            self._connection_pool.is_transaction_active
+            or (condition is not None and not condition)
+        ):
+            yield
+            return
+
+        self._connection_pool.begin()
+        self.cursor.execute("BEGIN TRANSACTION")
+        self.cursor.fetchall()
+        try:
+            yield
+        except Exception as e:
+            self.cursor.execute("ROLLBACK")
+            self.cursor.fetchall()
+            self._connection_pool.rollback()
+            raise e
+        else:
+            self.cursor.execute("COMMIT")
+            self.cursor.fetchall()
+            self._connection_pool.commit()
+
+    def set_current_catalog(self, catalog: str) -> None:
+        """Sets the catalog name of the current connection."""
+        self.execute(exp.Use(this=exp.to_identifier(catalog)))
+
+    def _create_catalog(self, catalog_name: exp.Identifier) -> None:
+        """Creates a new catalog (database) in GizmoSQL."""
+        self.execute(
+            exp.Create(this=exp.Table(this=catalog_name), kind="DATABASE", exists=True)
+        )
+
+    def _drop_catalog(self, catalog_name: exp.Identifier) -> None:
+        """Drops a catalog (database) from GizmoSQL."""
+        self.execute(
+            exp.Drop(
+                this=exp.Table(this=catalog_name), kind="DATABASE", cascade=True, exists=True
+            )
+        )
+
+    def _df_to_source_queries(
+        self,
+        df: DF,
+        target_columns_to_types: t.Dict[str, exp.DataType],
+        batch_size: int,
+        target_table: TableName,
+        source_columns: t.Optional[t.List[str]] = None,
+    ) -> t.List[SourceQuery]:
+        """
+        Convert a DataFrame to source queries for insertion.
+
+        Uses ADBC bulk ingestion (adbc_ingest) for efficient Arrow-native data transfer
+        to GizmoSQL, avoiding row-by-row insertion overhead.
+        """
+        import pyarrow as pa
+
+        temp_table = self._get_temp_table(target_table)
+
+        # Select only the source columns in the right order
+        source_columns_to_types = (
+            {col: target_columns_to_types[col] for col in source_columns}
+            if source_columns
+            else target_columns_to_types
+        )
+        ordered_df = df[list(source_columns_to_types.keys())]
+
+        # Convert DataFrame to PyArrow Table for bulk ingestion
+        arrow_table = pa.Table.from_pandas(ordered_df)
+
+        # Use ADBC bulk ingestion with temporary table
+        # Note: DuckDB temporary tables cannot have catalog/schema prefixes,
+        # so we only pass the table name when temporary=True
+        self.cursor.adbc_ingest(
+            table_name=temp_table.name,
+            data=arrow_table,
+            mode="create",
+            temporary=True,
+        )
+
+        # Reference temp table by name only (no schema prefix for temporary tables)
+        temp_table_name = exp.to_table(temp_table.name)
+
+        return [
+            SourceQuery(
+                query_factory=lambda: self._select_columns(target_columns_to_types).from_(
+                    temp_table_name
+                ),
+                cleanup_func=lambda: self.drop_table(temp_table_name),
+            )
+        ]
+
+    def _get_data_objects(
+        self, schema_name: SchemaName, object_names: t.Optional[t.Set[str]] = None
+    ) -> t.List[DataObject]:
+        """
+        Returns all the data objects that exist in the given schema and optionally catalog.
+        """
+        catalog = self.get_current_catalog()
+
+        if isinstance(schema_name, exp.Table):
+            # Ensures we don't generate identifier quotes
+            schema_name = ".".join(part.name for part in schema_name.parts)
+
+        query = (
+            exp.select(
+                exp.column("table_name").as_("name"),
+                exp.column("table_schema").as_("schema"),
+                exp.case(exp.column("table_type"))
+                .when(
+                    exp.Literal.string("BASE TABLE"),
+                    exp.Literal.string("table"),
+                )
+                .when(
+                    exp.Literal.string("VIEW"),
+                    exp.Literal.string("view"),
+                )
+                .when(
+                    exp.Literal.string("LOCAL TEMPORARY"),
+                    exp.Literal.string("table"),
+                )
+                .as_("type"),
+            )
+            .from_(exp.to_table("information_schema.tables"))
+            .where(
+                exp.column("table_catalog").eq(catalog), exp.column("table_schema").eq(schema_name)
+            )
+        )
+        if object_names:
+            query = query.where(exp.column("table_name").isin(*object_names))
+        df = self.fetchdf(query)
+        return [
+            DataObject(
+                catalog=catalog,  # type: ignore
+                schema=row.schema,  # type: ignore
+                name=row.name,  # type: ignore
+                type=DataObjectType.from_str(row.type),  # type: ignore
+            )
+            for row in df.itertuples()
+        ]
+
+    def _normalize_decimal_value(self, col: exp.Expression, precision: int) -> exp.Expression:
+        """
+        GizmoSQL (via DuckDB) truncates instead of rounding when casting to decimal.
+
+        other databases: select cast(3.14159 as decimal(38,3)) -> 3.142
+        duckdb: select cast(3.14159 as decimal(38,3)) -> 3.141
+
+        however, we can get the behaviour of other databases by casting to double first:
+        select cast(cast(3.14159 as double) as decimal(38, 3)) -> 3.142
+        """
+        return exp.cast(
+            exp.cast(col, "DOUBLE"),
+            f"DECIMAL(38, {precision})",
+        )
+
+    def _fetch_native_df(
+        self, query: t.Union[exp.Expression, str], quote_identifiers: bool = False
+    ) -> DF:
+        """
+        Fetches a Pandas DataFrame from a SQL query.
+
+        GizmoSQL returns Arrow tables which can be efficiently converted to Pandas.
+        """
+        import pandas as pd
+
+        sql = (
+            self._to_sql(query, quote=quote_identifiers)
+            if isinstance(query, exp.Expression)
+            else query
+        )
+
+        cursor = self.cursor
+        cursor.execute(sql)
+
+        # ADBC cursors support fetch_arrow_table() for efficient Arrow->Pandas conversion
+        if hasattr(cursor, "fetch_arrow_table"):
+            arrow_table = cursor.fetch_arrow_table()
+            return arrow_table.to_pandas()
+
+        # Fallback to standard fetchall + DataFrame construction
+        result = cursor.fetchall()
+        columns = [desc[0] for desc in cursor.description] if cursor.description else []
+        return pd.DataFrame(result, columns=columns)

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql/connection.py

@@ -0,0 +1,134 @@
+"""
+GizmoSQL Connection Configuration for SQLMesh.
+
+Provides the connection configuration class for GizmoSQL servers.
+"""
+from __future__ import annotations
+
+import typing as t
+
+from pydantic import Field
+from sqlmesh.core.config.common import concurrent_tasks_validator
+from sqlmesh.core.config.connection import ConnectionConfig
+from sqlmesh.core.engine_adapter import EngineAdapter
+from sqlmesh.utils.errors import ConfigError
+from sqlmesh.utils.pydantic import model_validator
+
+from sqlmesh_gizmosql.adapter import GizmoSQLEngineAdapter
+
+
+def _gizmosql_import_validator(cls: t.Any, data: t.Any) -> t.Any:
+    """Validate that ADBC Flight SQL driver is installed."""
+    check_import = (
+        data.pop("check_import", True) if isinstance(data, dict) else True
+    )
+    if not check_import:
+        return data
+    try:
+        import adbc_driver_flightsql  # noqa: F401
+    except ImportError:
+        raise ConfigError(
+            "Failed to import the 'adbc_driver_flightsql' library. "
+            "Please install it with: pip install sqlmesh-gizmosql[gizmosql] "
+            "or pip install adbc-driver-flightsql pyarrow"
+        )
+    return data
+
+
+class GizmoSQLConnectionConfig(ConnectionConfig):
+    """
+    GizmoSQL connection configuration.
+
+    GizmoSQL is a database server that uses DuckDB as its execution engine and
+    exposes an Arrow Flight SQL interface for remote connections. This configuration
+    uses ADBC (Arrow Database Connectivity) with the Flight SQL driver.
+
+    Args:
+        host: The hostname of the GizmoSQL server.
+        port: The port of the GizmoSQL server (default: 31337).
+        username: The username for authentication.
+        password: The password for authentication.
+        use_encryption: Whether to use TLS encryption (default: True).
+        disable_certificate_verification: Whether to skip TLS certificate verification.
+            Useful for self-signed certificates in development (default: False).
+        database: The default database/catalog to use.
+        concurrent_tasks: The maximum number of concurrent tasks.
+        register_comments: Whether to register model comments.
+        pre_ping: Whether to pre-ping the connection.
+    """
+
+    host: str = "localhost"
+    port: int = 31337
+    username: str
+    password: str
+    use_encryption: bool = True
+    disable_certificate_verification: bool = False
+    database: t.Optional[str] = None
+
+    concurrent_tasks: int = 4
+    register_comments: bool = True
+    pre_ping: bool = False
+
+    type_: t.Literal["gizmosql"] = Field(alias="type", default="gizmosql")
+    DIALECT: t.ClassVar[t.Literal["duckdb"]] = "duckdb"
+    DISPLAY_NAME: t.ClassVar[t.Literal["GizmoSQL"]] = "GizmoSQL"
+    DISPLAY_ORDER: t.ClassVar[t.Literal[17]] = 17
+
+    _engine_import_validator = model_validator(mode="before")(_gizmosql_import_validator)
+    _concurrent_tasks_validator = concurrent_tasks_validator
+
+    @property
+    def _connection_kwargs_keys(self) -> t.Set[str]:
+        # ADBC uses a different connection pattern, so we don't pass these directly
+        return set()
+
+    @property
+    def _engine_adapter(self) -> t.Type[EngineAdapter]:
+        return GizmoSQLEngineAdapter
+
+    @property
+    def _connection_factory(self) -> t.Callable:
+        """
+        Create a connection factory for GizmoSQL using ADBC Flight SQL driver.
+
+        The connection is established using the Arrow Flight SQL protocol over gRPC.
+        """
+        import re
+
+        from adbc_driver_flightsql import DatabaseOptions
+        from adbc_driver_flightsql import dbapi as flightsql
+
+        def connect() -> t.Any:
+            # Build the URI for the Flight SQL connection
+            protocol = "grpc+tls" if self.use_encryption else "grpc"
+            uri = f"{protocol}://{self.host}:{self.port}"
+
+            # ADBC database-level options (passed to the driver)
+            db_kwargs: t.Dict[str, str] = {
+                "username": self.username,
+                "password": self.password,
+            }
+
+            # Add TLS skip verify option using the proper DatabaseOptions enum
+            if self.use_encryption and self.disable_certificate_verification:
+                db_kwargs[DatabaseOptions.TLS_SKIP_VERIFY.value] = "true"
+
+            # Create the connection - uri is first positional arg, db_kwargs is for driver options
+            # Explicit autocommit=True since GizmoSQL doesn't support manual transaction commits
+            conn = flightsql.connect(uri, db_kwargs=db_kwargs, autocommit=True)
+
+            # Verify the backend is DuckDB - this adapter only supports the DuckDB backend
+            vendor_version = conn.adbc_get_info().get("vendor_version", "")
+            if not re.search(pattern=r"^duckdb ", string=vendor_version):
+                conn.close()
+                raise ConfigError(
+                    f"Unsupported GizmoSQL server backend: '{vendor_version}'. "
+                    "This adapter only supports the DuckDB backend for GizmoSQL."
+                )
+
+            return conn
+
+        return connect
+
+    def get_catalog(self) -> t.Optional[str]:
+        return self.database

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql.egg-info/PKG-INFO

@@ -0,0 +1,134 @@
+Metadata-Version: 2.4
+Name: sqlmesh-gizmosql
+Version: 0.1.0
+Summary: GizmoSQL engine adapter for SQLMesh
+Author: Philip
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/gizmodata/sqlmesh-gizmosql
+Project-URL: Repository, https://github.com/gizmodata/sqlmesh-gizmosql
+Project-URL: Documentation, https://github.com/gizmodata/sqlmesh-gizmosql#readme
+Keywords: sqlmesh,gizmosql,duckdb,arrow,flight-sql,data-engineering
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: sqlmesh>=0.100.0
+Requires-Dist: adbc-driver-flightsql>=1.10.0
+Requires-Dist: pyarrow>=22.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+
+# SQLMesh GizmoSQL Adapter
+
+A [SQLMesh](https://sqlmesh.com) engine adapter for [GizmoSQL](https://github.com/gizmodata/gizmosql) - a database server that uses DuckDB as its execution engine and exposes an Arrow Flight SQL interface for remote connections.
+
+## Installation
+
+```bash
+pip install sqlmesh-gizmosql
+```
+
+This will install `sqlmesh`, `adbc-driver-flightsql`, and `pyarrow` as dependencies.
+
+## Usage
+
+### 1. Import the adapter
+
+Simply import the package before using SQLMesh. The adapter registers itself automatically:
+
+```python
+import sqlmesh_gizmosql  # Registers GizmoSQL adapter
+from sqlmesh import Context
+
+context = Context(paths="path/to/project")
+```
+
+### 2. Configure your connection
+
+Add a GizmoSQL connection to your `config.yaml`:
+
+```yaml
+gateways:
+  my_gizmosql:
+    connection:
+      type: gizmosql
+      host: localhost
+      port: 31337
+      username: your_username
+      password: your_password
+      database: my_database  # optional, default catalog
+      use_encryption: true   # default: true (uses TLS)
+      disable_certificate_verification: false  # for self-signed certs
+```
+
+### 3. Use SQLMesh as normal
+
+```bash
+sqlmesh plan
+sqlmesh run
+```
+
+## Configuration Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `host` | str | `localhost` | GizmoSQL server hostname |
+| `port` | int | `31337` | GizmoSQL server port |
+| `username` | str | *required* | Authentication username |
+| `password` | str | *required* | Authentication password |
+| `database` | str | `None` | Default database/catalog |
+| `use_encryption` | bool | `True` | Use TLS encryption |
+| `disable_certificate_verification` | bool | `False` | Skip TLS cert verification |
+| `concurrent_tasks` | int | `4` | Max concurrent tasks |
+| `register_comments` | bool | `True` | Register model comments |
+| `pre_ping` | bool | `False` | Pre-ping connections |
+
+## Features
+
+- **Arrow Flight SQL**: Efficient data transfer using Arrow's columnar format
+- **Full Catalog Support**: Create, drop, and switch between databases
+- **Transaction Support**: Full transaction control via SQL statements
+- **ADBC Bulk Ingestion**: Fast data loading using Arrow-native bulk operations
+- **DuckDB Compatibility**: Uses DuckDB SQL dialect for query generation
+
+## Requirements
+
+- Python >= 3.10
+- SQLMesh >= 0.100.0
+- A running GizmoSQL server with DuckDB backend
+
+## Development
+
+```bash
+# Clone the repository
+git clone https://github.com/philip/sqlmesh-gizmosql.git
+cd sqlmesh-gizmosql
+
+# Install dev dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run linting
+ruff check .
+mypy sqlmesh_gizmosql
+```
+
+## License
+
+Apache 2.0

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+README.md
+pyproject.toml
+sqlmesh_gizmosql/__init__.py
+sqlmesh_gizmosql/adapter.py
+sqlmesh_gizmosql/connection.py
+sqlmesh_gizmosql.egg-info/PKG-INFO
+sqlmesh_gizmosql.egg-info/SOURCES.txt
+sqlmesh_gizmosql.egg-info/dependency_links.txt
+sqlmesh_gizmosql.egg-info/requires.txt
+sqlmesh_gizmosql.egg-info/top_level.txt
+tests/test_gizmosql.py

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql.egg-info/requires.txt

@@ -0,0 +1,9 @@
+sqlmesh>=0.100.0
+adbc-driver-flightsql>=1.10.0
+pyarrow>=22.0.0
+
+[dev]
+pytest>=7.0.0
+pytest-cov>=4.0.0
+mypy>=1.0.0
+ruff>=0.1.0

sqlmesh_gizmosql-0.1.0/sqlmesh_gizmosql.egg-info/top_level.txt

@@ -0,0 +1 @@
+sqlmesh_gizmosql

sqlmesh_gizmosql-0.1.0/tests/test_gizmosql.py

@@ -0,0 +1,235 @@
+"""Unit tests for GizmoSQL engine adapter and connection config.
+
+These tests don't require a running GizmoSQL server - they test
+configuration validation and adapter properties.
+"""
+
+from unittest.mock import MagicMock, patch
+
+import pytest
+from sqlmesh.utils.errors import ConfigError
+
+from sqlmesh_gizmosql.adapter import GizmoSQLEngineAdapter
+from sqlmesh_gizmosql.connection import GizmoSQLConnectionConfig
+
+
+class TestGizmoSQLConnectionConfig:
+    """Tests for GizmoSQLConnectionConfig."""
+
+    def test_default_values(self):
+        """Test default configuration values."""
+        config = GizmoSQLConnectionConfig(
+            username="user",
+            password="pass",
+        )
+
+        assert config.host == "localhost"
+        assert config.port == 31337
+        assert config.use_encryption is True
+        assert config.disable_certificate_verification is False
+        assert config.database is None
+        assert config.concurrent_tasks == 4
+
+    def test_custom_values(self):
+        """Test custom configuration values."""
+        config = GizmoSQLConnectionConfig(
+            host="gizmosql.example.com",
+            port=12345,
+            username="testuser",
+            password="testpass",
+            use_encryption=False,
+            disable_certificate_verification=True,
+            database="mydb",
+            concurrent_tasks=8,
+        )
+
+        assert config.host == "gizmosql.example.com"
+        assert config.port == 12345
+        assert config.username == "testuser"
+        assert config.password == "testpass"
+        assert config.use_encryption is False
+        assert config.disable_certificate_verification is True
+        assert config.database == "mydb"
+        assert config.concurrent_tasks == 8
+
+    def test_dialect_is_duckdb(self):
+        """Test that the dialect is set to duckdb."""
+        config = GizmoSQLConnectionConfig(
+            username="user",
+            password="pass",
+        )
+        assert config.DIALECT == "duckdb"
+
+    def test_type_is_gizmosql(self):
+        """Test that the type is set to gizmosql."""
+        config = GizmoSQLConnectionConfig(
+            username="user",
+            password="pass",
+        )
+        assert config.type_ == "gizmosql"
+
+    def test_get_catalog(self):
+        """Test get_catalog returns database."""
+        config = GizmoSQLConnectionConfig(
+            username="user",
+            password="pass",
+            database="mydb",
+        )
+        assert config.get_catalog() == "mydb"
+
+    def test_get_catalog_none(self):
+        """Test get_catalog returns None when database not set."""
+        config = GizmoSQLConnectionConfig(
+            username="user",
+            password="pass",
+        )
+        assert config.get_catalog() is None
+
+    def test_connection_factory_builds_correct_uri_with_tls(self):
+        """Test connection factory builds correct URI with TLS."""
+        mock_flightsql = MagicMock()
+        mock_conn = MagicMock()
+        mock_conn.adbc_get_info.return_value = {"vendor_version": "duckdb v1.0.0"}
+        mock_flightsql.connect.return_value = mock_conn
+
+        # Mock DatabaseOptions enum
+        mock_db_options = MagicMock()
+        mock_db_options.TLS_SKIP_VERIFY.value = "adbc.flight.sql.client_option.tls_skip_verify"
+
+        config = GizmoSQLConnectionConfig(
+            host="example.com",
+            port=31337,
+            username="user",
+            password="pass",
+            use_encryption=True,
+            disable_certificate_verification=False,
+        )
+
+        mock_module = MagicMock()
+        mock_module.dbapi = mock_flightsql
+        mock_module.DatabaseOptions = mock_db_options
+
+        with patch.dict("sys.modules", {"adbc_driver_flightsql": mock_module}):
+            factory = config._connection_factory
+            factory()
+
+        # Verify connect was called with correct URI (first positional arg)
+        call_args = mock_flightsql.connect.call_args
+        assert call_args[0][0] == "grpc+tls://example.com:31337"
+        # Verify db_kwargs contains credentials
+        db_kwargs = call_args[1]["db_kwargs"]
+        assert db_kwargs["username"] == "user"
+        assert db_kwargs["password"] == "pass"
+
+    def test_connection_factory_builds_correct_uri_without_tls(self):
+        """Test connection factory builds correct URI without TLS."""
+        mock_flightsql = MagicMock()
+        mock_conn = MagicMock()
+        mock_conn.adbc_get_info.return_value = {"vendor_version": "duckdb v1.0.0"}
+        mock_flightsql.connect.return_value = mock_conn
+
+        # Mock DatabaseOptions enum
+        mock_db_options = MagicMock()
+        mock_db_options.TLS_SKIP_VERIFY.value = "adbc.flight.sql.client_option.tls_skip_verify"
+
+        config = GizmoSQLConnectionConfig(
+            host="example.com",
+            port=31337,
+            username="user",
+            password="pass",
+            use_encryption=False,
+        )
+
+        mock_module = MagicMock()
+        mock_module.dbapi = mock_flightsql
+        mock_module.DatabaseOptions = mock_db_options
+
+        with patch.dict("sys.modules", {"adbc_driver_flightsql": mock_module}):
+            factory = config._connection_factory
+            factory()
+
+        call_args = mock_flightsql.connect.call_args
+        assert call_args[0][0] == "grpc://example.com:31337"
+
+    def test_connection_factory_rejects_non_duckdb_backend(self):
+        """Test connection factory raises error for non-DuckDB backend."""
+        mock_flightsql = MagicMock()
+        mock_conn = MagicMock()
+        mock_conn.adbc_get_info.return_value = {"vendor_version": "sqlite v3.40.0"}
+        mock_flightsql.connect.return_value = mock_conn
+
+        # Mock DatabaseOptions enum
+        mock_db_options = MagicMock()
+        mock_db_options.TLS_SKIP_VERIFY.value = "adbc.flight.sql.client_option.tls_skip_verify"
+
+        config = GizmoSQLConnectionConfig(
+            host="example.com",
+            port=31337,
+            username="user",
+            password="pass",
+        )
+
+        mock_module = MagicMock()
+        mock_module.dbapi = mock_flightsql
+        mock_module.DatabaseOptions = mock_db_options
+
+        with patch.dict("sys.modules", {"adbc_driver_flightsql": mock_module}):
+            factory = config._connection_factory
+            with pytest.raises(ConfigError, match="Unsupported GizmoSQL server backend"):
+                factory()
+
+        # Verify connection was closed after rejection
+        mock_conn.close.assert_called_once()
+
+
+class TestGizmoSQLEngineAdapter:
+    """Tests for GizmoSQLEngineAdapter properties."""
+
+    def test_dialect(self):
+        """Test that dialect is duckdb."""
+        assert GizmoSQLEngineAdapter.DIALECT == "duckdb"
+
+    def test_supports_transactions(self):
+        """Test transactions are supported via SQL statements."""
+        assert GizmoSQLEngineAdapter.SUPPORTS_TRANSACTIONS is True
+
+    def test_supports_create_drop_catalog(self):
+        """Test catalog operations are supported."""
+        assert GizmoSQLEngineAdapter.SUPPORTS_CREATE_DROP_CATALOG is True
+
+    def test_supported_drop_cascade_object_kinds(self):
+        """Test cascade drop is supported for schemas, tables, and views."""
+        assert "SCHEMA" in GizmoSQLEngineAdapter.SUPPORTED_DROP_CASCADE_OBJECT_KINDS
+        assert "TABLE" in GizmoSQLEngineAdapter.SUPPORTED_DROP_CASCADE_OBJECT_KINDS
+        assert "VIEW" in GizmoSQLEngineAdapter.SUPPORTED_DROP_CASCADE_OBJECT_KINDS
+
+
+class TestRegistration:
+    """Tests for automatic registration with SQLMesh."""
+
+    def test_adapter_registered(self):
+        """Test that adapter is registered in SQLMesh's dialect map."""
+        from sqlmesh.core import engine_adapter
+
+        import sqlmesh_gizmosql  # noqa: F401
+
+        assert "gizmosql" in engine_adapter.DIALECT_TO_ENGINE_ADAPTER
+        assert engine_adapter.DIALECT_TO_ENGINE_ADAPTER["gizmosql"] is GizmoSQLEngineAdapter
+
+    def test_connection_config_registered(self):
+        """Test that connection config is registered in SQLMesh."""
+        from sqlmesh.core.config import connection as conn_module
+
+        import sqlmesh_gizmosql  # noqa: F401
+
+        assert "gizmosql" in conn_module.CONNECTION_CONFIG_TO_TYPE
+        assert conn_module.CONNECTION_CONFIG_TO_TYPE["gizmosql"] is GizmoSQLConnectionConfig
+
+    def test_dialect_type_registered(self):
+        """Test that dialect type is registered."""
+        from sqlmesh.core.config import connection as conn_module
+
+        import sqlmesh_gizmosql  # noqa: F401
+
+        assert "gizmosql" in conn_module.DIALECT_TO_TYPE
+        assert conn_module.DIALECT_TO_TYPE["gizmosql"] == "duckdb"