boti-data 1.0.6__tar.gz → 1.2.0__tar.gz

{boti_data-1.0.6/src/boti_data.egg-info → boti_data-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boti-data
-Version: 1.0.6
+Version: 1.2.0
 Summary: Data infrastructure for the Boti ecosystem
 Author: Luis Valverde
 Maintainer-email: Luis Valverde <lvalverdeb@gmail.com>
@@ -17,14 +17,21 @@ Classifier: Programming Language :: Python :: 3 :: Only
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.13
 Description-Content-Type: text/markdown
-Requires-Dist: asyncmy<0.3.0,>=0.2.11
 Requires-Dist: boti<2.0.0,>=1.0.2
-Requires-Dist: boti-dask<2.0.0,>=1.0.0
-Requires-Dist: dask[dataframe,distributed]<2027.0.0,>=2026.3.0
-Requires-Dist: pandas<4.0.0,>=3.0.2
-Requires-Dist: polars<2.0.0,>=1.29.0
-Requires-Dist: pymysql<2.0.0,>=1.1.2
 Requires-Dist: sqlalchemy[asyncio]<3.0.0,>=2.0.49
+Provides-Extra: mysql
+Requires-Dist: asyncmy<0.3.0,>=0.2.11; extra == "mysql"
+Requires-Dist: pymysql<2.0.0,>=1.1.2; extra == "mysql"
+Provides-Extra: postgres
+Requires-Dist: asyncpg<0.32.0,>=0.30.0; extra == "postgres"
+Requires-Dist: psycopg[binary]<4.0.0,>=3.2.0; extra == "postgres"
+Provides-Extra: pgvector
+Requires-Dist: pgvector<0.6.0,>=0.3.0; extra == "pgvector"
+Provides-Extra: dataframes
+Requires-Dist: boti-dask<2.0.0,>=1.0.0; extra == "dataframes"
+Requires-Dist: dask[dataframe,distributed]<2027.0.0,>=2026.3.0; extra == "dataframes"
+Requires-Dist: pandas<4.0.0,>=3.0.2; extra == "dataframes"
+Requires-Dist: polars<2.0.0,>=1.29.0; extra == "dataframes"
 
 # boti-data
 
@@ -119,6 +126,27 @@ Or install through the core package extra:
 pip install "boti[data]"
 ```
 
+`boti-data` doesn't bundle a database driver by default — SQLAlchemy resolves the driver package from your connection URL's scheme (`mysql+pymysql://`, `postgresql+asyncpg://`, ...), so install the one matching your database:
+
+```bash
+pip install "boti-data[mysql]"      # asyncmy + pymysql
+pip install "boti-data[postgres]"   # asyncpg + psycopg[binary]
+```
+
+If a Postgres table has a [pgvector](https://github.com/pgvector/pgvector) `vector` column, install `boti-data[pgvector]` too — this registers pgvector's `Vector` type with SQLAlchemy's reflection so `SqlAlchemyModelBuilder`/`SqlRepository`/`DataGateway` map that column to a usable `list[float]` (with its dimension preserved) instead of an unreflectable `NullType`:
+
+```bash
+pip install "boti-data[pgvector]"
+```
+
+The base install is deliberately light: single-row transactional access (`SqlRepository`/`AsyncSqlRepository`, see below) works out of the box, with no dask/pandas/polars pulled in. The dataframe-shaped, bulk-oriented side of the package (`DataGateway`/`DataHelper`, `ParquetPipeline`, `HybridDataset`, the sink pipelines, ...) needs the `dataframes` extra:
+
+```bash
+pip install "boti-data[dataframes]"  # dask + pandas + polars + boti-dask
+```
+
+Every name importable from `boti_data` is loaded lazily on first access, so `import boti_data` alone never touches dask/pandas/polars — only accessing a dataframe-shaped name does.
+
 ## Imports
 
 `boti-data` uses the top-level Python package `boti_data`. Dask runtime/session/resilience helpers live in `boti_dask` and are not re-exported here.
@@ -146,9 +174,11 @@ from boti_data import (
     available_sinks,
     create_sink,
     register_sink,
+    AsyncSqlRepository,
     SqlAlchemyModelBuilder,
     SqlDatabaseConfig,
     SqlDatabaseResource,
+    SqlRepository,
 )
 ```
 
@@ -855,6 +885,61 @@ Use the following as a guide:
 
 ## Examples
 
+### Repository (lightest-weight, no dataframe dependencies)
+
+For single-row transactional access — get/insert/update/delete by primary key — without pulling in dask/pandas/polars at all. Rows come back as plain `dict[str, Any]`, never a DataFrame.
+
+`query_only` defaults to `True` on `SqlRepository`/`AsyncSqlRepository` themselves and always wins over whatever the passed-in `config.query_only` says — insert/update/delete raise until you opt in explicitly with `query_only=False`. This is deliberate: a `config` object might be shared with or reused from a context where writes are already enabled, and a repository shouldn't silently inherit write access nobody asked it for.
+
+```python
+from boti_data import SqlDatabaseConfig, SqlRepository
+
+config = SqlDatabaseConfig(connection_url="sqlite:///example.db")
+
+with SqlRepository(config, "users", query_only=False) as users:
+    row = users.insert({"id": 1, "name": "Ada"})
+    row = users.get(1)              # -> dict | None
+    row = users.update(1, {"name": "Grace"})
+    deleted = users.delete(1)       # -> bool
+```
+
+Omit `query_only=False` (or read-only access is all you need) and writes raise instead of silently succeeding:
+
+```python
+with SqlRepository(config, "users") as users:
+    users.get(1)                    # fine
+    users.insert({"id": 2, "name": "Bob"})  # raises SQLAlchemyError
+```
+
+An async counterpart, `AsyncSqlRepository`, mirrors the same four methods as coroutines and is used as an async context manager:
+
+```python
+from boti_data import AsyncSqlRepository
+
+async with AsyncSqlRepository(config, "users", query_only=False) as users:
+    row = await users.get(1)
+```
+
+### Nearest-neighbour search (pgvector)
+
+`boti_data.db.nearest_neighbors`/`vector_distance` build an `ORDER BY <distance> LIMIT k` statement against a pgvector column, with the query vector bound as a driver parameter — never string-interpolated into the SQL, regardless of how large the vector is. Requires `boti-data[pgvector]` (see [Installation](#installation)) and a table reflected via `SqlAlchemyModelBuilder`/`SqlRepository`/`DataGateway`.
+
+```python
+from boti_data import DataGateway, SqlDatabaseConfig
+from boti_data.db import SqlAlchemyModelBuilder, nearest_neighbors
+
+config = SqlDatabaseConfig(connection_url="postgresql+psycopg://user:pass@host/mydb", query_only=True)
+gateway = DataGateway(config)
+
+model = SqlAlchemyModelBuilder(gateway.resource.engine, "embeddings").build_model()
+query_vector = [...]  # e.g. a 512-float face embedding
+
+stmt = nearest_neighbors(model, model.vector, query_vector, k=10, metric="cosine")
+matches = gateway.load(statement=stmt, model=model, as_pandas=True)
+```
+
+Supported `metric` values map directly to pgvector's operators: `"cosine"` (`<=>`), `"l2"` (`<->`), `"inner_product"` (`<#>`), `"l1"` (`<+>`). Add ordinary filtering with `.where(...)` on the statement before passing it to `gateway.load(...)`.
+
 ### SQL resource (low-level)
 
 ```python

boti_data-1.0.6/PKG-INFO → boti_data-1.2.0/README.md

@@ -1,31 +1,3 @@
-Metadata-Version: 2.4
-Name: boti-data
-Version: 1.0.6
-Summary: Data infrastructure for the Boti ecosystem
-Author: Luis Valverde
-Maintainer-email: Luis Valverde <lvalverdeb@gmail.com>
-License-Expression: MIT
-Project-URL: Homepage, https://github.com/lvalverdeb/boti-data
-Project-URL: Repository, https://github.com/lvalverdeb/boti-data
-Project-URL: Documentation, https://github.com/lvalverdeb/boti-data#readme
-Project-URL: Issues, https://github.com/lvalverdeb/boti-data/issues
-Classifier: Development Status :: 3 - Alpha
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.13
-Classifier: Programming Language :: Python :: 3 :: Only
-Classifier: Topic :: Software Development :: Libraries :: Python Modules
-Requires-Python: >=3.13
-Description-Content-Type: text/markdown
-Requires-Dist: asyncmy<0.3.0,>=0.2.11
-Requires-Dist: boti<2.0.0,>=1.0.2
-Requires-Dist: boti-dask<2.0.0,>=1.0.0
-Requires-Dist: dask[dataframe,distributed]<2027.0.0,>=2026.3.0
-Requires-Dist: pandas<4.0.0,>=3.0.2
-Requires-Dist: polars<2.0.0,>=1.29.0
-Requires-Dist: pymysql<2.0.0,>=1.1.2
-Requires-Dist: sqlalchemy[asyncio]<3.0.0,>=2.0.49
-
 # boti-data
 
 `boti-data` is the **data access and data transformation layer** of the Boti ecosystem.
@@ -119,6 +91,27 @@ Or install through the core package extra:
 pip install "boti[data]"
 ```
 
+`boti-data` doesn't bundle a database driver by default — SQLAlchemy resolves the driver package from your connection URL's scheme (`mysql+pymysql://`, `postgresql+asyncpg://`, ...), so install the one matching your database:
+
+```bash
+pip install "boti-data[mysql]"      # asyncmy + pymysql
+pip install "boti-data[postgres]"   # asyncpg + psycopg[binary]
+```
+
+If a Postgres table has a [pgvector](https://github.com/pgvector/pgvector) `vector` column, install `boti-data[pgvector]` too — this registers pgvector's `Vector` type with SQLAlchemy's reflection so `SqlAlchemyModelBuilder`/`SqlRepository`/`DataGateway` map that column to a usable `list[float]` (with its dimension preserved) instead of an unreflectable `NullType`:
+
+```bash
+pip install "boti-data[pgvector]"
+```
+
+The base install is deliberately light: single-row transactional access (`SqlRepository`/`AsyncSqlRepository`, see below) works out of the box, with no dask/pandas/polars pulled in. The dataframe-shaped, bulk-oriented side of the package (`DataGateway`/`DataHelper`, `ParquetPipeline`, `HybridDataset`, the sink pipelines, ...) needs the `dataframes` extra:
+
+```bash
+pip install "boti-data[dataframes]"  # dask + pandas + polars + boti-dask
+```
+
+Every name importable from `boti_data` is loaded lazily on first access, so `import boti_data` alone never touches dask/pandas/polars — only accessing a dataframe-shaped name does.
+
 ## Imports
 
 `boti-data` uses the top-level Python package `boti_data`. Dask runtime/session/resilience helpers live in `boti_dask` and are not re-exported here.
@@ -146,9 +139,11 @@ from boti_data import (
     available_sinks,
     create_sink,
     register_sink,
+    AsyncSqlRepository,
     SqlAlchemyModelBuilder,
     SqlDatabaseConfig,
     SqlDatabaseResource,
+    SqlRepository,
 )
 ```
 
@@ -855,6 +850,61 @@ Use the following as a guide:
 
 ## Examples
 
+### Repository (lightest-weight, no dataframe dependencies)
+
+For single-row transactional access — get/insert/update/delete by primary key — without pulling in dask/pandas/polars at all. Rows come back as plain `dict[str, Any]`, never a DataFrame.
+
+`query_only` defaults to `True` on `SqlRepository`/`AsyncSqlRepository` themselves and always wins over whatever the passed-in `config.query_only` says — insert/update/delete raise until you opt in explicitly with `query_only=False`. This is deliberate: a `config` object might be shared with or reused from a context where writes are already enabled, and a repository shouldn't silently inherit write access nobody asked it for.
+
+```python
+from boti_data import SqlDatabaseConfig, SqlRepository
+
+config = SqlDatabaseConfig(connection_url="sqlite:///example.db")
+
+with SqlRepository(config, "users", query_only=False) as users:
+    row = users.insert({"id": 1, "name": "Ada"})
+    row = users.get(1)              # -> dict | None
+    row = users.update(1, {"name": "Grace"})
+    deleted = users.delete(1)       # -> bool
+```
+
+Omit `query_only=False` (or read-only access is all you need) and writes raise instead of silently succeeding:
+
+```python
+with SqlRepository(config, "users") as users:
+    users.get(1)                    # fine
+    users.insert({"id": 2, "name": "Bob"})  # raises SQLAlchemyError
+```
+
+An async counterpart, `AsyncSqlRepository`, mirrors the same four methods as coroutines and is used as an async context manager:
+
+```python
+from boti_data import AsyncSqlRepository
+
+async with AsyncSqlRepository(config, "users", query_only=False) as users:
+    row = await users.get(1)
+```
+
+### Nearest-neighbour search (pgvector)
+
+`boti_data.db.nearest_neighbors`/`vector_distance` build an `ORDER BY <distance> LIMIT k` statement against a pgvector column, with the query vector bound as a driver parameter — never string-interpolated into the SQL, regardless of how large the vector is. Requires `boti-data[pgvector]` (see [Installation](#installation)) and a table reflected via `SqlAlchemyModelBuilder`/`SqlRepository`/`DataGateway`.
+
+```python
+from boti_data import DataGateway, SqlDatabaseConfig
+from boti_data.db import SqlAlchemyModelBuilder, nearest_neighbors
+
+config = SqlDatabaseConfig(connection_url="postgresql+psycopg://user:pass@host/mydb", query_only=True)
+gateway = DataGateway(config)
+
+model = SqlAlchemyModelBuilder(gateway.resource.engine, "embeddings").build_model()
+query_vector = [...]  # e.g. a 512-float face embedding
+
+stmt = nearest_neighbors(model, model.vector, query_vector, k=10, metric="cosine")
+matches = gateway.load(statement=stmt, model=model, as_pandas=True)
+```
+
+Supported `metric` values map directly to pgvector's operators: `"cosine"` (`<=>`), `"l2"` (`<->`), `"inner_product"` (`<#>`), `"l1"` (`<+>`). Add ordinary filtering with `.where(...)` on the statement before passing it to `gateway.load(...)`.
+
 ### SQL resource (low-level)
 
 ```python

boti_data-1.2.0/pyproject.toml

@@ -0,0 +1,141 @@
+[project]
+name = "boti-data"
+version = "1.2.0"
+description = "Data infrastructure for the Boti ecosystem"
+readme = "README.md"
+requires-python = ">=3.13"
+license = "MIT"
+authors = [
+    {name = "Luis Valverde"}
+]
+maintainers = [
+    {name = "Luis Valverde", email = "lvalverdeb@gmail.com"}
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3 :: Only",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "boti>=1.0.2,<2.0.0",
+    "sqlalchemy[asyncio]>=2.0.49,<3.0.0",
+]
+
+# Neither driver is imported directly by boti_data's own code — SQLAlchemy
+# lazily imports the DBAPI module named by the connection URL's scheme
+# (e.g. "mysql+pymysql://", "postgresql+asyncpg://") when the engine/dialect
+# is actually constructed. So these are purely "make sure pip installed the
+# right package for the DSN you're using" extras, not code-level optionality.
+[project.optional-dependencies]
+mysql = [
+    "asyncmy>=0.2.11,<0.3.0",
+    "pymysql>=1.1.2,<2.0.0",
+]
+postgres = [
+    "asyncpg>=0.30.0,<0.32.0",
+    "psycopg[binary]>=3.2.0,<4.0.0",
+]
+# Merely importable, not called directly: importing pgvector.sqlalchemy has the
+# side effect of registering the 'vector' type name into SQLAlchemy's Postgres
+# dialect (sqlalchemy.dialects.postgresql.base.ischema_names), which is what
+# boti_data.db.sql_model_registry's reflection needs to recognize pgvector
+# Vector columns instead of falling back to an unusable NullType.
+pgvector = [
+    "pgvector>=0.3.0,<0.6.0",
+]
+# The dataframe-shaped, bulk-oriented side of the package (DataGateway/
+# DataHelper, ParquetPipeline, HybridDataset, the sink pipelines, etc.) — not
+# needed by the lightweight SqlRepository/AsyncSqlRepository single-row API,
+# which only needs the base "boti-data" install.
+dataframes = [
+    "boti-dask>=1.0.0,<2.0.0",
+    "dask[dataframe,distributed]>=2026.3.0,<2027.0.0",
+    "pandas>=3.0.2,<4.0.0",
+    "polars>=1.29.0,<2.0.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/lvalverdeb/boti-data"
+Repository = "https://github.com/lvalverdeb/boti-data"
+Documentation = "https://github.com/lvalverdeb/boti-data#readme"
+Issues = "https://github.com/lvalverdeb/boti-data/issues"
+
+[build-system]
+requires = ["setuptools>=80", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[dependency-groups]
+dev = [
+    # boti-data itself no longer requires these (see [project.optional-dependencies]
+    # mysql), but tests/security/test_regressions_engine.py and
+    # tests/data/test_sql_manager_registry.py construct real mysql+pymysql/
+    # mysql+asyncmy engines and expect them to succeed, not raise a
+    # missing-driver error — so the dev/test environment still needs both.
+    "asyncmy>=0.2.11,<0.3.0",
+    "pymysql>=1.1.2,<2.0.0",
+    # Real async-SQLite driver used only by tests/data/test_sql_repository.py
+    # for genuine end-to-end AsyncSqlRepository round-trips — not a runtime
+    # dependency of boti_data itself, and not part of [mysql]/[postgres]/
+    # [dataframes] since SQLite isn't one of boti-data's supported backends.
+    "aiosqlite>=0.21.0,<0.23.0",
+    # boti-data itself no longer requires these (see [project.optional-dependencies]
+    # dataframes), but the bulk of the test suite exercises DataGateway/
+    # DataHelper/ParquetPipeline/HybridDataset and friends, which do — so the
+    # dev/test environment still needs the full dataframe stack installed.
+    "boti-dask>=1.0.0,<2.0.0",
+    "dask[dataframe,distributed]>=2026.3.0,<2027.0.0",
+    "pandas>=3.0.2,<4.0.0",
+    "polars>=1.29.0,<2.0.0",
+    # Not part of [project.optional-dependencies] pgvector's runtime need — it's
+    # here so tests/data/test_sql_model_registry_pgvector.py can verify the
+    # registration side effect actually happens when the package is installed.
+    "pgvector>=0.3.0,<0.6.0",
+    "mypy>=1.14.0",
+    "pytest>=9.0.3",
+    "pytest-asyncio>=1.3.0",
+    "pytest-benchmark>=5.1.0",
+    "pytest-cov>=6.0.0",
+    "ruff>=0.11.0",
+]
+
+[tool.ruff]
+line-length = 100
+target-version = "py313"
+
+[tool.ruff.lint]
+select = ["E", "F", "W", "I", "UP"]
+ignore = ["E501"]
+
+[tool.ruff.lint.per-file-ignores]
+"tests/**" = ["E402", "F841"]
+"src/**" = ["UP007", "E701"]
+# __init__.py has a deliberate non-alphabetical import order to break a circular import cycle
+"src/boti_data/__init__.py" = ["I001"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+testpaths = ["tests"]
+markers = [
+    "perf: performance benchmark (not run in default test suite)",
+    "security_regression: focused regression coverage for security fixes and audit findings",
+]
+filterwarnings = [
+    'ignore:Distributed SQL task payloads will carry the raw DSN credential.*worker_connection_env_var.*:UserWarning',
+    'ignore:Closing managed Dask session with 1 live persisted collection\(s\).*Missing dependency\.:RuntimeWarning',
+]
+
+[tool.setuptools]
+include-package-data = true
+
+[tool.setuptools.package-data]
+boti_data = ["py.typed"]
+
+[tool.setuptools.package-dir]
+"" = "src"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["boti_data*"]

boti_data-1.2.0/src/boti_data/__init__.py

@@ -0,0 +1,177 @@
+"""
+Data modules and interfaces for the Boti pipeline context.
+"""
+
+from __future__ import annotations
+
+import importlib
+from typing import Any
+
+__all__ = [
+    "And",
+    "AsyncFrameEnricher",
+    "AsyncSqlDatabaseResource",
+    "AsyncSqlRepository",
+    "AttachmentSpec",
+    "BaseDataCube",
+    "BaseArtifact",
+    "BuilderConfig",
+    "ConnectionCatalog",
+    "CsvSink",
+    "CsvSinkConfig",
+    "DataFrameOptions",
+    "DataFrameParams",
+    "DataGateway",
+    "DataHelper",
+    "DatacubeConfig",
+    "DatacubeContract",
+    "DefaultBase",
+    "EngineRegistry",
+    "Expr",
+    "FieldMap",
+    "FileWatermarkStore",
+    "FsspecWatermarkStore",
+    "FilterHandler",
+    "FrameEnricher",
+    "HybridDataset",
+    "IncrementalResult",
+    "JsonlSink",
+    "JsonlSinkConfig",
+    "Not",
+    "Or",
+    "ParquetDataConfig",
+    "ParquetLoadRequest",
+    "ParquetDataResource",
+    "ParquetMaterializationResult",
+    "ParquetPipeline",
+    "ParquetReader",
+    "ParquetSink",
+    "RegistryConfig",
+    "SchemaValidationError",
+    "SinkPipeline",
+    "SinkRegistry",
+    "SinkWriteResult",
+    "SqlLoadRequest",
+    "SqlAlchemyModelBuilder",
+    "SqlDatabaseConfig",
+    "SqlDatabaseResource",
+    "SqlPartitionPlan",
+    "SqlPartitionSpec",
+    "SqlPartitionedLoadRequest",
+    "SqlPartitionedLoader",
+    "SqlModelRegistry",
+    "SqlRepository",
+    "TrueExpr",
+    "VectorMetric",
+    "WatermarkStore",
+    "advance_watermark",
+    "align_frames_for_join",
+    "apply_schema_map",
+    "available_sinks",
+    "build_incremental_filters",
+    "create_sink",
+    "ensure_greenlet_available",
+    "get_global_registry",
+    "indexed_left_join",
+    "infer_schema_map",
+    "left_join_frames",
+    "nearest_neighbors",
+    "normalize_dtype_alias",
+    "normalize_schema_map",
+    "register_sink",
+    "validate_schema",
+    "vector_distance",
+]
+
+# Every public name is loaded lazily (PEP 562) so that importing boti_data (or
+# any of its dependency-clean pieces, e.g. SqlRepository) never pulls in the
+# dask/pandas/polars/pyarrow chain used by the dataframe-shaped, bulk-oriented
+# side of the package (DataGateway/DataHelper, ParquetPipeline, etc.) unless a
+# consumer actually touches one of those names.
+_LAZY = {
+    # db.* — dependency-clean.
+    "AsyncSqlDatabaseResource": ("boti_data.db.sql_manager", "AsyncSqlDatabaseResource"),
+    "EngineRegistry": ("boti_data.db.sql_manager", "EngineRegistry"),
+    "SqlDatabaseConfig": ("boti_data.db.sql_manager", "SqlDatabaseConfig"),
+    "SqlDatabaseResource": ("boti_data.db.sql_manager", "SqlDatabaseResource"),
+    "BuilderConfig": ("boti_data.db.sql_model_builder", "BuilderConfig"),
+    "SqlAlchemyModelBuilder": ("boti_data.db.sql_model_builder", "SqlAlchemyModelBuilder"),
+    "DefaultBase": ("boti_data.db.sql_model_registry", "DefaultBase"),
+    "RegistryConfig": ("boti_data.db.sql_model_registry", "RegistryConfig"),
+    "SqlModelRegistry": ("boti_data.db.sql_model_registry", "SqlModelRegistry"),
+    "get_global_registry": ("boti_data.db.sql_model_registry", "get_global_registry"),
+    "ensure_greenlet_available": ("boti_data.db.sqlalchemy_async", "ensure_greenlet_available"),
+    "SqlPartitionPlan": ("boti_data.db.partitioned_types", "SqlPartitionPlan"),
+    "SqlPartitionSpec": ("boti_data.db.partitioned_types", "SqlPartitionSpec"),
+    "SqlRepository": ("boti_data.db.sql_repository", "SqlRepository"),
+    "AsyncSqlRepository": ("boti_data.db.sql_repository", "AsyncSqlRepository"),
+    "nearest_neighbors": ("boti_data.db.vector_search", "nearest_neighbors"),
+    "vector_distance": ("boti_data.db.vector_search", "vector_distance"),
+    "VectorMetric": ("boti_data.db.vector_search", "VectorMetric"),
+    "FieldMap": ("boti_data.field_map", "FieldMap"),
+    # dask+pandas — the one heavy import inside the otherwise-clean db package.
+    "SqlPartitionedLoader": ("boti_data.db.partitioned_loader", "SqlPartitionedLoader"),
+    "SqlPartitionedLoadRequest": ("boti_data.db.partitioned_loader", "SqlPartitionedLoadRequest"),
+    # dask/pandas/polars/pyarrow-touching — the dataframe-shaped, bulk-oriented side.
+    "ConnectionCatalog": ("boti_data.connection_catalog", "ConnectionCatalog"),
+    "ParquetDataConfig": ("boti_data.parquet", "ParquetDataConfig"),
+    "ParquetDataResource": ("boti_data.parquet", "ParquetDataResource"),
+    "ParquetReader": ("boti_data.parquet", "ParquetReader"),
+    "FilterHandler": ("boti_data.filters", "FilterHandler"),
+    "Expr": ("boti_data.filters", "Expr"),
+    "TrueExpr": ("boti_data.filters", "TrueExpr"),
+    "And": ("boti_data.filters", "And"),
+    "Or": ("boti_data.filters", "Or"),
+    "Not": ("boti_data.filters", "Not"),
+    "DataGateway": ("boti_data.gateway", "DataGateway"),
+    "ParquetLoadRequest": ("boti_data.gateway", "ParquetLoadRequest"),
+    "SqlLoadRequest": ("boti_data.gateway", "SqlLoadRequest"),
+    "DataFrameOptions": ("boti_data.gateway", "DataFrameOptions"),
+    "DataFrameParams": ("boti_data.gateway", "DataFrameParams"),
+    "DataHelper": ("boti_data.helper", "DataHelper"),
+    "indexed_left_join": ("boti_data.joins", "indexed_left_join"),
+    "left_join_frames": ("boti_data.joins", "left_join_frames"),
+    "SchemaValidationError": ("boti_data.schema", "SchemaValidationError"),
+    "align_frames_for_join": ("boti_data.schema", "align_frames_for_join"),
+    "apply_schema_map": ("boti_data.schema", "apply_schema_map"),
+    "infer_schema_map": ("boti_data.schema", "infer_schema_map"),
+    "normalize_dtype_alias": ("boti_data.schema", "normalize_dtype_alias"),
+    "normalize_schema_map": ("boti_data.schema", "normalize_schema_map"),
+    "validate_schema": ("boti_data.schema", "validate_schema"),
+    "BaseDataCube": ("boti_data.datacube", "BaseDataCube"),
+    "BaseArtifact": ("boti_data.datacube", "BaseArtifact"),
+    "DatacubeConfig": ("boti_data.datacube", "DatacubeConfig"),
+    "DatacubeContract": ("boti_data.datacube", "DatacubeContract"),
+    "HybridDataset": ("boti_data.dataset", "HybridDataset"),
+    "AsyncFrameEnricher": ("boti_data.enrichment", "AsyncFrameEnricher"),
+    "AttachmentSpec": ("boti_data.enrichment", "AttachmentSpec"),
+    "FrameEnricher": ("boti_data.enrichment", "FrameEnricher"),
+    "CsvSink": ("boti_data.pipelines", "CsvSink"),
+    "CsvSinkConfig": ("boti_data.pipelines", "CsvSinkConfig"),
+    "JsonlSink": ("boti_data.pipelines", "JsonlSink"),
+    "JsonlSinkConfig": ("boti_data.pipelines", "JsonlSinkConfig"),
+    "ParquetMaterializationResult": ("boti_data.pipelines", "ParquetMaterializationResult"),
+    "ParquetPipeline": ("boti_data.pipelines", "ParquetPipeline"),
+    "ParquetSink": ("boti_data.pipelines", "ParquetSink"),
+    "SinkPipeline": ("boti_data.pipelines", "SinkPipeline"),
+    "SinkRegistry": ("boti_data.pipelines", "SinkRegistry"),
+    "SinkWriteResult": ("boti_data.pipelines", "SinkWriteResult"),
+    "available_sinks": ("boti_data.pipelines", "available_sinks"),
+    "create_sink": ("boti_data.pipelines", "create_sink"),
+    "register_sink": ("boti_data.pipelines", "register_sink"),
+    "FileWatermarkStore": ("boti_data.watermark", "FileWatermarkStore"),
+    "FsspecWatermarkStore": ("boti_data.watermark", "FsspecWatermarkStore"),
+    "IncrementalResult": ("boti_data.watermark", "IncrementalResult"),
+    "WatermarkStore": ("boti_data.watermark", "WatermarkStore"),
+    "advance_watermark": ("boti_data.watermark", "advance_watermark"),
+    "build_incremental_filters": ("boti_data.watermark", "build_incremental_filters"),
+}
+
+
+def __getattr__(name: str) -> Any:
+    if name in _LAZY:
+        module_name, attr = _LAZY[name]
+        value = getattr(importlib.import_module(module_name), attr)
+        globals()[name] = value
+        return value
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

{boti_data-1.0.6 → boti_data-1.2.0}/src/boti_data/arrow_schema.py

@@ -7,6 +7,7 @@ Provides a canonical ``ArrowSchema`` class that wraps ``pa.Schema`` with:
 - Dict-compatible constructor for migration from existing schema maps
 - DataFrame ↔ Table conversion helpers
 """
+
 from __future__ import annotations
 
 from collections.abc import Mapping
@@ -66,10 +67,7 @@ class ArrowSchema:
         from boti_data.schema import normalize_schema_map
 
         normalized = normalize_schema_map(dtype_map)
-        fields = [
-            (col, pandas_dtype_to_arrow(dtype))
-            for col, dtype in normalized.items()
-        ]
+        fields = [(col, pandas_dtype_to_arrow(dtype)) for col, dtype in normalized.items()]
         return cls(pa.schema(fields))
 
     @classmethod
@@ -135,6 +133,7 @@ class ArrowSchema:
         pandas operations. Handles type mismatches via safe/unsafe casting.
         """
         from boti_data.db.arrow_schema_mapper import coerce_arrow_table
+
         return coerce_arrow_table(table, self._schema)
 
     def to_arrow_table(self, df: Any) -> pa.Table:
@@ -268,9 +267,11 @@ class ArrowSchema:
 # Helper functions
 # ---------------------------------------------------------------------------
 
+
 def _pandas_dtype_to_arrow(dtype_str: str) -> pa.DataType:
     """Convert a pandas dtype string to PyArrow type."""
     from boti_data.db.arrow_schema_mapper import pandas_dtype_to_arrow
+
     return pandas_dtype_to_arrow(dtype_str)
 
 
@@ -288,6 +289,7 @@ def _pandas_meta_to_arrow_schema(df: Any) -> pa.Schema:
 # Convenience functions (drop-in replacements for schema.py functions)
 # ---------------------------------------------------------------------------
 
+
 def apply_arrow_schema_map(
     dataframe: Any,
     schema_map: Mapping[str, str],
@@ -304,10 +306,12 @@ def apply_arrow_schema_map(
     # Validate required columns
     if require_columns:
         missing = arrow_schema.validate_columns(
-            dataframe, require_all=True,
+            dataframe,
+            require_all=True,
         )
         if missing:
             from boti_data.schema import SchemaValidationError
+
             raise SchemaValidationError(f"Missing required column(s): {missing}.")
 
     return arrow_schema.coerce_dataframe(dataframe)
@@ -325,8 +329,10 @@ def validate_arrow_schema(
     """
     arrow_schema = ArrowSchema.from_dict(expected_schema_map)
     errors = arrow_schema.validate_dataframe(
-        dataframe, require_columns=require_columns,
+        dataframe,
+        require_columns=require_columns,
     )
     if errors:
         from boti_data.schema import SchemaValidationError
+
         raise SchemaValidationError("Schema validation failed:\n" + "\n".join(errors))