boti 0.1.0__tar.gz

boti-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 boti contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

boti-0.1.0/PKG-INFO

@@ -0,0 +1,397 @@
+Metadata-Version: 2.4
+Name: boti
+Version: 0.1.0
+Summary: Core infrastructure for the Boti ecosystem
+Author: lvalverdeb
+License: MIT
+Project-URL: Homepage, https://github.com/lvalverdeb/boti
+Project-URL: Repository, https://github.com/lvalverdeb/boti
+Project-URL: Documentation, https://github.com/lvalverdeb/boti#readme
+Project-URL: Bug Tracker, https://github.com/lvalverdeb/boti/issues
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fsspec>=2026.3.0
+Requires-Dist: pyarrow>=23.0.1
+Requires-Dist: pydantic>=2.12.5
+Requires-Dist: pydantic-settings>=2.13.1
+Provides-Extra: data
+Requires-Dist: boti-data<0.2.0,>=0.1.0; extra == "data"
+Dynamic: license-file
+
+# boti
+
+`boti` stands for **Base Object Transformation Interface**.
+
+It is a Python library for building **reliable, reusable transformation-oriented software**: scripts, services, data pipelines, batch jobs, notebook helpers, and internal tooling that all need the same operational foundations.
+
+At its core, `boti` is about giving transformation code a consistent runtime model:
+
+- how resources are opened and closed
+- how file access is constrained and validated
+- how projects discover their root and runtime configuration
+- how logs are emitted in a predictable way
+
+The repository also contains the companion package **`boti-data`**, which extends that foundation with SQL, parquet, schema, gateway, and distributed data capabilities.
+
+## What problem `boti` solves
+
+A lot of data and automation code starts small and quickly becomes operationally messy:
+
+- ad hoc setup and teardown logic
+- duplicated path and file handling
+- environment loading spread across scripts and notebooks
+- inconsistent logging and diagnostics
+- brittle assumptions about where code is running from
+
+That usually leads to code that works in one notebook or one machine, but becomes fragile when reused in pipelines, packaged services, shared libraries, or scheduled jobs.
+
+`boti` gives those projects a small set of **opinionated runtime primitives** so the same code can move more cleanly between local development, automation, and production workflows.
+
+## Why `boti` is useful
+
+`boti` is useful when you want transformation code to behave like a real software component instead of a collection of one-off scripts.
+
+It helps by:
+
+- standardising resource lifecycle with `ManagedResource`
+- making constrained file access explicit with `SecureResource`
+- centralising project-root and environment discovery with `ProjectService`
+- giving the codebase a shared logging model with `Logger`
+
+This is especially valuable when multiple teams or notebooks interact with the same codebase, because it reduces hidden assumptions and makes behaviour more predictable.
+
+## What `boti-data` adds
+
+`boti-data` is the data layer for the Boti ecosystem.
+
+Where `boti` solves the runtime and application-structure problems, `boti-data` solves the **data access and data movement problems** that appear once teams need to work across databases, parquet files, schemas, and distributed workloads.
+
+It provides:
+
+- SQL database resources and session management
+- SQLAlchemy model reflection and model registries
+- connection catalogues for named data sources
+- gateway-style loading APIs
+- parquet resources and readers
+- schema normalisation, validation, and field mapping
+- filter expressions and join helpers
+- partitioned and distributed loading workflows
+
+In practice, it helps teams replace repetitive, hand-rolled access code with a consistent interface for loading, validating, shaping, and moving data.
+
+## Where `boti-data` can make a big difference
+
+`boti-data` is useful anywhere teams need to bridge operational systems and analytical workflows without rewriting the same infrastructure over and over.
+
+It can be especially impactful in domains such as:
+
+- **analytics engineering**: consistent loading from source systems into analysis-ready frames
+- **business intelligence**: reusable connection catalogues, filters, and schema handling across reports
+- **operations and supply chain**: joining transactional data from multiple systems with safer loading patterns
+- **finance and risk**: explicit schemas, reproducible transformations, and controlled access to structured data
+- **customer, product, and growth analytics**: repeatable extraction and normalisation across many upstream tables
+- **ML and feature pipelines**: partitioned loads, parquet workflows, and predictable resource management
+- **research and notebook-heavy teams**: moving from exploratory code to reusable library code without losing speed
+
+The value is largest when data work sits in the gap between raw infrastructure and business logic: not just querying tables, but building maintainable, reusable data interfaces.
+
+## Packages
+
+### Core package
+
+```bash
+pip install boti
+```
+
+Core imports:
+
+```python
+from boti import Logger, ManagedResource, ProjectService, SecureResource
+from boti.core import is_secure_path
+```
+
+You can also import from `boti.core` directly:
+
+```python
+from boti.core import Logger, ManagedResource, ProjectService, SecureResource
+```
+
+### Core + data package
+
+```bash
+pip install "boti[data]"
+```
+
+or:
+
+```bash
+pip install boti-data
+```
+
+Data imports live under the separate top-level package:
+
+```python
+from boti_data import DataGateway, DataHelper, SqlDatabaseConfig, SqlDatabaseResource
+```
+
+## Quick start
+
+### Managed resource
+
+```python
+from boti import ManagedResource
+
+
+class MyResource(ManagedResource):
+    def _cleanup(self) -> None:
+        print("cleaning up")
+
+
+with MyResource() as resource:
+    print(resource.closed)  # False
+```
+
+### Filesystem configuration
+
+`FilesystemConfig` provides a typed way to describe where a resource should read and write data. It uses `fsspec` underneath, so `boti` can work with the local filesystem, S3-compatible object storage, and any other backend supported by your installed `fsspec` drivers.
+
+#### Local files
+
+```python
+from boti.core.filesystem import FilesystemConfig, create_filesystem
+
+config = FilesystemConfig(
+    fs_type="file",
+    fs_path="/srv/boti/data",
+)
+
+fs = create_filesystem(config)
+with fs.open("/srv/boti/data/example.txt", "w") as handle:
+    handle.write("hello")
+```
+
+#### S3 server connections
+
+Use this pattern when connecting to AWS S3 or to an S3-compatible server such as MinIO, Ceph, or another internal object-storage endpoint.
+
+```python
+from boti.core.filesystem import FilesystemConfig, FilesystemAdapter
+
+config = FilesystemConfig(
+    fs_type="s3",
+    fs_path="analytics-bucket/raw/events",
+    fs_key="ACCESS_KEY",
+    fs_secret="SECRET_KEY",
+    fs_endpoint="https://minio.internal.example",
+    fs_region="eu-west-1",
+)
+
+adapter = FilesystemAdapter(config)
+fs = adapter.get_filesystem()
+
+with fs.open("analytics-bucket/raw/events/2026-04-15.json", "rb") as handle:
+    payload = handle.read()
+```
+
+`fs_endpoint` points at the S3 server, while `fs_path` identifies the bucket and prefix you want to work with.
+
+#### Other supported filesystems
+
+Any backend recognised by the installed `fsspec` stack can be used through `fs_type`. Common examples include:
+
+- `memory` for tests and ephemeral workflows
+- `gcs` for Google Cloud Storage
+- `az` or `abfs` for Azure storage
+- `ftp`, `sftp`, or `http` where the relevant driver is installed
+
+```python
+from boti.core.filesystem import FilesystemConfig
+
+memory_config = FilesystemConfig(fs_type="memory", fs_path="scratch")
+gcs_config = FilesystemConfig(fs_type="gcs", fs_path="my-bucket/datasets")
+azure_config = FilesystemConfig(fs_type="az", fs_path="container/path")
+```
+
+### Project service
+
+```python
+from boti import ProjectService
+
+project_root = ProjectService.detect_project_root()
+env_file = ProjectService.setup_environment(project_root)
+```
+
+### Secure file access
+
+`SecureResource` wraps file operations in a sandbox. By default it allows paths under the detected project root and the system temporary directory, and you can add extra allowlisted paths explicitly.
+
+```python
+from pathlib import Path
+
+from boti import SecureResource
+from boti.core.models import ResourceConfig
+
+config = ResourceConfig(project_root=Path.cwd())
+
+with SecureResource(config=config) as resource:
+    contents = resource.read_text_secure("README.md")
+```
+
+#### Allow an additional trusted directory
+
+```python
+from pathlib import Path
+
+from boti import SecureResource
+from boti.core.models import ResourceConfig
+
+config = ResourceConfig(
+    project_root=Path("/workspace/project"),
+    extra_allowed_paths=[Path("/srv/shared/reference-data")],
+)
+
+with SecureResource(config=config) as resource:
+    reference = resource.read_text_secure("/srv/shared/reference-data/lookup.csv")
+```
+
+#### Block unsafe paths
+
+```python
+from pathlib import Path
+
+from boti import SecureResource
+from boti.core.models import ResourceConfig
+
+config = ResourceConfig(project_root=Path("/workspace/project"))
+
+with SecureResource(config=config) as resource:
+    try:
+        resource.read_text_secure("/etc/passwd")
+    except PermissionError:
+        print("outside the configured sandbox roots")
+```
+
+### Logger
+
+`Logger` provides a thread-safe, non-blocking logging layer with secure file handling and sensitive-data redaction.
+
+#### Quick logger
+
+```python
+from pathlib import Path
+
+from boti import Logger
+
+logger = Logger.default_logger(
+    logger_name="daily_job",
+    log_file="daily_job",
+    base_dir=Path("/workspace/project"),
+)
+
+logger.info("starting extraction")
+logger.warning("retrying after transient error")
+```
+
+#### Explicit logger configuration
+
+```python
+from pathlib import Path
+
+from boti.core.logger import Logger
+from boti.core.models import LoggerConfig
+
+config = LoggerConfig(
+    log_dir=Path("/workspace/project/logs"),
+    logger_name="etl.pipeline",
+    log_file="etl_pipeline",
+    verbose=True,
+)
+
+logger = Logger(config)
+logger.set_level(Logger.INFO)
+logger.info("rows loaded=%s", 1200)
+```
+
+### Subclassing `ManagedResource`
+
+`ManagedResource` supports both synchronous and asynchronous cleanup patterns, so custom resources can expose the same lifecycle contract whether they wrap filesystems, clients, sockets, or other runtime state.
+
+#### Synchronous resource
+
+```python
+from boti import ManagedResource
+
+
+class FilesystemResource(ManagedResource):
+    def write_text(self, path: str, content: str) -> None:
+        fs = self.require_fs()
+        with fs.open(path, "w", encoding="utf-8") as handle:
+            handle.write(content)
+
+    def read_text(self, path: str) -> str:
+        fs = self.require_fs()
+        with fs.open(path, "r", encoding="utf-8") as handle:
+            return handle.read()
+
+    def _cleanup(self) -> None:
+        if self._owns_fs and self.fs is not None:
+            self.fs = None
+```
+
+```python
+import fsspec
+
+resource = FilesystemResource(fs_factory=lambda: fsspec.filesystem("memory"))
+
+with resource:
+    resource.write_text("memory://example.txt", "hello from fsspec")
+    print(resource.read_text("memory://example.txt"))
+```
+
+#### Asynchronous resource
+
+```python
+import asyncio
+
+from boti import ManagedResource
+
+
+class AsyncClientResource(ManagedResource):
+    def __init__(self, client) -> None:
+        super().__init__()
+        self.client = client
+
+    async def _acleanup(self) -> None:
+        await self.client.aclose()
+
+
+async def main(client) -> None:
+    async with AsyncClientResource(client) as resource:
+        await asyncio.sleep(0)
+```
+
+If a subclass only implements `_cleanup()`, `await resource.aclose()` will fall back to running the synchronous cleanup safely.
+
+## More package-specific docs
+
+- [`packages/boti/README.md`](packages/boti/README.md)
+- [`packages/boti-data/README.md`](packages/boti-data/README.md)
+- [`examples/`](examples/)
+- [`docs/`](docs/)
+
+## Development
+
+Run tests with the project interpreter:
+
+```bash
+PYTHONPATH=src python -m pytest -q
+```