hybrid-frame 0.3.0__tar.gz

hybrid_frame-0.3.0/MANIFEST.in

@@ -0,0 +1,2 @@
+include hybrid_frame.pyi
+include README.md

hybrid_frame-0.3.0/PKG-INFO

@@ -0,0 +1,33 @@
+Metadata-Version: 2.4
+Name: hybrid-frame
+Version: 0.3.0
+Summary: High-performance DataFrame unifying DuckDB (out-of-core) and Pandas (ML feature engineering)
+Author: mukesh
+License: MIT
+Project-URL: Homepage, https://github.com/shell-bay/hybrid-frame
+Project-URL: Repository, https://github.com/shell-bay/hybrid-frame
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Database :: Database Engines/Servers
+Requires-Python: >=3.9
+Requires-Dist: duckdb>=0.8.0
+Requires-Dist: pandas>=1.5.0
+Provides-Extra: arrow
+Requires-Dist: pyarrow>=6.0.0; extra == "arrow"
+Provides-Extra: memory-guard
+Requires-Dist: psutil>=5.9.0; extra == "memory-guard"
+Provides-Extra: ml
+Requires-Dist: scikit-learn>=1.2.0; extra == "ml"
+Provides-Extra: all
+Requires-Dist: pyarrow>=6.0.0; extra == "all"
+Requires-Dist: psutil>=5.9.0; extra == "all"
+Requires-Dist: scikit-learn>=1.2.0; extra == "all"

hybrid_frame-0.3.0/README.md

@@ -0,0 +1,55 @@
+# HybridFrame
+
+A high-performance dual-engine DataFrame that transparently shifts between DuckDB (lazy/out-of-core) and Pandas (materialised/in-memory).
+
+## Installation
+
+```bash
+pip install hybrid-frame
+```
+
+With optional dependencies:
+
+```bash
+pip install hybrid-frame[arrow]       # zero-copy Arrow materialisation
+pip install hybrid-frame[memory-guard] # psutil-based OOM protection
+pip install hybrid-frame[ml]           # scikit-learn for ML export
+pip install hybrid-frame[all]          # everything
+```
+
+## Quick Start
+
+```python
+from hybrid_frame import HybridFrame
+import pandas as pd
+
+# Wrap an existing DataFrame
+hf = HybridFrame.from_pandas(pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]}))
+
+# Lazy DuckDB operations — zero Pandas copies
+hf.filter("x > 1").sort_values("y", ascending=False)
+
+# Chain into Pandas for ML feature engineering
+result = hf.assign(x2=lambda df: df["x"] * 2).to_pandas()
+
+# Stream a CSV lazily from disk
+hf = HybridFrame.from_csv("large_file.csv")
+print(hf.filter("age > 21").shape)
+```
+
+## Features
+
+- **Zero-copy engine transitions** — stays in Pandas mode for simple operations (head/tail/select/rename/drop), auto-transitions to DuckDB for complex SQL operations (filter/sort/groupby/join)
+- **Lazy DuckDB connections** — no connection overhead until DuckDB is actually needed
+- **Out-of-core streaming** — `fetch_chunked()`, `to_pandas_iter()`, `to_arrow_reader()` for memory-efficient processing
+- **ML-ready export** — `to_ml_ready(target_column)` returns `(X, y)` tuple
+- **File I/O** — `from_csv()`, `from_parquet()`, `write_csv()`, `write_parquet()`
+- **Set operations** — `union`, `intersect`, `except_`
+- **Cumulative operations** — `diff`, `cumsum`, `cumprod`, `cummin`, `cummax`
+- **Value imputation** — `fillna`, `dropna`, `replace`, `clip`, `time_series_impute`
+- **DuckDB SQL passthrough** — `sql("SELECT ... FROM self WHERE ...")`
+- **Memory guard** — automatic OOM protection for large materialisations
+
+## Documentation
+
+Full API reference is available in the class docstrings. Run `help(HybridFrame)` or visit [GitHub](https://github.com/shell-bay/hybrid-frame).