mxframe 0.2.1__tar.gz → 0.3.0__tar.gz

mxframe-0.3.0/CHANGELOG.md

@@ -0,0 +1,199 @@
+# 📋 Changelog
+
+All notable changes to MXFrame are documented here.  
+Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
+
+---
+
+## [0.3.0] — 2026-07-10
+
+### 📚 Docs & Benchmarks
+- README restructured: 543 lines → 119 lines. Intro, quick start, and 4-row teaser only — all detail moved to dedicated docs.
+- New `docs/benchmarks.md`: full 22-query TPC-H tables (1 M + 10 M rows), Mojo kernel catalogue, current limitations, roadmap, and reproduce instructions.
+- New `docs/api.md`: LazyFrame method reference, expression syntax, SQL frontend, supported operations, project structure.
+- `docs/vision-and-architecture.md`: fixed garbled sections 2.5/2.6/3; success criteria table repaired; all milestones marked ✅.
+- `docs/archive/` added to `.gitignore` — old dev notes moved there, not published.
+- All stale `Mojo 0.26.2` / `MAX Graph shape-cached join` references updated to reflect v0.2.3 reality (Mojo 26.4, AOT ctypes everywhere).
+
+### 📊 Benchmark updates
+- Q5/Q7/Q8/Q10/Q13/Q19 re-measured with per-run join-cache clearing (fresh-join). Previous v0.2.0 numbers for those queries were warm join-cache hits.
+- Summary corrected: CPU 20/22 (Q13 loses fresh-join), GPU 12/22 (Q5/Q7 no longer win at 1 M without cache warmup).
+- Q8 GPU marked `—` (MAX Graph `year()` groupby JIT too slow to benchmark; Phase 6 target).
+
+### 🔨 Infrastructure
+- `modular >= 26.4` requirement added to `pyproject.toml` runtime deps and Dependencies table in README.
+- `docs/archive/` gitignored; previously committed archive files removed from history.
+
+---
+
+## [0.2.3] — 2026-07-09
+
+### ⚡ Performance
+- **Phase 4 — Device-resident GPU join pipeline** (`join_count_i32_gpu` + `join_scatter_i32_gpu`):
+  GPU inner joins now bypass MAX Graph JIT entirely. Cold-start join latency drops from ~300 ms (MAX session init + JIT) to ~70 ms. `AOTKernelsGPU.hash_join()` caches key arrays via `_cached_upload`; `AOTKernelsGPU.gather_table()` gathers all numeric columns with a single shared device-resident index upload.
+- **Phase 5 — GPU sort + top-k** (`sort_topk_i32_gpu`): `ORDER BY … LIMIT k` fuses sort and limit in a single AOT bitonic sort kernel, downloading only `k × 4` bytes instead of `N × 4`. `_apply_post_ops_custom` peek-ahead passes `topk=k` directly to the sort kernel.
+- **Phase 3 — Vectorised rank mapping**: `_encode_sort_key` replaces the per-unique-value Python `for` loop with a single vectorised NumPy scatter (`rank_of[sorted_order] = np.arange(n_unique)`). ~1.5× speedup for sort-key encoding at 5 M rows.
+
+### 🛠 Fixed
+- **GPU sort+limit crash**: `_apply_sort_custom` was calling `_gpu_gather_table` which referenced `gather_f32` as a MAX Graph custom op — that kernel was never registered in the mojopkg. Any `ORDER BY` on `device="gpu"` would crash. Now routes through `_aot_gpu.gather_table` (AOT ctypes, no MAX Graph).
+- `_apply_sort_custom` CPU path now uses `np.argpartition` (O(N) partial sort) when `topk < N`, instead of a full O(N log N) argsort.
+
+### 🔨 Infrastructure
+- **Mojo/MAX 26.4 migration**: all `kernels/*.mojo` and `kernels_aot/*.mojo` migrated from 26.2 API (`fn` → `def`, `from tensor import` → `from extensibility import`, `DeviceContextPtr` → `DeviceContext`, `enqueue_function_experimental` → `enqueue_function`).
+- **`kernels/_atomic.mojo`** (new): drop-in replacement for the missing `std.os.atomic.Atomic`. Implements `fetch_add`, `min`, `max`, `compare_exchange` via MLIR `llvm.atomicrmw` / `llvm.cmpxchg` instructions — portable across NVIDIA and AMD; works in both `mojo package` (JIT) and `mojo build --emit shared-lib` (AOT) contexts.
+- All compiled artifacts rebuilt with Mojo 1.0.0b2 (modular 26.4): `kernels.mojopkg`, `libmxkernels_aot.so`, `libmxkernels_aot_gpu.so`.
+
+---
+
+## [0.2.2] — 2026-07-08
+
+### 🔨 Infrastructure
+- First release targeting **Mojo/MAX 26.4** (`modular = "26.4.*"` in `pixi.toml`).
+- `_atomic.mojo` shim introduced to replace the removed `std.os.atomic` module.
+- All kernel source files migrated to 26.4 syntax; compiled artefacts rebuilt.
+- `pyproject.toml` `runtime` extra pinned to `modular>=26.4` to prevent 26.2 wheels from being installed by Colab/pip users.
+
+---
+
+## [0.2.1] — 2026-06-xx
+
+### 🔨 Infrastructure
+- TestPyPI re-upload to validate wheel packaging. No functional changes.
+
+---
+
+## [0.2.0] — 2026-05-xx
+
+### ✨ Added
+- **Phase 1 — GPU buffer cache + masked global min/max**:
+  - `masked_global_min_f32_gpu` / `masked_global_max_f32_gpu` Mojo AOT kernels.
+  - `AOTKernelsGPU` device-buffer cache (`_cached_upload`, eviction, `clear_buf_cache`) — eliminates redundant PCIe uploads on repeated hot queries.
+  - `group_mean_f32` GPU kernel (sum + count on GPU, divide on CPU).
+  - Masked global min/max fast-paths wired in `custom_ops.py`.
+- **Phase 2 — Masked global aggregation for any expression**:
+  - `_compute_masked_global_agg` extended to handle any inner expression via `_eval_expr_arrow` (e.g. `a*(1-b)`, `case_when`, `startswith`, `isin`) — SUM reduction stays GPU-resident.
+  - Join-only plans emit `"join_mojo_shortcut"` provenance (not `"pyarrow_shortcut"`) — Q19 restructured to single-pass join+filter+global_agg.
+  - `audit_gpu_paths.py` updated: `join_mojo_shortcut` counted as Mojo-backed.
+- **Phase 3 — GPU integer key encoding**:
+  - `group_encode_i32_gpu` open-addressing hash-table kernel for integer group keys; replaces PyArrow `dictionary_encode` for integer-keyed group-by on GPU.
+- **Audit tool** (`scripts/audit_gpu_paths.py`): classifies each of 22 TPC-H queries as `GPU-CLEAN` / `MOJO-CPU` / `FALLBACK` via `last_compile_provenance["path"]`.
+- **Interactive visualiser** (`visualize/mxframe_pipeline.html`): step-through of how a Python query becomes a plan tree and dispatches to GPU kernels.
+
+### ⚡ Performance
+- **22/22 TPC-H queries GPU-CLEAN** (100% Mojo-backed on GPU, zero PyArrow fallbacks in the reduction and join hot paths).
+
+---
+
+## [0.1.2] — 2026-04-22
+
+### 🛠 Fixed
+- `pip install "mxframe[runtime]"` now gives a working install. The `[runtime]` extra pulls `modular>=25.5`, which provides the `max.engine` / `max.driver` modules that `mxframe.compiler` imports at module load. (Pixi users get `modular` from the Modular conda channel and should install `mxframe` without the extra.)
+- `sqlglot` import in `mxframe.sql_frontend` is now lazy: `import mxframe` no longer requires the `[sql]` extra. Calling `mxframe.sql(…)` without `sqlglot` installed raises a clear `ImportError` with the install hint.
+- `__init__.py` `__version__` now matches `pyproject.toml`.
+
+---
+
+## [0.1.1] — 2026-04-22
+
+### 🛠 Fixed
+- Packaging: `readme` now points to `README.md` so PyPI shows the full project page with benchmarks and quickstart (was accidentally showing the internal vision/architecture doc)
+- CI: `AOTKernelsGPU` init no longer hard-fails on CPU-only runners (`CUDA driver unavailable` is caught, `self._aot_gpu` falls back to `None`)
+- CI: `_find_kernels_path()` uses `Path(__file__).resolve().parent` directly, removing the stale `/home/ablearn/mxdf/…` dev fallback that surfaced in installed wheels
+- Bug: `NameError: cache_key_count` in `_hash_join_mojo_cpu` MAX Graph fallback
+- Tests: `scripts/_test_phase0_custom_ops.py` and `_test_phase1.py` no longer hard-code a developer's kernels path; they use the auto-detected default
+
+---
+
+## [0.1.0] — 2026-04-20
+
+### ✨ Added
+- **All 22 TPC-H queries** on CPU and GPU AOT paths
+- **GPU LEFT JOIN** — `join_count_left` + `join_scatter_left` Mojo kernels wired to Python (Q13)
+- `bench_simple.py` — clean 4-column benchmark (Pandas | Polars | MX CPU | MX GPU)
+- `--queries` and `--runs` flags for targeted benchmarking
+
+### ⚡ Performance
+- **Q22** phone prefix anti-join: vectorized via `pc.utf8_slice_codeunits` + `pc.is_in` + `np.isin` — 3.6× faster than Polars CPU (was 10× **slower**)
+- **Q20** semi-join chain: eliminated `.to_pandas()` Pandas detour — 8.7× faster than Polars (was comparable)
+- **Q21** EXISTS+NOT EXISTS: replaced `.to_pandas().groupby.nunique()` with NumPy composite key dedup
+- **Q18** large volume customers: replaced `.to_pylist()` semi-join with Mojo join — semi-join now uses AOT kernel path
+- **Q15** argmax supplier: replaced `0.9999 * max_rev` tolerance hack with exact `pc.equal()` on float32
+- **Q13** LEFT JOIN: removed `.to_pandas()` groupby detour — now fully on GPU AOT path (19ms CPU, 29ms GPU vs 25ms Polars)
+
+### 🐛 Fixed
+- `not use_gpu_filter` gate bug in `custom_ops.py` section 4.6 that blocked GPU filter path
+- Q7 nation pre-join cache key now uses `id()` preventing cache misses on repeated hot calls
+
+### 🏗️ Infrastructure
+- `pyproject.toml` updated with full metadata, dependencies, optional extras, package-data for `.so` files
+- `CONTRIBUTING.md` — developer guide with kernel writing tutorial
+- `PUBLISHING.md` — step-by-step pip release guide
+- GitHub Actions: `test.yml` (CI), `publish.yml` (PyPI via OIDC)
+
+### 📊 Benchmark Summary (1M and 10M rows, all 22 queries, warm median of 3)
+
+See the full table in [README.md § TPC-H Benchmark](README.md#-tpc-h-benchmark--all-22-queries),
+sourced from `scripts/bench_results_1M.csv` and `scripts/bench_results_10M.csv`.
+
+- **Correctness:** 22/22 queries pass on CPU and GPU paths
+- **MX CPU wins vs Polars:** **21/22** at 1 M, **18/22** at 10 M
+  — headline: **Q9 128× · Q12 89× · Q7 42× · Q8 31× · Q17 24× · Q5 22×** (at 10 M)
+- **MX GPU wins vs Polars:** **16/22** at 1 M, **15/22** at 10 M
+  — headline: **Q12 26.5× · Q9 12× · Q8 10.8× · Q17 9.7× · Q7 4.7×** (at 10 M)
+- **Remaining losses (Q4, Q6, Q13, Q21):** ops that still route through PyArrow fallback — kernel replacements tracked in `roadmap.md`
+
+### 🛠 Fixed
+- `pip install "mxframe[runtime]"` now gives a working install. The `[runtime]` extra pulls `modular>=25.5`, which provides the `max.engine` / `max.driver` modules that `mxframe.compiler` imports at module load. (Pixi users get `modular` from the Modular conda channel and should install `mxframe` without the extra.)
+- `sqlglot` import in `mxframe.sql_frontend` is now lazy: `import mxframe` no longer requires the `[sql]` extra. Calling `mxframe.sql(…)` without `sqlglot` installed raises a clear `ImportError` with the install hint.
+- `__init__.py` `__version__` now matches `pyproject.toml`.
+
+---
+
+## [0.1.1] — 2026-04-22
+
+### 🛠 Fixed
+- Packaging: `readme` now points to `README.md` so PyPI shows the full project page with benchmarks and quickstart (was accidentally showing the internal vision/architecture doc)
+- CI: `AOTKernelsGPU` init no longer hard-fails on CPU-only runners (`CUDA driver unavailable` is caught, `self._aot_gpu` falls back to `None`)
+- CI: `_find_kernels_path()` uses `Path(__file__).resolve().parent` directly, removing the stale `/home/ablearn/mxdf/…` dev fallback that surfaced in installed wheels
+- Bug: `NameError: cache_key_count` in `_hash_join_mojo_cpu` MAX Graph fallback
+- Tests: `scripts/_test_phase0_custom_ops.py` and `_test_phase1.py` no longer hard-code a developer's kernels path; they use the auto-detected default
+
+---
+
+## [0.1.0] — 2026-04-20
+
+### ✨ Added
+- **All 22 TPC-H queries** on CPU and GPU AOT paths
+- **GPU LEFT JOIN** — `join_count_left` + `join_scatter_left` Mojo kernels wired to Python (Q13)
+- `bench_simple.py` — clean 4-column benchmark (Pandas | Polars | MX CPU | MX GPU)
+- `--queries` and `--runs` flags for targeted benchmarking
+
+### ⚡ Performance
+- **Q22** phone prefix anti-join: vectorized via `pc.utf8_slice_codeunits` + `pc.is_in` + `np.isin` — 3.6× faster than Polars CPU (was 10× **slower**)
+- **Q20** semi-join chain: eliminated `.to_pandas()` Pandas detour — 8.7× faster than Polars (was comparable)
+- **Q21** EXISTS+NOT EXISTS: replaced `.to_pandas().groupby.nunique()` with NumPy composite key dedup
+- **Q18** large volume customers: replaced `.to_pylist()` semi-join with Mojo join — semi-join now uses AOT kernel path
+- **Q15** argmax supplier: replaced `0.9999 * max_rev` tolerance hack with exact `pc.equal()` on float32
+- **Q13** LEFT JOIN: removed `.to_pandas()` groupby detour — now fully on GPU AOT path (19ms CPU, 29ms GPU vs 25ms Polars)
+
+### 🐛 Fixed
+- `not use_gpu_filter` gate bug in `custom_ops.py` section 4.6 that blocked GPU filter path
+- Q7 nation pre-join cache key now uses `id()` preventing cache misses on repeated hot calls
+
+### 🏗️ Infrastructure
+- `pyproject.toml` updated with full metadata, dependencies, optional extras, package-data for `.so` files
+- `CONTRIBUTING.md` — developer guide with kernel writing tutorial
+- `PUBLISHING.md` — step-by-step pip release guide
+- GitHub Actions: `test.yml` (CI), `publish.yml` (PyPI via OIDC)
+
+### 📊 Benchmark Summary (1M and 10M rows, all 22 queries, warm median of 3)
+
+See the full table in [README.md § TPC-H Benchmark](README.md#-tpc-h-benchmark--all-22-queries),
+sourced from `scripts/bench_results_1M.csv` and `scripts/bench_results_10M.csv`.
+
+- **Correctness:** 22/22 queries pass on CPU and GPU paths
+- **MX CPU wins vs Polars:** **21/22** at 1 M, **18/22** at 10 M
+  — headline: **Q9 128× · Q12 89× · Q7 42× · Q8 31× · Q17 24× · Q5 22×** (at 10 M)
+- **MX GPU wins vs Polars:** **16/22** at 1 M, **15/22** at 10 M
+  — headline: **Q12 26.5× · Q9 12× · Q8 10.8× · Q17 9.7× · Q7 4.7×** (at 10 M)
+- **Remaining losses (Q4, Q6, Q13, Q21):** ops that still route through PyArrow fallback — kernel replacements tracked in `roadmap.md`

mxframe-0.3.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: mxframe
+Version: 0.3.0
+Summary: GPU-accelerated DataFrame engine: Python ergonomics + Mojo AOT kernels, cross-vendor (NVIDIA/AMD/Apple).
+Author-email: Abhishek Sreesaila <abhisheksreesaila@users.noreply.github.com>
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/abhisheksreesaila/mxframe
+Project-URL: Repository, https://github.com/abhisheksreesaila/mxframe
+Keywords: dataframe,gpu,mojo,arrow,analytics,tpch
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Database
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: pyarrow>=14.0
+Requires-Dist: numpy>=1.24
+Requires-Dist: pandas>=2.0
+Provides-Extra: runtime
+Requires-Dist: modular>=26.4; extra == "runtime"
+Provides-Extra: polars
+Requires-Dist: polars>=0.20; extra == "polars"
+Provides-Extra: sql
+Requires-Dist: sqlglot>=25.0; extra == "sql"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: polars>=0.20; extra == "dev"
+Requires-Dist: sqlglot>=25.0; extra == "dev"
+Requires-Dist: duckdb>=0.9; extra == "dev"
+
+# 🚀 MXFrame
+
+> **GPU-accelerated DataFrames — Python ergonomics, Mojo speed, every GPU.**
+
+MXFrame is a DataFrame query engine that pairs a Polars-style Python API with
+pre-compiled Mojo AOT kernels. The same code runs on **NVIDIA, AMD, and Apple Silicon** —
+no CUDA required, no JIT compilation at query time.
+
+[![TPC-H](https://img.shields.io/badge/TPC--H-22%2F22%20queries-brightgreen)](docs/benchmarks.md)
+[![Python](https://img.shields.io/badge/python-3.12%2B-blue)](pyproject.toml)
+[![Platform](https://img.shields.io/badge/platform-linux--x86__64-lightgrey)](pyproject.toml)
+[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
+
+---
+
+## ✨ Why MXFrame?
+
+| | pandas | Polars | cuDF (Rapids) | **MXFrame** |
+|---|---|---|---|---|
+| GPU support | ❌ | ❌ | ✅ NVIDIA only | ✅ **Any GPU** |
+| Compiled kernels | ❌ | ✅ Rust | ✅ CUDA | ✅ **Mojo AOT** |
+| Install complexity | pip | pip | CUDA + Rapids stack | **pixi install** |
+| TPC-H competitive | ❌ | ✅ | ✅ | ✅ |
+| Cross-vendor | ❌ | ❌ | ❌ | ✅ NVIDIA/AMD/Apple |
+
+MXFrame is the **cuDF architecture without the CUDA lock-in**.  
+Kernels are compiled **once** to a `.so` at build time — loaded in ~1 ms, then pure dispatch on every query.
+
+---
+
+## ⚡ Quick Start
+
+```sh
+# 1. Install pixi (Modular's package manager)
+curl -fsSL https://pixi.sh/install.sh | bash
+
+# 2. Clone and set up
+git clone https://github.com/abhisheksreesaila/mxframe
+cd mxframe
+pixi install
+
+# 3. Verify GPU is working
+pixi run python3 scripts/_check_gpu.py
+```
+
+```python
+import pyarrow as pa
+from mxframe import LazyFrame, Scan, col, lit
+
+data = pa.table({
+    "dept":   pa.array(["eng", "eng", "mkt", "mkt", "eng"]),
+    "salary": pa.array([120.0, 95.0, 80.0, 110.0, 130.0], pa.float32()),
+    "age":    pa.array([32, 28, 35, 29, 40], pa.int32()),
+})
+
+result = (
+    LazyFrame(Scan(data))
+    .filter(col("age") > lit(28))
+    .groupby("dept")
+    .agg(
+        col("salary").sum().alias("total_salary"),
+        col("age").count().alias("headcount"),
+    )
+    .sort(col("total_salary"), descending=True)
+    .compute(device="gpu")   # or "cpu"
+)
+print(result.to_pandas())
+```
+
+```
+  dept  total_salary  headcount
+0  eng         345.0          3
+1  mkt         110.0          1
+```
+
+---
+
+## 📊 Performance — TPC-H at 1 M rows *(v0.3.0)*
+
+MX CPU beats Polars on **20/22** queries. GPU advantage compounds at 10 M+ rows where kernel parallelism dominates PCIe upload cost.
+
+| Query | MX CPU | MX GPU | Polars | CPU vs Polars |
+|---|---:|---:|---:|---:|
+| Q9 · Product profit (6-table join) | **0.6 ms** | 6.6 ms | 39.9 ms | **67×** |
+| Q12 · 2-table join + agg           | **0.5 ms** | 3.4 ms | 23.2 ms | **46×** |
+| Q3 · 3-table join + agg            | **2.6 ms** | 8.7 ms | 23.3 ms | **9×** |
+| Q1 · Filter + 8 aggregations       | **10.7 ms** | 42.1 ms | 33.3 ms | **3.1×** |
+
+→ [Full 22-query tables (1 M + 10 M rows), methodology, and reproduce instructions](docs/benchmarks.md)
+
+---
+
+## 📚 Docs
+
+| | |
+|---|---|
+| [**API Reference**](docs/api.md) | LazyFrame, expressions, SQL frontend, supported operations, running tests |
+| [**Benchmarks**](docs/benchmarks.md) | Full TPC-H tables, kernel catalogue, limitations, roadmap, reproduce instructions |
+| [**Architecture**](docs/vision-and-architecture.md) | Design philosophy, internal layers, how MAX Graph fits in |
+| [**Contributing**](CONTRIBUTING.md) | Dev setup, writing Mojo kernels, adding queries |
+
+---
+
+## 📦 Dependencies
+
+| Package | Required | Purpose |
+|---|---|---|
+| `pyarrow >= 14` | ✅ | Column storage, zero-copy NumPy bridge |
+| `numpy >= 1.24` | ✅ | Vectorized pre/post processing |
+| `pandas >= 2.0` | ✅ | Reference implementations |
+| `modular >= 26.4` | GPU only | MAX Engine runtime, Mojo GPU dispatch |
+| `polars >= 0.20` | optional | Benchmark comparison |
+| `sqlglot >= 25` | optional | SQL frontend parsing |
+
+---
+
+## 📄 License
+
+Apache 2.0 — see [LICENSE](LICENSE).

mxframe-0.3.0/README.md

@@ -0,0 +1,119 @@
+# 🚀 MXFrame
+
+> **GPU-accelerated DataFrames — Python ergonomics, Mojo speed, every GPU.**
+
+MXFrame is a DataFrame query engine that pairs a Polars-style Python API with
+pre-compiled Mojo AOT kernels. The same code runs on **NVIDIA, AMD, and Apple Silicon** —
+no CUDA required, no JIT compilation at query time.
+
+[![TPC-H](https://img.shields.io/badge/TPC--H-22%2F22%20queries-brightgreen)](docs/benchmarks.md)
+[![Python](https://img.shields.io/badge/python-3.12%2B-blue)](pyproject.toml)
+[![Platform](https://img.shields.io/badge/platform-linux--x86__64-lightgrey)](pyproject.toml)
+[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)
+
+---
+
+## ✨ Why MXFrame?
+
+| | pandas | Polars | cuDF (Rapids) | **MXFrame** |
+|---|---|---|---|---|
+| GPU support | ❌ | ❌ | ✅ NVIDIA only | ✅ **Any GPU** |
+| Compiled kernels | ❌ | ✅ Rust | ✅ CUDA | ✅ **Mojo AOT** |
+| Install complexity | pip | pip | CUDA + Rapids stack | **pixi install** |
+| TPC-H competitive | ❌ | ✅ | ✅ | ✅ |
+| Cross-vendor | ❌ | ❌ | ❌ | ✅ NVIDIA/AMD/Apple |
+
+MXFrame is the **cuDF architecture without the CUDA lock-in**.  
+Kernels are compiled **once** to a `.so` at build time — loaded in ~1 ms, then pure dispatch on every query.
+
+---
+
+## ⚡ Quick Start
+
+```sh
+# 1. Install pixi (Modular's package manager)
+curl -fsSL https://pixi.sh/install.sh | bash
+
+# 2. Clone and set up
+git clone https://github.com/abhisheksreesaila/mxframe
+cd mxframe
+pixi install
+
+# 3. Verify GPU is working
+pixi run python3 scripts/_check_gpu.py
+```
+
+```python
+import pyarrow as pa
+from mxframe import LazyFrame, Scan, col, lit
+
+data = pa.table({
+    "dept":   pa.array(["eng", "eng", "mkt", "mkt", "eng"]),
+    "salary": pa.array([120.0, 95.0, 80.0, 110.0, 130.0], pa.float32()),
+    "age":    pa.array([32, 28, 35, 29, 40], pa.int32()),
+})
+
+result = (
+    LazyFrame(Scan(data))
+    .filter(col("age") > lit(28))
+    .groupby("dept")
+    .agg(
+        col("salary").sum().alias("total_salary"),
+        col("age").count().alias("headcount"),
+    )
+    .sort(col("total_salary"), descending=True)
+    .compute(device="gpu")   # or "cpu"
+)
+print(result.to_pandas())
+```
+
+```
+  dept  total_salary  headcount
+0  eng         345.0          3
+1  mkt         110.0          1
+```
+
+---
+
+## 📊 Performance — TPC-H at 1 M rows *(v0.3.0)*
+
+MX CPU beats Polars on **20/22** queries. GPU advantage compounds at 10 M+ rows where kernel parallelism dominates PCIe upload cost.
+
+| Query | MX CPU | MX GPU | Polars | CPU vs Polars |
+|---|---:|---:|---:|---:|
+| Q9 · Product profit (6-table join) | **0.6 ms** | 6.6 ms | 39.9 ms | **67×** |
+| Q12 · 2-table join + agg           | **0.5 ms** | 3.4 ms | 23.2 ms | **46×** |
+| Q3 · 3-table join + agg            | **2.6 ms** | 8.7 ms | 23.3 ms | **9×** |
+| Q1 · Filter + 8 aggregations       | **10.7 ms** | 42.1 ms | 33.3 ms | **3.1×** |
+
+→ [Full 22-query tables (1 M + 10 M rows), methodology, and reproduce instructions](docs/benchmarks.md)
+
+---
+
+## 📚 Docs
+
+| | |
+|---|---|
+| [**API Reference**](docs/api.md) | LazyFrame, expressions, SQL frontend, supported operations, running tests |
+| [**Benchmarks**](docs/benchmarks.md) | Full TPC-H tables, kernel catalogue, limitations, roadmap, reproduce instructions |
+| [**Architecture**](docs/vision-and-architecture.md) | Design philosophy, internal layers, how MAX Graph fits in |
+| [**Contributing**](CONTRIBUTING.md) | Dev setup, writing Mojo kernels, adding queries |
+
+---
+
+## 📦 Dependencies
+
+| Package | Required | Purpose |
+|---|---|---|
+| `pyarrow >= 14` | ✅ | Column storage, zero-copy NumPy bridge |
+| `numpy >= 1.24` | ✅ | Vectorized pre/post processing |
+| `pandas >= 2.0` | ✅ | Reference implementations |
+| `modular >= 26.4` | GPU only | MAX Engine runtime, Mojo GPU dispatch |
+| `polars >= 0.20` | optional | Benchmark comparison |
+| `sqlglot >= 25` | optional | SQL frontend parsing |
+
+---
+
+## 📄 License
+
+Apache 2.0 — see [LICENSE](LICENSE).

{mxframe-0.2.1 → mxframe-0.3.0}/__init__.py

@@ -1,6 +1,6 @@
 """mxframe - GPU-accelerated DataFrames with MAX Engine"""
 
-__version__ = "0.1.2"
+__version__ = "0.3.0"
 
 # Lazy expression layer
 from .lazy_expr import Expr, col, lit, when, row_number