typedframes 0.1.0__tar.gz

typedframes-0.1.0/PKG-INFO

@@ -0,0 +1,901 @@
+Metadata-Version: 2.4
+Name: typedframes
+Version: 0.1.0
+Requires-Dist: mypy ; extra == 'mypy'
+Requires-Dist: pandas>=2.3.3 ; extra == 'pandas'
+Requires-Dist: pandera>=0.22.0 ; extra == 'pandera'
+Requires-Dist: polars>=1.37.1 ; extra == 'polars'
+Provides-Extra: mypy
+Provides-Extra: pandas
+Provides-Extra: pandera
+Provides-Extra: polars
+Summary: Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime.
+Home-Page: https://github.com/w-martin/typedframes
+Author: William Martin
+License: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+
+# typedframes
+
+> ⚠️ **Project Status: Proof of Concept**
+>
+> `typedframes` (v0.1.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
+> integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
+> schema validation over production-grade stability.
+>
+**Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime.**
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+
+
+class UserData(BaseSchema):
+    user_id = Column(type=int)
+    email = Column(type=str)
+    signup_date = Column(type=str)
+
+
+def process(df: PandasFrame[UserData]) -> None:
+  df[UserData.user_id]  # ✓ Schema descriptor — autocomplete, refactor-safe
+  df['user_id']  # ✓ String access — also validated by checker
+  df['username']  # ✗ Error: Column 'username' not in UserData
+```
+
+**Why descriptors?** Rename a column in ONE place (`user_id = Column(type=int)`), and all references — `Schema.user_id`,
+`str(Schema.user_id)`, `Schema.user_id.col` — update automatically. No find-and-replace across string literals.
+
+---
+
+## Table of Contents
+
+- [Why typedframes?](#why-typedframes)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Static Analysis](#static-analysis)
+- [Static Analysis Performance](#static-analysis-performance)
+- [Comparison](#comparison)
+- [Type Safety With Multiple Backends](#type-safety-with-multiple-backends)
+- [Features](#features)
+- [Advanced Usage](#advanced-usage)
+- [Pandera Integration](#pandera-integration)
+- [Examples](#examples)
+- [Philosophy](#philosophy)
+- [FAQ](#faq)
+
+---
+
+## Why typedframes?
+
+**The problem:** Many pandas bugs are column mismatches. You access a column that doesn't exist, pass the wrong schema to a function, or make a typo. These errors only surface at runtime, often in production.
+
+**The solution:** Define your DataFrame schemas as Python classes. Get static type checking that catches column errors before you even run your code.
+
+**What you get:**
+
+- ✅ **Static analysis** - Catch column errors at lint-time with mypy or the standalone checker
+- ✅ **Beautiful runtime UX** - `df[Schema.column_group].mean()` (pandas) instead of ugly column lists
+- ✅ **Works with pandas AND polars** - Same schema API, explicit backend types
+- ✅ **Dynamic column matching** - Regex-based ColumnSets for time-series data
+- ✅ **Zero runtime overhead** - No validation, no slowdown
+- ✅ **Type-safe backends** - Type checker knows pandas vs polars methods
+
+---
+
+## Installation
+
+```shell
+pip install typedframes
+```
+or
+```shell
+uv add typedframes
+```
+
+The Rust-based checker is included — no separate install needed.
+
+---
+
+## Quick Start
+
+### Define Your Schema (Once)
+
+```python
+from typedframes import BaseSchema, Column, ColumnSet
+
+
+class SalesData(BaseSchema):
+    date = Column(type=str)
+    revenue = Column(type=float)
+    customer_id = Column(type=int)
+
+    # Dynamic columns with regex
+    metrics = ColumnSet(type=float, members=r"metric_\d+", regex=True)
+```
+
+### Use With Pandas
+
+```python
+from typedframes.pandas import PandasFrame
+
+# Load data with schema — one line
+df = PandasFrame.read_csv("sales.csv", SalesData)
+
+# Access columns via schema descriptors
+print(df[SalesData.revenue].sum())
+print(df[SalesData.metrics].mean())  # All metric_* columns
+
+
+# Type-safe pandas operations
+def analyze(data: PandasFrame[SalesData]) -> float:
+  data[SalesData.revenue]  # ✓ Validated by type checker
+  data['profit']  # ✗ Error at lint-time: 'profit' not in SalesData
+  return data[SalesData.revenue].mean()
+
+
+# Standard pandas access still works
+filtered = df[df[SalesData.revenue] > 1000]
+grouped = df.groupby(SalesData.customer_id)[str(SalesData.revenue)].sum()
+```
+
+### Use With Polars
+
+```python
+from typedframes.polars import PolarsFrame
+import polars as pl
+
+# Load data with schema — one line
+df = PolarsFrame.read_csv("sales.csv", SalesData)
+
+# Use schema column references for type-safe expressions
+print(df.select(SalesData.revenue.col).sum())
+
+
+# Type-safe polars operations
+def analyze_polars(data: PolarsFrame[SalesData]) -> pl.DataFrame:
+  data.select(SalesData.revenue.col)  # ✓ OK
+  data.select(['profit'])  # ✗ Error at lint-time: 'profit' not in SalesData
+  return data.select(SalesData.revenue.col).mean()
+
+
+# Polars methods work as expected
+filtered = df.filter(SalesData.revenue.col > 1000)
+grouped = df.group_by('customer_id').agg(SalesData.revenue.col.sum())
+```
+
+---
+
+## Static Analysis
+
+typedframes provides **two ways** to check your code:
+
+### Option 1: Standalone Checker (Fast)
+
+```shell
+# Blazing fast Rust-based checker
+typedframes check src/
+
+# Output:
+# ✓ Checked 47 files in 0.0s
+# ✗ src/analysis.py:23 - Column 'profit' not in SalesData
+# ✗ src/pipeline.py:56 - Column 'user_name' not in UserData
+```
+
+**Features:**
+- Catches column name errors
+- Validates schema mismatches between functions
+- Checks both pandas and polars code
+- 10-100x faster than mypy
+
+**Use this for:**
+- Fast feedback during development
+- CI/CD pipelines
+- Pre-commit hooks
+
+**Configuration:**
+```shell
+# Check specific files
+typedframes check src/pipeline.py
+
+# Check directory
+typedframes check src/
+
+# Fail on any error (for CI)
+typedframes check src/ --strict
+
+# JSON output
+typedframes check src/ --json
+```
+
+### Option 2: Mypy Plugin (Comprehensive)
+
+```shell
+# Add to pyproject.toml
+[tool.mypy]
+plugins = ["typedframes.mypy"]
+
+# Or mypy.ini
+[mypy]
+plugins = typedframes.mypy
+
+# Run mypy
+mypy src/
+```
+
+**Features:**
+- Full type checking across your codebase
+- Catches column errors AND regular type errors
+- IDE integration (VSCode, PyCharm)
+- Works with existing mypy configuration
+
+**Use this for:**
+- Comprehensive type checking
+- Integration with existing mypy setup
+- IDE error highlighting
+
+---
+
+## Static Analysis Performance
+
+Fast feedback reduces development time. The typedframes Rust binary provides near-instant column checking.
+
+**Benchmark results** (10 runs, 3 warmup, caches cleared between runs):
+
+| Tool               | Version | What it does                  | typedframes (11 files) | great_expectations (490 files) |
+|--------------------|---------|-------------------------------|------------------------|--------------------------------|
+| typedframes        | 0.1.0   | DataFrame column checker      | 961µs ±56µs            | 930µs ±89µs                    |
+| ruff               | 0.15.0  | Linter (no type checking)     | 39ms ±12ms             | 360ms ±18ms                    |
+| ty                 | 0.0.16  | Type checker                  | 146ms ±13ms            | 1.65s ±26ms                    |
+| pyrefly            | 0.52.0  | Type checker                  | 152ms ±7ms             | 693ms ±33ms                    |
+| mypy               | 1.19.1  | Type checker (no plugin)      | 9.15s ±218ms           | 12.13s ±400ms                  |
+| mypy + typedframes | 1.19.1  | Type checker + column checker | 9.34s ±331ms           | 13.89s ±491ms                  |
+| pyright            | 1.1.408 | Type checker                  | 2.34s ±335ms           | 8.37s ±253ms                   |
+
+*Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
+
+The typedframes binary performs lexical column name resolution within a single file. It does not perform cross-file type
+inference. Full type checkers (mypy, pyright, ty) analyze all Python types across your entire codebase. Use both: the
+binary for fast iteration, mypy for comprehensive checking.
+
+The standalone checker is built with [`ruff_python_parser`](https://github.com/astral-sh/ruff) for Python AST
+parsing.
+
+**Note:** ty (Astral) does not currently support mypy plugins, so use the standalone binary for column checking with ty.
+
+---
+
+## Comparison
+
+### Feature Matrix (Static Analysis Focus)
+
+Comprehensive comparison of pandas/DataFrame typing and validation tools. **typedframes focuses on static analysis**
+—catching errors at lint-time before your code runs.
+
+| Feature                         | typedframes            | Pandera     | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame      | narwhals |
+|---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|
+| **Version tested**              | 0.1.0                  | 0.29.0      | 1.4.3              | 0.3.6                 | 3.0.0        | 0.1.2       | 1.1.3              | 3.7.0            | 2.16.0   |
+| **Analysis Type**               |
+| When errors are caught          | **Static (lint-time)** | Runtime     | Runtime            | Static + Runtime      | Static       | Runtime     | Runtime            | Static + Runtime | Runtime  |
+| **Static Analysis (our focus)** |
+| Mypy plugin                     | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ✅ Yes                 | ✅ Yes        | ❌ No        | ❌ No               | ⚠️ Basic         | ❌ No     |
+| Standalone checker              | ✅ Rust (~1ms)          | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     |
+| Column name checking            | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ✅ Yes                 | ❌ No         | ❌ No        | ❌ No               | ✅ Yes            | ❌ No     |
+| Column type checking            | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ✅ Yes                 | ❌ No         | ❌ No        | ❌ No               | ✅ Yes            | ❌ No     |
+| Typo suggestions                | ✅ Yes                  | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     |
+| **Runtime Validation**          |
+| Data validation                 | ❌ No                   | ✅ Excellent | ✅ Excellent        | ✅ typeguard           | ❌ No         | ✅ Yes       | ✅ Yes              | ✅ Yes            | ❌ No     |
+| Value constraints               | ❌ No                   | ✅ Yes       | ✅ Excellent        | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ✅ Yes            | ❌ No     |
+| **Schema Features**             |
+| Column grouping                 | ✅ ColumnGroup          | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     |
+| Regex column matching           | ✅ Yes                  | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     |
+| **Backend Support**             |
+| Pandas                          | ✅ Yes                  | ✅ Yes       | ✅ Yes              | ✅ Yes                 | ✅ Yes        | ✅ Yes       | ✅ Yes              | ❌ Own            | ✅ Yes    |
+| Polars                          | ✅ Yes                  | ✅ Yes       | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ Own            | ✅ Yes    |
+| DuckDB, cuDF, etc.              | ❌ No                   | ❌ No        | ✅ Spark, SQL       | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ✅ Yes    |
+| **Project Status (Feb 2026)**   |
+| Active development              | ✅ Yes                  | ✅ Yes       | ✅ Yes              | ⚠️ Low                | ✅ Yes        | ❌ Inactive  | ⚠️ Low             | ✅ Yes            | ✅ Yes    |
+
+**Legend:** ✅ Full support | ⚠️ Limited/Partial | ❌ Not supported
+
+### Tool Descriptions
+
+- **[Pandera](https://pandera.readthedocs.io/)** (v0.29.0): Excellent runtime validation. Static analysis support exists
+  but has limitations—column access via `df["column"]` is not validated, and schema mismatches between functions may not
+  be caught.
+
+- **[strictly_typed_pandas](https://strictly-typed-pandas.readthedocs.io/)** (v0.3.6): Provides `DataSet[Schema]` type
+  hints with mypy support. No standalone checker. No polars support. Runtime validation via typeguard.
+
+- **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.0): Official pandas type stubs. Provides
+  API-level types but no column-level checking.
+
+- **[dataenforce](https://github.com/CedricFR/dataenforce)** (v0.1.2): Runtime validation via decorator. Marked as
+  experimental/not production-ready. Appears inactive.
+
+- **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
+  static analysis.
+
+- **[StaticFrame](https://github.com/static-frame/static-frame)** (v3.7.0): Alternative immutable DataFrame library with
+  built-in static typing. Not compatible with pandas/polars—requires using StaticFrame's own DataFrame implementation.
+
+- **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.16.0): Compatibility layer that provides a unified API
+  across pandas, polars, DuckDB, cuDF, and more. Solves a different problem—write-once-run-anywhere portability, not
+  type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
+  below.
+
+- **[Great Expectations](https://greatexpectations.io/)** (v1.4.3): Comprehensive data quality framework. Defines
+  "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
+  validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
+  code. Supports pandas, Spark, and SQL backends.
+
+### Type Checkers (Not DataFrame-Specific)
+
+These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
+typedframes for comprehensive type checking:
+
+- **[mypy](https://mypy-lang.org/)** (v1.19.1): The original Python type checker. typedframes provides a mypy plugin for
+  column checking. See [performance benchmarks](#static-analysis-performance).
+
+- **[ty](https://github.com/astral-sh/ty)** (v0.0.14, Astral): New Rust-based type checker, 10-60x faster than mypy on
+  large codebases. Does not support mypy plugins—use typedframes standalone checker.
+
+- **[pyrefly](https://pyrefly.org/)** (v0.51.1, Meta): Rust-based type checker from Meta, replacement for Pyre. Fast,
+  but no DataFrame column checking.
+
+- **[pyright](https://github.com/microsoft/pyright)** (v1.1.408, Microsoft): Type checker powering Pylance/VSCode. No
+  mypy plugin support—use typedframes standalone checker.
+
+### Not Directly Comparable
+
+These tools serve different purposes:
+
+- **[pandas_lint](https://github.com/Jean-EstevezT/pandas_lint)**: Lints pandas code patterns (performance, best
+  practices). Does not check column names/types.
+- **[pandas-vet](https://github.com/deppen8/pandas-vet)**: Flake8 plugin for pandas best practices. Does not check
+  column names/types.
+
+### When to Use What
+
+| Use Case                                             | Recommended Tool                    |
+|------------------------------------------------------|-------------------------------------|
+| Static column checking (existing pandas/polars)      | **typedframes**                     |
+| Runtime data validation                              | Pandera                             |
+| Both static + runtime                                | typedframes + `to_pandera_schema()` |
+| Cross-library portability (write once, run anywhere) | narwhals                            |
+| Data quality monitoring / pipeline validation        | Great Expectations                  |
+| Immutable DataFrames from scratch                    | StaticFrame                         |
+| Pandas API type hints only                           | pandas-stubs                        |
+
+---
+
+## Type Safety With Multiple Backends
+
+typedframes uses **explicit backend types** to ensure complete type safety:
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+from typedframes.polars import PolarsFrame
+
+
+class UserData(BaseSchema):
+    user_id = Column(type=int)
+    email = Column(type=str)
+
+
+# Pandas pipeline - type checker knows pandas methods
+def pandas_analyze(df: PandasFrame[UserData]) -> PandasFrame[UserData]:
+  return df[df[UserData.user_id] > 100]  # ✓ Pandas syntax
+
+
+# Polars pipeline - type checker knows polars methods
+def polars_analyze(df: PolarsFrame[UserData]) -> PolarsFrame[UserData]:
+  return df.filter(UserData.user_id.col > 100)  # ✓ Polars syntax
+
+
+# Type checker prevents mixing backends
+df_pandas = PandasFrame.read_csv("data.csv", UserData)
+df_polars = PolarsFrame.read_csv("data.csv", UserData)
+
+pandas_analyze(df_pandas)  # ✓ OK
+polars_analyze(df_polars)  # ✓ OK
+pandas_analyze(df_polars)  # ✗ Type error: Expected PandasFrame, got PolarsFrame
+```
+
+---
+
+## Features
+
+### Clean Schema Definition
+
+```python
+from typedframes import BaseSchema, Column, ColumnSet, ColumnGroup
+
+
+class TimeSeriesData(BaseSchema):
+    timestamp = Column(type=str)
+    temperature = ColumnSet(type=float, members=r"temp_sensor_\d+", regex=True)
+    pressure = ColumnSet(type=float, members=r"pressure_\d+", regex=True)
+
+    # Logical grouping
+    sensors = ColumnGroup(members=[temperature, pressure])
+```
+
+### Beautiful Runtime API
+
+```python
+from typedframes.pandas import PandasFrame
+
+df = PandasFrame.read_csv("sensors.csv", TimeSeriesData)
+
+# Access column groups as DataFrames
+temps = df[TimeSeriesData.temperature]  # All temp_sensor_* columns
+all_sensors = df[TimeSeriesData.sensors]  # All sensor columns
+
+# Clean operations
+avg_temp = df[TimeSeriesData.temperature].mean()
+max_pressure = df[TimeSeriesData.pressure].max()
+
+# Standard pandas access still works
+df['timestamp']  # Single column
+df[['timestamp', 'temp_sensor_1']]  # Multi-column select
+```
+
+### Column-Level Static Checking
+
+```python
+from typedframes.pandas import PandasFrame
+
+
+def daily_summary(data: PandasFrame[TimeSeriesData]) -> PandasFrame[DailySummary]:
+    # Type checker validates column access
+    data['timestamp']  # ✓ OK - column exists
+    data['date']  # ✗ Error: Column 'date' not in TimeSeriesData
+
+    # Type checker validates ColumnSet access
+    temps = data[TimeSeriesData.temperature]  # ✓ OK - ColumnSet exists
+    summary = temps.mean()
+    return summary
+```
+
+### Dynamic Column Matching
+
+Perfect for time-series data where column counts change. Regex ColumnSets match actual DataFrame columns at
+`from_schema()` time — the matched columns are stored and used when you access `df[Schema.sensors]`. Static analysis
+validates that the ColumnSet *name* exists in the schema, but cannot verify which columns the regex will match at
+runtime.
+
+```python
+class SensorReadings(BaseSchema):
+    timestamp = Column(type=str)
+    # Matches: sensor_1, sensor_2, ..., sensor_N
+    sensors = ColumnSet(type=float, members=r"sensor_\d+", regex=True)
+
+# Works regardless of how many sensor columns exist
+df = PandasFrame.read_csv("readings_2024_01.csv", SensorReadings)  # 50 sensors
+df[SensorReadings.sensors].mean()  # All sensor columns
+
+df = PandasFrame.read_csv("readings_2024_02.csv", SensorReadings)  # 75 sensors
+df[SensorReadings.sensors].mean()  # All sensor columns (different count, same code)
+```
+
+---
+
+## Advanced Usage
+
+### Merges, Joins, and Filters
+
+Schema-typed DataFrames preserve their type through common operations:
+
+**Pandas:**
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+import pandas as pd
+
+
+class UserSchema(BaseSchema):
+    user_id = Column(type=int)
+    email = Column(type=str)
+
+
+class OrderSchema(BaseSchema):
+    order_id = Column(type=int)
+    user_id = Column(type=int)
+    total = Column(type=float)
+
+
+# Schema preserved through filtering
+def get_active_users(df: PandasFrame[UserSchema]) -> PandasFrame[UserSchema]:
+  return df[df[UserSchema.user_id] > 100]  # ✓ Still PandasFrame[UserSchema]
+
+
+# Schema preserved through merges
+users: PandasFrame[UserSchema] = ...
+orders: PandasFrame[OrderSchema] = ...
+merged = users.merge(orders, on=str(UserSchema.user_id))
+```
+
+**Polars:**
+```python
+from typedframes.polars import PolarsFrame
+import polars as pl
+
+
+# Schema columns work in filter expressions
+def filter_users(df: PolarsFrame[UserSchema]) -> pl.DataFrame:
+    return df.filter(UserSchema.user_id.col > 100)
+
+
+# Schema columns work in join expressions
+def join_data(
+    users: PolarsFrame[UserSchema],
+    orders: PolarsFrame[OrderSchema]
+) -> pl.DataFrame:
+    return users.join(
+        orders,
+        left_on=UserSchema.user_id.col,
+        right_on=OrderSchema.user_id.col
+    )
+
+
+# Schema columns work in select expressions
+def select_columns(df: PolarsFrame[UserSchema]) -> pl.DataFrame:
+    return df.select([UserSchema.user_id.col, UserSchema.email.col])
+```
+
+### Schema Composition
+
+Compose upward — build bigger schemas from smaller ones via inheritance. Type checkers see all columns natively.
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+
+
+# Start with the smallest useful schema
+class UserPublic(BaseSchema):
+    user_id = Column(type=int)
+    email = Column(type=str)
+    name = Column(type=str)
+
+
+# Extend it — never strip down
+class UserFull(UserPublic):
+    password_hash = Column(type=str)
+
+
+class Orders(BaseSchema):
+    order_id = Column(type=int)
+    user_id = Column(type=int)
+    total = Column(type=float)
+
+
+# Combine via multiple inheritance
+class UserOrders(UserPublic, Orders):
+  """Type checkers see all columns from both parents."""
+  ...
+
+
+# Or use the + operator
+UserOrdersDynamic = UserPublic + Orders
+
+users: PandasFrame[UserPublic] = ...
+orders: PandasFrame[Orders] = ...
+merged: PandasFrame[UserOrders] = PandasFrame.from_schema(
+  users.merge(orders, on=str(UserPublic.user_id)), UserOrders
+)
+```
+
+Overlapping columns with the same type are allowed (common after merges). Conflicting types raise `SchemaConflictError`.
+
+See [`examples/schema_algebra_example.py`](examples/schema_algebra_example.py) for a complete walkthrough.
+
+---
+
+## Pandera Integration
+
+Convert typedframes schemas to [Pandera](https://pandera.readthedocs.io/) schemas for runtime validation. Define your
+schema once, get both static and runtime checking.
+
+```shell
+pip install typedframes[pandera]
+```
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandera import to_pandera_schema
+import pandas as pd
+
+
+class UserData(BaseSchema):
+  user_id = Column(type=int)
+  email = Column(type=str)
+  age = Column(type=int, nullable=True)
+
+
+# Convert to pandera schema
+pandera_schema = to_pandera_schema(UserData)
+
+# Validate data at runtime
+df = pd.read_csv("users.csv")
+validated_df = pandera_schema.validate(df)  # Raises SchemaError on failure
+```
+
+The conversion maps:
+
+- `Column` type/nullable/alias to `pa.Column` dtype/nullable/name
+- `ColumnSet` with explicit members to individual `pa.Column` entries
+- `ColumnSet` with regex to `pa.Column(regex=True)`
+- `allow_extra_columns` to pandera's `strict` mode
+
+---
+
+## Examples
+
+### Basic CSV Processing
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+
+
+class Orders(BaseSchema):
+    order_id = Column(type=int)
+    customer_id = Column(type=int)
+    total = Column(type=float)
+    date = Column(type=str)
+
+
+def calculate_revenue(orders: PandasFrame[Orders]) -> float:
+  return orders[Orders.total].sum()
+
+
+df = PandasFrame.read_csv("orders.csv", Orders)
+revenue = calculate_revenue(df)
+```
+
+### Time Series Analysis
+
+```python
+from typedframes import BaseSchema, Column, ColumnSet, ColumnGroup
+from typedframes.pandas import PandasFrame
+
+
+class SensorData(BaseSchema):
+    timestamp = Column(type=str)
+    temperature = ColumnSet(type=float, members=r"temp_\d+", regex=True)
+    humidity = ColumnSet(type=float, members=r"humidity_\d+", regex=True)
+
+    all_sensors = ColumnGroup(members=[temperature, humidity])
+
+
+df = PandasFrame.read_csv("sensors.csv", SensorData)
+
+# Clean, type-safe operations
+avg_temp_per_row = df[SensorData.temperature].mean(axis=1)
+all_readings_stats = df[SensorData.all_sensors].describe()
+```
+
+### Multi-Step Pipeline
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.pandas import PandasFrame
+
+
+class RawSales(BaseSchema):
+    date = Column(type=str)
+    product_id = Column(type=int)
+    quantity = Column(type=int)
+    price = Column(type=float)
+
+
+class AggregatedSales(BaseSchema):
+    date = Column(type=str)
+    total_revenue = Column(type=float)
+    total_quantity = Column(type=int)
+
+
+def aggregate_daily(df: PandasFrame[RawSales]) -> PandasFrame[AggregatedSales]:
+  result = df.groupby(RawSales.date).agg({
+    str(RawSales.price): 'sum',
+    str(RawSales.quantity): 'sum',
+    }).reset_index()
+
+    result.columns = ['date', 'total_revenue', 'total_quantity']
+    return PandasFrame.from_schema(result, AggregatedSales)
+
+
+# Type-safe pipeline
+raw = PandasFrame.read_csv("sales.csv", RawSales)
+aggregated = aggregate_daily(raw)
+
+
+# Type checker validates schema transformations
+def analyze(df: PandasFrame[AggregatedSales]) -> float:
+  df[AggregatedSales.total_revenue]  # ✓ OK
+  df['price']  # ✗ Error: 'price' not in AggregatedSales
+  return df[AggregatedSales.total_revenue].mean()
+```
+
+### Polars Performance Pipeline
+
+```python
+from typedframes import BaseSchema, Column
+from typedframes.polars import PolarsFrame
+import polars as pl
+
+
+class LargeDataset(BaseSchema):
+    id = Column(type=int)
+    value = Column(type=float)
+    category = Column(type=str)
+
+
+def efficient_aggregation(df: PolarsFrame[LargeDataset]) -> pl.DataFrame:
+    return (
+        df.filter(LargeDataset.value.col > 100)
+        .group_by('category')
+        .agg(LargeDataset.value.col.mean())
+    )
+
+
+# Polars handles large files efficiently
+df = PolarsFrame.read_csv("huge_file.csv", LargeDataset)
+result = efficient_aggregation(df)
+```
+
+---
+
+## Philosophy
+
+### Type Safety Over Validation
+
+We believe static analysis catches bugs earlier and cheaper than runtime validation.
+
+**typedframes focuses on:**
+- ✅ Catching errors at lint-time
+- ✅ Zero runtime overhead
+- ✅ Developer experience
+
+**We explicitly don't focus on:**
+- ❌ Runtime data validation (use Pandera)
+- ❌ Statistical checks (use Pandera)
+- ❌ Data quality monitoring (use Great Expectations)
+
+**Important:** `PandasFrame.from_schema()` is a *trust assertion*, not a validation step. It tells the type checker "
+this DataFrame conforms to this schema" without verifying the actual data. The linter catches mistakes in your code (
+wrong
+column names, schema mismatches between functions), but it cannot verify that a CSV file contains the expected columns.
+For runtime validation of external data, use [`to_pandera_schema()`](#pandera-integration) to convert your typedframes
+schemas to Pandera schemas.
+
+### Explicit Backend Types
+
+We use explicit `PandasFrame` and `PolarsFrame` types because:
+- Pandas and polars have different APIs
+- Type safety requires knowing which methods are available
+- Being explicit prevents bugs
+
+**Trade-offs we avoid:**
+- ❌ "Universal DataFrame" abstractions (you lose features)
+- ❌ Implicit backend detection (runtime errors)
+- ❌ Lowest-common-denominator APIs
+
+The reason to choose polars over pandas is its lazy evaluation, native parallelism, and expressive query syntax.
+Abstraction layers often must expose a lowest-common-denominator API. By using explicit backend types instead,
+typedframes lets you use each library's full, native API while still getting schema-level type safety.
+
+### Why Abstraction Layers Don't Solve Type Safety
+
+Tools like [narwhals](https://narwhals-dev.github.io/narwhals/) solve a different problem: writing portable code that runs on pandas, polars, DuckDB, cuDF, and other backends. This is useful for library authors who want to support multiple backends without maintaining separate codebases.
+
+However, abstraction layers don't provide column-level type safety:
+
+```python
+import narwhals as nw
+
+def process(df: nw.DataFrame) -> nw.DataFrame:
+    # No static checking - "revenue" typo won't be caught until runtime
+    return df.filter(nw.col("revnue") > 100)  # Typo: "revnue" vs "revenue"
+```
+
+**The fundamental issue:** Abstraction layers abstract over *which library* you're using, not *what columns* your data has. They can't know at lint-time whether "revenue" is a valid column in your DataFrame.
+
+typedframes solves the orthogonal problem of schema safety:
+
+```python
+from typedframes.polars import PolarsFrame
+
+class SalesData(BaseSchema):
+    revenue = Column(type=float)
+
+def process(df: PolarsFrame[SalesData]) -> PolarsFrame[SalesData]:
+    return df.filter(df['revnue'] > 100)  # ✗ Error at lint-time: 'revnue' not in SalesData
+```
+
+**Use narwhals when:** You're writing a library that needs to work with multiple DataFrame backends.
+
+**Use typedframes when:** You want to catch column name/type errors before your code runs.
+
+### Why No Built-in Validation?
+
+Ideally, validation happens at the point of data ingestion rather than in Python application code. If you're validating
+DataFrames in Python, consider whether your data pipeline could enforce constraints earlier. Use Pandera for cases where
+runtime validation is genuinely necessary.
+
+---
+
+## License
+
+MIT License - see [LICENSE](LICENSE)
+
+---
+
+## Roadmap
+
+**Shipped:**
+- [x] Schema definition API
+- [x] Pandas support
+- [x] Polars support
+- [x] Mypy plugin
+- [x] Standalone checker (Rust)
+- [x] Explicit backend types
+- [x] Merge/join schema preservation
+- [x] Schema Composition (multiple inheritance, `SchemaA + SchemaB`)
+- [x] Column name collision warnings
+- [x] Pandera integration (`to_pandera_schema()`)
+
+**Planned:**
+
+- [ ] **Opt-in data loading constraints** - `Field` class with constraints (`gt`, `ge`, `lt`, `le`), strictly isolated
+  to `from_schema()` ingestion boundaries
+
+---
+
+## FAQ
+
+**Q: Do I need to choose between pandas and polars?**
+A: No. Define your schema once, use it with both. Just use the appropriate type (`PandasFrame` or `PolarsFrame`) in your function signatures.
+
+**Q: Does this replace Pandera?**
+A: No, it complements it. Use typedframes for static analysis, and `to_pandera_schema()` to convert your schemas to
+Pandera for runtime validation. See [Pandera Integration](#pandera-integration).
+
+**Q: Is the standalone checker required?**
+A: No. You can use just the mypy plugin, just the standalone checker, or both. They catch the same errors.
+
+**Q: What works without any plugin?**
+A: The `__getitem__` overloads on `PandasFrame` mean that any type checker (mypy, pyright, ty) understands
+`df[Schema.column]` returns `pd.Series` and `df[Schema.column_set]` returns `pd.DataFrame` — no plugin or stubs needed.
+Column *name* validation (catching typos like `df["revnue"]` in string-based access) still requires the standalone
+checker or mypy plugin.
+
+**Q: What about pyright/pylance users?**
+A: The mypy plugin doesn't work with pyright. Use the standalone checker (`typedframes check`) for column name
+validation. Schema descriptor access (`df[Schema.column]`) works natively in pyright without any plugin.
+
+**Q: Does this work with existing pandas/polars code?**
+A: Yes. You can gradually adopt typedframes by adding schemas to new code. Existing code continues to work.
+
+**Q: What if my column name conflicts with a pandas/polars method?**
+A: No problem. Since column access uses bracket syntax with schema descriptors (`df[Schema.mean]`), there is no conflict
+with DataFrame methods (`df.mean()`). Both work independently.
+
+---
+
+## Credits
+
+Built by developers who believe DataFrame bugs should be caught at lint-time, not in production.
+
+Inspired by the needs of ML/data science teams working with complex data pipelines.
+
+---
+
+**Questions? Issues? Ideas?** [Open an issue](https://github.com/yourusername/typedframes/issues)
+
+**Ready to catch DataFrame bugs before runtime?** `pip install typedframes`
+