typedframes 0.3.0__tar.gz → 0.4.0__tar.gz

{typedframes-0.3.0 → typedframes-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: typedframes
-Version: 0.3.0
+Version: 0.4.0
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
@@ -42,32 +42,32 @@ Project-URL: Repository, https://github.com/w-martin/typedframes
 
 > ⚠️ **Project Status: Proof of Concept**
 >
-> `typedframes` (v0.3.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
+> `typedframes` (v0.4.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
 > integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
-> schema validation over production-grade stability.
+> column checking over production-grade stability.
 >
-**Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime.**
+**A Rust-fast linter for pandas and polars DataFrames. Catches column errors at lint-time, and gates CI on how
+much of your codebase it can actually see — no schema classes required to start.**
 
 ```python
-from typing import Annotated
 import pandas as pd
-from typedframes import BaseSchema, Column
-
-
-class UserData(BaseSchema):
-    user_id = Column(type=int)
-    email = Column(type=str)
-    signup_date = Column(type=str)
 
+# Checker infers {order_id, amount, status} from usecols= — no schema class needed
+orders = pd.read_csv("orders.csv", usecols=["order_id", "amount", "status"])
+print(orders["amount"])  # ✓ OK
+print(orders["revenue"])  # ✗ unknown-column — 'revenue' not in inferred column set
+```
 
-df: Annotated[pd.DataFrame, UserData] = pd.read_csv("users.csv")
-df['user_id']    # ✓ Validated by checker
-df['username']   # ✗ unknown-column: Column 'username' not in UserData
+```shell
+typedframes check src/ --fail-under=90
+# src/pipeline.py:7:8: error[unknown-column] Column 'revenue' does not exist in inferred column set (defined at line 6)
+# ✗ Found 1 error in 12 files (0.0s)
+# ✗ DataFrame schema coverage 82.0% is below the required 90.0% (9/11 DataFrames had column info)
 ```
 
-**Descriptors as a bridge:** define once in `Column(type=int)`, access as `df[UserData.user_id.s]` (pandas string
-access) or `df.select(UserData.revenue.col)` (polars expression). Refactor by changing the descriptor definition —
-all `.s` and `.col` references update automatically. No find-and-replace across string literals.
+`--fail-under=N` is the same idea as a test-coverage or mypy type-coverage gate, applied to how many DataFrames
+the checker can resolve columns for — see [DataFrame Schema Coverage Thresholds](#dataframe-schema-coverage-thresholds-opt-in).
+Add `BaseSchema` classes later for cross-file awareness and IDE autocomplete — see [Quick Start](#quick-start).
 
 ---
 
@@ -78,6 +78,7 @@ all `.s` and `.col` references update automatically. No find-and-replace across
 - [Quick Start](#quick-start)
 - [Column Inference](#column-inference)
 - [Static Analysis](#static-analysis)
+- [DataFrame Schema Coverage Thresholds](#dataframe-schema-coverage-thresholds-opt-in)
 - [Static Analysis Performance](#static-analysis-performance)
 - [Type Safety With Multiple Backends](#type-safety-with-multiple-backends)
 - [Advanced Usage](#advanced-usage)
@@ -91,15 +92,21 @@ all `.s` and `.col` references update automatically. No find-and-replace across
 
 ## Why typedframes?
 
-**The problem:** Many pandas bugs are column mismatches. You access a column that doesn't exist, pass the wrong schema to a function, or make a typo. These errors only surface at runtime, often in production.
+**The problem:** Many pandas bugs are column mismatches — you access a column that doesn't exist, pass a
+DataFrame missing a column a function needs, or make a typo. These errors only surface at runtime, often in
+production, and it's hard to know how much of a codebase is actually protected against them.
 
-**The solution:** Define your DataFrame schemas as Python classes. Get static type checking that catches column errors before you even run your code.
+**The solution:** A fast standalone linter that infers column sets from your existing code (`usecols=`,
+`dtype=`, method chains) and catches mismatches at lint-time — plus an opt-in coverage threshold so CI fails
+when too much of the codebase is invisible to the checker. Add `BaseSchema` classes where you want cross-file
+tracking and IDE autocomplete; they're a progressive enhancement, not a prerequisite.
 
 **What you get:**
 
 - ✅ **Works without schema annotations** - Column inference from `usecols=`, `dtype=`, and method chains catches errors on unannotated code
+- ✅ **CI gate via coverage threshold** - `--fail-under=N` fails the build when too much of your codebase is invisible to the checker, the same way you'd gate on test coverage or a type checker's type-coverage number
+- ✅ **Rust-fast** - Milliseconds, not seconds, even on hundreds of files; fast enough for pre-commit hooks and CI (see [benchmarks](#static-analysis-performance))
 - ✅ **Cross-file awareness** - Add `BaseSchema` and typed return annotations to follow schemas across module boundaries
-- ✅ **Static analysis** - Catch column errors at lint-time with mypy or the standalone checker
 - ✅ **Refactor-safe access** - `df[Schema.column_group.s].mean()` (pandas) or `df.select(Schema.col.col)` (polars) instead of scattered string literals
 - ✅ **Works with pandas AND polars** - Same schema API, native backend types
 - ✅ **Dynamic column matching** - Regex-based ColumnSets for time-series data
@@ -134,7 +141,7 @@ import pandas as pd
 
 # Checker infers {order_id, amount, status} from usecols=
 orders = pd.read_csv("orders.csv", usecols=["order_id", "amount", "status"])
-print(orders["amount"])   # ✓ OK
+print(orders["amount"])  # ✓ OK
 print(orders["revenue"])  # ✗ unknown-column — 'revenue' not in inferred set
 ```
 
@@ -144,13 +151,17 @@ typedframes check src/
 # ✗ Found 1 error in 12 files (0.0s)
 ```
 
-See [`examples/multi_file_inference/`](examples/multi_file_inference/) for a multi-file example with no `BaseSchema`
+See [`examples/features/multi_file_inference/`](examples/features/multi_file_inference/) for a multi-file example with no `BaseSchema`
 classes at all.
 
 ### Define Your Schema (Once)
 
 Add `BaseSchema` classes when you want cross-file awareness and IDE autocomplete. Schemas travel with function return
-types across module boundaries — the checker validates call sites even in files that have no `usecols=` of their own:
+types across module boundaries — the checker validates call sites even in files that have no `usecols=` of their own.
+
+**Descriptors as a bridge:** define once in `Column(type=int)`, access as `df[UserData.user_id.s]` (pandas string
+access) or `df.select(UserData.revenue.col)` (polars expression). Refactor by changing the descriptor definition —
+all `.s` and `.col` references update automatically. No find-and-replace across string literals.
 
 ```python
 from typedframes import BaseSchema, Column, ColumnSet
@@ -175,17 +186,17 @@ import pandas as pd
 df: Annotated[pd.DataFrame, SalesData] = pd.read_csv("sales.csv")
 
 # String access — validated by the standalone checker
-print(df['revenue'].sum())
-print(df['profit'])  # ✗ unknown-column: Column 'profit' not in SalesData
+print(df["revenue"].sum())
+print(df["profit"])  # ✗ unknown-column: Column 'profit' not in SalesData
 
 # .s gives a refactor-safe string name from the descriptor
-print(df[SalesData.revenue.s].sum())   # same as df['revenue'].sum()
+print(df[SalesData.revenue.s].sum())  # same as df['revenue'].sum()
 
 
 # Type-safe function signature
 def analyze(data: Annotated[pd.DataFrame, SalesData]) -> float:
-    data['revenue']  # ✓ Validated by checker
-    data['profit']   # ✗ unknown-column: 'profit' not in SalesData
+    data["revenue"]  # ✓ Validated by checker
+    data["profit"]  # ✗ unknown-column: 'profit' not in SalesData
     return data[SalesData.revenue.s].mean()
 ```
 
@@ -199,12 +210,12 @@ import polars as pl
 df: Annotated[pl.DataFrame, SalesData] = pl.read_csv("sales.csv")
 
 # pl.col() references are now validated by the standalone checker
-print(df.filter(pl.col('revenue') > 1000))
-print(df.select(pl.col('profit')))  # ✗ unknown-column: Column 'profit' not in SalesData
+print(df.filter(pl.col("revenue") > 1000))
+print(df.select(pl.col("profit")))  # ✗ unknown-column: Column 'profit' not in SalesData
 
 # .col gives a refactor-safe polars expression from the descriptor
 filtered = df.filter(SalesData.revenue.col > 1000)
-grouped = df.group_by('customer_id').agg(SalesData.revenue.col.sum())
+grouped = df.group_by("customer_id").agg(SalesData.revenue.col.sum())
 ```
 
 ---
@@ -224,7 +235,7 @@ and validates all subscript access against it — no schema annotation required:
 ```python
 # Checker infers {user_id, email} from usecols= — no annotation needed
 df = pd.read_csv("users.csv", usecols=["user_id", "email"])
-print(df["user_id"])   # ✓ OK — in usecols
+print(df["user_id"])  # ✓ OK — in usecols
 # print(df["age"])     # ✗ Error: 'age' not in inferred column set
 ```
 
@@ -234,6 +245,7 @@ unchanged. Structural operations update it:
 
 ```python
 from typing import Annotated
+
 df: Annotated[pd.DataFrame, UserData] = pd.read_csv("users.csv")
 
 # Subscript slice — inferred column set {user_id, email}
@@ -255,29 +267,31 @@ print(augmented["created_at"])  # ✓ OK
 
 ### Inference Gaps and Warnings
 
-**untracked-dataframe — unannotated data ingestion (off by default)**
+**untracked-dataframe — unannotated data ingestion (on by default)**
 
-By default, typedframes supports permissive Exploratory Data Analysis (EDA). When a DataFrame is loaded
-via `pd.read_csv()` without `usecols=` or a schema annotation, the checker assumes an *Unknown* state
-and bypasses strict column validation to avoid nagging you during discovery.
+When a DataFrame is loaded via `pd.read_csv()` without `usecols=` or a schema annotation, the checker
+assumes an *Unknown* state, bypasses strict column validation on it (to avoid false positives on columns
+it simply can't see), and flags the load itself as a warning-level diagnostic.
 
-To lock down production CI/CD pipelines, opt in to `untracked-dataframe` warnings with `--strict-ingest`:
+For permissive Exploratory Data Analysis (EDA) work where you don't want that noise yet, downgrade it to
+a quiet info-level note with `--lenient-ingest`:
 
 ```shell
-typedframes check src/ --strict-ingest
+typedframes check src/ --lenient-ingest
 ```
 
-With strict ingestion enabled, loading a DataFrame without a schema or `usecols=` produces:
+By default, loading a DataFrame without a schema or `usecols=` produces:
 
 ```python
 df = pd.read_csv("users.csv")
-# ⚠ untracked-dataframe: columns unknown at lint time; specify `usecols`/`columns` or
-#   annotate: `df: Annotated[pd.DataFrame, MySchema] = pd.read_csv(...)`
+# ⚠ untracked-dataframe: columns unknown at lint time; specify `usecols`/`columns`, or
+#   annotate the variable's type, e.g. `df: Annotated[pd.DataFrame, MySchema] = pd.read_csv(...)`
 ```
 
 Fix option 1 — annotate with a schema:
 ```python
 from typing import Annotated
+
 df: Annotated[pd.DataFrame, UserData] = pd.read_csv("users.csv")
 ```
 
@@ -292,6 +306,7 @@ Emitted when `drop(columns=[...])` names a column that isn't in the inferred set
 
 ```python
 from typing import Annotated
+
 df: Annotated[pd.DataFrame, UserData] = pd.read_csv("users.csv")
 trimmed = df.drop(columns=["nonexistent"])
 # ⚠ dropped-unknown-column: Dropped column 'nonexistent' does not exist in UserData
@@ -323,15 +338,44 @@ The contract is resolved *transitively*: if a function only forwards its paramet
 requirements, catching a missing column even when no single function in the chain touches it directly.
 Column-list slices (`df[["a", "b"]]`) contribute to the contract too.
 
+**Known limitations:**
+- Cross-file delegate/schema resolution follows `from module import name`, plain `import module` +
+  `module.helper(df)` attribute access, and `from module import *` wildcard imports. A dotted import with no
+  alias (`import a.b.c`) only binds the first segment (`a`), matching Python's own binding rules, so a
+  deeply nested submodule accessed without an alias is not tracked.
+- Contract inference is a single top-to-bottom pass over a function body, not full control-flow analysis.
+  Deeply nested control flow (nested `try`/`except`, `match`, comprehensions) may under-report a function's
+  true requirements.
+- A cycle in the delegate graph (mutually- or self-delegating helpers) contributes only each function's own
+  direct requirements to the cycle, not the full transitive union — conservative rather than exhaustive.
+- If two plainly-imported modules both define a same-named function, an attribute-style delegate call
+  (`module.helper(df)`) resolves to whichever one is found first — the checker doesn't disambiguate by which
+  module the call site actually used.
+
 ### See Also
 
-- [`examples/inference_example.py`](examples/inference_example.py) — single-file walkthrough of all four inference
+- [`examples/features/inference_example.py`](examples/features/inference_example.py) — single-file walkthrough of all four inference
   scenarios with annotated ✓/✗ comments.
-- [`examples/multi_file_inference/`](examples/multi_file_inference/) — multi-file project checked with
-  `typedframes check examples/multi_file_inference/`; no `BaseSchema` anywhere. Includes a function
+- [`examples/features/multi_file_inference/`](examples/features/multi_file_inference/) — multi-file project checked with
+  `typedframes check examples/features/multi_file_inference/`; no `BaseSchema` anywhere. Includes a function
   parameter contract violation caught at the call site (`missing-column`).
-- [`examples/multi_file_with_schema/`](examples/multi_file_with_schema/) — same scenario with `BaseSchema`
+- [`examples/features/multi_file_with_schema/`](examples/features/multi_file_with_schema/) — same scenario with `BaseSchema`
   classes; the checker follows schemas across module boundaries via the project index.
+- SQL / data-warehouse column inference — the column set is inferred from a query's `SELECT` list instead of
+  `usecols=`/`columns=`, including tracing the query back through a single-assignment variable or a `.sql` file,
+  and dialect-aware identifier case folding (`sql_dialect` in `pyproject.toml` — see [Project-level
+  configuration](docs/api/cli.md#project-level-configuration)):
+  [`examples/sql_connectors/snowflake/`](examples/sql_connectors/snowflake/), [`examples/sql_connectors/bigquery/`](examples/sql_connectors/bigquery/),
+  [`examples/sql_connectors/athena/`](examples/sql_connectors/athena/), [`examples/sql_connectors/redshift/`](examples/sql_connectors/redshift/),
+  [`examples/sql_connectors/databricks/`](examples/sql_connectors/databricks/), [`examples/sql_connectors/pyspark/`](examples/sql_connectors/pyspark/),
+  [`examples/sql_connectors/duckdb/`](examples/sql_connectors/duckdb/), [`examples/sql_connectors/connectorx/`](examples/sql_connectors/connectorx/),
+  [`examples/sql_connectors/sqlalchemy/`](examples/sql_connectors/sqlalchemy/) (Core `select()` and declarative models, not just raw SQL text),
+  [`examples/sql_connectors/feast/`](examples/sql_connectors/feast/) (feature-store retrieval, registered as an *open* schema since
+  `entity_df`'s own columns aren't enumerable in general), and
+  [`examples/sql_connectors/azure_synapse/`](examples/sql_connectors/azure_synapse/) (Azure's closest analog to Athena, including T-SQL's
+  `[bracket-quoted]` identifier convention). A wrapper function that case-folds a connector's result before
+  returning it (`.rename(columns=str.lower)`, `df.columns = df.columns.str.lower()`) is traced cross-file
+  too — see [docs/usage.md's "Supported column-set transforms"](docs/usage.md#supported-column-set-transforms).
 
 ---
 
@@ -376,13 +420,19 @@ typedframes check src/
 typedframes check src/ --strict
 
 # JSON output
-typedframes check src/ --json
+typedframes check src/ --output-format=json
 
 # Skip cross-file index (single-file mode, faster for quick checks)
 typedframes check src/ --no-index
 
 # Suppress all warnings (untracked-dataframe, dropped-unknown-column)
 typedframes check src/ --no-warnings
+
+# Enforce minimum DataFrame schema coverage (see below)
+typedframes check src/ --fail-under=90
+
+# Show which DataFrames lack column info, per file
+typedframes check src/ --coverage-report=term-missing
 ```
 
 To suppress warnings project-wide, add to `pyproject.toml`:
@@ -432,21 +482,126 @@ code reference.
 
 ---
 
+## DataFrame Schema Coverage Thresholds (Opt-In)
+
+**DataFrame schema coverage** is the fraction of DataFrames `typedframes check` could
+resolve column information for — the analogue of the "type coverage" reported by mypy,
+pyright, and pyre, and unrelated to test coverage. That number is informational by
+default. If you want it enforced — failing the run when too much of your code is
+invisible to the checker — enable a threshold.
+
+This is **entirely opt-in**. With no `[tool.typedframes.coverage]` table and no
+`--fail-under`, nothing changes: no threshold, no exit-code difference.
+
+Every supported key, at its default value:
+
+```toml
+[tool.typedframes.coverage]
+# Master switch. Coverage enforcement is off unless this is true, so adding this
+# table without setting it changes nothing.
+enabled = false
+
+# Minimum percentage of DataFrames that must have recognized column/schema info
+# before `typedframes check` exits 1. Only consulted when `enabled = true`.
+# Applies to every file not captured by a glob in [overrides] below.
+fail_under = 100.0
+
+# How much coverage detail to print after each check. One of:
+#   "summary"      one line (the default, unchanged from before this feature)
+#   "term-missing" per-file table plus the DataFrame sites lacking column info
+#   "json"         machine-readable document, for CI tooling
+# Independent of `enabled` — a detailed report is useful without a gate, and
+# vice versa. Overridden by `--coverage-report`.
+report = "summary"
+
+[tool.typedframes.coverage.overrides]
+# Per-path glob overrides of `fail_under`, for holding legacy code to a lower bar
+# than new code. Each glob is graded on its own files as a separate group, so a
+# lenient legacy bucket can't drag down (or rescue) the rest of the project.
+# Paths are matched project-relative: `**` spans any number of directories,
+# `*` and `?` stay within one path segment.
+# When several globs match one file the most specific wins — longest literal
+# prefix before the first `*` or `?`. Files matching no glob use `fail_under`.
+# "legacy/**" = 50.0
+# "src/new_module/**" = 100.0
+```
+
+Prefer to keep `pyproject.toml` clean? The same settings work in a standalone
+`typedframes.toml` at the project root, with the `[tool.typedframes]` prefix dropped
+(the way `ruff.toml` drops `[tool.ruff]`):
+
+```toml
+# typedframes.toml
+[coverage]
+enabled = false
+fail_under = 100.0
+report = "summary"
+
+[coverage.overrides]
+# "legacy/**" = 50.0
+```
+
+If both files exist, `typedframes.toml` wins **entirely** — the two are never merged,
+so exactly one file explains the whole configuration.
+
+### Seeing What's Missing
+
+The default one-line DataFrame schema coverage summary tells you the ratio but not what
+to fix. `--coverage-report=term-missing` names the DataFrames that cost you coverage:
+
+```shell
+typedframes check src/ --coverage-report=term-missing
+```
+
+```
+Name           Typed  Total   Cover   Missing
+---------------------------------------------
+legacy/old.py      0      2      0%   old_one:2, old_two:3
+src/new.py         1      2     50%   bad:3
+---------------------------------------------
+TOTAL              1      4     25%
+```
+
+Each `Missing` entry is `variable:line` — the assignment where the checker recognized a
+DataFrame but couldn't resolve its columns. Fix those (add `usecols=`, name the columns
+in the `SELECT`, or annotate the variable) and coverage rises.
+
+For CI tooling, `--coverage-report=json` emits the same data as a document. Combine it
+with `--output-format=json` and the coverage report is nested under a `coverage` key so
+stdout stays a single valid JSON document:
+
+```shell
+typedframes check src/ --output-format=json --coverage-report=json
+```
+
+Notes:
+
+- Coverage is a **separate gate from `--strict`**. `--strict` fails on errors
+  (correctness); a threshold fails on missing column information (completeness).
+  Enabling one never implies the other.
+- `--fail-under=N` is a **total override**: it applies one threshold everywhere and
+  ignores the config table, per-path overrides included. Handy for a one-off CI run.
+- A group with no recognized DataFrames passes: 0/0 means there was nothing to
+  measure, not that something failed.
+- A failed threshold exits **1** and is reported even under `--no-info` — that flag
+  silences the informational summary line, not a gate result.
+
 ## Static Analysis Performance
 
 Fast feedback reduces development time. The typedframes Rust binary provides near-instant column checking.
 
-**Benchmark results** (10 runs, 3 warmup, caches cleared between runs):
+**Benchmark results** (20 runs, 3 warmup, caches cleared between runs):
+*2026-08-12 · Darwin 25.6.0 · arm · CPython 3.14.4 · 64GiB RAM · Great Expectations pinned @ 1.9.3*
 
-| Tool | Version | What it does | typedframes (13 files) | great_expectations (485 files) |
+| Tool | Version | What it does | typedframes (13 files) | great_expectations (482 files) |
 |------|---------|--------------|------------------------|--------------------------------|
-| typedframes | 0.3.0 | DataFrame column checker | 81ms ±6ms | 3.64s ±9ms |
-| ruff | 0.15.21 | Linter (no type checking) | 48ms ±13ms | 237ms ±31ms |
-| ty | 0.0.58 | Type checker | 119ms ±14ms | 287ms ±33ms |
-| pyrefly | 1.1.1 | Type checker | 162ms ±18ms | 390ms ±21ms |
-| mypy | 2.2.0 | Type checker (no plugin) | 4.56s ±22ms | 7.31s ±35ms |
-| mypy + typedframes | 2.2.0 | Type checker + column checker | 4.69s ±61ms | 10.89s ±356ms |
-| pyright | 1.1.411 | Type checker | 1.39s ±17ms | 8.06s ±193ms |
+| typedframes | 0.4.0 | DataFrame column checker | 47ms ±998µs (IQR 1ms) | 183ms ±2ms (IQR 2ms) |
+| ruff | 0.16.2 | Linter (no type checking) | 29ms ±677µs (IQR 729µs) | 208ms ±2ms (IQR 3ms) |
+| ty | 0.0.69 | Type checker | 74ms ±900µs (IQR 1ms) | 941ms ±14ms (IQR 18ms) |
+| pyrefly | 1.2.0 | Type checker | 96ms ±2ms (IQR 2ms) | 275ms ±7ms (IQR 14ms) |
+| mypy | 2.3.0 | Type checker (no plugin) | 2.78s ±47ms (IQR 84ms) | 4.25s ±48ms (IQR 76ms) |
+| mypy + typedframes | 2.3.0 | Type checker + column checker | 2.74s ±29ms (IQR 34ms) | 4.44s ±54ms (IQR 105ms) |
+| pyright | 1.1.411 | Type checker | 808ms ±24ms (IQR 21ms) | 3.45s ±93ms (IQR 138ms) |
 
 *Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
 
@@ -481,12 +636,12 @@ class UserData(BaseSchema):
 
 # Pandas pipeline - type checker knows pandas methods
 def pandas_analyze(df: Annotated[pd.DataFrame, UserData]) -> Annotated[pd.DataFrame, UserData]:
-    return df[df['user_id'] > 100]  # ✓ Pandas syntax
+    return df[df["user_id"] > 100]  # ✓ Pandas syntax
 
 
 # Polars pipeline - type checker knows polars methods
 def polars_analyze(df: Annotated[pl.DataFrame, UserData]) -> Annotated[pl.DataFrame, UserData]:
-    return df.filter(pl.col('user_id') > 100)  # ✓ Polars syntax
+    return df.filter(pl.col("user_id") > 100)  # ✓ Polars syntax
 
 
 # Use native types throughout
@@ -526,7 +681,7 @@ class OrderSchema(BaseSchema):
 
 # Schema preserved through filtering
 def get_active_users(df: Annotated[pd.DataFrame, UserSchema]) -> Annotated[pd.DataFrame, UserSchema]:
-    return df[df['user_id'] > 100]  # ✓ Validated by checker
+    return df[df["user_id"] > 100]  # ✓ Validated by checker
 
 
 # Schema preserved through merges
@@ -543,7 +698,7 @@ import polars as pl
 
 # Schema columns work in filter expressions
 def filter_users(df: Annotated[pl.DataFrame, UserSchema]) -> pl.DataFrame:
-    return df.filter(pl.col('user_id') > 100)
+    return df.filter(pl.col("user_id") > 100)
 
 
 # Schema columns work in join expressions
@@ -582,7 +737,7 @@ class SensorReadings(BaseSchema):
 
 
 df: Annotated[pd.DataFrame, SensorReadings] = pd.read_csv("readings.csv")
-df[SensorReadings.sensors.s].mean()    # ✓ Expands to df[["sensor_1", "sensor_2", "sensor_3"]].mean()
+df[SensorReadings.sensors.s].mean()  # ✓ Expands to df[["sensor_1", "sensor_2", "sensor_3"]].mean()
 ```
 
 For logical grouping across multiple ColumnSets:
@@ -633,6 +788,7 @@ class Orders(BaseSchema):
 # Combine via multiple inheritance
 class UserOrders(UserPublic, Orders):
     """Type checkers see all columns from both parents."""
+
     ...
 
 
@@ -646,7 +802,7 @@ merged: Annotated[pd.DataFrame, UserOrders] = users.merge(orders, on=UserPublic.
 
 Overlapping columns with the same type are allowed (common after merges). Conflicting types raise `SchemaConflictError`.
 
-See [`examples/schema_algebra_example.py`](examples/schema_algebra_example.py) for a complete walkthrough.
+See [`examples/features/schema_algebra_example.py`](examples/features/schema_algebra_example.py) for a complete walkthrough.
 
 ---
 
@@ -659,12 +815,12 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 
 | Feature                         | typedframes            | Pandera     | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame      | narwhals | dataframely      | patito           |
 |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|------------------|------------------|
-| **Version tested**              | 0.3.0                  | 0.32.1      | 1.18.2             | 0.3.7                 | 3.0.3        | 0.1.2       | 1.1.3              | 5.0.0            | 2.23.0   | 2.13.0           | 0.8.6            |
+| **Version tested**              | 0.4.0                  | 0.32.1      | 1.19.1             | 0.3.7                 | 3.0.5        | 0.1.2       | 1.1.3              | 5.0.0            | 2.24.0   | 3.0.0            | 0.8.6            |
 | **Analysis Type**               |
 | When errors are caught          | **Static (lint-time)** | Runtime     | Runtime            | Runtime               | Static       | Runtime     | Runtime            | Runtime          | Runtime  | Runtime          | Runtime          |
 | **Static Analysis (our focus)** |
 | Mypy plugin                     | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ❌ No                  | ✅ Yes        | ❌ No        | ❌ No               | ⚠️ Basic         | ❌ No     | ❌ No             | ❌ No             |
-| Standalone checker              | ✅ Rust (~1ms)          | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     | ❌ No             | ❌ No             |
+| Standalone checker              | ✅ Rust (ms-scale)      | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     | ❌ No             | ❌ No             |
 | Column name checking            | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     | ❌ No             | ❌ No             |
 | Column type checking            | ✅ Yes                  | ⚠️ Limited  | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     | ❌ No             | ❌ No             |
 | Typo suggestions                | ✅ Yes                  | ❌ No        | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ❌ No     | ❌ No             | ❌ No             |
@@ -678,7 +834,7 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
 | Pandas                          | ✅ Yes                  | ✅ Yes       | ✅ Yes              | ✅ Yes                 | ✅ Yes        | ✅ Yes       | ✅ Yes              | ❌ Own            | ✅ Yes    | ❌ No             | ⚠️ Limited        |
 | Polars                          | ✅ Yes                  | ✅ Yes       | ❌ No               | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ Own            | ✅ Yes    | ✅ Yes (only)     | ✅ Yes            |
 | DuckDB, cuDF, etc.              | ❌ No                   | ❌ No        | ✅ Spark, SQL       | ❌ No                  | ❌ No         | ❌ No        | ❌ No               | ❌ No             | ✅ Yes    | ❌ No             | ❌ No             |
-| **Project Status (Feb 2026)**   |
+| **Project Status (Aug 2026)**   |
 | Active development              | ✅ Yes                  | ✅ Yes       | ✅ Yes              | ⚠️ Low                | ✅ Yes        | ❌ Inactive  | ⚠️ Low             | ✅ Yes            | ✅ Yes    | ✅ Yes            | ✅ Yes            |
 
 **Legend:** ✅ Full support | ⚠️ Limited/Partial | ❌ Not supported
@@ -693,11 +849,13 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
   hints for runtime validation via typeguard. Despite documentation implying mypy support, there is no mypy plugin —
   column access errors are not caught statically. No standalone checker. No polars support.
 
-- **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.3): Official pandas type stubs. Provides
+- **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.5): Official pandas type stubs. Provides
   API-level types but no column-level checking.
 
-- **[dataenforce](https://github.com/CedricFR/dataenforce)** (v0.1.2): Runtime validation via decorator. Marked as
-  experimental/not production-ready. Appears inactive. Broken on Python 3.13+ due to removal of internal typing APIs.
+- **[dataenforce](https://github.com/CedricFR/dataenforce)** (v0.1.2, the only release ever published): Runtime
+  validation via decorator. Appears inactive/abandoned. Broken on every currently-supported Python version
+  (3.11 through 3.14) due to removal of internal typing APIs (`typing._TypingEmpty`) it depends on — confirmed
+  working only as far back as Python 3.9.
 
 - **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
   static analysis.
@@ -706,43 +864,44 @@ Comprehensive comparison of pandas/DataFrame typing and validation tools. **type
   Not compatible with pandas/polars — requires a full rewrite to StaticFrame's own API. Column access is still
   string-based; mypy does not catch column name typos. Type safety comes from immutability guarantees, not schema checking.
 
-- **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.23.0): Compatibility layer that provides a unified API
+- **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.24.0): Compatibility layer that provides a unified API
   across pandas, polars, DuckDB, cuDF, and more. Solves a different problem—write-once-run-anywhere portability, not
   type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
   below.
 
-- **[Great Expectations](https://greatexpectations.io/)** (v1.18.2): Comprehensive data quality framework. Defines
+- **[Great Expectations](https://greatexpectations.io/)** (v1.19.1): Comprehensive data quality framework. Defines
   "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
   validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
   code. Supports pandas, Spark, and SQL backends.
 
-- **[dataframely](https://github.com/Quantco/dataframely)** (v2.13.0): Polars-only runtime validation library from Quantco.
+- **[dataframely](https://github.com/Quantco/dataframely)** (v3.0.0): Polars-only runtime validation library from Quantco.
   Schemas are defined as classes inheriting `dy.Schema` with typed descriptor fields (`dy.String()`, `dy.Float64()`)
   and `@dy.rule()` decorators for cross-column and group-level constraints. Returns `dy.DataFrame[Schema]` generic
   types that give call-site narrowing to type checkers, but does not validate column subscript access inside function
-  bodies. No lint-time or static analysis capability. Supports nullability, string constraints, numeric bounds,
-  cross-column rules, soft validation, test data generation, and SQLAlchemy/PyArrow export.
+  bodies, and (as of 3.0) that narrowing doesn't even survive a `.filter()` call — it returns a plain `pl.DataFrame`.
+  3.0 also removed the `dy.Series` type entirely; column access now returns a plain `pl.Series`. No lint-time or
+  static analysis capability. Supports nullability, string constraints, numeric bounds, cross-column rules, soft
+  validation, test data generation, and SQLAlchemy/PyArrow export.
 
 - **[patito](https://github.com/JakobGM/patito)** (v0.8.6): Runtime validation library using a Pydantic-style `patito.Model`
   class. Polars is the primary backend; pandas is supported but works by converting to Polars via PyArrow (an
-  undeclared dependency). DuckDB is not supported despite appearing in some documentation—validation crashes immediately
-  on DuckDB relations. No static analysis or standalone checker.
+  undeclared dependency). No static analysis or standalone checker.
 
 ### Type Checkers (Not DataFrame-Specific)
 
 These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
 typedframes for comprehensive type checking:
 
-- **[mypy](https://mypy-lang.org/)** (v1.19.1): The original Python type checker. typedframes provides a mypy plugin for
+- **[mypy](https://mypy-lang.org/)** (v2.3.0): The original Python type checker. typedframes provides a mypy plugin for
   column checking. See [performance benchmarks](#static-analysis-performance).
 
-- **[ty](https://github.com/astral-sh/ty)** (v0.0.19, Astral): New Rust-based type checker, 10-60x faster than mypy on
+- **[ty](https://github.com/astral-sh/ty)** (v0.0.69, Astral): New Rust-based type checker, faster than mypy on
   large codebases. Does not support mypy plugins—use typedframes standalone checker.
 
-- **[pyrefly](https://pyrefly.org/)** (v0.54.0, Meta): Rust-based type checker from Meta, replacement for Pyre. Fast,
+- **[pyrefly](https://pyrefly.org/)** (v1.2.0, Meta): Rust-based type checker from Meta, replacement for Pyre. Fast,
   but no DataFrame column checking.
 
-- **[pyright](https://github.com/microsoft/pyright)** (v1.1.408, Microsoft): Type checker powering Pylance/VSCode. No
+- **[pyright](https://github.com/microsoft/pyright)** (v1.1.411, Microsoft): Type checker powering Pylance/VSCode. No
   mypy plugin support—use typedframes standalone checker.
 
 ### Not Directly Comparable
@@ -784,9 +943,9 @@ import pandas as pd
 
 
 class UserData(BaseSchema):
-  user_id = Column(type=int)
-  email = Column(type=str)
-  age = Column(type=int, nullable=True)
+    user_id = Column(type=int)
+    email = Column(type=str)
+    age = Column(type=int, nullable=True)
 
 
 # Convert to pandera schema
@@ -824,7 +983,7 @@ class Orders(BaseSchema):
 
 
 def calculate_revenue(orders: Annotated[pd.DataFrame, Orders]) -> float:
-    return orders['total'].sum()
+    return orders["total"].sum()
 
 
 df: Annotated[pd.DataFrame, Orders] = pd.read_csv("orders.csv")
@@ -876,11 +1035,17 @@ class AggregatedSales(BaseSchema):
 
 
 def aggregate_daily(df: Annotated[pd.DataFrame, RawSales]) -> Annotated[pd.DataFrame, AggregatedSales]:
-    result = df.groupby(RawSales.date.s).agg({
-        RawSales.price.s: 'sum',
-        RawSales.quantity.s: 'sum',
-    }).reset_index()
-    result.columns = pd.Index(['date', 'total_revenue', 'total_quantity'])
+    result = (
+        df.groupby(RawSales.date.s)
+        .agg(
+            {
+                RawSales.price.s: "sum",
+                RawSales.quantity.s: "sum",
+            }
+        )
+        .reset_index()
+    )
+    result.columns = pd.Index(["date", "total_revenue", "total_quantity"])
     return result  # type: ignore[return-value]
 
 
@@ -891,8 +1056,8 @@ aggregated = aggregate_daily(raw)
 
 # Type checker validates schema transformations
 def analyze(df: Annotated[pd.DataFrame, AggregatedSales]) -> float:
-    df['total_revenue']  # ✓ OK
-    df['price']          # ✗ Error: 'price' not in AggregatedSales
+    df["total_revenue"]  # ✓ OK
+    df["price"]  # ✗ Error: 'price' not in AggregatedSales
     return df[AggregatedSales.total_revenue.s].mean()
 ```
 
@@ -911,11 +1076,7 @@ class LargeDataset(BaseSchema):
 
 
 def efficient_aggregation(df: Annotated[pl.DataFrame, LargeDataset]) -> pl.DataFrame:
-    return (
-        df.filter(pl.col('value') > 100)
-        .group_by('category')
-        .agg(pl.col('value').mean())
-    )
+    return df.filter(pl.col("value") > 100).group_by("category").agg(pl.col("value").mean())
 
 
 # Polars handles large files efficiently
@@ -967,6 +1128,7 @@ However, abstraction layers don't provide column-level type safety:
 ```python
 import narwhals as nw
 
+
 def process(df: nw.DataFrame) -> nw.DataFrame:
     # No static checking - "revenue" typo won't be caught until runtime
     return df.filter(nw.col("revnue") > 100)  # Typo: "revnue" vs "revenue"
@@ -981,11 +1143,13 @@ from typing import Annotated
 import polars as pl
 from typedframes import BaseSchema, Column
 
+
 class SalesData(BaseSchema):
     revenue = Column(type=float)
 
+
 def process(df: Annotated[pl.DataFrame, SalesData]) -> pl.DataFrame:
-    return df.filter(pl.col('revnue') > 100)  # ✗ Error at lint-time: 'revnue' not in SalesData
+    return df.filter(pl.col("revnue") > 100)  # ✗ Error at lint-time: 'revnue' not in SalesData
 ```
 
 **Use narwhals when:** You're writing a library that needs to work with multiple DataFrame backends.
@@ -1023,6 +1187,10 @@ MIT License - see [LICENSE](LICENSE)
 - [x] Aggressive column inference (untracked-dataframe/dropped-unknown-column warnings, method chain propagation)
 - [x] Function parameter contracts (`missing-column`), resolved transitively across chains of helper
   functions and cross-file calls; schema-annotated parameters take priority over body-scanning
+- [x] SQL / data-warehouse column inference (`SELECT` list parsing across Snowflake, BigQuery, Athena, Redshift,
+  Databricks, PySpark, DuckDB, SQLAlchemy Core/ORM, Feast, and T-SQL/Synapse/Fabric dialects)
+- [x] Jupyter notebook (`.ipynb`) checking — code cells are checked directly, with errors reported as
+  `notebook.ipynb:cell N:line:col`
 
 **Planned:**
 
@@ -1058,7 +1226,7 @@ calls give it enough information to validate column access and propagate that kn
 (`rename`, `drop`, `assign`, `select`, …). `BaseSchema` is a progressive enhancement that unlocks cross-file
 awareness (schemas travel with function return types across module boundaries) and IDE autocomplete via
 descriptors — but the checker catches real column errors from day one without it. See
-[`examples/multi_file_inference/`](examples/multi_file_inference/) for a complete demo with no schema classes.
+[`examples/features/multi_file_inference/`](examples/features/multi_file_inference/) for a complete demo with no schema classes.
 
 **Q: Does this work with existing pandas/polars code?**
 A: Yes. You can gradually adopt typedframes by adding schemas to new code. Existing code continues to work.