typedframes 0.2.1__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (24) hide show
  1. typedframes-0.2.1/README.md → typedframes-0.3.0/PKG-INFO +121 -43
  2. typedframes-0.2.1/PKG-INFO → typedframes-0.3.0/README.md +86 -67
  3. {typedframes-0.2.1 → typedframes-0.3.0}/pyproject.toml +36 -7
  4. {typedframes-0.2.1 → typedframes-0.3.0}/rust/Cargo.lock +17 -47
  5. {typedframes-0.2.1 → typedframes-0.3.0}/rust/Cargo.toml +2 -2
  6. {typedframes-0.2.1 → typedframes-0.3.0}/rust/src/lib.rs +1679 -166
  7. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/__init__.py +1 -1
  8. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/pandas.py +2 -2
  9. {typedframes-0.2.1 → typedframes-0.3.0}/rust/README.md +0 -0
  10. {typedframes-0.2.1 → typedframes-0.3.0}/rust/benches/parser_bench.rs +0 -0
  11. {typedframes-0.2.1 → typedframes-0.3.0}/rust/src/main.rs +0 -0
  12. {typedframes-0.2.1 → typedframes-0.3.0}/rust/tests/integration_test.rs +0 -0
  13. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/base_schema.py +0 -0
  14. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/cli.py +0 -0
  15. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/column.py +0 -0
  16. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/column_group.py +0 -0
  17. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/column_group_error.py +0 -0
  18. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/column_set.py +0 -0
  19. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/missing_dependency_error.py +0 -0
  20. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/mypy.py +0 -0
  21. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/pandera.py +0 -0
  22. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/polars.py +0 -0
  23. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/py.typed +0 -0
  24. {typedframes-0.2.1 → typedframes-0.3.0}/src/typedframes/schema_algebra.py +0 -0
@@ -1,14 +1,48 @@
1
+ Metadata-Version: 2.4
2
+ Name: typedframes
3
+ Version: 0.3.0
4
+ Classifier: Development Status :: 3 - Alpha
5
+ Classifier: Intended Audience :: Developers
6
+ Classifier: License :: OSI Approved :: MIT License
7
+ Classifier: Operating System :: OS Independent
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Programming Language :: Python :: 3.11
10
+ Classifier: Programming Language :: Python :: 3.12
11
+ Classifier: Programming Language :: Python :: 3.13
12
+ Classifier: Programming Language :: Python :: 3.14
13
+ Classifier: Topic :: Software Development :: Quality Assurance
14
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
15
+ Requires-Dist: mypy ; extra == 'mypy'
16
+ Requires-Dist: pandas>=2.0 ; extra == 'pandas'
17
+ Requires-Dist: pandera>=0.20.0 ; extra == 'pandera'
18
+ Requires-Dist: polars>=1.0 ; extra == 'polars'
19
+ Provides-Extra: mypy
20
+ Provides-Extra: pandas
21
+ Provides-Extra: pandera
22
+ Provides-Extra: polars
23
+ Summary: Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime.
24
+ Keywords: pandas,polars,type-checking,static-analysis,dataframe,linter,mypy-plugin
25
+ Home-Page: https://github.com/w-martin/typedframes
26
+ Author: William Martin
27
+ License-Expression: MIT
28
+ Requires-Python: >=3.11
29
+ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
30
+ Project-URL: Documentation, https://typedframes.readthedocs.io/en/latest/
31
+ Project-URL: Homepage, https://github.com/w-martin/typedframes
32
+ Project-URL: Issue Tracker, https://github.com/w-martin/typedframes/issues
33
+ Project-URL: Repository, https://github.com/w-martin/typedframes
34
+
1
35
  # typedframes
2
36
 
3
37
  [![CI](https://github.com/w-martin/typedframes/actions/workflows/publish.yml/badge.svg)](https://github.com/w-martin/typedframes/actions/workflows/publish.yml)
4
38
  [![PyPI version](https://img.shields.io/pypi/v/typedframes.svg)](https://pypi.org/project/typedframes/)
5
39
  [![Python versions](https://img.shields.io/pypi/pyversions/typedframes.svg)](https://pypi.org/project/typedframes/)
6
- [![Coverage](https://codecov.io/gh/w-martin/typedframes/branch/main/graph/badge.svg)](https://codecov.io/gh/w-martin/typedframes)
40
+ [![Coverage](https://coveralls.io/repos/github/w-martin/typedframes/badge.svg?branch=main)](https://coveralls.io/github/w-martin/typedframes?branch=main)
7
41
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
8
42
 
9
43
  > ⚠️ **Project Status: Proof of Concept**
10
44
  >
11
- > `typedframes` (v0.2.1) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
45
+ > `typedframes` (v0.3.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
12
46
  > integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
13
47
  > schema validation over production-grade stability.
14
48
  >
@@ -106,7 +140,7 @@ print(orders["revenue"]) # ✗ unknown-column — 'revenue' not in inferred set
106
140
 
107
141
  ```shell
108
142
  typedframes check src/
109
- # src/pipeline.py:7:8: error[unknown-column] Column 'revenue' not in inferred set {order_id, amount, status}
143
+ # src/pipeline.py:7:8: error[unknown-column] Column 'revenue' does not exist in inferred column set (defined at line 6)
110
144
  # ✗ Found 1 error in 12 files (0.0s)
111
145
  ```
112
146
 
@@ -263,12 +297,39 @@ trimmed = df.drop(columns=["nonexistent"])
263
297
  # ⚠ dropped-unknown-column: Dropped column 'nonexistent' does not exist in UserData
264
298
  ```
265
299
 
300
+ ### Function Parameter Contracts
301
+
302
+ Beyond validating access at the point it happens, the checker infers a *contract* for any function's
303
+ first parameter: every column the function needs, drawn from what its body accesses or — taking priority —
304
+ from a schema annotation on the parameter itself. Calling that function with a DataFrame that doesn't
305
+ satisfy the contract is caught at the **call site**, across files:
306
+
307
+ ```python
308
+ # transforms.py
309
+ def contact_label(customers):
310
+ return customers["name"] + customers["email"]
311
+ ```
312
+
313
+ ```python
314
+ # pipeline.py
315
+ customers = load_customers(path) # inferred columns: {customer_id, name, region}
316
+ contact_label(customers)
317
+ # ✗ missing-column: 'customers' passed to contact_label (transforms.py:2) is missing
318
+ # column(s) {email} — available: {customer_id, name, region}, required: {email, name}
319
+ ```
320
+
321
+ The contract is resolved *transitively*: if a function only forwards its parameter to other functions
322
+ (`step1 = preprocess(df); step2 = enrich(step1)`), the checker follows the chain and unions their
323
+ requirements, catching a missing column even when no single function in the chain touches it directly.
324
+ Column-list slices (`df[["a", "b"]]`) contribute to the contract too.
325
+
266
326
  ### See Also
267
327
 
268
328
  - [`examples/inference_example.py`](examples/inference_example.py) — single-file walkthrough of all four inference
269
329
  scenarios with annotated ✓/✗ comments.
270
330
  - [`examples/multi_file_inference/`](examples/multi_file_inference/) — multi-file project checked with
271
- `typedframes check examples/multi_file_inference/`; no `BaseSchema` anywhere.
331
+ `typedframes check examples/multi_file_inference/`; no `BaseSchema` anywhere. Includes a function
332
+ parameter contract violation caught at the call site (`missing-column`).
272
333
  - [`examples/multi_file_with_schema/`](examples/multi_file_with_schema/) — same scenario with `BaseSchema`
273
334
  classes; the checker follows schemas across module boundaries via the project index.
274
335
 
@@ -293,8 +354,10 @@ typedframes check src/
293
354
  **Features:**
294
355
  - Catches column name errors
295
356
  - Validates schema mismatches between functions
357
+ - Validates function parameter contracts across files, including transitively through chains of
358
+ helper functions (`missing-column`)
296
359
  - Checks both pandas and polars code
297
- - 10-100x faster than mypy
360
+ - Significantly faster than mypy (see benchmarks below)
298
361
 
299
362
  **Use this for:**
300
363
  - Fast feedback during development
@@ -375,17 +438,15 @@ Fast feedback reduces development time. The typedframes Rust binary provides nea
375
438
 
376
439
  **Benchmark results** (10 runs, 3 warmup, caches cleared between runs):
377
440
 
378
- | Tool | Version | What it does | typedframes (13 files) | great_expectations (490 files)† |
379
- |--------------------|---------|-------------------------------|------------------------|---------------------------------|
380
- | typedframes | 0.2.0 | DataFrame column checker | 9ms ±2ms | 930µs ±89µs |
381
- | ruff | 0.15.4 | Linter (no type checking) | 64ms ±16ms | 360ms ±18ms |
382
- | ty | 0.0.19 | Type checker | 115ms ±22ms | 1.65s ±26ms |
383
- | pyrefly | 0.54.0 | Type checker | 3.78s ±7.53s | 693ms ±33ms |
384
- | mypy | 1.19.1 | Type checker (no plugin) | 13.85s ±1.08s | 12.13s ±400ms |
385
- | mypy + typedframes | 1.19.1 | Type checker + column checker | 13.51s ±273ms | 13.89s ±491ms |
386
- | pyright | 1.1.408 | Type checker | 2.10s ±422ms | 8.37s ±253ms |
387
-
388
- *† great_expectations column from previous benchmark run.*
441
+ | Tool | Version | What it does | typedframes (13 files) | great_expectations (485 files) |
442
+ |------|---------|--------------|------------------------|--------------------------------|
443
+ | typedframes | 0.3.0 | DataFrame column checker | 81ms ±6ms | 3.64s ±9ms |
444
+ | ruff | 0.15.21 | Linter (no type checking) | 48ms ±13ms | 237ms ±31ms |
445
+ | ty | 0.0.58 | Type checker | 119ms ±14ms | 287ms ±33ms |
446
+ | pyrefly | 1.1.1 | Type checker | 162ms ±18ms | 390ms ±21ms |
447
+ | mypy | 2.2.0 | Type checker (no plugin) | 4.56s ±22ms | 7.31s ±35ms |
448
+ | mypy + typedframes | 2.2.0 | Type checker + column checker | 4.69s ±61ms | 10.89s ±356ms |
449
+ | pyright | 1.1.411 | Type checker | 1.39s ±17ms | 8.06s ±193ms |
389
450
 
390
451
  *Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
391
452
 
@@ -596,63 +657,77 @@ See [`examples/schema_algebra_example.py`](examples/schema_algebra_example.py) f
596
657
  Comprehensive comparison of pandas/DataFrame typing and validation tools. **typedframes focuses on static analysis**
597
658
  —catching errors at lint-time before your code runs.
598
659
 
599
- | Feature | typedframes | Pandera | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame | narwhals |
600
- |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|
601
- | **Version tested** | 0.2.0 | 0.29.0 | 1.4.3 | 0.3.6 | 3.0.0 | 0.1.2 | 1.1.3 | 3.7.0 | 2.16.0 |
660
+ | Feature | typedframes | Pandera | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame | narwhals | dataframely | patito |
661
+ |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|------------------|------------------|
662
+ | **Version tested** | 0.3.0 | 0.32.1 | 1.18.2 | 0.3.7 | 3.0.3 | 0.1.2 | 1.1.3 | 5.0.0 | 2.23.0 | 2.13.0 | 0.8.6 |
602
663
  | **Analysis Type** |
603
- | When errors are caught | **Static (lint-time)** | Runtime | Runtime | Static + Runtime | Static | Runtime | Runtime | Static + Runtime | Runtime |
664
+ | When errors are caught | **Static (lint-time)** | Runtime | Runtime | Runtime | Static | Runtime | Runtime | Runtime | Runtime | Runtime | Runtime |
604
665
  | **Static Analysis (our focus)** |
605
- | Mypy plugin | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ⚠️ Basic | ❌ No |
606
- | Standalone checker | ✅ Rust (~1ms) | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
607
- | Column name checking | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
608
- | Column type checking | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
609
- | Typo suggestions | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
666
+ | Mypy plugin | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ✅ Yes | ❌ No | ❌ No | ⚠️ Basic | ❌ No | ❌ No | ❌ No |
667
+ | Standalone checker | ✅ Rust (~1ms) | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
668
+ | Column name checking | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
669
+ | Column type checking | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
670
+ | Typo suggestions | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
610
671
  | **Runtime Validation** |
611
- | Data validation | ❌ No | ✅ Excellent | ✅ Excellent | ✅ typeguard | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No |
612
- | Value constraints | ❌ No | ✅ Yes | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
672
+ | Data validation | ❌ No | ✅ Excellent | ✅ Excellent | ✅ typeguard | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |
673
+ | Value constraints | ❌ No | ✅ Yes | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |
613
674
  | **Schema Features** |
614
- | Column grouping | ✅ ColumnGroup | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
615
- | Regex column matching | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
675
+ | Column grouping | ✅ ColumnGroup | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
676
+ | Regex column matching | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
616
677
  | **Backend Support** |
617
- | Pandas | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Own | ✅ Yes |
618
- | Polars | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ Own | ✅ Yes |
619
- | DuckDB, cuDF, etc. | ❌ No | ❌ No | ✅ Spark, SQL | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes |
678
+ | Pandas | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Own | ✅ Yes | ❌ No | ⚠️ Limited |
679
+ | Polars | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ Own | ✅ Yes | ✅ Yes (only) | ✅ Yes |
680
+ | DuckDB, cuDF, etc. | ❌ No | ❌ No | ✅ Spark, SQL | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No | ❌ No |
620
681
  | **Project Status (Feb 2026)** |
621
- | Active development | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Low | ✅ Yes | ❌ Inactive | ⚠️ Low | ✅ Yes | ✅ Yes |
682
+ | Active development | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Low | ✅ Yes | ❌ Inactive | ⚠️ Low | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
622
683
 
623
684
  **Legend:** ✅ Full support | ⚠️ Limited/Partial | ❌ Not supported
624
685
 
625
686
  ### Tool Descriptions
626
687
 
627
- - **[Pandera](https://pandera.readthedocs.io/)** (v0.29.0): Excellent runtime validation. Static analysis support exists
688
+ - **[Pandera](https://pandera.readthedocs.io/)** (v0.32.1): Excellent runtime validation. Static analysis support exists
628
689
  but has limitations—column access via `df["column"]` is not validated, and schema mismatches between functions may not
629
690
  be caught.
630
691
 
631
- - **[strictly_typed_pandas](https://strictly-typed-pandas.readthedocs.io/)** (v0.3.6): Provides `DataSet[Schema]` type
632
- hints with mypy support. No standalone checker. No polars support. Runtime validation via typeguard.
692
+ - **[strictly_typed_pandas](https://strictly-typed-pandas.readthedocs.io/)** (v0.3.7): Provides `DataSet[Schema]` type
693
+ hints for runtime validation via typeguard. Despite documentation implying mypy support, there is no mypy plugin —
694
+ column access errors are not caught statically. No standalone checker. No polars support.
633
695
 
634
- - **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.0): Official pandas type stubs. Provides
696
+ - **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.3): Official pandas type stubs. Provides
635
697
  API-level types but no column-level checking.
636
698
 
637
699
  - **[dataenforce](https://github.com/CedricFR/dataenforce)** (v0.1.2): Runtime validation via decorator. Marked as
638
- experimental/not production-ready. Appears inactive.
700
+ experimental/not production-ready. Appears inactive. Broken on Python 3.13+ due to removal of internal typing APIs.
639
701
 
640
702
  - **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
641
703
  static analysis.
642
704
 
643
- - **[StaticFrame](https://github.com/static-frame/static-frame)** (v3.7.0): Alternative immutable DataFrame library with
644
- built-in static typing. Not compatible with pandas/polars—requires using StaticFrame's own DataFrame implementation.
705
+ - **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.0.0): Alternative immutable DataFrame library.
706
+ Not compatible with pandas/polars — requires a full rewrite to StaticFrame's own API. Column access is still
707
+ string-based; mypy does not catch column name typos. Type safety comes from immutability guarantees, not schema checking.
645
708
 
646
- - **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.16.0): Compatibility layer that provides a unified API
709
+ - **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.23.0): Compatibility layer that provides a unified API
647
710
  across pandas, polars, DuckDB, cuDF, and more. Solves a different problem—write-once-run-anywhere portability, not
648
711
  type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
649
712
  below.
650
713
 
651
- - **[Great Expectations](https://greatexpectations.io/)** (v1.4.3): Comprehensive data quality framework. Defines
714
+ - **[Great Expectations](https://greatexpectations.io/)** (v1.18.2): Comprehensive data quality framework. Defines
652
715
  "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
653
716
  validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
654
717
  code. Supports pandas, Spark, and SQL backends.
655
718
 
719
+ - **[dataframely](https://github.com/Quantco/dataframely)** (v2.13.0): Polars-only runtime validation library from Quantco.
720
+ Schemas are defined as classes inheriting `dy.Schema` with typed descriptor fields (`dy.String()`, `dy.Float64()`)
721
+ and `@dy.rule()` decorators for cross-column and group-level constraints. Returns `dy.DataFrame[Schema]` generic
722
+ types that give call-site narrowing to type checkers, but does not validate column subscript access inside function
723
+ bodies. No lint-time or static analysis capability. Supports nullability, string constraints, numeric bounds,
724
+ cross-column rules, soft validation, test data generation, and SQLAlchemy/PyArrow export.
725
+
726
+ - **[patito](https://github.com/JakobGM/patito)** (v0.8.6): Runtime validation library using a Pydantic-style `patito.Model`
727
+ class. Polars is the primary backend; pandas is supported but works by converting to Polars via PyArrow (an
728
+ undeclared dependency). DuckDB is not supported despite appearing in some documentation—validation crashes immediately
729
+ on DuckDB relations. No static analysis or standalone checker.
730
+
656
731
  ### Type Checkers (Not DataFrame-Specific)
657
732
 
658
733
  These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
@@ -946,6 +1021,8 @@ MIT License - see [LICENSE](LICENSE)
946
1021
  - [x] Pandera integration (`to_pandera_schema()`)
947
1022
  - [x] Cross-file schema inference (project-level index, `--no-index` flag)
948
1023
  - [x] Aggressive column inference (untracked-dataframe/dropped-unknown-column warnings, method chain propagation)
1024
+ - [x] Function parameter contracts (`missing-column`), resolved transitively across chains of helper
1025
+ functions and cross-file calls; schema-annotated parameters take priority over body-scanning
949
1026
 
950
1027
  **Planned:**
951
1028
 
@@ -1002,6 +1079,7 @@ Inspired by the needs of ML/data science teams working with complex data pipelin
1002
1079
 
1003
1080
  ---
1004
1081
 
1005
- **Questions? Issues? Ideas?** [Open an issue](https://github.com/yourusername/typedframes/issues)
1082
+ **Questions? Issues? Ideas?** [Open an issue](https://github.com/w-martin/typedframes/issues)
1006
1083
 
1007
1084
  **Ready to catch DataFrame bugs before runtime?** `pip install typedframes`
1085
+
@@ -1,37 +1,14 @@
1
- Metadata-Version: 2.4
2
- Name: typedframes
3
- Version: 0.2.1
4
- Classifier: Programming Language :: Python :: 3
5
- Classifier: Programming Language :: Python :: 3.11
6
- Classifier: Programming Language :: Python :: 3.12
7
- Classifier: Programming Language :: Python :: 3.13
8
- Classifier: Programming Language :: Python :: 3.14
9
- Requires-Dist: mypy ; extra == 'mypy'
10
- Requires-Dist: pandas>=2.3.3 ; extra == 'pandas'
11
- Requires-Dist: pandera>=0.29.0 ; extra == 'pandera'
12
- Requires-Dist: polars>=1.38.1 ; extra == 'polars'
13
- Provides-Extra: mypy
14
- Provides-Extra: pandas
15
- Provides-Extra: pandera
16
- Provides-Extra: polars
17
- Summary: Static analysis for pandas and polars DataFrames. Catch column errors at lint-time, not runtime.
18
- Home-Page: https://github.com/w-martin/typedframes
19
- Author: William Martin
20
- License: MIT
21
- Requires-Python: >=3.11
22
- Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
23
-
24
1
  # typedframes
25
2
 
26
3
  [![CI](https://github.com/w-martin/typedframes/actions/workflows/publish.yml/badge.svg)](https://github.com/w-martin/typedframes/actions/workflows/publish.yml)
27
4
  [![PyPI version](https://img.shields.io/pypi/v/typedframes.svg)](https://pypi.org/project/typedframes/)
28
5
  [![Python versions](https://img.shields.io/pypi/pyversions/typedframes.svg)](https://pypi.org/project/typedframes/)
29
- [![Coverage](https://codecov.io/gh/w-martin/typedframes/branch/main/graph/badge.svg)](https://codecov.io/gh/w-martin/typedframes)
6
+ [![Coverage](https://coveralls.io/repos/github/w-martin/typedframes/badge.svg?branch=main)](https://coveralls.io/github/w-martin/typedframes?branch=main)
30
7
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
31
8
 
32
9
  > ⚠️ **Project Status: Proof of Concept**
33
10
  >
34
- > `typedframes` (v0.2.1) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
11
+ > `typedframes` (v0.3.0) is currently an experimental proof-of-concept. The core static analysis and mypy/Rust
35
12
  > integrations work, but expect rough edges. The codebase prioritizes demonstrating the viability of static DataFrame
36
13
  > schema validation over production-grade stability.
37
14
  >
@@ -129,7 +106,7 @@ print(orders["revenue"]) # ✗ unknown-column — 'revenue' not in inferred set
129
106
 
130
107
  ```shell
131
108
  typedframes check src/
132
- # src/pipeline.py:7:8: error[unknown-column] Column 'revenue' not in inferred set {order_id, amount, status}
109
+ # src/pipeline.py:7:8: error[unknown-column] Column 'revenue' does not exist in inferred column set (defined at line 6)
133
110
  # ✗ Found 1 error in 12 files (0.0s)
134
111
  ```
135
112
 
@@ -286,12 +263,39 @@ trimmed = df.drop(columns=["nonexistent"])
286
263
  # ⚠ dropped-unknown-column: Dropped column 'nonexistent' does not exist in UserData
287
264
  ```
288
265
 
266
+ ### Function Parameter Contracts
267
+
268
+ Beyond validating access at the point it happens, the checker infers a *contract* for any function's
269
+ first parameter: every column the function needs, drawn from what its body accesses or — taking priority —
270
+ from a schema annotation on the parameter itself. Calling that function with a DataFrame that doesn't
271
+ satisfy the contract is caught at the **call site**, across files:
272
+
273
+ ```python
274
+ # transforms.py
275
+ def contact_label(customers):
276
+ return customers["name"] + customers["email"]
277
+ ```
278
+
279
+ ```python
280
+ # pipeline.py
281
+ customers = load_customers(path) # inferred columns: {customer_id, name, region}
282
+ contact_label(customers)
283
+ # ✗ missing-column: 'customers' passed to contact_label (transforms.py:2) is missing
284
+ # column(s) {email} — available: {customer_id, name, region}, required: {email, name}
285
+ ```
286
+
287
+ The contract is resolved *transitively*: if a function only forwards its parameter to other functions
288
+ (`step1 = preprocess(df); step2 = enrich(step1)`), the checker follows the chain and unions their
289
+ requirements, catching a missing column even when no single function in the chain touches it directly.
290
+ Column-list slices (`df[["a", "b"]]`) contribute to the contract too.
291
+
289
292
  ### See Also
290
293
 
291
294
  - [`examples/inference_example.py`](examples/inference_example.py) — single-file walkthrough of all four inference
292
295
  scenarios with annotated ✓/✗ comments.
293
296
  - [`examples/multi_file_inference/`](examples/multi_file_inference/) — multi-file project checked with
294
- `typedframes check examples/multi_file_inference/`; no `BaseSchema` anywhere.
297
+ `typedframes check examples/multi_file_inference/`; no `BaseSchema` anywhere. Includes a function
298
+ parameter contract violation caught at the call site (`missing-column`).
295
299
  - [`examples/multi_file_with_schema/`](examples/multi_file_with_schema/) — same scenario with `BaseSchema`
296
300
  classes; the checker follows schemas across module boundaries via the project index.
297
301
 
@@ -316,8 +320,10 @@ typedframes check src/
316
320
  **Features:**
317
321
  - Catches column name errors
318
322
  - Validates schema mismatches between functions
323
+ - Validates function parameter contracts across files, including transitively through chains of
324
+ helper functions (`missing-column`)
319
325
  - Checks both pandas and polars code
320
- - 10-100x faster than mypy
326
+ - Significantly faster than mypy (see benchmarks below)
321
327
 
322
328
  **Use this for:**
323
329
  - Fast feedback during development
@@ -398,17 +404,15 @@ Fast feedback reduces development time. The typedframes Rust binary provides nea
398
404
 
399
405
  **Benchmark results** (10 runs, 3 warmup, caches cleared between runs):
400
406
 
401
- | Tool | Version | What it does | typedframes (13 files) | great_expectations (490 files)† |
402
- |--------------------|---------|-------------------------------|------------------------|---------------------------------|
403
- | typedframes | 0.2.0 | DataFrame column checker | 9ms ±2ms | 930µs ±89µs |
404
- | ruff | 0.15.4 | Linter (no type checking) | 64ms ±16ms | 360ms ±18ms |
405
- | ty | 0.0.19 | Type checker | 115ms ±22ms | 1.65s ±26ms |
406
- | pyrefly | 0.54.0 | Type checker | 3.78s ±7.53s | 693ms ±33ms |
407
- | mypy | 1.19.1 | Type checker (no plugin) | 13.85s ±1.08s | 12.13s ±400ms |
408
- | mypy + typedframes | 1.19.1 | Type checker + column checker | 13.51s ±273ms | 13.89s ±491ms |
409
- | pyright | 1.1.408 | Type checker | 2.10s ±422ms | 8.37s ±253ms |
410
-
411
- *† great_expectations column from previous benchmark run.*
407
+ | Tool | Version | What it does | typedframes (13 files) | great_expectations (485 files) |
408
+ |------|---------|--------------|------------------------|--------------------------------|
409
+ | typedframes | 0.3.0 | DataFrame column checker | 81ms ±6ms | 3.64s ±9ms |
410
+ | ruff | 0.15.21 | Linter (no type checking) | 48ms ±13ms | 237ms ±31ms |
411
+ | ty | 0.0.58 | Type checker | 119ms ±14ms | 287ms ±33ms |
412
+ | pyrefly | 1.1.1 | Type checker | 162ms ±18ms | 390ms ±21ms |
413
+ | mypy | 2.2.0 | Type checker (no plugin) | 4.56s ±22ms | 7.31s ±35ms |
414
+ | mypy + typedframes | 2.2.0 | Type checker + column checker | 4.69s ±61ms | 10.89s ±356ms |
415
+ | pyright | 1.1.411 | Type checker | 1.39s ±17ms | 8.06s ±193ms |
412
416
 
413
417
  *Run `uv run python benchmarks/benchmark_checkers.py` to reproduce.*
414
418
 
@@ -619,63 +623,77 @@ See [`examples/schema_algebra_example.py`](examples/schema_algebra_example.py) f
619
623
  Comprehensive comparison of pandas/DataFrame typing and validation tools. **typedframes focuses on static analysis**
620
624
  —catching errors at lint-time before your code runs.
621
625
 
622
- | Feature | typedframes | Pandera | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame | narwhals |
623
- |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|
624
- | **Version tested** | 0.2.0 | 0.29.0 | 1.4.3 | 0.3.6 | 3.0.0 | 0.1.2 | 1.1.3 | 3.7.0 | 2.16.0 |
626
+ | Feature | typedframes | Pandera | Great Expectations | strictly_typed_pandas | pandas-stubs | dataenforce | pandas-type-checks | StaticFrame | narwhals | dataframely | patito |
627
+ |---------------------------------|------------------------|-------------|--------------------|-----------------------|--------------|-------------|--------------------|------------------|----------|------------------|------------------|
628
+ | **Version tested** | 0.3.0 | 0.32.1 | 1.18.2 | 0.3.7 | 3.0.3 | 0.1.2 | 1.1.3 | 5.0.0 | 2.23.0 | 2.13.0 | 0.8.6 |
625
629
  | **Analysis Type** |
626
- | When errors are caught | **Static (lint-time)** | Runtime | Runtime | Static + Runtime | Static | Runtime | Runtime | Static + Runtime | Runtime |
630
+ | When errors are caught | **Static (lint-time)** | Runtime | Runtime | Runtime | Static | Runtime | Runtime | Runtime | Runtime | Runtime | Runtime |
627
631
  | **Static Analysis (our focus)** |
628
- | Mypy plugin | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ⚠️ Basic | ❌ No |
629
- | Standalone checker | ✅ Rust (~1ms) | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
630
- | Column name checking | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
631
- | Column type checking | ✅ Yes | ⚠️ Limited | ❌ No | ✅ Yes | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
632
- | Typo suggestions | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
632
+ | Mypy plugin | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ✅ Yes | ❌ No | ❌ No | ⚠️ Basic | ❌ No | ❌ No | ❌ No |
633
+ | Standalone checker | ✅ Rust (~1ms) | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
634
+ | Column name checking | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
635
+ | Column type checking | ✅ Yes | ⚠️ Limited | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
636
+ | Typo suggestions | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
633
637
  | **Runtime Validation** |
634
- | Data validation | ❌ No | ✅ Excellent | ✅ Excellent | ✅ typeguard | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No |
635
- | Value constraints | ❌ No | ✅ Yes | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No |
638
+ | Data validation | ❌ No | ✅ Excellent | ✅ Excellent | ✅ typeguard | ❌ No | ✅ Yes | ✅ Yes | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |
639
+ | Value constraints | ❌ No | ✅ Yes | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No | ✅ Yes | ✅ Yes |
636
640
  | **Schema Features** |
637
- | Column grouping | ✅ ColumnGroup | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
638
- | Regex column matching | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
641
+ | Column grouping | ✅ ColumnGroup | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
642
+ | Regex column matching | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
639
643
  | **Backend Support** |
640
- | Pandas | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Own | ✅ Yes |
641
- | Polars | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ Own | ✅ Yes |
642
- | DuckDB, cuDF, etc. | ❌ No | ❌ No | ✅ Spark, SQL | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes |
644
+ | Pandas | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ❌ Own | ✅ Yes | ❌ No | ⚠️ Limited |
645
+ | Polars | ✅ Yes | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ Own | ✅ Yes | ✅ Yes (only) | ✅ Yes |
646
+ | DuckDB, cuDF, etc. | ❌ No | ❌ No | ✅ Spark, SQL | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ✅ Yes | ❌ No | ❌ No |
643
647
  | **Project Status (Feb 2026)** |
644
- | Active development | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Low | ✅ Yes | ❌ Inactive | ⚠️ Low | ✅ Yes | ✅ Yes |
648
+ | Active development | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Low | ✅ Yes | ❌ Inactive | ⚠️ Low | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
645
649
 
646
650
  **Legend:** ✅ Full support | ⚠️ Limited/Partial | ❌ Not supported
647
651
 
648
652
  ### Tool Descriptions
649
653
 
650
- - **[Pandera](https://pandera.readthedocs.io/)** (v0.29.0): Excellent runtime validation. Static analysis support exists
654
+ - **[Pandera](https://pandera.readthedocs.io/)** (v0.32.1): Excellent runtime validation. Static analysis support exists
651
655
  but has limitations—column access via `df["column"]` is not validated, and schema mismatches between functions may not
652
656
  be caught.
653
657
 
654
- - **[strictly_typed_pandas](https://strictly-typed-pandas.readthedocs.io/)** (v0.3.6): Provides `DataSet[Schema]` type
655
- hints with mypy support. No standalone checker. No polars support. Runtime validation via typeguard.
658
+ - **[strictly_typed_pandas](https://strictly-typed-pandas.readthedocs.io/)** (v0.3.7): Provides `DataSet[Schema]` type
659
+ hints for runtime validation via typeguard. Despite documentation implying mypy support, there is no mypy plugin —
660
+ column access errors are not caught statically. No standalone checker. No polars support.
656
661
 
657
- - **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.0): Official pandas type stubs. Provides
662
+ - **[pandas-stubs](https://github.com/pandas-dev/pandas-stubs)** (v3.0.3): Official pandas type stubs. Provides
658
663
  API-level types but no column-level checking.
659
664
 
660
665
  - **[dataenforce](https://github.com/CedricFR/dataenforce)** (v0.1.2): Runtime validation via decorator. Marked as
661
- experimental/not production-ready. Appears inactive.
666
+ experimental/not production-ready. Appears inactive. Broken on Python 3.13+ due to removal of internal typing APIs.
662
667
 
663
668
  - **[pandas-type-checks](https://pypi.org/project/pandas-type-checks/)** (v1.1.3): Runtime validation decorator. No
664
669
  static analysis.
665
670
 
666
- - **[StaticFrame](https://github.com/static-frame/static-frame)** (v3.7.0): Alternative immutable DataFrame library with
667
- built-in static typing. Not compatible with pandas/polars—requires using StaticFrame's own DataFrame implementation.
671
+ - **[StaticFrame](https://github.com/static-frame/static-frame)** (v5.0.0): Alternative immutable DataFrame library.
672
+ Not compatible with pandas/polars — requires a full rewrite to StaticFrame's own API. Column access is still
673
+ string-based; mypy does not catch column name typos. Type safety comes from immutability guarantees, not schema checking.
668
674
 
669
- - **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.16.0): Compatibility layer that provides a unified API
675
+ - **[narwhals](https://narwhals-dev.github.io/narwhals/)** (v2.23.0): Compatibility layer that provides a unified API
670
676
  across pandas, polars, DuckDB, cuDF, and more. Solves a different problem—write-once-run-anywhere portability, not
671
677
  type safety. See [Why Abstraction Layers Don't Solve Type Safety](#why-abstraction-layers-dont-solve-type-safety)
672
678
  below.
673
679
 
674
- - **[Great Expectations](https://greatexpectations.io/)** (v1.4.3): Comprehensive data quality framework. Defines
680
+ - **[Great Expectations](https://greatexpectations.io/)** (v1.18.2): Comprehensive data quality framework. Defines
675
681
  "expectations" (assertions) about data values, distributions, and schema properties. Excellent for runtime
676
682
  validation, data documentation, and data quality monitoring. No static analysis or column-level type checking in
677
683
  code. Supports pandas, Spark, and SQL backends.
678
684
 
685
+ - **[dataframely](https://github.com/Quantco/dataframely)** (v2.13.0): Polars-only runtime validation library from Quantco.
686
+ Schemas are defined as classes inheriting `dy.Schema` with typed descriptor fields (`dy.String()`, `dy.Float64()`)
687
+ and `@dy.rule()` decorators for cross-column and group-level constraints. Returns `dy.DataFrame[Schema]` generic
688
+ types that give call-site narrowing to type checkers, but does not validate column subscript access inside function
689
+ bodies. No lint-time or static analysis capability. Supports nullability, string constraints, numeric bounds,
690
+ cross-column rules, soft validation, test data generation, and SQLAlchemy/PyArrow export.
691
+
692
+ - **[patito](https://github.com/JakobGM/patito)** (v0.8.6): Runtime validation library using a Pydantic-style `patito.Model`
693
+ class. Polars is the primary backend; pandas is supported but works by converting to Polars via PyArrow (an
694
+ undeclared dependency). DuckDB is not supported despite appearing in some documentation—validation crashes immediately
695
+ on DuckDB relations. No static analysis or standalone checker.
696
+
679
697
  ### Type Checkers (Not DataFrame-Specific)
680
698
 
681
699
  These are general Python type checkers. They don't validate DataFrame column names, but they can be used alongside
@@ -969,6 +987,8 @@ MIT License - see [LICENSE](LICENSE)
969
987
  - [x] Pandera integration (`to_pandera_schema()`)
970
988
  - [x] Cross-file schema inference (project-level index, `--no-index` flag)
971
989
  - [x] Aggressive column inference (untracked-dataframe/dropped-unknown-column warnings, method chain propagation)
990
+ - [x] Function parameter contracts (`missing-column`), resolved transitively across chains of helper
991
+ functions and cross-file calls; schema-annotated parameters take priority over body-scanning
972
992
 
973
993
  **Planned:**
974
994
 
@@ -1025,7 +1045,6 @@ Inspired by the needs of ML/data science teams working with complex data pipelin
1025
1045
 
1026
1046
  ---
1027
1047
 
1028
- **Questions? Issues? Ideas?** [Open an issue](https://github.com/yourusername/typedframes/issues)
1048
+ **Questions? Issues? Ideas?** [Open an issue](https://github.com/w-martin/typedframes/issues)
1029
1049
 
1030
1050
  **Ready to catch DataFrame bugs before runtime?** `pip install typedframes`
1031
-