tracepipe 0.3.0__tar.gz → 0.3.1__tar.gz

{tracepipe-0.3.0 → tracepipe-0.3.1}/CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to TracePipe will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 0.3.1 - 2026-02-03
+
+### Fixed
+- Cell history now correctly chains through merge operations via lineage traversal
+- `tp.why()` and `tp.trace()` show pre-merge changes for post-merge rows
+- `enable()` resets accumulated state when called multiple times (fixes duplicate warnings in notebooks/IDEs)
+
+### Added
+- `get_row_history_with_lineage()` and `get_cell_history_with_lineage()` methods for lineage-aware queries
+- `follow_lineage` parameter in `explain_value()` for opt-out of lineage traversal
+- Integration tests for cell provenance through merge operations
+
 ## 0.3.0 - 2026-02-03
 
 ### Added

tracepipe-0.3.1/PKG-INFO

@@ -0,0 +1,308 @@
+Metadata-Version: 2.4
+Name: tracepipe
+Version: 0.3.1
+Summary: Row-level data lineage tracking for pandas pipelines
+Project-URL: Homepage, https://github.com/tracepipe/tracepipe
+Project-URL: Documentation, https://tracepipe.github.io/tracepipe/
+Project-URL: Repository, https://github.com/tracepipe/tracepipe.git
+Project-URL: Issues, https://github.com/tracepipe/tracepipe/issues
+Project-URL: Changelog, https://tracepipe.github.io/tracepipe/changelog/
+Author: Gauthier Piarrette
+License: MIT License
+        
+        Copyright (c) 2026 Gauthier Piarrette
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: data-engineering,data-lineage,data-quality,debugging,observability,pandas
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Requires-Dist: numpy>=1.20.0
+Requires-Dist: pandas>=1.5.0
+Provides-Extra: all
+Requires-Dist: psutil>=5.9.0; extra == 'all'
+Requires-Dist: pyarrow>=10.0.0; extra == 'all'
+Provides-Extra: arrow
+Requires-Dist: pyarrow>=10.0.0; extra == 'arrow'
+Provides-Extra: dev
+Requires-Dist: black>=23.0.0; extra == 'dev'
+Requires-Dist: pre-commit>=3.5.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest>=7.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.1.0; extra == 'dev'
+Requires-Dist: taskipy>=1.12.0; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
+Requires-Dist: mkdocs>=1.5.0; extra == 'docs'
+Requires-Dist: mkdocstrings[python]>=0.24.0; extra == 'docs'
+Requires-Dist: pymdown-extensions>=10.0.0; extra == 'docs'
+Provides-Extra: memory
+Requires-Dist: psutil>=5.9.0; extra == 'memory'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# TracePipe
+
+### Row-level data lineage for pandas pipelines
+
+**Know exactly where every row went, why values changed, and how your data transformed.**
+
+[![PyPI version](https://img.shields.io/pypi/v/tracepipe.svg)](https://pypi.org/project/tracepipe/)
+[![Python 3.9+](https://img.shields.io/pypi/pyversions/tracepipe.svg)](https://pypi.org/project/tracepipe/)
+[![CI](https://github.com/gauthierpiarrette/tracepipe/actions/workflows/ci.yml/badge.svg)](https://github.com/gauthierpiarrette/tracepipe/actions/workflows/ci.yml)
+[![codecov](https://codecov.io/gh/gauthierpiarrette/tracepipe/branch/main/graph/badge.svg)](https://codecov.io/gh/gauthierpiarrette/tracepipe)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Docs](https://img.shields.io/badge/docs-mkdocs-blue.svg)](https://gauthierpiarrette.github.io/tracepipe/)
+
+[Getting Started](#getting-started) · [Documentation](https://gauthierpiarrette.github.io/tracepipe/) · [Examples](#real-world-example)
+
+</div>
+
+---
+
+## Why TracePipe?
+
+Data pipelines are black boxes. Rows vanish. Values change. You're left guessing.
+
+```python
+df = pd.read_csv("customers.csv")
+df = df.dropna()                      # Some rows disappear
+df = df.merge(regions, on="zip")      # New rows appear, some vanish
+df["income"] = df["income"].fillna(0) # Values change silently
+df = df[df["age"] >= 18]              # More rows gone
+# What happened to customer C-789? 🤷
+```
+
+**TracePipe gives you the complete audit trail — zero code changes required.**
+
+---
+
+## Getting Started
+
+```bash
+pip install tracepipe
+```
+
+```python
+import tracepipe as tp
+import pandas as pd
+
+tp.enable(mode="debug", watch=["income"])
+
+df = pd.read_csv("customers.csv")
+df = df.dropna()
+df["income"] = df["income"].fillna(0)
+df = df[df["age"] >= 18]
+
+tp.check(df)  # See what happened
+```
+
+```
+TracePipe Check: [OK] Pipeline healthy
+
+Retention: 847/1000 (84.7%)
+Dropped: 153 rows
+  • DataFrame.dropna: 42
+  • DataFrame.__getitem__[mask]: 111
+
+Value changes: 23 cells modified
+  • DataFrame.fillna: 23 (income)
+```
+
+That's it. **One import, full visibility.**
+
+---
+
+## Core API
+
+| Function | What it does |
+|----------|--------------|
+| `tp.enable()` | Start tracking |
+| `tp.check(df)` | Health check — retention, drops, changes |
+| `tp.trace(df, where={"id": "C-789"})` | Follow a row's complete journey |
+| `tp.why(df, col="income", row=5)` | Explain why a cell has its current value |
+| `tp.report(df, "audit.html")` | Export interactive HTML report |
+
+---
+
+## Key Features
+
+<table>
+<tr>
+<td width="50%">
+
+### 🔍 Zero-Code Instrumentation
+TracePipe patches pandas at runtime. Your existing code works unchanged.
+
+### 📊 Complete Provenance
+Track drops, transforms, merges, and cell-level changes with before/after values.
+
+</td>
+<td width="50%">
+
+### 🎯 Business-Key Lookups
+Find rows by their values: `tp.trace(df, where={"email": "alice@example.com"})`
+
+### ⚡ Production-Ready
+1.0-2.8x overhead (varies by operation). Tested on DataFrames up to 1M rows.
+
+</td>
+</tr>
+</table>
+
+---
+
+## Real-World Example
+
+```python
+import tracepipe as tp
+import pandas as pd
+
+tp.enable(mode="debug", watch=["age", "income", "label"])
+
+# Load and clean
+df = pd.read_csv("training_data.csv")
+df = df.dropna(subset=["label"])
+df["income"] = df["income"].fillna(df["income"].median())
+df = df[df["age"] >= 18]
+
+# Audit
+print(tp.check(df))
+```
+
+```
+Retention: 8234/10000 (82.3%)
+Dropped: 1766 rows
+  • DataFrame.dropna: 423
+  • DataFrame.__getitem__[mask]: 1343
+
+Value changes: 892 cells
+  • DataFrame.fillna: 892 (income)
+```
+
+```python
+# Why does this customer have a filled income?
+tp.why(df, col="income", where={"customer_id": "C-789"})
+```
+
+```
+Cell History: row 156, column 'income'
+  Current value: 45000.0
+  [i] Was null at step 1 (later recovered)
+
+  History (1 change):
+    None -> 45000.0
+      by: DataFrame.fillna
+```
+
+---
+
+## Two Modes
+
+| Mode | Use Case | What's Tracked |
+|------|----------|----------------|
+| **CI** (default) | Production pipelines | Step counts, retention rates, merge warnings |
+| **Debug** | Development | Full row history, cell diffs, merge parents, group membership |
+
+```python
+tp.enable(mode="ci")     # Lightweight
+tp.enable(mode="debug")  # Full lineage
+```
+
+---
+
+## What's Tracked
+
+| Operation | Coverage |
+|-----------|----------|
+| `dropna`, `drop_duplicates`, `query`, `df[mask]` | ✅ Full |
+| `fillna`, `replace`, `loc[]=`, `iloc[]=` | ✅ Full (cell diffs) |
+| `merge`, `join` | ✅ Full (parent tracking) |
+| `groupby().agg()` | ✅ Full (group membership) |
+| `sort_values`, `head`, `tail`, `sample` | ✅ Full |
+| `apply`, `pipe` | ⚠️ Partial |
+
+---
+
+## Data Quality Contracts
+
+```python
+(tp.contract()
+    .expect_unique("customer_id")
+    .expect_no_nulls("email")
+    .expect_retention(min_rate=0.9)
+    .check(df)
+    .raise_if_failed())
+```
+
+---
+
+## Documentation
+
+📚 **[Full Documentation](https://gauthierpiarrette.github.io/tracepipe/)**
+
+- [Quickstart](https://gauthierpiarrette.github.io/tracepipe/getting-started/quickstart/)
+- [User Guide](https://gauthierpiarrette.github.io/tracepipe/guide/concepts/)
+- [API Reference](https://gauthierpiarrette.github.io/tracepipe/api/)
+- [Examples](https://gauthierpiarrette.github.io/tracepipe/examples/ml-pipeline/)
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/gauthierpiarrette/tracepipe.git
+cd tracepipe
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+See [CONTRIBUTING](https://gauthierpiarrette.github.io/tracepipe/contributing/) for guidelines.
+
+---
+
+## License
+
+MIT License. See [LICENSE](LICENSE).
+
+---
+
+<div align="center">
+
+**Stop guessing where your rows went.**
+
+```bash
+pip install tracepipe
+```
+
+⭐ Star us on GitHub if TracePipe helps your data work!
+
+</div>

tracepipe-0.3.1/README.md

@@ -0,0 +1,239 @@
+<div align="center">
+
+# TracePipe
+
+### Row-level data lineage for pandas pipelines
+
+**Know exactly where every row went, why values changed, and how your data transformed.**
+
+[![PyPI version](https://img.shields.io/pypi/v/tracepipe.svg)](https://pypi.org/project/tracepipe/)
+[![Python 3.9+](https://img.shields.io/pypi/pyversions/tracepipe.svg)](https://pypi.org/project/tracepipe/)
+[![CI](https://github.com/gauthierpiarrette/tracepipe/actions/workflows/ci.yml/badge.svg)](https://github.com/gauthierpiarrette/tracepipe/actions/workflows/ci.yml)
+[![codecov](https://codecov.io/gh/gauthierpiarrette/tracepipe/branch/main/graph/badge.svg)](https://codecov.io/gh/gauthierpiarrette/tracepipe)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
+[![Docs](https://img.shields.io/badge/docs-mkdocs-blue.svg)](https://gauthierpiarrette.github.io/tracepipe/)
+
+[Getting Started](#getting-started) · [Documentation](https://gauthierpiarrette.github.io/tracepipe/) · [Examples](#real-world-example)
+
+</div>
+
+---
+
+## Why TracePipe?
+
+Data pipelines are black boxes. Rows vanish. Values change. You're left guessing.
+
+```python
+df = pd.read_csv("customers.csv")
+df = df.dropna()                      # Some rows disappear
+df = df.merge(regions, on="zip")      # New rows appear, some vanish
+df["income"] = df["income"].fillna(0) # Values change silently
+df = df[df["age"] >= 18]              # More rows gone
+# What happened to customer C-789? 🤷
+```
+
+**TracePipe gives you the complete audit trail — zero code changes required.**
+
+---
+
+## Getting Started
+
+```bash
+pip install tracepipe
+```
+
+```python
+import tracepipe as tp
+import pandas as pd
+
+tp.enable(mode="debug", watch=["income"])
+
+df = pd.read_csv("customers.csv")
+df = df.dropna()
+df["income"] = df["income"].fillna(0)
+df = df[df["age"] >= 18]
+
+tp.check(df)  # See what happened
+```
+
+```
+TracePipe Check: [OK] Pipeline healthy
+
+Retention: 847/1000 (84.7%)
+Dropped: 153 rows
+  • DataFrame.dropna: 42
+  • DataFrame.__getitem__[mask]: 111
+
+Value changes: 23 cells modified
+  • DataFrame.fillna: 23 (income)
+```
+
+That's it. **One import, full visibility.**
+
+---
+
+## Core API
+
+| Function | What it does |
+|----------|--------------|
+| `tp.enable()` | Start tracking |
+| `tp.check(df)` | Health check — retention, drops, changes |
+| `tp.trace(df, where={"id": "C-789"})` | Follow a row's complete journey |
+| `tp.why(df, col="income", row=5)` | Explain why a cell has its current value |
+| `tp.report(df, "audit.html")` | Export interactive HTML report |
+
+---
+
+## Key Features
+
+<table>
+<tr>
+<td width="50%">
+
+### 🔍 Zero-Code Instrumentation
+TracePipe patches pandas at runtime. Your existing code works unchanged.
+
+### 📊 Complete Provenance
+Track drops, transforms, merges, and cell-level changes with before/after values.
+
+</td>
+<td width="50%">
+
+### 🎯 Business-Key Lookups
+Find rows by their values: `tp.trace(df, where={"email": "alice@example.com"})`
+
+### ⚡ Production-Ready
+1.0-2.8x overhead (varies by operation). Tested on DataFrames up to 1M rows.
+
+</td>
+</tr>
+</table>
+
+---
+
+## Real-World Example
+
+```python
+import tracepipe as tp
+import pandas as pd
+
+tp.enable(mode="debug", watch=["age", "income", "label"])
+
+# Load and clean
+df = pd.read_csv("training_data.csv")
+df = df.dropna(subset=["label"])
+df["income"] = df["income"].fillna(df["income"].median())
+df = df[df["age"] >= 18]
+
+# Audit
+print(tp.check(df))
+```
+
+```
+Retention: 8234/10000 (82.3%)
+Dropped: 1766 rows
+  • DataFrame.dropna: 423
+  • DataFrame.__getitem__[mask]: 1343
+
+Value changes: 892 cells
+  • DataFrame.fillna: 892 (income)
+```
+
+```python
+# Why does this customer have a filled income?
+tp.why(df, col="income", where={"customer_id": "C-789"})
+```
+
+```
+Cell History: row 156, column 'income'
+  Current value: 45000.0
+  [i] Was null at step 1 (later recovered)
+
+  History (1 change):
+    None -> 45000.0
+      by: DataFrame.fillna
+```
+
+---
+
+## Two Modes
+
+| Mode | Use Case | What's Tracked |
+|------|----------|----------------|
+| **CI** (default) | Production pipelines | Step counts, retention rates, merge warnings |
+| **Debug** | Development | Full row history, cell diffs, merge parents, group membership |
+
+```python
+tp.enable(mode="ci")     # Lightweight
+tp.enable(mode="debug")  # Full lineage
+```
+
+---
+
+## What's Tracked
+
+| Operation | Coverage |
+|-----------|----------|
+| `dropna`, `drop_duplicates`, `query`, `df[mask]` | ✅ Full |
+| `fillna`, `replace`, `loc[]=`, `iloc[]=` | ✅ Full (cell diffs) |
+| `merge`, `join` | ✅ Full (parent tracking) |
+| `groupby().agg()` | ✅ Full (group membership) |
+| `sort_values`, `head`, `tail`, `sample` | ✅ Full |
+| `apply`, `pipe` | ⚠️ Partial |
+
+---
+
+## Data Quality Contracts
+
+```python
+(tp.contract()
+    .expect_unique("customer_id")
+    .expect_no_nulls("email")
+    .expect_retention(min_rate=0.9)
+    .check(df)
+    .raise_if_failed())
+```
+
+---
+
+## Documentation
+
+📚 **[Full Documentation](https://gauthierpiarrette.github.io/tracepipe/)**
+
+- [Quickstart](https://gauthierpiarrette.github.io/tracepipe/getting-started/quickstart/)
+- [User Guide](https://gauthierpiarrette.github.io/tracepipe/guide/concepts/)
+- [API Reference](https://gauthierpiarrette.github.io/tracepipe/api/)
+- [Examples](https://gauthierpiarrette.github.io/tracepipe/examples/ml-pipeline/)
+
+---
+
+## Contributing
+
+```bash
+git clone https://github.com/gauthierpiarrette/tracepipe.git
+cd tracepipe
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+See [CONTRIBUTING](https://gauthierpiarrette.github.io/tracepipe/contributing/) for guidelines.
+
+---
+
+## License
+
+MIT License. See [LICENSE](LICENSE).
+
+---
+
+<div align="center">
+
+**Stop guessing where your rows went.**
+
+```bash
+pip install tracepipe
+```
+
+⭐ Star us on GitHub if TracePipe helps your data work!
+
+</div>

{tracepipe-0.3.0 → tracepipe-0.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "tracepipe"
-version = "0.3.0"
+version = "0.3.1"
 description = "Row-level data lineage tracking for pandas pipelines"
 readme = "README.md"
 license = {file = "LICENSE"}