InfoTracker 0.3.0__tar.gz → 0.4.0__tar.gz

infotracker-0.4.0/.cursorrules

@@ -0,0 +1,105 @@
+# InfoTracker – Cursor Rules
+# Krótkie reguły dla automatycznych zmian w projekcie (parser/engine/CLI/diff/OL).
+
+# ✱ ZASADY OGÓLNE
+- Nie zmieniaj publicznych interfejsów CLI (nazwy komend i flag): extract, impact, diff.
+- Zachowaj kompatybilność plików wyjściowych (OpenLineage JSON, format tabel tekstowych).
+- Nie usuwaj dokumentacji ani komentarzy TODO; dopisuj „// NOTE(cursor): …” gdzie uzasadniasz decyzję.
+- Logowanie: use WARNING dla problemów użytkownika (parse/cycle), DEBUG dla szczegółów; nie spamuj INFO.
+- Deterministyczność: zachowaj stabilne sortowanie (po object_name, ordinal) i stabilne ID runów.
+- Każda zmiana, która dotyka semantyki, musi mieć test(y) w `tests/` i przechodzić `pytest -q`.
+- Nie dodawaj twardych zależności zewnętrznych; trzymaj się stdlib + obecnych pakietów.
+
+# ✱ PARSER (T-SQL)
+- Zawsze uruchamiaj pre-processing przed sqlglot.parse:
+  - Usuń linie zaczynające się od DECLARE/SET/PRINT.
+  - Usuń GO (puste batche).
+  - Usuń jednowierszowe IF OBJECT_ID('tempdb..#…') … DROP TABLE #… i gołe DROP TABLE #… (temp tables housekeeping).
+  - Sklej dwuliniowe „INSERT INTO #tmp” + następna „EXEC …” do jednej linii: „INSERT INTO #tmp EXEC …”.
+- Fallback gdy parse się wywala:
+  - Rozpoznaj `INSERT INTO <tabela|#temp> EXEC <proc>` (obsłuż zarówno temp jak i zwykłe tabele).
+  - Wyczyść nazwę procedury (_bez_ końcowego `;` i bez `()`).
+  - Zwróć ObjectInfo:
+    - name = nazwa tabeli (temp → `#…`), object_type = temp_table|table,
+    - schema: 2 placeholderowe kolumny (nullable=True),
+    - dependencies = {proc_name},
+    - lineage: ColumnReference(table_name=proc_name, column_name="*", transformation_type=EXEC).
+- SELECT/CAST:
+  - Projekcje pobieraj defensywnie: expressions | projections | args["expressions"].
+  - Dla `CAST(... AS TYPE)` ustaw ColumnSchema.data_type na docelowy typ (np. DECIMAL(10,2), NVARCHAR(50)).
+- Kwalifikacja:
+  - Temp-tabele zawsze pod `namespace="tempdb"`.
+  - Mssql dataset namespace: `mssql://localhost/InfoTrackerDW` (chyba że user ustawi w configu).
+
+# ✱ ENGINE
+- Graf zależności buduj z `ObjectInfo.dependencies`. Jeśli puste → fallback na tabele z `lineage.input_fields`.
+- Topologiczne sortowanie nie może polegać na kolejności plików.
+- Przy „circular/missing deps” loguj WARNING i przetwarzaj mimo wszystko (best-effort).
+
+# ✱ OPENLINEAGE
+- Dla outputs dodawaj facet `schema` dla wszystkich obiektów z kolumnami (tabele, widoki, proc/fn z wynikami).
+- Dodawaj facet `columnLineage` gdy istnieje lineage.
+- Namespace i nazwy datasetów muszą być spójne w całym repo.
+
+# ✱ DIFF
+- Klasyfikacja zmian:
+  - COLUMN_TYPE_CHANGED → BREAKING.
+  - COLUMN_RENAMED → POTENTIALLY_BREAKING (wykrywanie: typ/nullability/lineage zgodne; heurystyka łącząca pary).
+  - COLUMN_ADDED → NON_BREAKING jeśli nullable, w przeciwnym razie POTENTIALLY_BREAKING.
+  - OBJECT_REMOVED/ADDED → BREAKING.
+- Honoruj `severity_threshold` z configu w filtrze wyświetlania i exit code:
+  - BREAKING → tylko breaking zmiany.
+  - POTENTIALLY_BREAKING → breaking + potentially.
+  - NON_BREAKING → wszystkie zmiany.
+
+# ✱ WILDCARDY (ColumnGraph.find_columns_wildcard)
+- Pusty/pół-pusty wzorzec (`"."`, "`..`", "`ns..`" bez patternu kolumny) → zwracaj pustą listę.
+- `schema.table.*`:
+  - Dopasowuj case-insensitive po końcówce nazwy tabeli (sufiks 3/2/1 segmentów), niezależnie od obecności bazy.
+- `..pattern` i `prefix..pattern`:
+  - `pattern` bez metaznaków działa jako „contains” (case-insensitive); z metaznakami → `fnmatch`.
+  - `prefix` z `://` filtruje po namespace, bez `://` po `table_name` (startswith).
+- Zawsze deduplikuj węzły przed zwrotem.
+
+# ✱ CLI
+- Nie dodawaj nowych flag bez uzasadnienia. Dozwolone: `--graph-dir`, `--out`, `--format`, `--config`.
+- (Opcjonalnie) możesz dodać `--threshold` do `diff`, ale musi honorować wartości z configu (flagą tylko nadpisujesz).
+
+# ✱ TESTY
+- Dodawaj test przy każdej zmianie heurystyki parsera/diff/wildcard.
+- W testach `INSERT … EXEC` nazwę procedury porównuj bez `;`.
+- W testach wildcardów – sprawdzaj zarówno tryb exact (`schema.table.*`), jak i contains (`..customer*`), z i bez namespace.
+
+# ✱ KOMITY / PR-Y
+- Commity: konwencja
+  - feat(parser|engine|cli|diff|models): …
+  - fix(parser|engine|…): …
+  - chore(logging|tests|ci): …
+- PR ma być mały i atomowy; zawiera opis zmiany, wpływ na diff/impact, link do testów.
+- Nie łącz refaktoru z funkcjonalną zmianą w jednym PR (chyba że to mechaniczny lint bez ryzyka).
+
+# ✱ TYPOWE AUTO-NAPRAWY DOZWOLONE
+- Usuwanie martwego kodu po `return`.
+- Wstawienie brakujących `GO` między wieloma `CREATE` w jednym pliku przykładowym.
+- Dodanie brakującego `CREATE` przed `VIEW` gdy wykryto wzorzec `;VIEW … AS` (demo).
+- Normalizacja nazw procedur (usunięcie `;`/`()`).
+
+# ✱ CO JEST ZABRONIONE
+- Zmiana semantyki outputu OpenLineage (np. inny kształt JSON).
+- Wyciszanie ostrzeżeń bez adresowania przyczyny (np. „pass” w except).
+- Wymiana parsera SQL lub dodawanie ciężkich zależności.
+
+# ✱ SZYBKIE POLECENIA (smoke)
+- extract: `infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage`
+- impact: `infotracker impact -s "+dbo.orders.total_amount+" --graph-dir build/lineage --max-depth 3`
+- diff: `infotracker --config tmp.yml diff --base build/ol_base --head build/ol_head --format text`
+- tests: `pytest -q` lub selektywnie `pytest -k wildcard -q`
+
+# ✱ KONTEKST PROJEKTU (dla agenta)
+- Projekt: InfoTracker – lineage/diff dla MSSQL z OL artifacts; moduły: parser.py, engine.py, diff.py, models.py, lineage.py, cli.py.
+- Najważniejsze kontrakty: ObjectInfo, ColumnSchema, ColumnLineage, ColumnGraph.
+- Kluczowe inwarianty:
+  - parser: preprocess + fallback dla INSERT INTO … EXEC,
+  - engine: deps z ObjectInfo.dependencies,
+  - diff: filtr po severity_threshold,
+  - models: wildcardy case-insensitive + dopasowanie po sufiksie tabeli.

{infotracker-0.3.0 → infotracker-0.4.0}/.gitignore

@@ -36,7 +36,6 @@ docs/_build/
 (?i)*phase*.py
 (?i)*test*.md
 (?i)*phase*.md
-InfoTracker/PHASE2_COMPLETE.md
-InfoTracker/PHASE3_PLAN.md
-InfoTracker/test_phase2.py
-InfoTracker/test_comprehensive.py
+examples/warehouse/sql2
+
+*.txt

infotracker-0.4.0/PKG-INFO

@@ -0,0 +1,301 @@
+Metadata-Version: 2.4
+Name: InfoTracker
+Version: 0.4.0
+Summary: Column-level SQL lineage, impact analysis, and breaking-change detection (MS SQL first)
+Project-URL: homepage, https://example.com/infotracker
+Project-URL: documentation, https://example.com/infotracker/docs
+Author: InfoTracker Authors
+License: MIT
+Keywords: data-lineage,impact-analysis,lineage,mssql,openlineage,sql
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Database
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Requires-Dist: click
+Requires-Dist: networkx>=3.3
+Requires-Dist: packaging>=24.0
+Requires-Dist: pydantic>=2.8.2
+Requires-Dist: pyyaml>=6.0.1
+Requires-Dist: rich
+Requires-Dist: shellingham
+Requires-Dist: sqlglot>=23.0.0
+Requires-Dist: typer
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=7.4.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# InfoTracker
+
+**Column-level SQL lineage extraction and impact analysis for MS SQL Server**
+
+InfoTracker is a powerful command-line tool that parses T-SQL files and generates detailed column-level lineage in OpenLineage format. It supports advanced SQL Server features including table-valued functions, stored procedures, temp tables, and EXEC patterns.
+
+[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
+[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+[![PyPI](https://img.shields.io/badge/PyPI-InfoTracker-blue.svg)](https://pypi.org/project/InfoTracker/)
+
+## 🚀 Features
+
+- **Column-level lineage** - Track data flow at the column level with precise transformations
+- **Advanced SQL support** - T-SQL dialect with temp tables, variables, CTEs, and window functions
+- **Impact analysis** - Find upstream and downstream dependencies with flexible selectors
+- **Wildcard matching** - Support for table wildcards (`schema.table.*`) and column wildcards (`..pattern`)
+- **Breaking change detection** - Detect schema changes that could break downstream processes
+- **Multiple output formats** - Text tables or JSON for integration with other tools
+- **OpenLineage compatible** - Standard format for data lineage interoperability
+- **Advanced SQL objects** - Table-valued functions (TVF) and dataset-returning procedures
+- **Temp table tracking** - Full lineage through EXEC into temp tables
+
+## 📦 Installation
+
+### From PyPI (Recommended)
+```bash
+pip install InfoTracker
+```
+
+### From GitHub
+```bash
+# Latest stable release
+pip install git+https://github.com/InfoMatePL/InfoTracker.git
+
+# Development version
+git clone https://github.com/InfoMatePL/InfoTracker.git
+cd InfoTracker
+pip install -e .
+```
+
+### Verify Installation
+```bash
+infotracker --help
+```
+
+## ⚡ Quick Start
+
+### 1. Extract Lineage
+```bash
+# Extract lineage from SQL files
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+```
+
+### 2. Run Impact Analysis
+```bash
+# Find what feeds into a column (upstream)
+infotracker impact -s "+STG.dbo.Orders.OrderID"
+
+# Find what uses a column (downstream)  
+infotracker impact -s "STG.dbo.Orders.OrderID+"
+
+# Both directions
+infotracker impact -s "+dbo.fct_sales.Revenue+"
+```
+
+### 3. Detect Breaking Changes
+```bash
+# Compare two versions of your schema
+infotracker diff --base build/lineage --head build/lineage_new
+```
+## 📖 Selector Syntax
+
+InfoTracker supports flexible column selectors for precise impact analysis:
+
+| Selector Format | Description | Example |
+|-----------------|-------------|---------|
+| `table.column` | Simple format (adds default `dbo` schema) | `Orders.OrderID` |
+| `schema.table.column` | Schema-qualified format | `dbo.Orders.OrderID` |
+| `database.schema.table.column` | Database-qualified format | `STG.dbo.Orders.OrderID` |
+| `schema.table.*` | Table wildcard (all columns) | `dbo.fct_sales.*` |
+| `..pattern` | Column wildcard (name contains pattern) | `..revenue` |
+| `..pattern*` | Column wildcard with fnmatch | `..customer*` |
+
+### Direction Control
+- `selector` - downstream dependencies (default)
+- `+selector` - upstream sources  
+- `selector+` - downstream dependencies (explicit)
+- `+selector+` - both upstream and downstream
+
+## 💡 Examples
+
+### Basic Usage
+```bash
+# Extract lineage first (always run this before impact analysis)
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+
+# Basic column lineage
+infotracker impact -s "+dbo.fct_sales.Revenue"        # What feeds this column?
+infotracker impact -s "STG.dbo.Orders.OrderID+"      # What uses this column?
+```
+
+### Wildcard Selectors
+```bash
+# All columns from a specific table
+infotracker impact -s "dbo.fct_sales.*"
+infotracker impact -s "STG.dbo.Orders.*"
+
+# Find all columns containing "revenue" (case-insensitive)
+infotracker impact -s "..revenue"
+
+# Find all columns starting with "customer" 
+infotracker impact -s "..customer*"
+```
+
+### Advanced SQL Objects
+```bash
+# Table-valued function columns (upstream)
+infotracker impact -s "+dbo.fn_customer_orders_tvf.*"
+
+# Procedure dataset columns (upstream)  
+infotracker impact -s "+dbo.usp_customer_metrics_dataset.*"
+
+# Temp table lineage from EXEC
+infotracker impact -s "+#temp_table.*"
+```
+
+### Output Formats
+```bash
+# Text output (default, human-readable)
+infotracker impact -s "+..revenue"
+
+# JSON output (machine-readable)
+infotracker --format json impact -s "..customer*" > customer_lineage.json
+
+# Control traversal depth
+infotracker impact -s "+dbo.Orders.OrderID" --max-depth 2
+```
+
+### Breaking Change Detection
+```bash
+# Extract baseline
+infotracker extract --sql-dir sql_v1 --out-dir build/baseline
+
+# Extract new version  
+infotracker extract --sql-dir sql_v2 --out-dir build/current
+
+# Detect breaking changes
+infotracker diff --base build/baseline --head build/current
+
+# Filter by severity
+infotracker diff --base build/baseline --head build/current --threshold BREAKING
+```
+
+
+## Output Format
+
+Impact analysis returns these columns:
+- **from** - Source column (fully qualified)
+- **to** - Target column (fully qualified)  
+- **direction** - `upstream` or `downstream`
+- **transformation** - Type of transformation (`IDENTITY`, `ARITHMETIC`, `AGGREGATION`, `CASE_AGGREGATION`, `DATE_FUNCTION`, `WINDOW`, etc.)
+- **description** - Human-readable transformation description
+
+Results are automatically deduplicated. Use `--format json` for machine-readable output.
+
+### New Transformation Types
+
+The enhanced transformation taxonomy includes:
+- `ARITHMETIC_AGGREGATION` - Arithmetic operations combined with aggregation functions
+- `COMPLEX_AGGREGATION` - Multi-step calculations involving multiple aggregations  
+- `DATE_FUNCTION` - Date/time calculations like DATEDIFF, DATEADD
+- `DATE_FUNCTION_AGGREGATION` - Date functions applied to aggregated results
+- `CASE_AGGREGATION` - CASE statements applied to aggregated results
+
+### Advanced Object Support
+
+InfoTracker now supports advanced SQL Server objects:
+
+**Table-Valued Functions (TVF):**
+- Inline TVF (`RETURN AS SELECT`) - Parsed directly from SELECT statement
+- Multi-statement TVF (`RETURN @table TABLE`) - Extracts schema from table variable definition
+- Function parameters are tracked as filter metadata (don't create columns)
+
+**Dataset-Returning Procedures:**
+- Procedures ending with SELECT statement are treated as dataset sources
+- Output schema extracted from the final SELECT statement  
+- Parameters tracked as filter metadata affecting lineage scope
+
+**EXEC into Temp Tables:**
+- `INSERT INTO #temp EXEC procedure` patterns create edges from procedure columns to temp table columns
+- Temp table lineage propagates downstream to final targets
+- Supports complex workflow patterns combining functions, procedures, and temp tables
+
+## Configuration
+
+InfoTracker follows this configuration precedence:
+1. **CLI flags** (highest priority) - override everything
+2. **infotracker.yml** config file - project defaults  
+3. **Built-in defaults** (lowest priority) - fallback values
+
+## 🔧 Configuration
+
+Create an `infotracker.yml` file in your project root:
+
+```yaml
+sql_dirs:
+  - "sql/"
+  - "models/"
+out_dir: "build/lineage"
+exclude_dirs: 
+  - "__pycache__"
+  - ".git"
+severity_threshold: "POTENTIALLY_BREAKING"
+```
+
+### Configuration Options
+
+| Setting | Description | Default | Examples |
+|---------|-------------|---------|----------|
+| `sql_dirs` | Directories to scan for SQL files | `["."]` | `["sql/", "models/"]` |
+| `out_dir` | Output directory for lineage files | `"lineage"` | `"build/artifacts"` |
+| `exclude_dirs` | Directories to skip | `[]` | `["__pycache__", "node_modules"]` |
+| `severity_threshold` | Breaking change detection level | `"NON_BREAKING"` | `"BREAKING"` |
+
+## 📚 Documentation
+
+- **[Architecture](docs/architecture.md)** - Core concepts and design
+- **[Lineage Concepts](docs/lineage_concepts.md)** - Data lineage fundamentals  
+- **[CLI Usage](docs/cli_usage.md)** - Complete command reference
+- **[Configuration](docs/configuration.md)** - Advanced configuration options
+- **[DBT Integration](docs/dbt_integration.md)** - Using with DBT projects
+- **[OpenLineage Mapping](docs/openlineage_mapping.md)** - Output format specification
+- **[Breaking Changes](docs/breaking_changes.md)** - Change detection and severity levels
+- **[Advanced Use Cases](docs/advanced_use_cases.md)** - TVFs, stored procedures, and complex scenarios
+- **[Edge Cases](docs/edge_cases.md)** - SELECT *, UNION, temp tables handling
+- **[FAQ](docs/faq.md)** - Common questions and troubleshooting
+
+## 🧪 Testing
+
+```bash
+# Run all tests
+pytest
+
+# Run specific test categories
+pytest tests/test_parser.py     # Parser functionality
+pytest tests/test_wildcard.py   # Wildcard selectors
+pytest tests/test_adapter.py    # SQL dialect adapters
+
+# Run with coverage
+pytest --cov=infotracker --cov-report=html
+```
+
+
+
+
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 🙏 Acknowledgments
+
+- [SQLGlot](https://github.com/tobymao/sqlglot) - SQL parsing library
+- [OpenLineage](https://openlineage.io/) - Data lineage standard
+- [Typer](https://typer.tiangolo.com/) - CLI framework
+- [Rich](https://rich.readthedocs.io/) - Terminal formatting
+
+---
+
+**InfoTracker** - Making database schema evolution safer, one column at a time. 🎯 

infotracker-0.4.0/README.md

@@ -0,0 +1,270 @@
+# InfoTracker
+
+**Column-level SQL lineage extraction and impact analysis for MS SQL Server**
+
+InfoTracker is a powerful command-line tool that parses T-SQL files and generates detailed column-level lineage in OpenLineage format. It supports advanced SQL Server features including table-valued functions, stored procedures, temp tables, and EXEC patterns.
+
+[![Python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://python.org)
+[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
+[![PyPI](https://img.shields.io/badge/PyPI-InfoTracker-blue.svg)](https://pypi.org/project/InfoTracker/)
+
+## 🚀 Features
+
+- **Column-level lineage** - Track data flow at the column level with precise transformations
+- **Advanced SQL support** - T-SQL dialect with temp tables, variables, CTEs, and window functions
+- **Impact analysis** - Find upstream and downstream dependencies with flexible selectors
+- **Wildcard matching** - Support for table wildcards (`schema.table.*`) and column wildcards (`..pattern`)
+- **Breaking change detection** - Detect schema changes that could break downstream processes
+- **Multiple output formats** - Text tables or JSON for integration with other tools
+- **OpenLineage compatible** - Standard format for data lineage interoperability
+- **Advanced SQL objects** - Table-valued functions (TVF) and dataset-returning procedures
+- **Temp table tracking** - Full lineage through EXEC into temp tables
+
+## 📦 Installation
+
+### From PyPI (Recommended)
+```bash
+pip install InfoTracker
+```
+
+### From GitHub
+```bash
+# Latest stable release
+pip install git+https://github.com/InfoMatePL/InfoTracker.git
+
+# Development version
+git clone https://github.com/InfoMatePL/InfoTracker.git
+cd InfoTracker
+pip install -e .
+```
+
+### Verify Installation
+```bash
+infotracker --help
+```
+
+## ⚡ Quick Start
+
+### 1. Extract Lineage
+```bash
+# Extract lineage from SQL files
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+```
+
+### 2. Run Impact Analysis
+```bash
+# Find what feeds into a column (upstream)
+infotracker impact -s "+STG.dbo.Orders.OrderID"
+
+# Find what uses a column (downstream)  
+infotracker impact -s "STG.dbo.Orders.OrderID+"
+
+# Both directions
+infotracker impact -s "+dbo.fct_sales.Revenue+"
+```
+
+### 3. Detect Breaking Changes
+```bash
+# Compare two versions of your schema
+infotracker diff --base build/lineage --head build/lineage_new
+```
+## 📖 Selector Syntax
+
+InfoTracker supports flexible column selectors for precise impact analysis:
+
+| Selector Format | Description | Example |
+|-----------------|-------------|---------|
+| `table.column` | Simple format (adds default `dbo` schema) | `Orders.OrderID` |
+| `schema.table.column` | Schema-qualified format | `dbo.Orders.OrderID` |
+| `database.schema.table.column` | Database-qualified format | `STG.dbo.Orders.OrderID` |
+| `schema.table.*` | Table wildcard (all columns) | `dbo.fct_sales.*` |
+| `..pattern` | Column wildcard (name contains pattern) | `..revenue` |
+| `..pattern*` | Column wildcard with fnmatch | `..customer*` |
+
+### Direction Control
+- `selector` - downstream dependencies (default)
+- `+selector` - upstream sources  
+- `selector+` - downstream dependencies (explicit)
+- `+selector+` - both upstream and downstream
+
+## 💡 Examples
+
+### Basic Usage
+```bash
+# Extract lineage first (always run this before impact analysis)
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+
+# Basic column lineage
+infotracker impact -s "+dbo.fct_sales.Revenue"        # What feeds this column?
+infotracker impact -s "STG.dbo.Orders.OrderID+"      # What uses this column?
+```
+
+### Wildcard Selectors
+```bash
+# All columns from a specific table
+infotracker impact -s "dbo.fct_sales.*"
+infotracker impact -s "STG.dbo.Orders.*"
+
+# Find all columns containing "revenue" (case-insensitive)
+infotracker impact -s "..revenue"
+
+# Find all columns starting with "customer" 
+infotracker impact -s "..customer*"
+```
+
+### Advanced SQL Objects
+```bash
+# Table-valued function columns (upstream)
+infotracker impact -s "+dbo.fn_customer_orders_tvf.*"
+
+# Procedure dataset columns (upstream)  
+infotracker impact -s "+dbo.usp_customer_metrics_dataset.*"
+
+# Temp table lineage from EXEC
+infotracker impact -s "+#temp_table.*"
+```
+
+### Output Formats
+```bash
+# Text output (default, human-readable)
+infotracker impact -s "+..revenue"
+
+# JSON output (machine-readable)
+infotracker --format json impact -s "..customer*" > customer_lineage.json
+
+# Control traversal depth
+infotracker impact -s "+dbo.Orders.OrderID" --max-depth 2
+```
+
+### Breaking Change Detection
+```bash
+# Extract baseline
+infotracker extract --sql-dir sql_v1 --out-dir build/baseline
+
+# Extract new version  
+infotracker extract --sql-dir sql_v2 --out-dir build/current
+
+# Detect breaking changes
+infotracker diff --base build/baseline --head build/current
+
+# Filter by severity
+infotracker diff --base build/baseline --head build/current --threshold BREAKING
+```
+
+
+## Output Format
+
+Impact analysis returns these columns:
+- **from** - Source column (fully qualified)
+- **to** - Target column (fully qualified)  
+- **direction** - `upstream` or `downstream`
+- **transformation** - Type of transformation (`IDENTITY`, `ARITHMETIC`, `AGGREGATION`, `CASE_AGGREGATION`, `DATE_FUNCTION`, `WINDOW`, etc.)
+- **description** - Human-readable transformation description
+
+Results are automatically deduplicated. Use `--format json` for machine-readable output.
+
+### New Transformation Types
+
+The enhanced transformation taxonomy includes:
+- `ARITHMETIC_AGGREGATION` - Arithmetic operations combined with aggregation functions
+- `COMPLEX_AGGREGATION` - Multi-step calculations involving multiple aggregations  
+- `DATE_FUNCTION` - Date/time calculations like DATEDIFF, DATEADD
+- `DATE_FUNCTION_AGGREGATION` - Date functions applied to aggregated results
+- `CASE_AGGREGATION` - CASE statements applied to aggregated results
+
+### Advanced Object Support
+
+InfoTracker now supports advanced SQL Server objects:
+
+**Table-Valued Functions (TVF):**
+- Inline TVF (`RETURN AS SELECT`) - Parsed directly from SELECT statement
+- Multi-statement TVF (`RETURN @table TABLE`) - Extracts schema from table variable definition
+- Function parameters are tracked as filter metadata (don't create columns)
+
+**Dataset-Returning Procedures:**
+- Procedures ending with SELECT statement are treated as dataset sources
+- Output schema extracted from the final SELECT statement  
+- Parameters tracked as filter metadata affecting lineage scope
+
+**EXEC into Temp Tables:**
+- `INSERT INTO #temp EXEC procedure` patterns create edges from procedure columns to temp table columns
+- Temp table lineage propagates downstream to final targets
+- Supports complex workflow patterns combining functions, procedures, and temp tables
+
+## Configuration
+
+InfoTracker follows this configuration precedence:
+1. **CLI flags** (highest priority) - override everything
+2. **infotracker.yml** config file - project defaults  
+3. **Built-in defaults** (lowest priority) - fallback values
+
+## 🔧 Configuration
+
+Create an `infotracker.yml` file in your project root:
+
+```yaml
+sql_dirs:
+  - "sql/"
+  - "models/"
+out_dir: "build/lineage"
+exclude_dirs: 
+  - "__pycache__"
+  - ".git"
+severity_threshold: "POTENTIALLY_BREAKING"
+```
+
+### Configuration Options
+
+| Setting | Description | Default | Examples |
+|---------|-------------|---------|----------|
+| `sql_dirs` | Directories to scan for SQL files | `["."]` | `["sql/", "models/"]` |
+| `out_dir` | Output directory for lineage files | `"lineage"` | `"build/artifacts"` |
+| `exclude_dirs` | Directories to skip | `[]` | `["__pycache__", "node_modules"]` |
+| `severity_threshold` | Breaking change detection level | `"NON_BREAKING"` | `"BREAKING"` |
+
+## 📚 Documentation
+
+- **[Architecture](docs/architecture.md)** - Core concepts and design
+- **[Lineage Concepts](docs/lineage_concepts.md)** - Data lineage fundamentals  
+- **[CLI Usage](docs/cli_usage.md)** - Complete command reference
+- **[Configuration](docs/configuration.md)** - Advanced configuration options
+- **[DBT Integration](docs/dbt_integration.md)** - Using with DBT projects
+- **[OpenLineage Mapping](docs/openlineage_mapping.md)** - Output format specification
+- **[Breaking Changes](docs/breaking_changes.md)** - Change detection and severity levels
+- **[Advanced Use Cases](docs/advanced_use_cases.md)** - TVFs, stored procedures, and complex scenarios
+- **[Edge Cases](docs/edge_cases.md)** - SELECT *, UNION, temp tables handling
+- **[FAQ](docs/faq.md)** - Common questions and troubleshooting
+
+## 🧪 Testing
+
+```bash
+# Run all tests
+pytest
+
+# Run specific test categories
+pytest tests/test_parser.py     # Parser functionality
+pytest tests/test_wildcard.py   # Wildcard selectors
+pytest tests/test_adapter.py    # SQL dialect adapters
+
+# Run with coverage
+pytest --cov=infotracker --cov-report=html
+```
+
+
+
+
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 🙏 Acknowledgments
+
+- [SQLGlot](https://github.com/tobymao/sqlglot) - SQL parsing library
+- [OpenLineage](https://openlineage.io/) - Data lineage standard
+- [Typer](https://typer.tiangolo.com/) - CLI framework
+- [Rich](https://rich.readthedocs.io/) - Terminal formatting
+
+---
+
+**InfoTracker** - Making database schema evolution safer, one column at a time. 🎯 

{infotracker-0.3.0 → infotracker-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "InfoTracker"
-version = "0.3.0"
+version = "0.4.0"
 description = "Column-level SQL lineage, impact analysis, and breaking-change detection (MS SQL first)"
 readme = "README.md"
 requires-python = ">=3.10"