InfoTracker 0.1.0__py3-none-any.whl → 0.2.0__py3-none-any.whl

infotracker-0.2.0.dist-info/METADATA

@@ -0,0 +1,285 @@
+Metadata-Version: 2.4
+Name: InfoTracker
+Version: 0.2.0
+Summary: Column-level SQL lineage, impact analysis, and breaking-change detection (MS SQL first)
+Project-URL: homepage, https://example.com/infotracker
+Project-URL: documentation, https://example.com/infotracker/docs
+Author: InfoTracker Authors
+License: MIT
+Keywords: data-lineage,impact-analysis,lineage,mssql,openlineage,sql
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Topic :: Database
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Requires-Dist: click
+Requires-Dist: networkx>=3.3
+Requires-Dist: packaging>=24.0
+Requires-Dist: pydantic>=2.8.2
+Requires-Dist: pyyaml>=6.0.1
+Requires-Dist: rich
+Requires-Dist: shellingham
+Requires-Dist: sqlglot>=23.0.0
+Requires-Dist: typer
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
+Requires-Dist: pytest>=7.4.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# InfoTracker
+
+Column-level SQL lineage extraction and impact analysis for MS SQL Server
+
+## Features
+
+- **Column-level lineage** - Track data flow at the column level
+- **Parse SQL files** and generate OpenLineage-compatible JSON
+- **Impact analysis** - Find upstream and downstream column dependencies with flexible selectors
+- **Wildcard matching** - Support for table wildcards (`schema.table.*`) and column wildcards (`..pattern`)
+- **Direction control** - Query upstream (`+selector`), downstream (`selector+`), or both (`+selector+`)
+- **Configurable depth** - Control traversal depth with `--max-depth`
+- **Multiple output formats** - Text tables or JSON for scripting
+- **MSSQL support** - T-SQL dialect with temp tables, variables, and stored procedures
+- **Advanced SQL objects** - Support for table-valued functions (TVF) and dataset-returning procedures
+- **Temp table lineage** - Track EXEC into temp tables and propagate lineage downstream
+
+## Requirements
+- Python 3.10+
+- Virtual environment (activated)
+- Basic SQL knowledge
+- Git and shell
+
+## Troubleshooting
+- **Error tracebacks on help commands**: Make sure you're running in an activated virtual environment
+- **Command not found**: Activate your virtual environment first
+- **Import errors**: Ensure all dependencies are installed with `pip install -e .`
+- **Column not found**: Use full URI format or check column_graph.json for exact names
+
+## Quickstart
+
+### Setup & Installation
+```bash
+# Activate virtual environment first (REQUIRED)
+
+# Install dependencies
+pip install -e .
+
+# Verify installation
+infotracker --help
+```
+
+### Basic Usage
+```bash
+# 1. Extract lineage from SQL files (builds column graph)
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+
+# 2. Run impact analysis
+infotracker impact -s "STG.dbo.Orders.OrderID"  # downstream dependencies
+infotracker impact -s "+STG.dbo.Orders.OrderID" # upstream sources
+```
+
+## Selector Syntax
+
+InfoTracker supports flexible column selectors:
+
+| Selector Format | Description | Example |
+|-----------------|-------------|---------|
+| `table.column` | Simple format (adds default `dbo` schema) | `Orders.OrderID` |
+| `schema.table.column` | Schema-qualified format | `dbo.Orders.OrderID` |
+| `database.schema.table.column` | Database-qualified format | `STG.dbo.Orders.OrderID` |
+| `schema.table.*` | Table wildcard (all columns) | `dbo.fct_sales.*` |
+| `..pattern` | Column wildcard (name contains pattern) | `..revenue` |
+| `.pattern` | Alias for column wildcard | `.orderid` |
+| Full URI | Complete namespace format | `mssql://localhost/InfoTrackerDW.STG.dbo.Orders.OrderID` |
+
+### Direction Control
+- `selector` - downstream dependencies (default)
+- `+selector` - upstream sources  
+- `selector+` - downstream dependencies (explicit)
+- `+selector+` - both upstream and downstream
+
+### Selector Cheat Sheet
+
+**Table wildcards:**
+```bash
+# All columns from a specific table
+infotracker impact -s "dbo.fct_sales.*"
+infotracker impact -s "STG.dbo.Orders.*"
+```
+
+**Column name matching:**
+```bash
+# Find all columns containing "revenue" (case-insensitive)
+infotracker impact -s "..revenue"
+
+# Find all columns containing "id" 
+infotracker impact -s "..id"
+
+# Use wildcards for pattern matching
+infotracker impact -s "..customer*"
+```
+
+**Direction examples:**
+```bash
+# Upstream: what feeds into this column
+infotracker impact -s "+dbo.fct_sales.Revenue"
+
+# Downstream: what uses this column
+infotracker impact -s "STG.dbo.Orders.OrderID+"
+
+# Both directions
+infotracker impact -s "+dbo.dim_customer.CustomerID+"
+```
+
+**Advanced SQL objects:**
+```bash
+# Table-valued function columns (upstream)
+infotracker impact -s "+dbo.fn_customer_orders_tvf.*"
+
+# Procedure dataset columns (upstream)  
+infotracker impact -s "+dbo.usp_customer_metrics_dataset.*"
+
+# Temp table lineage from EXEC
+infotracker impact -s "+#temp_table.*"
+```
+
+## Examples
+
+```bash
+# Extract lineage (run this first)
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+
+# Basic column lineage
+infotracker impact -s "+dbo.fct_sales.Revenue"        # upstream sources
+infotracker impact -s "STG.dbo.Orders.OrderID+"      # downstream usage
+
+# Wildcard selectors
+infotracker impact -s "+..revenue+"                   # all revenue columns (both directions)
+infotracker impact -s "dbo.fct_sales.*"              # all columns from table
+infotracker --format json impact -s "..customer*"     # customer columns (JSON output)
+
+# Advanced SQL objects (NEW)
+infotracker impact -s "+dbo.fn_customer_orders_tvf.*"      # TVF columns (upstream)
+infotracker impact -s "+dbo.usp_customer_metrics_dataset.*" # procedure columns (upstream)
+
+# Depth control
+infotracker impact -s "+dbo.Orders.OrderID" --max-depth 1
+
+# Demo the new features with the included examples
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+infotracker impact -s "+dbo.fn_customer_orders_inline.*"
+infotracker impact -s "+dbo.usp_customer_metrics_dataset.TotalRevenue"
+```
+
+### Copy-Paste Demo Commands
+
+Test the new TVF and procedure lineage features:
+
+```bash
+# 1. Extract all lineage (including new TVF/procedure support)
+infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
+
+# 2. Test TVF lineage 
+infotracker --format text impact -s "+dbo.fn_customer_orders_tvf.*"
+
+# 3. Test procedure lineage
+infotracker --format text impact -s "+dbo.usp_customer_metrics_dataset.*"
+
+# 4. Test column name contains wildcard
+infotracker --format text impact -s "+..revenue"
+
+# 5. Show results in JSON format
+infotracker --format json impact -s "..total*" > tvf_lineage.json
+```
+
+## Output Format
+
+Impact analysis returns these columns:
+- **from** - Source column (fully qualified)
+- **to** - Target column (fully qualified)  
+- **direction** - `upstream` or `downstream`
+- **transformation** - Type of transformation (`IDENTITY`, `ARITHMETIC`, `AGGREGATION`, `CASE_AGGREGATION`, `DATE_FUNCTION`, `WINDOW`, etc.)
+- **description** - Human-readable transformation description
+
+Results are automatically deduplicated. Use `--format json` for machine-readable output.
+
+### New Transformation Types
+
+The enhanced transformation taxonomy includes:
+- `ARITHMETIC_AGGREGATION` - Arithmetic operations combined with aggregation functions
+- `COMPLEX_AGGREGATION` - Multi-step calculations involving multiple aggregations  
+- `DATE_FUNCTION` - Date/time calculations like DATEDIFF, DATEADD
+- `DATE_FUNCTION_AGGREGATION` - Date functions applied to aggregated results
+- `CASE_AGGREGATION` - CASE statements applied to aggregated results
+
+### Advanced Object Support
+
+InfoTracker now supports advanced SQL Server objects:
+
+**Table-Valued Functions (TVF):**
+- Inline TVF (`RETURN AS SELECT`) - Parsed directly from SELECT statement
+- Multi-statement TVF (`RETURN @table TABLE`) - Extracts schema from table variable definition
+- Function parameters are tracked as filter metadata (don't create columns)
+
+**Dataset-Returning Procedures:**
+- Procedures ending with SELECT statement are treated as dataset sources
+- Output schema extracted from the final SELECT statement  
+- Parameters tracked as filter metadata affecting lineage scope
+
+**EXEC into Temp Tables:**
+- `INSERT INTO #temp EXEC procedure` patterns create edges from procedure columns to temp table columns
+- Temp table lineage propagates downstream to final targets
+- Supports complex workflow patterns combining functions, procedures, and temp tables
+
+## Configuration
+
+InfoTracker follows this configuration precedence:
+1. **CLI flags** (highest priority) - override everything
+2. **infotracker.yml** config file - project defaults  
+3. **Built-in defaults** (lowest priority) - fallback values
+
+Create an `infotracker.yml` file in your project root:
+```yaml
+default_adapter: mssql
+sql_dir: examples/warehouse/sql
+out_dir: build/lineage
+include: ["*.sql"]
+exclude: ["*_wip.sql"]
+```
+
+## Documentation
+
+For detailed information:
+- `docs/overview.md` — what it is, goals, scope
+- `docs/algorithm.md` — how extraction works
+- `docs/lineage_concepts.md` — core concepts with visuals
+- `docs/cli_usage.md` — commands and options
+- `docs/breaking_changes.md` — definition and detection
+- `docs/edge_cases.md` — SELECT *, UNION, temp tables, etc.
+- `docs/adapters.md` — interface and MSSQL specifics
+- `docs/architecture.md` — system and sequence diagrams
+- `docs/configuration.md` — configuration reference
+- `docs/openlineage_mapping.md` — how outputs map to OpenLineage
+- `docs/faq.md` — common questions
+
+#### Documentation
+- `docs/overview.md` — what it is, goals, scope
+- `docs/algorithm.md` — how extraction works
+- `docs/lineage_concepts.md` — core concepts with visuals
+- `docs/cli_usage.md` — commands and options
+- `docs/breaking_changes.md` — definition and detection
+- `docs/edge_cases.md` — SELECT *, UNION, temp tables, etc.
+- `docs/advanced_use_cases.md` — tabular functions, procedures returning datasets
+- `docs/adapters.md` — interface and MSSQL specifics
+- `docs/architecture.md` — system and sequence diagrams
+- `docs/configuration.md` — configuration reference
+- `docs/openlineage_mapping.md` — how outputs map to OpenLineage
+- `docs/faq.md` — common questions
+- `docs/dbt_integration.md` — how to use with dbt projects
+
+
+## License
+MIT (or your team’s preferred license) 

infotracker-0.2.0.dist-info/RECORD

@@ -0,0 +1,15 @@
+infotracker/__init__.py,sha256=XkoK2R_QULA1UDQqgaLbmKQ2bdsi-lO3mo_wi7dy9Gg,57
+infotracker/__main__.py,sha256=_iCom0ddZ1myy6ly3ID1dBlLzzjf7iV7Kq9uUfkat74,121
+infotracker/adapters.py,sha256=UEQeGSS3_fMOc5_Jsrw5aTtmIXlOdqqbHWL2uSgqkGM,3011
+infotracker/cli.py,sha256=PQQoxqSmu8fSFTeGCdLKIKiY7WTcCzddiANYGc1qqe8,5666
+infotracker/config.py,sha256=pYriKZxwHNdU_nkk3n3R6d_h45izUj-BIkpLeFGASzE,1861
+infotracker/diff.py,sha256=LmIl3FL5NVxil6AFefrqQBkCCRonueg6BEXrnleVpw8,19796
+infotracker/engine.py,sha256=JlsrzPoB4Xe4qnTrEZ7emYP0K-zkqTqYOGzZiEZesks,23441
+infotracker/lineage.py,sha256=GcNflXSO5QhqJj9eJewlWwfL_86N4aHdEgoY3ESD6_U,4863
+infotracker/models.py,sha256=aQwU_4V69CnnHdgsybd99uvE3fzoQoW-nwn5aMhxdbU,14796
+infotracker/openlineage_utils.py,sha256=-g9Pkl5hOMQP2Rtu47ItHBC13z6Y0K3gEG6x9GrTJH8,5845
+infotracker/parser.py,sha256=8NVtCMvyt7l_dIfAydR_VJGB7A_NBLb2T827ac8uMXc,70255
+infotracker-0.2.0.dist-info/METADATA,sha256=E36Q9HtPU3eY4lNMRFhwCl3WtpePiwN8KimoEhF-VOo,10449
+infotracker-0.2.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+infotracker-0.2.0.dist-info/entry_points.txt,sha256=5ulAYRSvW3SohjeMwlYRX6LoWIHkEtc1qnwxWJQgN2Y,59
+infotracker-0.2.0.dist-info/RECORD,,

infotracker-0.1.0.dist-info/METADATA

@@ -1,108 +0,0 @@
-Metadata-Version: 2.4
-Name: InfoTracker
-Version: 0.1.0
-Summary: Column-level SQL lineage, impact analysis, and breaking-change detection (MS SQL first)
-Project-URL: homepage, https://example.com/infotracker
-Project-URL: documentation, https://example.com/infotracker/docs
-Author: InfoTracker Authors
-License: MIT
-Keywords: data-lineage,impact-analysis,lineage,mssql,openlineage,sql
-Classifier: Environment :: Console
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Topic :: Database
-Classifier: Topic :: Software Development :: Libraries
-Requires-Python: <3.13,>=3.10
-Requires-Dist: click<9.0.0,>=8.1.3
-Requires-Dist: networkx>=3.3
-Requires-Dist: packaging>=24.0
-Requires-Dist: pydantic>=2.8.2
-Requires-Dist: pyyaml>=6.0.1
-Requires-Dist: sqlglot>=23.0.0
-Requires-Dist: typer[all]==0.12.3
-Provides-Extra: dev
-Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
-Requires-Dist: pytest>=7.4.0; extra == 'dev'
-Description-Content-Type: text/markdown
-
-### InfoTracker
-
-This is a Python CLI that extracts column-level lineage from SQL, runs impact analysis, and detects breaking changes. First adapter targets MS SQL.
-
-#### For Students
-Start with a simple command: `infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage`. This analyzes SQL files in the directory.
-
-#### Setup & Installation
-```bash
-# Activate virtual environment first (REQUIRED)
-source infotracker-env/bin/activate  # or your venv path
-
-# Install dependencies
-pip install -e .
-
-# Verify installation
-infotracker --help
-```
-
-#### Quickstart
-```bash
-# IMPORTANT: Always run InfoTracker commands in the activated virtual environment
-
-# Extract lineage from all SQL files
-infotracker extract --sql-dir examples/warehouse/sql --out-dir build/lineage
-
-# Impact analysis (downstream dependencies)
-infotracker impact -s dbo.fct_sales.Revenue+
-
-# Impact analysis (upstream sources)
-infotracker impact -s +dbo.Orders.OrderID
-
-# Branch diff for breaking changes
-infotracker diff --base main --head feature/x --sql-dir examples/warehouse/sql
-```
-
-#### Configuration
-InfoTracker follows this configuration precedence:
-1. **CLI flags** (highest priority) - override everything
-2. **infotracker.yml** config file - project defaults  
-3. **Built-in defaults** (lowest priority) - fallback values
-
-Create an `infotracker.yml` file in your project root:
-```yaml
-default_adapter: mssql
-sql_dir: examples/warehouse/sql
-out_dir: build/lineage
-include: ["*.sql"]
-exclude: ["*_wip.sql"]
-severity_threshold: BREAKING
-```
-
-#### Documentation
-- `docs/overview.md` — what it is, goals, scope
-- `docs/algorithm.md` — how extraction works
-- `docs/lineage_concepts.md` — core concepts with visuals
-- `docs/cli_usage.md` — commands and options
-- `docs/breaking_changes.md` — definition and detection
-- `docs/edge_cases.md` — SELECT *, UNION, temp tables, etc.
-- `docs/adapters.md` — interface and MSSQL specifics
-- `docs/architecture.md` — system and sequence diagrams
-- `docs/configuration.md` — configuration reference
-- `docs/openlineage_mapping.md` — how outputs map to OpenLineage
-- `docs/faq.md` — common questions
-- `docs/dbt_integration.md` — how to use with dbt projects
-
-#### Requirements
-- Python 3.10+
-- Virtual environment (activated)
-- Basic SQL knowledge
-- Git and shell
-
-#### Troubleshooting
-- **Error tracebacks on help commands**: Make sure you're running in an activated virtual environment
-- **Command not found**: Activate your virtual environment first
-- **Import errors**: Ensure all dependencies are installed with `pip install -e .`
-
-#### License
-MIT (or your team’s preferred license) 

infotracker-0.1.0.dist-info/RECORD

@@ -1,14 +0,0 @@
-infotracker/__init__.py,sha256=XkoK2R_QULA1UDQqgaLbmKQ2bdsi-lO3mo_wi7dy9Gg,57
-infotracker/__main__.py,sha256=_iCom0ddZ1myy6ly3ID1dBlLzzjf7iV7Kq9uUfkat74,121
-infotracker/adapters.py,sha256=mfP5uep8HN4Xi-78rAKA8PqUY6q17amXLd2Zf0Jl1xU,2535
-infotracker/cli.py,sha256=NER5jByK2AwAZKbqdxPsE713TdGkirE_tarKLs-aMjg,4841
-infotracker/config.py,sha256=EKlETgAvbSQJbty87TsKMTbFjwI16RMBpOPrduw3pMk,1725
-infotracker/diff.py,sha256=ziNFmNW6SEppdeD_iIdEsNHVYHT3AQdB2IQVPzhIRIk,12470
-infotracker/engine.py,sha256=xW-10Jfxm0-MvPUBe1il37GBkNdchc8IC0-EZfZzPfk,13557
-infotracker/lineage.py,sha256=Gb8NQj-igf8wjdyMhfkSmB1_bVS__cipMivXJCRmiJQ,4645
-infotracker/models.py,sha256=cBsGOGmmGFfyz26TE-endA4AZc0qxzVMQ_7nZCJlcVA,10398
-infotracker/parser.py,sha256=Mr4pWGCaodwEA6jKzf6HkLHjQGHNaHlqCFXv9jDjVtA,36269
-infotracker-0.1.0.dist-info/METADATA,sha256=y9rMRDCAQ8faIngE7fU9gUT-26e6GopjJ-c3AxOM16A,3749
-infotracker-0.1.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-infotracker-0.1.0.dist-info/entry_points.txt,sha256=5ulAYRSvW3SohjeMwlYRX6LoWIHkEtc1qnwxWJQgN2Y,59
-infotracker-0.1.0.dist-info/RECORD,,