coltrace 0.1.0.dev4__py3-none-any.whl

coltrace-0.1.0.dev4.dist-info/METADATA

@@ -0,0 +1,241 @@
+Metadata-Version: 2.4
+Name: coltrace
+Version: 0.1.0.dev4
+Summary: Static Pandas column lineage, impact analysis, and local dependency graphs
+Author: Cristiano Diniz
+License-Expression: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: networkx<4,>=3.2
+Requires-Dist: typer<1,>=0.12
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: ruff>=0.9; extra == "dev"
+Requires-Dist: mypy>=1.14; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: types-networkx>=3.2; extra == "dev"
+Provides-Extra: viz
+Requires-Dist: pyvis<0.4,>=0.3.2; extra == "viz"
+Dynamic: license-file
+
+# ColTrace
+
+Static Pandas column lineage and potential impact analysis for Python projects.
+ColTrace reads source files, identifies supported transformations, and connects
+column versions across direct function calls. It never imports or executes the
+project or reads the project's input data.
+
+**Development preview: `0.1.0.dev4`.** The supported subset is deliberately small.
+Unsupported operations produce diagnostics and may interrupt lineage. A graph
+with no edge is not proof that a dependency does not exist.
+
+## Install locally
+
+Requires Python 3.11+. From this checkout:
+
+```sh
+python -m venv .venv
+# Activate .venv using your shell, then:
+python -m pip install -e ".[dev,viz]"
+coltrace --help
+```
+
+On Windows without activation, use `.\.venv\Scripts\python.exe -m coltrace`.
+The `viz` extra supplies PyVis for HTML export; basic analysis and JSON need only
+NetworkX and Typer. Pandas is not a runtime dependency. Publication to PyPI is a
+separate release step; these commands install the local project.
+
+## Try the complete example
+
+```sh
+coltrace scan examples/pricing_pipeline
+coltrace trace preco_referencia
+coltrace impact preco_referencia
+coltrace graph preco_referencia --open
+coltrace export preco_referencia --format json --output .coltrace/pricing.json
+```
+
+The example connects extraction, pricing, and feature functions through explicit
+calls in `pipeline.py`. No `prices.csv` file is needed to analyze it.
+
+```text
+valor_produto -> preco_bruto -> preco_referencia -> margem
+                                      |              |
+                                      +--------------+-> margem_percentual
+custo ---------------------------------------------> margem
+```
+
+`impact preco_referencia` identifies `margem` and `margem_percentual` as direct
+potential dependents. There is also a longer path through `margem`. A changed
+input does not necessarily change every downstream value.
+
+`scan` writes `.coltrace/scan.json` by default. `--out DIRECTORY` selects another
+snapshot directory. Queries load that snapshot explicitly; they do not silently
+rescan the project. Rescan after editing source, or use `--project DIRECTORY` to
+analyze current files in memory. `--from FILE` selects an existing snapshot.
+
+```sh
+coltrace trace margem --project examples/pricing_pipeline --upstream --depth 2
+coltrace impact preco_referencia --from .coltrace/scan.json --evidence confirmed
+coltrace graph --from .coltrace/scan.json --output .coltrace/all.html
+```
+
+HTML files contain their visualization resources and work locally without a
+server or CDN. A selected column opens its journey: direct reads, applications,
+writes, derived columns and destinations. Frame passage is labeled separately
+from a direct column read. Callback control and configuration appear in the
+expandable creation history, rather than as direct uses of the selected column.
+The graph uses generously spaced, numbered blocks in a vertical flow. The
+scrollable sidebar contains the trace history, source code, and locations;
+selecting a step highlights its matching graph block, and vice versa. Open Details
+or Diagnostics without losing the current selection. The Outcomes tab shows
+derived-column destinations and the DataFrames carrying the selected column.
+Zoom in/out, use **Fit all**
+for an overview, or **Reset view** to return to a readable scale.
+`--view columns` shows the column dependency projection; `--operations` includes
+operation nodes in that projection. A graph without a selected column shows the
+whole project's column dependencies.
+
+```sh
+coltrace scan examples/text_pipeline
+coltrace trace comment_text --function process_train
+coltrace graph comment_text --function process_train --open
+```
+
+`trace` defaults to direct column actions, derived-column outcomes, and containing
+DataFrames. `--full-history` includes creation ancestry and callback internals,
+with control/configuration distinguished from value transformations.
+`--diagnostics` also displays dependency lists and relevant diagnostics. Rescan after an
+upgrade: schema 0.2 snapshots have no recorded history. See the
+[column history guide](docs/column-history.md) and [current checkpoint](docs/history-checkpoint.md).
+The [graph workspace checkpoint](docs/graph-workspace-checkpoint.md) describes the
+current layout and [additional column checks](docs/other-column-checkpoint.md).
+See the [column journey checkpoint](docs/column-journey-checkpoint.md) for the
+usage-first view and its acceptance cases.
+
+## Identity and ambiguity
+
+Different DataFrames keep different identities, even when both contain a column
+named `score`. Assignments create column versions. `frame["a"] = frame["a"] + 1`
+connects the old and new versions; `frame["a"] = 0` cuts the old value's lineage.
+History still records that overwrite without adding a false value dependency.
+
+Selection by name uses the latest observed version within each DataFrame. If
+several unrelated identities remain, the CLI lists candidates and exits with status 3.
+Same-name copies connected by proven pass-through relations form one selectable
+family, whose original observed column is selected. A rename, value derivation,
+or shared label alone does not establish that equivalence.
+Select one explicitly using `--column-id ID` or `--dataframe ID`. Use `--version`
+for an older version, and `--module` or `--function` for exact source filters.
+IDs are opaque, deterministic within equivalent scans, and may change when source
+locations or call contexts change. They are not permanent database keys.
+
+`--depth` counts column-to-column hops. `--relation derive|rename|pass` filters
+relations, while `--evidence confirmed|all` filters their evidence. Filtering out
+intermediate relations can disconnect a path. Direct and transitive lists use
+the shortest eligible distance, so a node reached both ways appears as direct.
+
+## Supported patterns
+
+| Pattern | Support |
+| --- | --- |
+| Explicit Pandas imports and aliases | Recognized origins; names such as `df` alone are insufficient |
+| `read_csv`, `read_parquet` | External schema is unknown; CSV chunk/iterator forms are excluded |
+| Literal dictionary `DataFrame` construction | String column keys and literal values |
+| `frame["column"]`, `frame.loc[:, "column"]` | Single literal string labels, all rows |
+| `frame["c"] = frame["a"] + frame["b"]` | Arithmetic, scalar constants, temporary values, versions |
+| `alias = frame` | Same abstract object and shared mutation |
+| `rename(columns={"old": "new"})` | Literal, non-colliding map; new DataFrame, or shared mutation with `inplace=True` |
+| `assign(b=lambda x: x["a"] * 2)` | Sequential simple lambdas; eager arguments use their original scope |
+| `Series.apply(clean_text)` | Resolved scalar callbacks; bounded summaries and inferred evidence |
+| `frame[["a", "b"]] = DataFrame(results.tolist(), index=frame.index)` | Fixed callback output width and matching source index |
+| `dropna`, literal projection, boolean row filters | Preserve value lineage; predicate consumption tracked separately |
+| `to_parquet` / `to_csv` | Located export consumption with literal options |
+| Direct synchronous function calls | Explicit arguments/returns, bounded acyclic contexts and explicit imports |
+| Annotated, uncalled functions | Separate inferred templates, not evidence of pipeline connections |
+
+See [the supported subset and boundaries](docs/supported-patterns.md) for the exact
+contract. Calls, branches, dynamic columns, and Pandas operations outside that
+contract produce located diagnostics rather than invented connections.
+
+The [text pipeline example](examples/text_pipeline/README.md) demonstrates
+independent train/evaluation flows, multi-output text cleaning, row filters, and
+exports. See the [NLP checkpoint](docs/nlp-checkpoint.md) for what was validated
+against a real project and which downstream third-party operations remain opaque.
+
+## Evidence and completeness
+
+- `confirmed`: supported syntax and receiver identity establish the relation.
+- `inferred`: the relation depends on an explicit assumption, such as an uncalled
+  function's DataFrame annotation or an unknown external schema during rename.
+- `unresolved`: insufficient evidence across an unsupported boundary.
+
+`complete`, `partial`, and `unresolved` describe how much of an operation is
+understood; evidence and completeness are separate. External column reads are
+marked as external/unknown origins. They do not identify the real upstream data
+source's column definition or guarantee its presence.
+
+Use `--diagnostics` to inspect warnings and errors. Query output also includes
+relevant gaps. An inconclusive trace prints `Not determined` for empty dependency
+lists and shows the recognized DataFrame origin separately from column evidence.
+See [troubleshooting incomplete traces](docs/troubleshooting.md).
+The analyzer is not a Python type checker or an execution
+validator. Its grammar is limited by the interpreter running ColTrace. Pandas
+imports are assumed to refer to the real library; environment/module impersonation
+is not resolved.
+
+## Python API
+
+```python
+from coltrace import analyze_project
+from coltrace.graph import LineageGraph
+
+report = analyze_project("examples/pricing_pipeline")
+graph = LineageGraph(report)
+column = graph.select("preco_referencia")
+impact = graph.impact(column.id)
+print(report.to_json())
+```
+
+The earlier `scan_project` API remains available for raw occurrences and receiver
+evidence. Its discovery report (`0.1`) is a different format from the lineage
+snapshot (`0.3`). See [discovery JSON](docs/discovery-json.md) and
+[lineage JSON](docs/lineage-json.md). Public facts are dataclasses; their serialized
+form is independent of NetworkX and contains no AST or executable objects.
+
+## Configuration
+
+Only the scanned root's `pyproject.toml` is read:
+
+```toml
+[tool.coltrace]
+exclude = ["generated", "legacy/*.py", "*_test.py"]
+```
+
+Glob patterns match case-sensitive, project-relative POSIX paths. Patterns with
+no `/` also match individual path components. Built-in exclusions cover `.venv`,
+`venv`, `.git`, `__pycache__`, `.coltrace`, pytest/mypy/Ruff caches, `build`, and
+`dist`. Linked files/directories and Windows reparse points are skipped.
+`ScanConfig(exclude=("generated",))` overrides TOML settings and extends built-ins.
+
+## Development
+
+```sh
+python -m pytest -q
+python -m ruff check src tests examples
+python -m ruff format --check src tests examples
+python -m mypy
+python -m build
+```
+
+Tests include independent expected dependencies, forbidden edges, alias/version
+semantics, cross-module calls, CLI, malformed snapshots, offline HTML, and a
+sentinel proving that analyzed source is not executed. See the
+[accuracy protocol](docs/accuracy.md), [current checkpoint](docs/nlp-checkpoint.md),
+[decision log](docs/decisions.md), and [backlog](docs/backlog.md).
+
+The GitHub Actions workflow configures Windows/Linux with Python 3.11–3.13. It
+must run in the repository before those combinations can be claimed as verified.
+
+Licensed under the [MIT License](LICENSE). Copyright 2026 Cristiano Diniz.

coltrace-0.1.0.dev4.dist-info/RECORD

@@ -0,0 +1,22 @@
+coltrace/__init__.py,sha256=94i70hEERBM8qDz0xPiNIHaF9extfuAI07oGh1hv6rE,358
+coltrace/__main__.py,sha256=XjZeA9-YaZ07jZrK-kQE7vmqTpG0dvFhYw_3nZlygSY,137
+coltrace/analysis.py,sha256=F3cbHMgDaX6wcE7hy-tZ9T67Dt-HDG2XLTiVHvdHXw4,93365
+coltrace/callbacks.py,sha256=YF-dqFKcKJrwpQ7Afy4EGMNDMuxiNedCQEOwOaNHrV8,24906
+coltrace/cli.py,sha256=_3QL9TlGxlE2iF4yfwGKeclYMRZ6dwjMVxzGP04z6oc,27088
+coltrace/collector.py,sha256=MKshuU6uG1QNFLAmjf6aJs9SXlew2NOtyvXqVJMr5Rs,21936
+coltrace/exporters.py,sha256=OyshvS-Rm4Q_zfb0OxCiZ1DASonmh8CYXukfdp7RyrM,28433
+coltrace/graph.py,sha256=6BhClA1VxBtEbq-J7iZJg09GTNISrx8Vp9x3mtHboWo,17529
+coltrace/history.py,sha256=JE7P_U9NIYtzQWARbaU6xIVIluQIiQySxEtWxMp0z1E,17675
+coltrace/journey.py,sha256=M60ZZm-74avYYEQCRGSyaHdVa-nNxKyxaftduyyPnyw,24827
+coltrace/lineage_models.py,sha256=QEpxJ1NbDMbsOPCrstDHepmS7wYbvrnPUA8VCw66o0Q,2665
+coltrace/models.py,sha256=8BJMdj9Z_WxWKyh3hrosaroYLghXJqoWvxlPHnO4lrQ,2246
+coltrace/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+coltrace/scanner.py,sha256=zdILpuEPcJpzugby8CN-uy34h6-k2KuAfVfoCqBa66Q,8413
+coltrace/visual_layout.py,sha256=1ICyfebm_cjixPrqfjv2FzLrf0k04GEDDw82bbcofDY,6708
+coltrace/templates/graph.html,sha256=66NKL6egh6emU6jFLnesjmax5xfbmC_pihdNGFq9Ij4,30141
+coltrace-0.1.0.dev4.dist-info/licenses/LICENSE,sha256=TLXAlpQbJLqXQZRfkicbF_sNmQDyL6yy04Rc1RT11Mo,1072
+coltrace-0.1.0.dev4.dist-info/METADATA,sha256=H-vV2POPgq--APZcdL0JUO3s2PnxkzciJxiX0i46MoY,11962
+coltrace-0.1.0.dev4.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
+coltrace-0.1.0.dev4.dist-info/entry_points.txt,sha256=nYK3AJlNnIRmGqc5nCR-bbGB90EoGtJcxE4uW00HnrE,46
+coltrace-0.1.0.dev4.dist-info/top_level.txt,sha256=cvliY6pFzj97aZEYfG3MHSOOzCWxReXFIzv8ELMCVOU,9
+coltrace-0.1.0.dev4.dist-info/RECORD,,

coltrace-0.1.0.dev4.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (84.2.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

coltrace-0.1.0.dev4.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+coltrace = coltrace.cli:app

coltrace-0.1.0.dev4.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Cristiano Diniz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

coltrace-0.1.0.dev4.dist-info/top_level.txt

@@ -0,0 +1 @@
+coltrace