tmdl-lens 0.4.0__tar.gz

tmdl_lens-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Marcin Mozol
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tmdl_lens-0.4.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.4
+Name: tmdl-lens
+Version: 0.4.0
+Summary: Automated documentation generator for Power BI projects.
+Author-email: Marcin Mozol <marcin.mozol@pm.me>
+License-Expression: MIT
+Keywords: powerbi,pbip,tmdl,power-bi,documentation,dax
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Documentation
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: gui
+Requires-Dist: customtkinter>=5.2.0; extra == "gui"
+Requires-Dist: watchdog>=4.0.0; extra == "gui"
+Dynamic: license-file
+
+# tmdl-lens
+
+Automated documentation generator for Power BI projects.
+
+Reads TMDL files from a folder of `.pbip` reports and generates structured HTML or Markdown documentation for each one - covering data sources, tables, measures, relationships, M parameters, security roles, and calculation groups.
+
+Runs as a standalone Windows desktop app. Output can be HTML (opens in any browser) or Markdown (README.md for GitHub). A live file watcher regenerates documentation automatically on every save.
+
+---
+
+## What it documents
+
+See a [real example output](sample/README.md) generated from the included sample report.
+
+Each generated README includes:
+
+- **Data Sources** - connector type and resolved source detail for every query, covering 30+ connectors including SQL, Dataflows, SharePoint, Excel, OData, Azure storage, Fabric, Databricks, Snowflake, Salesforce, and more. Unknown connectors are auto-detected and flagged for manual labelling.
+- **Table Details** - table type, source detail, and column list
+- **Measures** - full DAX (optional), display folder, and format string
+- **Relationships** - cardinality, cross-filter direction, and active/inactive state
+- **M Parameters** - current value and which source expressions reference each parameter
+- **Security Roles** - static and dynamic RLS rules per table, flagged by type
+- **Calculation Groups** - calculation items with DAX and ordinal order
+- **Model Statistics** - table, column, measure, and relationship counts
+
+---
+
+## Installation
+
+### Option A - Run from source
+
+Requires Python 3.12+.
+
+```
+git clone https://github.com/nox-magistralis/tmdl-lens.git
+cd tmdl-lens
+pip install -r requirements.txt
+python main.py
+```
+
+### Option B - Standalone executable
+
+Download the latest `tmdl-lens.exe` from the [Releases](https://github.com/nox-magistralis/tmdl-lens/releases) page. No Python or pip required - just run the `.exe` directly.
+
+A `config.json` file will be created alongside the executable on first run to store your settings.
+
+> **Note:** Windows may show a SmartScreen warning on first run. Click **More info** then **Run anyway**. This is expected for unsigned applications.
+
+---
+
+## Usage
+
+1. Launch the app
+2. Under **Configure**, set the **Reports folder** to the root of your `.pbip` projects
+3. Set an **Output folder** if you want READMEs written somewhere other than next to each `.pbip`
+4. Press **Run Now** to generate documentation for all reports
+5. Enable **Watch for TMDL changes** to auto-regenerate on every save
+
+If no output folder is set, each `README.md` is written next to its `.pbip` file. If a separate output folder is set, each report gets its own named subfolder inside it - for example `docs/SalesReport/README.md`, `docs/FinanceReport/README.md` - so multiple reports never collide.
+
+Optional documentation metadata (owner, team, refresh schedule) is set in the **Configure** tab and stored in `config.json` next to the app. Values left blank are omitted from the generated output.
+
+By default the app skips reports whose TMDL files and documentation settings have not changed since the last run - each report is hashed, and only reports that actually moved are regenerated. Turn this off with **Skip reports with no TMDL changes** in **Configure**.
+
+On startup the app validates the config and logs a warning for a missing or empty reports folder, a missing output folder, a watch debounce outside 1-300, or an output format other than `html` or `md`.
+
+---
+
+## Requirements
+
+| Package | Version |
+|---|---|
+| customtkinter | >= 5.2.0 |
+| watchdog | >= 4.0.0 |
+
+Python 3.12 or later required when running from source.
+
+---
+
+## Development
+
+```
+pip install -r requirements-dev.txt
+python -m pytest
+```
+
+`requirements-dev.txt` adds dev-only dependencies (pytest, pyinstaller) on top of the base `requirements.txt`. The test suite runs against the bundled sample report and covers the TMDL parser, source resolution, config validation, and README/HTML generation.
+
+---
+
+## Project structure
+
+```
+tmdl-lens/
+  main.py                   entry point
+  requirements.txt
+  requirements-dev.txt      dev-only dependencies (pytest, pyinstaller)
+  pytest.ini
+  src/
+    app.py                  CustomTkinter UI
+    pipeline.py             documentation generation pipeline
+    tmdl_parser.py          TMDL file parser
+    source_resolver.py      M expression source resolver
+    readme_generator.py     README markdown generator
+    watcher.py              file watcher (watchdog)
+    config.py               app settings + documentation metadata (config.json)
+  sample/                   sample .pbip project for testing
+  tests/                    pytest suite
+```
+
+---
+
+## Author
+
+**Marcin Mozol** - [github.com/nox-magistralis](https://github.com/nox-magistralis)
+
+---
+
+## License
+
+MIT - see [LICENSE](LICENSE)

tmdl_lens-0.4.0/README.md

@@ -0,0 +1,121 @@
+# tmdl-lens
+
+Automated documentation generator for Power BI projects.
+
+Reads TMDL files from a folder of `.pbip` reports and generates structured HTML or Markdown documentation for each one - covering data sources, tables, measures, relationships, M parameters, security roles, and calculation groups.
+
+Runs as a standalone Windows desktop app. Output can be HTML (opens in any browser) or Markdown (README.md for GitHub). A live file watcher regenerates documentation automatically on every save.
+
+---
+
+## What it documents
+
+See a [real example output](sample/README.md) generated from the included sample report.
+
+Each generated README includes:
+
+- **Data Sources** - connector type and resolved source detail for every query, covering 30+ connectors including SQL, Dataflows, SharePoint, Excel, OData, Azure storage, Fabric, Databricks, Snowflake, Salesforce, and more. Unknown connectors are auto-detected and flagged for manual labelling.
+- **Table Details** - table type, source detail, and column list
+- **Measures** - full DAX (optional), display folder, and format string
+- **Relationships** - cardinality, cross-filter direction, and active/inactive state
+- **M Parameters** - current value and which source expressions reference each parameter
+- **Security Roles** - static and dynamic RLS rules per table, flagged by type
+- **Calculation Groups** - calculation items with DAX and ordinal order
+- **Model Statistics** - table, column, measure, and relationship counts
+
+---
+
+## Installation
+
+### Option A - Run from source
+
+Requires Python 3.12+.
+
+```
+git clone https://github.com/nox-magistralis/tmdl-lens.git
+cd tmdl-lens
+pip install -r requirements.txt
+python main.py
+```
+
+### Option B - Standalone executable
+
+Download the latest `tmdl-lens.exe` from the [Releases](https://github.com/nox-magistralis/tmdl-lens/releases) page. No Python or pip required - just run the `.exe` directly.
+
+A `config.json` file will be created alongside the executable on first run to store your settings.
+
+> **Note:** Windows may show a SmartScreen warning on first run. Click **More info** then **Run anyway**. This is expected for unsigned applications.
+
+---
+
+## Usage
+
+1. Launch the app
+2. Under **Configure**, set the **Reports folder** to the root of your `.pbip` projects
+3. Set an **Output folder** if you want READMEs written somewhere other than next to each `.pbip`
+4. Press **Run Now** to generate documentation for all reports
+5. Enable **Watch for TMDL changes** to auto-regenerate on every save
+
+If no output folder is set, each `README.md` is written next to its `.pbip` file. If a separate output folder is set, each report gets its own named subfolder inside it - for example `docs/SalesReport/README.md`, `docs/FinanceReport/README.md` - so multiple reports never collide.
+
+Optional documentation metadata (owner, team, refresh schedule) is set in the **Configure** tab and stored in `config.json` next to the app. Values left blank are omitted from the generated output.
+
+By default the app skips reports whose TMDL files and documentation settings have not changed since the last run - each report is hashed, and only reports that actually moved are regenerated. Turn this off with **Skip reports with no TMDL changes** in **Configure**.
+
+On startup the app validates the config and logs a warning for a missing or empty reports folder, a missing output folder, a watch debounce outside 1-300, or an output format other than `html` or `md`.
+
+---
+
+## Requirements
+
+| Package | Version |
+|---|---|
+| customtkinter | >= 5.2.0 |
+| watchdog | >= 4.0.0 |
+
+Python 3.12 or later required when running from source.
+
+---
+
+## Development
+
+```
+pip install -r requirements-dev.txt
+python -m pytest
+```
+
+`requirements-dev.txt` adds dev-only dependencies (pytest, pyinstaller) on top of the base `requirements.txt`. The test suite runs against the bundled sample report and covers the TMDL parser, source resolution, config validation, and README/HTML generation.
+
+---
+
+## Project structure
+
+```
+tmdl-lens/
+  main.py                   entry point
+  requirements.txt
+  requirements-dev.txt      dev-only dependencies (pytest, pyinstaller)
+  pytest.ini
+  src/
+    app.py                  CustomTkinter UI
+    pipeline.py             documentation generation pipeline
+    tmdl_parser.py          TMDL file parser
+    source_resolver.py      M expression source resolver
+    readme_generator.py     README markdown generator
+    watcher.py              file watcher (watchdog)
+    config.py               app settings + documentation metadata (config.json)
+  sample/                   sample .pbip project for testing
+  tests/                    pytest suite
+```
+
+---
+
+## Author
+
+**Marcin Mozol** - [github.com/nox-magistralis](https://github.com/nox-magistralis)
+
+---
+
+## License
+
+MIT - see [LICENSE](LICENSE)

tmdl_lens-0.4.0/pyproject.toml

@@ -0,0 +1,37 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "tmdl-lens"
+description = "Automated documentation generator for Power BI projects."
+readme = "README.md"
+requires-python = ">=3.12"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Marcin Mozol", email = "marcin.mozol@pm.me" }]
+keywords = ["powerbi", "pbip", "tmdl", "power-bi", "documentation", "dax"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3 :: Only",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Documentation",
+]
+dynamic = ["version"]
+dependencies = []
+
+[project.optional-dependencies]
+gui = ["customtkinter>=5.2.0", "watchdog>=4.0.0"]
+
+[project.scripts]
+tmdl-lens = "tmdl_lens.cli:main"
+
+[tool.setuptools.dynamic]
+version = { attr = "tmdl_lens.version.__version__" }
+
+[tool.setuptools.packages.find]
+include = ["tmdl_lens*"]

tmdl_lens-0.4.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

tmdl_lens-0.4.0/tests/test_config.py

@@ -0,0 +1,49 @@
+"""
+test_config.py — pytest tests for src.config.
+"""
+
+from tmdl_lens.config import validate_config
+
+
+def test_valid_config_no_warnings(tmp_path):
+    warnings = validate_config({
+        "reports_folder": str(tmp_path),
+        "output_folder":  str(tmp_path),
+        "watch_debounce": 10,
+        "output_format":  "md",
+    })
+    assert warnings == []
+
+
+def test_empty_reports_folder_warns():
+    warnings = validate_config({"reports_folder": ""})
+    assert any("reports_folder" in w for w in warnings)
+
+
+def test_missing_reports_folder_warns(tmp_path):
+    missing = str(tmp_path / "does-not-exist")
+    warnings = validate_config({"reports_folder": missing})
+    assert any("does not exist" in w for w in warnings)
+
+
+def test_missing_output_folder_warns(tmp_path):
+    warnings = validate_config({
+        "reports_folder": str(tmp_path),
+        "output_folder":  str(tmp_path / "does-not-exist"),
+    })
+    assert any("output_folder" in w for w in warnings)
+
+
+def test_watch_debounce_out_of_range_warns():
+    warnings = validate_config({"watch_debounce": 500})
+    assert any("watch_debounce" in w for w in warnings)
+
+
+def test_watch_debounce_non_numeric_warns():
+    warnings = validate_config({"watch_debounce": "abc"})
+    assert any("watch_debounce" in w for w in warnings)
+
+
+def test_invalid_output_format_warns():
+    warnings = validate_config({"output_format": "pdf"})
+    assert any("output_format" in w for w in warnings)

tmdl_lens-0.4.0/tests/test_hidden_references.py

@@ -0,0 +1,115 @@
+"""
+test_hidden_references.py — pytest tests for hidden-object reference
+detection used by the README/HTML generators.
+"""
+
+from tmdl_lens.readme_generator import (
+    _build_hidden_reference_map,
+    _find_hidden_references,
+    _hidden_reference_note,
+)
+from tmdl_lens.tmdl_parser import Column, Measure, SemanticModel, Table
+
+
+def _build_test_model() -> SemanticModel:
+    """A small model with hidden columns, hidden tables and ambiguous names."""
+    sales = Table(
+        name="Sales",
+        table_type="fact",
+        columns=[
+            Column(name="Visible", data_type="int64"),
+            Column(name="Secret", data_type="int64", is_hidden=True),
+        ],
+        measures=[
+            Measure(name="Visible Measure", dax_expression="SUM('Sales'[Visible])"),
+            Measure(
+                name="Hidden Measure",
+                dax_expression="SUM('Sales'[Visible])",
+                is_hidden=True,
+            ),
+        ],
+    )
+    helper = Table(
+        name="Helper",
+        table_type="helper",
+        is_hidden=True,
+        columns=[Column(name="Key", data_type="int64")],
+        measures=[Measure(name="Open Count", dax_expression="COUNTROWS('Helper')")],
+    )
+    t1 = Table(
+        name="T1",
+        table_type="dim",
+        measures=[Measure(name="Amb", dax_expression="1", is_hidden=True)],
+    )
+    t2 = Table(
+        name="T2",
+        table_type="dim",
+        measures=[Measure(name="Amb", dax_expression="1", is_hidden=True)],
+    )
+    return SemanticModel(report_name="Test", tables=[sales, helper, t1, t2])
+
+
+def _maps(model):
+    return _build_hidden_reference_map(model)
+
+
+def test_build_map_hidden_objects():
+    tables, hidden_measures, hidden_columns = _maps(_build_test_model())
+    assert hidden_measures["Hidden Measure"] == "Sales"
+    assert hidden_measures["Open Count"] == "Helper"      # via hidden table
+    assert set(hidden_measures["Amb"]) == {"T1", "T2"}    # ambiguous
+    assert hidden_columns[("Sales", "Secret")] is True
+    assert hidden_columns[("Helper", "Key")] is True      # via hidden table
+    assert ("Sales", "Visible") not in hidden_columns
+
+
+def test_hidden_column_reference():
+    tables, hm, hc = _maps(_build_test_model())
+    refs = _find_hidden_references("X = 'Sales'[Secret]", tables, hm, hc)
+    assert refs == [("Sales[Secret]", "Sales")]
+
+
+def test_hidden_table_column_reference():
+    tables, hm, hc = _maps(_build_test_model())
+    refs = _find_hidden_references("X = 'Helper'[Key]", tables, hm, hc)
+    assert refs == [("Helper[Key]", "Helper")]
+
+
+def test_hidden_measure_reference():
+    tables, hm, hc = _maps(_build_test_model())
+    refs = _find_hidden_references("X = [Hidden Measure]", tables, hm, hc)
+    assert refs == [("[Hidden Measure]", "Sales")]
+
+
+def test_ambiguous_measure_skipped():
+    tables, hm, hc = _maps(_build_test_model())
+    refs = _find_hidden_references("X = [Amb]", tables, hm, hc)
+    assert refs == []
+
+
+def test_string_literals_ignored():
+    tables, hm, hc = _maps(_build_test_model())
+    dax = '"Sales[Secret]" & [Hidden Measure]'
+    refs = _find_hidden_references(dax, tables, hm, hc)
+    assert refs == [("[Hidden Measure]", "Sales")]
+
+
+def test_visible_references_not_flagged():
+    tables, hm, hc = _maps(_build_test_model())
+    dax = "'Sales'[Visible] + [Visible Measure]"
+    refs = _find_hidden_references(dax, tables, hm, hc)
+    assert refs == []
+
+
+def test_empty_expression():
+    tables, hm, hc = _maps(_build_test_model())
+    assert _find_hidden_references("", tables, hm, hc) == []
+
+
+def test_hidden_reference_note():
+    refs = [("Sales[Secret]", "Sales"), ("[Hidden Measure]", "Sales")]
+    note = _hidden_reference_note(refs)
+    assert note.startswith("⚠ References hidden:")
+    assert "`Sales[Secret]`" in note
+    assert "`[Hidden Measure]`" in note
+    assert _hidden_reference_note([]) == ""

tmdl_lens-0.4.0/tests/test_parse.py

@@ -0,0 +1,155 @@
+"""
+test_parse.py — pytest tests for tmdl_parser and source_resolver.
+
+Run from the repo root:  python -m pytest
+"""
+
+
+def test_report_name(sample_model):
+    assert sample_model.report_name == "tmdl-lens Test Report"
+
+
+def test_tables_count(sample_model):
+    assert len(sample_model.tables) == 8
+
+
+def test_table_names(sample_model):
+    names = sorted(t.name for t in sample_model.tables)
+    assert names == [
+        "_measures",
+        "cg-time-intelligence",
+        "dim-date",
+        "dim-product",
+        "fact-sales",
+        "helper-order-lookup",
+        "param-metric-selector",
+        "source-sql-staging",
+    ]
+
+
+def test_table_classification(sample_model):
+    types = {t.name: t.table_type for t in sample_model.tables}
+    assert types == {
+        "_measures":             "measures_only",
+        "cg-time-intelligence":  "calc_group",
+        "dim-date":              "calculated",
+        "dim-product":           "dim",
+        "fact-sales":            "fact",
+        "helper-order-lookup":   "helper",
+        "param-metric-selector": "field_parameter",
+        "source-sql-staging":    "staging",
+    }
+
+
+def test_hidden_and_not_loaded_tables(sample_model):
+    helper = next(t for t in sample_model.tables if t.name == "helper-order-lookup")
+    assert helper.is_hidden is True
+    staging = next(t for t in sample_model.tables if t.name == "source-sql-staging")
+    assert staging.is_loaded is False
+
+
+def test_m_parameters(sample_model):
+    assert len(sample_model.m_parameters) == 2
+    by_name = {p.name: p for p in sample_model.m_parameters}
+    assert by_name["ServerName"].value == "fake-server.database.windows.net"
+    assert by_name["DatabaseName"].value == "SalesDB"
+
+
+def test_source_expressions(sample_model):
+    assert len(sample_model.source_expressions) == 21
+
+
+def test_relationships(sample_model):
+    assert len(sample_model.relationships) == 3
+    pairs = {
+        (r.from_table, r.from_column, r.to_table, r.to_column)
+        for r in sample_model.relationships
+    }
+    assert ("fact-sales", "order_date", "dim-date", "Date") in pairs
+    assert ("fact-sales", "customer_id", "dim-product", "product_id") in pairs
+    assert ("fact-sales", "ship_date", "dim-date", "Date") in pairs
+    assert sum(1 for r in sample_model.relationships if not r.is_active) == 1
+
+
+def test_measures(sample_model):
+    by_table = {t.name: t.measures for t in sample_model.tables}
+    assert len(by_table["_measures"]) == 4
+    assert len(by_table["fact-sales"]) == 2
+    assert len(by_table["helper-order-lookup"]) == 1
+    names = {m.name for m in by_table["_measures"]}
+    assert names == {
+        "Total Sales Amount",
+        "Order Count",
+        "Avg Order Value",
+        "Sales YTD",
+    }
+
+
+def test_resolved_sources_tiers(sample_resolved):
+    assert len(sample_resolved) == 21
+    tier1 = sum(1 for rs in sample_resolved.values() if rs.resolution_tier == 1)
+    tier2 = sum(1 for rs in sample_resolved.values() if rs.resolution_tier == 2)
+    tier3 = sum(1 for rs in sample_resolved.values() if rs.resolution_tier == 3)
+    assert tier1 == 18
+    assert tier2 == 1
+    assert tier3 == 2
+
+
+def test_resolved_sql_source(sample_resolved):
+    rs = sample_resolved["source-sql-direct"]
+    assert rs.source_type == "connector"
+    assert rs.resolution_tier == 1
+    assert "SalesDB" in rs.label
+    assert "dbo.orders" in rs.label
+
+
+def test_resolved_derived_chain(sample_resolved):
+    rs = sample_resolved["source-derived"]
+    assert rs.resolution_tier == 2
+    assert rs.derived_from == "source-sql-direct"
+    assert rs.chain[0] == "source-sql-direct"
+
+
+def test_tier3_are_unresolved(sample_resolved):
+    tier3 = [rs for rs in sample_resolved.values() if rs.resolution_tier == 3]
+    assert len(tier3) == 2
+    assert all(rs.unresolved for rs in tier3)
+    names = sorted(rs.expression_name for rs in tier3)
+    assert names == ["source-dynamic", "source-via-custom-function"]
+
+
+def test_security_roles(sample_model):
+    assert len(sample_model.security_roles) == 5
+    names = {r.name for r in sample_model.security_roles}
+    assert names == {
+        "Regional Managers",
+        "Employees",
+        "Legacy Users",
+        "Area Supervisors",
+        "Administrators",
+    }
+    dynamic = {
+        r.name: r.dynamic_function
+        for r in sample_model.security_roles
+        if r.is_dynamic
+    }
+    assert dynamic == {
+        "Employees":   "USERPRINCIPALNAME",
+        "Legacy Users": "USERNAME",
+    }
+    admins = next(r for r in sample_model.security_roles if r.name == "Administrators")
+    assert admins.table_filters == []
+
+
+def test_calculation_groups(sample_model):
+    cgs = [t for t in sample_model.tables if t.table_type == "calc_group"]
+    assert len(cgs) == 1
+    cg = cgs[0]
+    assert cg.name == "cg-time-intelligence"
+    assert len(cg.calculation_items) == 5
+    assert [i.name for i in cg.calculation_items] == [
+        "YTD", "MTD", "Rolling 12M", "Prior Year", "YoY %",
+    ]
+    ytd = cg.calculation_items[0]
+    assert "TOTALYTD" in ytd.dax_expression
+    assert ytd.format_string_expression == "SELECTEDMEASUREFORMATSTRING()"