py-tbparse 0.3.0__tar.gz → 0.4.4__tar.gz

{py_tbparse-0.3.0 → py_tbparse-0.4.4}/AGENTS.md

@@ -14,14 +14,15 @@ type="join">` and 2020.2+ `<relationships>`), inferred relationships,
 dashboards/dashboard-sheets, `validate_relationships`, custom/initial SQL,
 and published-source detection. **Not yet ported** (v2): formatting,
 tooltips, colors, axes, sorts, dashboard layout/actions, analytics
-helpers (calc complexity, field usage, replication brief), and the
+helpers (calc complexity, replication brief), and the
 Shiny-inspector equivalent.
 
 Also added, with no R equivalent (Python-native extras -- see "Non-R
 modules" below): Graphviz DOT export of the relationship graph
 (replaces the R package's igraph/ggraph-based plotting with a
 dependency-free alternative), workbook-to-workbook diff, folder/batch
-analysis across many workbooks, and Jupyter rich display.
+analysis across many workbooks, field renaming, field usage, workbook
+templates, and Jupyter rich display.
 
 ## Setup
 
@@ -120,11 +121,14 @@ beyond the R package's scope:
 | Python module | What it is |
 |---|---|
 | `_tables.py` | Name → `TwbParser`-accessor registry shared by `cli.py`, `webgui.py`, `diff.py`, and `batch.py`. Adding a new extractor to `TwbParser`? Add it here too so it's automatically available everywhere else. |
-| `cli.py` | `py-tbparse` command-line entry point, plus the `diff`/`batch`/`rename` subcommands (dispatched on `sys.argv[1]` before the normal single-workbook argparse parser runs) |
-| `webgui.py` | `py-tbparse-gui`: stdlib-only (`http.server` + vanilla JS) local browser GUI, no GUI toolkit dependency. Loosely fills the role of the R package's `run_twbparser_app`/Shiny inspector. |
-| `graph.py` | `to_dot()`: Graphviz DOT export of joins/relationships (+ optional inferred, as dashed edges). Replaces the R package's igraph/ggraph-based `plot_dependency_graph`/`plot_relationship_graph` with a dependency-free text format any Graphviz-compatible tool can render. |
+| `cli.py` | `py-tbparse` command-line entry point, plus the `diff`/`batch`/`rename`/`template` subcommands (dispatched on `sys.argv[1]` before the normal single-workbook argparse parser runs) |
+| `webgui.py` + `webui/` | `py-tbparse-gui`: stdlib-only (`http.server` + vanilla JS) local browser GUI, no GUI toolkit dependency. The server is `webgui.py`; the page is `webui/` (`index.html`, `tokens.css`, `themes.css`, `app.css`, `table.js`, `graph.js`, `app.js`). Loosely fills the role of the R package's `run_twbparser_app`/Shiny inspector. |
+| `graph.py` | `to_dot()`: Graphviz DOT export of joins/relationships (+ optional inferred, as dashed edges); `graph_data()` is the same edges as plain data, which the GUI lays out and draws itself (`webui/graph.js`: layered layout, SVG, pan/zoom, keyboard). Replaces the R package's igraph/ggraph-based `plot_dependency_graph`/`plot_relationship_graph` with a dependency-free text format any Graphviz-compatible tool can render. |
 | `diff.py` | `diff_tables()`/`diff_workbooks()`: row-level added/removed diff between two workbooks' same-named table, via `_tables.TABLE_SPECS`. No "changed" classification without a natural key — a changed row shows as one removed + one added row. |
-| `rename.py` | `suggest_field_renames()` (clean-name suggestions, optionally matched against a "before" reference), `load_rename_mapping()` (read an edited CSV back), `compare_field_schemas()` (fields with no counterpart across a datasource switch) and `apply_field_renames()`/`build_renamed_workbook()` (write a copy with captions set; never overwrites). Also the `field-renames` table in `_tables.py`. |
+| `rename.py` | `suggest_field_renames()` (clean-name suggestions, optionally matched against a "before" reference), `suggest_renames()` (the same for every kind of object: field, parameter, worksheet, dashboard, datasource, folder, hierarchy; adds a `kind` column), `load_rename_mapping()` (read an edited CSV back), `compare_field_schemas()` (fields with no counterpart across a datasource switch) and `apply_field_renames()`/`build_renamed_workbook()` (write a copy with captions set; never overwrites). Also the `field-renames` and `report-renames` tables in `_tables.py`. A worksheet/dashboard rename must rewrite every reference (`_SHEET_REFERENCES`); if you learn of another place Tableau writes a sheet name, add it there. |
+| `report.py` | `workbook_report()`: the overview's report card (summary sentence, health checks, dashboards with their sheets) built from `validate_relationships`, `field_usage` and `usage.missing_references`. Every health item names the table and column filters that list exactly the rows it counted; a test keeps count and rows equal on all 200 corpus workbooks. |
+| `usage.py` | `field_usage()`: for every field, the worksheets, dashboards and calculations that use it, followed through calculations, groups/sets and bins (the `field-usage` table). Python-native rather than a port of the R package's field-usage helper. |
+| `templates.py` | Workbook templates. `make_template()` writes a `.twbx` with a `template.json` manifest (required fields from `usage.py`, parameters, connections; data, extracts and credentials stripped); `read_data()` describes new data (CSV, or a `.twb`/`.twbx`/`.tds`); `suggest_mapping()` pairs fields with columns (reuses `rename._match_key`, type checks against the field's *physical* type, `datatype-customized` fields keep their type); `apply_template()` replaces the datasource's connection, keeping every field's local name, sets parameters and stores `template-answers.json`. A CSV connection is written in the 2020.2+ object-model shape (both `_.fcp.ObjectModelEncapsulateLegacy` relations, object graph, table column, `object-id` per record) when the template uses it -- keep those in step if you touch one. Untested in Tableau itself. |
 | `batch.py` | `scan_folder()`: runs one table across every `.twb`/`.twbx` in a directory, concatenated with a `workbook` column. Skips (with a warning) any file that fails to load/extract rather than aborting the batch. |
 
 ## Porting conventions (read before adding/modifying a function)
@@ -174,6 +178,12 @@ beyond the R package's scope:
   synthetic XML snippet (see `tests/test_joins.py`,
   `tests/test_dashboards.py` for the pattern — many are lifted from the R
   functions' own `@examples` roxygen blocks).
+- `tests/corpus/` is 200 real workbooks (permissive licences, pinned by blob sha in
+  `manifest.csv`, licence texts in `licenses/`). The files are gitignored; fetch with
+  `python scripts/fetch_corpus.py`. `tests/test_corpus.py` skips without them. Run it when you
+  change anything that reads workbook XML: it found a Unicode matching bug the hand-made
+  fixtures could not. Add to the corpus only from repositories whose licence permits
+  redistribution, and record the licence text.
 - `tests/conftest.py` provides `wenjie_xml`, `wenjie_path`,
   `zip_twbx_path` fixtures and an `xml_from_string()` helper. Tests import
   it with `from conftest import xml_from_string` (no `tests/__init__.py`,
@@ -194,21 +204,38 @@ string became a literal newline, splitting a JS string literal across
 two lines) killed the entire script, so no handlers bound and the UI was
 inert — with the whole suite green.
 
-So: **any change to `webgui.py`'s `_PAGE` needs a browser test**, in
+So: **any change to the page (`py_tbparse/webui/`) needs a browser test**, in
+The browser suite is split by concern, all sharing the fixtures in `tests/test_gui_browser.py`:
+`test_gui_table.py` (windowing invariants, pipeline vs. a Python oracle, a seeded random walk;
+`PYTBPARSE_WALK_SEEDS`/`PYTBPARSE_WALK_STEPS` widen it), `test_gui_a11y.py` (roles, keyboard, focus, rendered
+contrast in both themes) and `test_gui_layout.py` (no sideways overflow from 320 px up, layout stability).
+Design decisions the tests pin: only sorting uses a view transition (Chromium sends clicks to the page root while
+one runs); column `MIN_WIDTH` is 80; per-table view settings are keyed by column name; disabled menu items use
+`aria-disabled` so they stay focusable.
+
 `tests/test_gui_browser.py`, which runs the page in real headless
 Chromium via Playwright and fails on any uncaught JS error. The cheap
 structural guards in `test_webgui.py` (unterminated string literals,
 bracket balance) are a backstop, not a substitute.
 
-`_PAGE` is declared as `r"""..."""` (a **raw** string) specifically so
-this can't recur: without `r`, any backslash escape meant for the
-*browser* (`\n`, `\t`, a future `\'`) would need doubling in the Python
-source, and forgetting to double it silently corrupts the embedded JS
-instead of erroring. Keep it raw — write JS escapes the normal JS way
-(`'\n'`, not `'\\n'`).
-
-Any server-side value spliced into `_PAGE` (currently `TABLE_NAMES` and
-the preloaded workbook path) must go through `_json_for_script()`, not
+The page is real files, not a Python string: `py_tbparse/webui/index.html`, `tokens.css` (the design
+tokens), `app.css` and `app.js`. They are served by `webgui.py` from a fixed whitelist under
+`/static/` (`_ASSETS`), so a request can never reach any other file; add a new asset to
+`_ASSETS` and to the `webui/*` package-data (pyproject and MANIFEST.in) or it will not ship.
+`index.html` is the only thing that gets server values: `_render_index()` replaces
+`<!--APP_CONFIG-->` (now in `<head>`, so the saved theme is on `<html>` before the first paint) with the single
+inline `<script>`. Keep it that way (one inline script plus the three external ones, `table.js`, `graph.js`, then
+`app.js`); a test counts them. `populateTables();` must appear exactly once
+in `app.js`, and the structural tests (unterminated string literals, bracket balance) read both served scripts.
+Keep apostrophes out of JS strings and comments (the test counts quotes per line).
+
+`POST /upload` (drag and drop, Open file) takes raw bytes with `Content-Type: application/octet-stream` and the name
+in `X-Filename`: neither is CORS-safelisted, so another site cannot send it without a preflight. It streams to a
+`mkdtemp` directory, refuses over `MAX_UPLOAD_BYTES` (413) and content that does not match the extension, and keeps
+only one upload at a time. An uploaded workbook cannot "create beside the original" (409), only download.
+
+Any server-side value spliced into the page (currently `TABLE_NAMES`, the preloaded workbook path
+and the version, all in `_render_index()`'s config script) must go through `_json_for_script()`, not
 bare `json.dumps()`. `json.dumps` doesn't escape `/`, so a value
 containing the literal text `</script>` closes the script tag early in
 the browser's HTML parser — this is real, not theoretical, since the
@@ -232,6 +259,35 @@ leaves inputs empty and tests time out) and may crash rendering. Don't add
 startup, which makes collection fail for anyone who doesn't have it
 installed.
 
+The table is windowed (`table.js`, class `VTable`): rows have a fixed height (`--row-h`) and only the rows
+near the viewport exist in the page, so code and tests that read rows from the DOM see a window, not the table.
+Use `aria-rowcount` for the size and the column menu's "Copy column values" to read a whole column (the tests do:
+`_copy_column`). Never render every row; the old table needed over 4 s to sort 20,000 rows and could not draw
+50,000. Data work (search index, typed sort) is in `app.js`; each render records the User Timing measure
+`py-tbparse:table`, which the speed tests read. The budgets (first paint of 1,000 rows 150 ms, filtering 50,000 rows
+100 ms) are enforced at twice the figure. Heavy tests skip below 1.5 GB of free memory (`_enough_memory`); this
+sandbox has only 2 CPUs and 7.9 GB, and a 50,000-row DOM render has crashed it before.
+
+The page's colours, spacing, type and motion are tokens in `webui/tokens.css`; use them rather than literals.
+Colour themes are `webui/themes.css`: each theme is a complete colour set in light and dark, selected by
+`<html data-theme data-mode>` (Auto is resolved to light/dark by the head script and kept in step by `app.js`;
+dark colours live under `[data-mode="dark"]`, there is no `prefers-color-scheme` query). To add a theme, add its
+two blocks there, its id to `webgui.THEMES` and its label to `THEME_LABELS` in `app.js`; the token test checks it.
+`tests/test_webui_tokens.py` reads those files and fails if any text/background pair drops below WCAG AA
+4.5:1, if the stylesheet uses an undefined variable, or if text is faded with `opacity` (use `--faint`).
+Motion durations come from the `--dur-*` tokens, which `prefers-reduced-motion` sets to instant; keep new
+animations on those tokens.
+
+`scripts/readme_screenshots.py` retakes the README's four images from the made-up `docs/demo/coffee-shop.twb`
+(built by `scripts/make_demo_workbook.py`, which is the one place that workbook is defined); run both after a
+visible change.
+
+`scripts/gui_screenshots.py WORKBOOK OUT_DIR [--compare BASELINE_DIR]` captures nine GUI states (start,
+overview and fields in light and dark, renames, graph, phone width) deterministically. Use it for GUI
+refactors: a pure refactor must compare all-identical to the baseline taken before it; a redesign is expected to
+differ, so review the new look and re-capture the baseline. The redesign plan and its decisions are in
+`docs/ui-redesign-plan.md`.
+
 ## Commit / PR conventions
 
 Nothing project-specific beyond the harness defaults — see repo commit

{py_tbparse-0.3.0 → py_tbparse-0.4.4}/MANIFEST.in

@@ -5,3 +5,4 @@ recursive-include tests *.py
 include tests/fixtures/*.twb
 include tests/fixtures/*.twbx
 recursive-include docs *.png
+recursive-include py_tbparse/webui *

py_tbparse-0.4.4/PKG-INFO

@@ -0,0 +1,137 @@
+Metadata-Version: 2.4
+Name: py-tbparse
+Version: 0.4.4
+Summary: Native Python port of the twbparser R package: parse Tableau .twb/.twbx workbooks into pandas DataFrames.
+Author: DDSNA
+License: MIT
+Project-URL: Homepage, https://github.com/DDSNA/py-tbparse
+Project-URL: Source, https://github.com/DDSNA/py-tbparse
+Project-URL: Issues, https://github.com/DDSNA/py-tbparse/issues
+Keywords: tableau,twb,twbx,workbook,parser,pandas
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Office/Business
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: lxml>=4.9
+Requires-Dist: pandas>=1.5
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Provides-Extra: browser
+Requires-Dist: playwright>=1.40; extra == "browser"
+Provides-Extra: dev
+Requires-Dist: build>=1.0; extra == "dev"
+Requires-Dist: twine>=5.0; extra == "dev"
+Dynamic: license-file
+
+# py-tbparse
+
+[![PyPI](https://img.shields.io/pypi/v/py-tbparse)](https://pypi.org/project/py-tbparse/)
+[![Python](https://img.shields.io/pypi/pyversions/py-tbparse)](https://pypi.org/project/py-tbparse/)
+[![CI](https://github.com/DDSNA/py-tbparse/actions/workflows/ci.yml/badge.svg)](https://github.com/DDSNA/py-tbparse/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/DDSNA/py-tbparse/blob/main/LICENSE)
+
+Read Tableau workbooks (`.twb` and `.twbx`) from Python. You get datasources, fields, calculated fields, joins, relationships and dashboards as pandas DataFrames. It is plain Python on top of `lxml` and `pandas`, so Tableau and R are not needed.
+
+It began as a port of PrigasG's R package [twbparser](https://github.com/PrigasG/twbparser). The command-line tool, the browser GUI, renaming, templates, workbook diffing and folder scanning are new here.
+
+![The overview report card for a small demo workbook](https://raw.githubusercontent.com/DDSNA/py-tbparse/main/docs/gui-overview.png)
+
+## What it does
+
+- Lists what is in a workbook: datasources, parameters, fields, calculations, joins, relationships, dashboards, custom SQL, published sources.
+- Checks relationships and finds calculations that refer to fields the workbook does not have.
+- Shows which worksheets, dashboards and calculations use each field.
+- Suggests clean names for fields, sheets, dashboards and other objects, and writes a renamed copy.
+- Turns a finished workbook into a template you can fill with other data.
+- Compares two workbooks, scans a folder of them, and draws the data model as a graph.
+- Works from Python, from the command line, or in a local browser page.
+
+## Install
+
+```bash
+pip install py-tbparse
+```
+
+Python 3.9 or newer. The package, the import and both commands all use the same name: `pip install py-tbparse`, `import py_tbparse`, `py-tbparse`, `py-tbparse-gui`. Releases up to 0.2.0 used `twbparser_py` and `twbparser`; those names are gone.
+
+## Quick start
+
+From Python:
+
+```python
+from py_tbparse import TwbParser
+
+p = TwbParser("workbook.twb")    # or a .twbx
+p.get_overview()                 # counts of everything
+p.get_calculated_fields()        # each calculation and its formula
+p.get_relationships()            # how the tables connect
+```
+
+Every getter returns a DataFrame. The others are `get_datasources`, `get_parameters`, `get_fields`, `get_raw_fields`, `get_joins`, `get_relations`, `get_inferred_relationships`, `get_dashboards`, `get_dashboard_sheets`, `get_custom_sql`, `get_initial_sql`, `get_published_refs` and `get_field_usage`. `get_relationship_graph_dot()` returns the data model as Graphviz text and `validate()` looks for problems in the relationships. In Jupyter, a bare `p` shows the overview.
+
+From the command line:
+
+```bash
+py-tbparse workbook.twb                    # overview
+py-tbparse workbook.twb calculated-fields
+py-tbparse workbook.twb fields --format csv -o fields.csv
+py-tbparse workbook.twb validate           # exit code 2 if it finds a problem
+```
+
+In the browser:
+
+```bash
+py-tbparse-gui workbook.twb
+```
+
+Across workbooks:
+
+```python
+from py_tbparse import TwbParser, diff_workbooks, scan_folder
+
+diff_workbooks(TwbParser("v1.twb"), TwbParser("v2.twb"), table="datasources")
+scan_folder("./workbooks", table="datasources")   # one table, every workbook in the folder
+```
+
+## Guides
+
+- [Command line](https://github.com/DDSNA/py-tbparse/blob/main/docs/cli.md): every table, the `diff`, `batch`, `rename` and `template` commands, and their options.
+- [Browser GUI](https://github.com/DDSNA/py-tbparse/blob/main/docs/gui.md): opening files, the table, the overview, the graph, themes.
+- [Renaming](https://github.com/DDSNA/py-tbparse/blob/main/docs/renaming.md): clean names after a datasource switch, editing the suggestions, what will stay broken.
+- [Templates](https://github.com/DDSNA/py-tbparse/blob/main/docs/templates.md): make a template from a workbook and apply it to new data.
+- [`.twbx` files](https://github.com/DDSNA/py-tbparse/blob/main/docs/twbx.md): they are read straight from the zip; how to extract the contents.
+- [Development](https://github.com/DDSNA/py-tbparse/blob/main/docs/development.md): running the tests, the workbook corpus, the browser tests.
+
+## Limits
+
+- **Nothing the tool writes has been opened in Tableau yet.** That covers renamed workbooks and workbooks made from templates. The XML follows what Tableau writes, but open one on a copy and check it before relying on it. The original file is never modified or overwritten.
+- **Only part of the R package is ported.** Missing: formatting, tooltips, colors, axes and sorts, dashboard layout and actions, calculation complexity, the replication brief and the Shiny inspector. The GUI covers some of what the inspector did.
+- Where the R version has a bug, this one does not copy it: joins and relationships on more than one key, nested joins, the include-parameters option, and calculations with brackets inside brackets.
+- The GUI is meant to run on your own machine for one person. It refuses requests that come from other websites, but there is no login, so do not put it on a shared network.
+
+## Why a Tableau parser
+
+A `.twb` is XML and a `.twbx` is a zip containing one, so reading them from Python is not hard. Power BI's `.pbix` is a binary format built on a proprietary storage engine, and getting into it from code took reverse-engineering projects like [PBIXRay](https://github.com/Hugoberry/pbixray) and [pbi-tools](https://github.com/pbi-tools/pbi-tools). On the server side it goes the other way: Tableau's own Python tooling ([tableauserverclient](https://pypi.org/project/tableauserverclient/) and `tabcmd`) is more mature and more open than what Microsoft has for the Power BI REST API.
+
+## Contributing
+
+Issues and pull requests are welcome at [github.com/DDSNA/py-tbparse](https://github.com/DDSNA/py-tbparse). The test setup is in the [development guide](https://github.com/DDSNA/py-tbparse/blob/main/docs/development.md). `AGENTS.md` describes the code layout and the rules for porting a function from the R package.
+
+## Credit
+
+Based on [twbparser](https://github.com/PrigasG/twbparser) by George Arthur, MIT licensed.
+
+## License
+
+MIT, see [LICENSE](https://github.com/DDSNA/py-tbparse/blob/main/LICENSE).

py_tbparse-0.4.4/README.md

@@ -0,0 +1,101 @@
+# py-tbparse
+
+[![PyPI](https://img.shields.io/pypi/v/py-tbparse)](https://pypi.org/project/py-tbparse/)
+[![Python](https://img.shields.io/pypi/pyversions/py-tbparse)](https://pypi.org/project/py-tbparse/)
+[![CI](https://github.com/DDSNA/py-tbparse/actions/workflows/ci.yml/badge.svg)](https://github.com/DDSNA/py-tbparse/actions/workflows/ci.yml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/DDSNA/py-tbparse/blob/main/LICENSE)
+
+Read Tableau workbooks (`.twb` and `.twbx`) from Python. You get datasources, fields, calculated fields, joins, relationships and dashboards as pandas DataFrames. It is plain Python on top of `lxml` and `pandas`, so Tableau and R are not needed.
+
+It began as a port of PrigasG's R package [twbparser](https://github.com/PrigasG/twbparser). The command-line tool, the browser GUI, renaming, templates, workbook diffing and folder scanning are new here.
+
+![The overview report card for a small demo workbook](https://raw.githubusercontent.com/DDSNA/py-tbparse/main/docs/gui-overview.png)
+
+## What it does
+
+- Lists what is in a workbook: datasources, parameters, fields, calculations, joins, relationships, dashboards, custom SQL, published sources.
+- Checks relationships and finds calculations that refer to fields the workbook does not have.
+- Shows which worksheets, dashboards and calculations use each field.
+- Suggests clean names for fields, sheets, dashboards and other objects, and writes a renamed copy.
+- Turns a finished workbook into a template you can fill with other data.
+- Compares two workbooks, scans a folder of them, and draws the data model as a graph.
+- Works from Python, from the command line, or in a local browser page.
+
+## Install
+
+```bash
+pip install py-tbparse
+```
+
+Python 3.9 or newer. The package, the import and both commands all use the same name: `pip install py-tbparse`, `import py_tbparse`, `py-tbparse`, `py-tbparse-gui`. Releases up to 0.2.0 used `twbparser_py` and `twbparser`; those names are gone.
+
+## Quick start
+
+From Python:
+
+```python
+from py_tbparse import TwbParser
+
+p = TwbParser("workbook.twb")    # or a .twbx
+p.get_overview()                 # counts of everything
+p.get_calculated_fields()        # each calculation and its formula
+p.get_relationships()            # how the tables connect
+```
+
+Every getter returns a DataFrame. The others are `get_datasources`, `get_parameters`, `get_fields`, `get_raw_fields`, `get_joins`, `get_relations`, `get_inferred_relationships`, `get_dashboards`, `get_dashboard_sheets`, `get_custom_sql`, `get_initial_sql`, `get_published_refs` and `get_field_usage`. `get_relationship_graph_dot()` returns the data model as Graphviz text and `validate()` looks for problems in the relationships. In Jupyter, a bare `p` shows the overview.
+
+From the command line:
+
+```bash
+py-tbparse workbook.twb                    # overview
+py-tbparse workbook.twb calculated-fields
+py-tbparse workbook.twb fields --format csv -o fields.csv
+py-tbparse workbook.twb validate           # exit code 2 if it finds a problem
+```
+
+In the browser:
+
+```bash
+py-tbparse-gui workbook.twb
+```
+
+Across workbooks:
+
+```python
+from py_tbparse import TwbParser, diff_workbooks, scan_folder
+
+diff_workbooks(TwbParser("v1.twb"), TwbParser("v2.twb"), table="datasources")
+scan_folder("./workbooks", table="datasources")   # one table, every workbook in the folder
+```
+
+## Guides
+
+- [Command line](https://github.com/DDSNA/py-tbparse/blob/main/docs/cli.md): every table, the `diff`, `batch`, `rename` and `template` commands, and their options.
+- [Browser GUI](https://github.com/DDSNA/py-tbparse/blob/main/docs/gui.md): opening files, the table, the overview, the graph, themes.
+- [Renaming](https://github.com/DDSNA/py-tbparse/blob/main/docs/renaming.md): clean names after a datasource switch, editing the suggestions, what will stay broken.
+- [Templates](https://github.com/DDSNA/py-tbparse/blob/main/docs/templates.md): make a template from a workbook and apply it to new data.
+- [`.twbx` files](https://github.com/DDSNA/py-tbparse/blob/main/docs/twbx.md): they are read straight from the zip; how to extract the contents.
+- [Development](https://github.com/DDSNA/py-tbparse/blob/main/docs/development.md): running the tests, the workbook corpus, the browser tests.
+
+## Limits
+
+- **Nothing the tool writes has been opened in Tableau yet.** That covers renamed workbooks and workbooks made from templates. The XML follows what Tableau writes, but open one on a copy and check it before relying on it. The original file is never modified or overwritten.
+- **Only part of the R package is ported.** Missing: formatting, tooltips, colors, axes and sorts, dashboard layout and actions, calculation complexity, the replication brief and the Shiny inspector. The GUI covers some of what the inspector did.
+- Where the R version has a bug, this one does not copy it: joins and relationships on more than one key, nested joins, the include-parameters option, and calculations with brackets inside brackets.
+- The GUI is meant to run on your own machine for one person. It refuses requests that come from other websites, but there is no login, so do not put it on a shared network.
+
+## Why a Tableau parser
+
+A `.twb` is XML and a `.twbx` is a zip containing one, so reading them from Python is not hard. Power BI's `.pbix` is a binary format built on a proprietary storage engine, and getting into it from code took reverse-engineering projects like [PBIXRay](https://github.com/Hugoberry/pbixray) and [pbi-tools](https://github.com/pbi-tools/pbi-tools). On the server side it goes the other way: Tableau's own Python tooling ([tableauserverclient](https://pypi.org/project/tableauserverclient/) and `tabcmd`) is more mature and more open than what Microsoft has for the Power BI REST API.
+
+## Contributing
+
+Issues and pull requests are welcome at [github.com/DDSNA/py-tbparse](https://github.com/DDSNA/py-tbparse). The test setup is in the [development guide](https://github.com/DDSNA/py-tbparse/blob/main/docs/development.md). `AGENTS.md` describes the code layout and the rules for porting a function from the R package.
+
+## Credit
+
+Based on [twbparser](https://github.com/PrigasG/twbparser) by George Arthur, MIT licensed.
+
+## License
+
+MIT, see [LICENSE](https://github.com/DDSNA/py-tbparse/blob/main/LICENSE).

{py_tbparse-0.3.0 → py_tbparse-0.4.4}/py_tbparse/__init__.py

@@ -10,7 +10,7 @@ from .dashboards import dashboard_sheets, list_dashboards
 from .datasources import extract_datasource_details, extract_named_connections, extract_parameters
 from .diff import diff_tables, diff_workbooks
 from .fields import extract_columns_with_table_source, infer_implicit_relationships
-from .graph import to_dot
+from .graph import graph_data, to_dot
 from .joins import extract_joins
 from .parser import TwbParser
 from .published import extract_published_refs
@@ -21,8 +21,20 @@ from .rename import (
     load_rename_mapping,
     normalize_name,
     suggest_field_renames,
+    suggest_renames,
 )
 from .sql import extract_custom_sql, extract_initial_sql
+from .templates import (
+    Template,
+    TemplateError,
+    apply_template,
+    load_mapping,
+    load_template,
+    make_template,
+    read_data,
+    suggest_mapping,
+)
+from .usage import field_usage
 from .validators import validate_relationships
 from ._xml import extract_twb_from_twbx, twbx_extract_files, twbx_list
 
@@ -39,6 +51,7 @@ __all__ = [
     "infer_implicit_relationships",
     "extract_joins",
     "to_dot",
+    "graph_data",
     "extract_relations",
     "extract_relationships",
     "extract_custom_sql",
@@ -56,6 +69,16 @@ __all__ = [
     "load_rename_mapping",
     "normalize_name",
     "suggest_field_renames",
+    "suggest_renames",
+    "field_usage",
+    "Template",
+    "TemplateError",
+    "make_template",
+    "load_template",
+    "read_data",
+    "suggest_mapping",
+    "load_mapping",
+    "apply_template",
 ]
 
 try:

{py_tbparse-0.3.0 → py_tbparse-0.4.4}/py_tbparse/_tables.py

@@ -39,12 +39,28 @@ def _calculated_fields(p: TwbParser, include_parameters: bool = False, **_kw) ->
 
 
 def _field_renames(p: TwbParser, style: str = "title", reference=None, only_changed: bool = False,
-                  datasource=None, **_kw) -> pd.DataFrame:
+                  datasource=None, kinds=None, **_kw) -> pd.DataFrame:
+    if kinds:  # the GUI's "everything in the report" switch
+        return p.get_renames(
+            reference=reference, style=style, only_changed=only_changed, datasource=datasource, kinds=kinds
+        )
     return p.get_field_renames(
         reference=reference, style=style, only_changed=only_changed, datasource=datasource
     )
 
 
+def _report_renames(p: TwbParser, style: str = "title", reference=None, only_changed: bool = False,
+                    datasource=None, **_kw) -> pd.DataFrame:
+    return p.get_renames(reference=reference, style=style, only_changed=only_changed, datasource=datasource)
+
+
+def _field_usage(p: TwbParser, **_kw) -> pd.DataFrame:
+    df = p.get_field_usage()
+    for col in ("sheets", "dashboards", "calculations"):
+        df[col] = df[col].map("; ".join)
+    return df
+
+
 def _joins(p: TwbParser, **_kw) -> pd.DataFrame:
     return p.get_joins()
 
@@ -77,6 +93,10 @@ def _initial_sql(p: TwbParser, **_kw) -> pd.DataFrame:
     return p.get_initial_sql()
 
 
+def _missing_references(p: TwbParser, **_kw) -> pd.DataFrame:
+    return p.get_missing_references()
+
+
 def _published_refs(p: TwbParser, **_kw) -> pd.DataFrame:
     return p.get_published_refs()
 
@@ -91,6 +111,9 @@ TABLE_SPECS: dict[str, Callable[..., pd.DataFrame]] = {
     "raw-fields": _raw_fields,
     "calculated-fields": _calculated_fields,
     "field-renames": _field_renames,
+    "report-renames": _report_renames,
+    "field-usage": _field_usage,
+    "missing-references": _missing_references,
     "joins": _joins,
     "relations": _relations,
     "relationships": _relationships,