dataform-dependency-visualizer 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- dataform_dependency_visualizer-0.1.0/LICENSE +21 -0
- dataform_dependency_visualizer-0.1.0/PKG-INFO +170 -0
- dataform_dependency_visualizer-0.1.0/README.md +144 -0
- dataform_dependency_visualizer-0.1.0/README_PACKAGE.md +145 -0
- dataform_dependency_visualizer-0.1.0/pyproject.toml +39 -0
- dataform_dependency_visualizer-0.1.0/setup.cfg +4 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_dependency_visualizer.egg-info/PKG-INFO +170 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_dependency_visualizer.egg-info/SOURCES.txt +16 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_dependency_visualizer.egg-info/dependency_links.txt +1 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_dependency_visualizer.egg-info/entry_points.txt +2 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_dependency_visualizer.egg-info/top_level.txt +1 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/__init__.py +14 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/cli.py +159 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/dataform_check.py +175 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/master_index.py +278 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/parser.py +69 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/svg_generator.py +381 -0
- dataform_dependency_visualizer-0.1.0/src/dataform_viz/visualizer.py +140 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 [Your Name]
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dataform-dependency-visualizer
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Visualize Dataform table dependencies as interactive SVG diagrams
|
|
5
|
+
Author-email: Thamo <thamo@example.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/thamo/dataform-dependency-visualizer
|
|
8
|
+
Project-URL: Repository, https://github.com/thamo/dataform-dependency-visualizer
|
|
9
|
+
Keywords: dataform,dependencies,visualization,bigquery,sql
|
|
10
|
+
Classifier: Development Status :: 4 - Beta
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
20
|
+
Classifier: Topic :: Database
|
|
21
|
+
Requires-Python: >=3.8
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Dynamic: license-file
|
|
25
|
+
|
|
26
|
+
# Dataform Dependency Visualizer
|
|
27
|
+
|
|
28
|
+
Generate beautiful, interactive SVG diagrams showing dependencies between Dataform tables.
|
|
29
|
+
|
|
30
|
+
## Features
|
|
31
|
+
|
|
32
|
+
- 📊 **Individual table diagrams** - One SVG per table showing immediate dependencies
|
|
33
|
+
- 🎨 **Color-coded by type** - Tables, views, and operations visually distinct
|
|
34
|
+
- 🔍 **Master index viewer** - Browse all tables in single interface
|
|
35
|
+
- 📁 **Schema organization** - Organized by schema with collapsible sections
|
|
36
|
+
- ⚡ **Pure Python SVGs** - No Graphviz required
|
|
37
|
+
- 🎯 **Orthogonal routing** - Clean, professional arrow paths
|
|
38
|
+
- 📝 **Smart text wrapping** - Long table names split across lines
|
|
39
|
+
|
|
40
|
+
## Installation
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
pip install dataform-dependency-visualizer
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## Quick Start
|
|
47
|
+
|
|
48
|
+
### 1. Generate dependency report
|
|
49
|
+
|
|
50
|
+
In your Dataform project directory:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
dataform compile --json > dependencies_report.txt
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### 2. Generate SVG diagrams
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
# Generate for specific schema
|
|
60
|
+
dataform-deps generate dashboard_wwim
|
|
61
|
+
|
|
62
|
+
# Generate for all schemas (excluding refined_*)
|
|
63
|
+
dataform-deps generate-all
|
|
64
|
+
|
|
65
|
+
# Generate master index
|
|
66
|
+
dataform-deps index
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### 3. View diagrams
|
|
70
|
+
|
|
71
|
+
Open `output/dependencies_master_index.html` in your browser to browse all tables and their dependencies.
|
|
72
|
+
|
|
73
|
+
## Usage
|
|
74
|
+
|
|
75
|
+
### Command Line
|
|
76
|
+
|
|
77
|
+
**Generate diagrams for a specific schema:**
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
dataform-deps generate SCHEMA_NAME
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Example:
|
|
84
|
+
```bash
|
|
85
|
+
dataform-deps generate dashboard_wwim
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Generate for all schemas:**
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
dataform-deps generate-all
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
By default, schemas starting with `refined_` are excluded. To include them:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
dataform-deps generate-all --include-refined
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
**Generate master index:**
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
dataform-deps index
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Creates `output/dependencies_master_index.html` with links to all tables.
|
|
107
|
+
|
|
108
|
+
**Check prerequisites:**
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
dataform-deps check
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
Verifies that:
|
|
115
|
+
- You're in a Dataform project directory
|
|
116
|
+
- `dependencies_report.txt` exists
|
|
117
|
+
- The report contains valid dependency data
|
|
118
|
+
|
|
119
|
+
## Example Output
|
|
120
|
+
|
|
121
|
+
### Individual Table Diagram
|
|
122
|
+
|
|
123
|
+
Each table gets its own SVG showing:
|
|
124
|
+
- **Dependencies (upstream)** - Tables this table depends on
|
|
125
|
+
- **Dependents (downstream)** - Tables that depend on this table
|
|
126
|
+
- **Color coding** - Tables (blue), views (green), operations (orange)
|
|
127
|
+
|
|
128
|
+
### Master Index
|
|
129
|
+
|
|
130
|
+
The master index organizes all tables by schema with:
|
|
131
|
+
- Clickable table names that open their SVG
|
|
132
|
+
- Expandable/collapsible schemas
|
|
133
|
+
- Type badges (table/view/incremental)
|
|
134
|
+
- Search functionality
|
|
135
|
+
|
|
136
|
+
## Requirements
|
|
137
|
+
|
|
138
|
+
- Python 3.8+
|
|
139
|
+
- Dataform project with compiled dependencies
|
|
140
|
+
|
|
141
|
+
## How It Works
|
|
142
|
+
|
|
143
|
+
1. Parse `dependencies_report.txt` generated by Dataform
|
|
144
|
+
2. Extract table dependencies and metadata
|
|
145
|
+
3. Generate SVG diagrams with orthogonal routing
|
|
146
|
+
4. Create master index HTML for easy navigation
|
|
147
|
+
|
|
148
|
+
## Project Structure
|
|
149
|
+
|
|
150
|
+
```
|
|
151
|
+
output/
|
|
152
|
+
├── dependencies_master_index.html # Main entry point
|
|
153
|
+
└── dependencies/
|
|
154
|
+
├── schema1_table1.svg
|
|
155
|
+
├── schema1_table2.svg
|
|
156
|
+
└── ...
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
## License
|
|
160
|
+
|
|
161
|
+
MIT License - see LICENSE file for details.
|
|
162
|
+
|
|
163
|
+
## Contributing
|
|
164
|
+
|
|
165
|
+
Contributions welcome! Please open an issue or PR on GitHub.
|
|
166
|
+
|
|
167
|
+
## Links
|
|
168
|
+
|
|
169
|
+
- [GitHub Repository](https://github.com/yourusername/dataform-dependency-visualizer)
|
|
170
|
+
- [Issue Tracker](https://github.com/yourusername/dataform-dependency-visualizer/issues)
|
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
# Dataform Dependency Visualizer
|
|
2
|
+
|
|
3
|
+
Generate beautiful, interactive SVG diagrams showing dependencies between Dataform tables.
|
|
4
|
+
|
|
5
|
+
## Features
|
|
6
|
+
|
|
7
|
+
- 📊 **Individual table diagrams** - One SVG per table showing immediate dependencies
|
|
8
|
+
- 🎨 **Color-coded by type** - Tables, views, and operations visually distinct
|
|
9
|
+
- 🔍 **Master index viewer** - Browse all 133+ tables in single interface
|
|
10
|
+
- 📁 **Schema organization** - Organized by schema with collapsible sections
|
|
11
|
+
- ⚡ **Pure Python SVGs** - No Graphviz required
|
|
12
|
+
- 🎯 **Orthogonal routing** - Clean, professional arrow paths
|
|
13
|
+
- 📝 **Smart text wrapping** - Long table names split across 2 lines
|
|
14
|
+
|
|
15
|
+
## Installation
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
pip install dataform-dependency-visualizer
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Quick Start
|
|
22
|
+
|
|
23
|
+
### 1. Generate dependency report
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
cd your-dataform-project
|
|
27
|
+
dataform compile --json > dependencies_report.txt
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
### 2. Generate SVG diagrams
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
# Generate for specific schema
|
|
34
|
+
dataform-deps generate dashboard_wwim
|
|
35
|
+
|
|
36
|
+
# Generate for all schemas (excluding refined_*)
|
|
37
|
+
dataform-deps generate-all
|
|
38
|
+
|
|
39
|
+
# Generate master index
|
|
40
|
+
dataform-deps index
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### 3. View diagrams
|
|
44
|
+
|
|
45
|
+
Open `output/dependencies_master_index.html` in your browser.
|
|
46
|
+
|
|
47
|
+
## Usage
|
|
48
|
+
|
|
49
|
+
### Command Line
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
# Generate SVGs for one schema
|
|
53
|
+
dataform-deps generate <schema_name>
|
|
54
|
+
|
|
55
|
+
# Generate for all schemas
|
|
56
|
+
dataform-deps generate-all
|
|
57
|
+
|
|
58
|
+
# Generate master index
|
|
59
|
+
dataform-deps index
|
|
60
|
+
|
|
61
|
+
# Full pipeline with prerequisites check
|
|
62
|
+
dataform-deps setup
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Python API
|
|
66
|
+
|
|
67
|
+
```python
|
|
68
|
+
from dataform_viz import DependencyVisualizer
|
|
69
|
+
|
|
70
|
+
# Initialize visualizer
|
|
71
|
+
viz = DependencyVisualizer('dependencies_report.txt')
|
|
72
|
+
|
|
73
|
+
# Generate SVGs for schema
|
|
74
|
+
viz.generate_schema_svgs('dashboard_wwim', output_dir='output')
|
|
75
|
+
|
|
76
|
+
# Generate master index
|
|
77
|
+
viz.generate_master_index('output')
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Output Structure
|
|
81
|
+
|
|
82
|
+
```
|
|
83
|
+
output/
|
|
84
|
+
├── dependencies_master_index.html # Interactive viewer
|
|
85
|
+
├── dependencies_dashboard_wwim/
|
|
86
|
+
│ ├── index.html
|
|
87
|
+
│ ├── table1.svg
|
|
88
|
+
│ └── table2.svg
|
|
89
|
+
├── dependencies_datamart_wwim/
|
|
90
|
+
│ └── ...
|
|
91
|
+
└── dependencies_report.txt
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Features
|
|
95
|
+
|
|
96
|
+
### SVG Diagrams
|
|
97
|
+
|
|
98
|
+
Each table gets its own diagram showing:
|
|
99
|
+
- **Yellow center node**: The table itself
|
|
100
|
+
- **Blue left nodes**: Dependencies (what it reads from)
|
|
101
|
+
- **Green right nodes**: Dependents (what reads from it)
|
|
102
|
+
- **Schema labels**: Show which schema each table belongs to
|
|
103
|
+
- **Type badges**: Distinguish tables, views, operations
|
|
104
|
+
|
|
105
|
+
### Master Index
|
|
106
|
+
|
|
107
|
+
Interactive HTML with:
|
|
108
|
+
- Sidebar navigation by schema
|
|
109
|
+
- Collapsible schema sections
|
|
110
|
+
- Search-friendly table list
|
|
111
|
+
- Click-to-view diagrams
|
|
112
|
+
- Statistics (total schemas/tables)
|
|
113
|
+
|
|
114
|
+
## Requirements
|
|
115
|
+
|
|
116
|
+
- Python 3.8+
|
|
117
|
+
- Dataform CLI (for generating reports)
|
|
118
|
+
- Node.js (for running Dataform)
|
|
119
|
+
|
|
120
|
+
## Development
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
# Clone repo
|
|
124
|
+
git clone https://github.com/yourusername/dataform-dependency-visualizer
|
|
125
|
+
cd dataform-dependency-visualizer
|
|
126
|
+
|
|
127
|
+
# Install in development mode
|
|
128
|
+
pip install -e .
|
|
129
|
+
|
|
130
|
+
# Run tests
|
|
131
|
+
pytest
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
## License
|
|
135
|
+
|
|
136
|
+
MIT License - See LICENSE file for details
|
|
137
|
+
|
|
138
|
+
## Contributing
|
|
139
|
+
|
|
140
|
+
Contributions welcome! Please open an issue or PR.
|
|
141
|
+
|
|
142
|
+
## Credits
|
|
143
|
+
|
|
144
|
+
Created for visualizing complex Dataform projects with 100+ table dependencies.
|
|
@@ -0,0 +1,145 @@
|
|
|
1
|
+
# Dataform Dependency Visualizer
|
|
2
|
+
|
|
3
|
+
Generate beautiful, interactive SVG diagrams showing dependencies between Dataform tables.
|
|
4
|
+
|
|
5
|
+
## Features
|
|
6
|
+
|
|
7
|
+
- 📊 **Individual table diagrams** - One SVG per table showing immediate dependencies
|
|
8
|
+
- 🎨 **Color-coded by type** - Tables, views, and operations visually distinct
|
|
9
|
+
- 🔍 **Master index viewer** - Browse all tables in single interface
|
|
10
|
+
- 📁 **Schema organization** - Organized by schema with collapsible sections
|
|
11
|
+
- ⚡ **Pure Python SVGs** - No Graphviz required
|
|
12
|
+
- 🎯 **Orthogonal routing** - Clean, professional arrow paths
|
|
13
|
+
- 📝 **Smart text wrapping** - Long table names split across lines
|
|
14
|
+
|
|
15
|
+
## Installation
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
pip install dataform-dependency-visualizer
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Quick Start
|
|
22
|
+
|
|
23
|
+
### 1. Generate dependency report
|
|
24
|
+
|
|
25
|
+
In your Dataform project directory:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
dataform compile --json > dependencies_report.txt
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
### 2. Generate SVG diagrams
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
# Generate for specific schema
|
|
35
|
+
dataform-deps generate dashboard_wwim
|
|
36
|
+
|
|
37
|
+
# Generate for all schemas (excluding refined_*)
|
|
38
|
+
dataform-deps generate-all
|
|
39
|
+
|
|
40
|
+
# Generate master index
|
|
41
|
+
dataform-deps index
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### 3. View diagrams
|
|
45
|
+
|
|
46
|
+
Open `output/dependencies_master_index.html` in your browser to browse all tables and their dependencies.
|
|
47
|
+
|
|
48
|
+
## Usage
|
|
49
|
+
|
|
50
|
+
### Command Line
|
|
51
|
+
|
|
52
|
+
**Generate diagrams for a specific schema:**
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
dataform-deps generate SCHEMA_NAME
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Example:
|
|
59
|
+
```bash
|
|
60
|
+
dataform-deps generate dashboard_wwim
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
**Generate for all schemas:**
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
dataform-deps generate-all
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
By default, schemas starting with `refined_` are excluded. To include them:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
dataform-deps generate-all --include-refined
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
**Generate master index:**
|
|
76
|
+
|
|
77
|
+
```bash
|
|
78
|
+
dataform-deps index
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Creates `output/dependencies_master_index.html` with links to all tables.
|
|
82
|
+
|
|
83
|
+
**Check prerequisites:**
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
dataform-deps check
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
Verifies that:
|
|
90
|
+
- You're in a Dataform project directory
|
|
91
|
+
- `dependencies_report.txt` exists
|
|
92
|
+
- The report contains valid dependency data
|
|
93
|
+
|
|
94
|
+
## Example Output
|
|
95
|
+
|
|
96
|
+
### Individual Table Diagram
|
|
97
|
+
|
|
98
|
+
Each table gets its own SVG showing:
|
|
99
|
+
- **Dependencies (upstream)** - Tables this table depends on
|
|
100
|
+
- **Dependents (downstream)** - Tables that depend on this table
|
|
101
|
+
- **Color coding** - Tables (blue), views (green), operations (orange)
|
|
102
|
+
|
|
103
|
+
### Master Index
|
|
104
|
+
|
|
105
|
+
The master index organizes all tables by schema with:
|
|
106
|
+
- Clickable table names that open their SVG
|
|
107
|
+
- Expandable/collapsible schemas
|
|
108
|
+
- Type badges (table/view/incremental)
|
|
109
|
+
- Search functionality
|
|
110
|
+
|
|
111
|
+
## Requirements
|
|
112
|
+
|
|
113
|
+
- Python 3.8+
|
|
114
|
+
- Dataform project with compiled dependencies
|
|
115
|
+
|
|
116
|
+
## How It Works
|
|
117
|
+
|
|
118
|
+
1. Parse `dependencies_report.txt` generated by Dataform
|
|
119
|
+
2. Extract table dependencies and metadata
|
|
120
|
+
3. Generate SVG diagrams with orthogonal routing
|
|
121
|
+
4. Create master index HTML for easy navigation
|
|
122
|
+
|
|
123
|
+
## Project Structure
|
|
124
|
+
|
|
125
|
+
```
|
|
126
|
+
output/
|
|
127
|
+
├── dependencies_master_index.html # Main entry point
|
|
128
|
+
└── dependencies/
|
|
129
|
+
├── schema1_table1.svg
|
|
130
|
+
├── schema1_table2.svg
|
|
131
|
+
└── ...
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
## License
|
|
135
|
+
|
|
136
|
+
MIT License - see LICENSE file for details.
|
|
137
|
+
|
|
138
|
+
## Contributing
|
|
139
|
+
|
|
140
|
+
Contributions welcome! Please open an issue or PR on GitHub.
|
|
141
|
+
|
|
142
|
+
## Links
|
|
143
|
+
|
|
144
|
+
- [GitHub Repository](https://github.com/yourusername/dataform-dependency-visualizer)
|
|
145
|
+
- [Issue Tracker](https://github.com/yourusername/dataform-dependency-visualizer/issues)
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61.0", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "dataform-dependency-visualizer"
|
|
7
|
+
version = "0.1.0"
|
|
8
|
+
description = "Visualize Dataform table dependencies as interactive SVG diagrams"
|
|
9
|
+
readme = "README_PACKAGE.md"
|
|
10
|
+
requires-python = ">=3.8"
|
|
11
|
+
license = {text = "MIT"}
|
|
12
|
+
authors = [
|
|
13
|
+
{name = "Thamo", email = "thamo@example.com"}
|
|
14
|
+
]
|
|
15
|
+
keywords = ["dataform", "dependencies", "visualization", "bigquery", "sql"]
|
|
16
|
+
classifiers = [
|
|
17
|
+
"Development Status :: 4 - Beta",
|
|
18
|
+
"Intended Audience :: Developers",
|
|
19
|
+
"License :: OSI Approved :: MIT License",
|
|
20
|
+
"Programming Language :: Python :: 3",
|
|
21
|
+
"Programming Language :: Python :: 3.8",
|
|
22
|
+
"Programming Language :: Python :: 3.9",
|
|
23
|
+
"Programming Language :: Python :: 3.10",
|
|
24
|
+
"Programming Language :: Python :: 3.11",
|
|
25
|
+
"Programming Language :: Python :: 3.12",
|
|
26
|
+
"Topic :: Software Development :: Libraries :: Python Modules",
|
|
27
|
+
"Topic :: Database",
|
|
28
|
+
]
|
|
29
|
+
|
|
30
|
+
[project.urls]
|
|
31
|
+
Homepage = "https://github.com/thamo/dataform-dependency-visualizer"
|
|
32
|
+
Repository = "https://github.com/thamo/dataform-dependency-visualizer"
|
|
33
|
+
|
|
34
|
+
[project.scripts]
|
|
35
|
+
dataform-deps = "dataform_viz.cli:main"
|
|
36
|
+
|
|
37
|
+
[tool.setuptools.packages.find]
|
|
38
|
+
where = ["src"]
|
|
39
|
+
include = ["dataform_viz*"]
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dataform-dependency-visualizer
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Visualize Dataform table dependencies as interactive SVG diagrams
|
|
5
|
+
Author-email: Thamo <thamo@example.com>
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/thamo/dataform-dependency-visualizer
|
|
8
|
+
Project-URL: Repository, https://github.com/thamo/dataform-dependency-visualizer
|
|
9
|
+
Keywords: dataform,dependencies,visualization,bigquery,sql
|
|
10
|
+
Classifier: Development Status :: 4 - Beta
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
20
|
+
Classifier: Topic :: Database
|
|
21
|
+
Requires-Python: >=3.8
|
|
22
|
+
Description-Content-Type: text/markdown
|
|
23
|
+
License-File: LICENSE
|
|
24
|
+
Dynamic: license-file
|
|
25
|
+
|
|
26
|
+
# Dataform Dependency Visualizer
|
|
27
|
+
|
|
28
|
+
Generate beautiful, interactive SVG diagrams showing dependencies between Dataform tables.
|
|
29
|
+
|
|
30
|
+
## Features
|
|
31
|
+
|
|
32
|
+
- 📊 **Individual table diagrams** - One SVG per table showing immediate dependencies
|
|
33
|
+
- 🎨 **Color-coded by type** - Tables, views, and operations visually distinct
|
|
34
|
+
- 🔍 **Master index viewer** - Browse all tables in single interface
|
|
35
|
+
- 📁 **Schema organization** - Organized by schema with collapsible sections
|
|
36
|
+
- ⚡ **Pure Python SVGs** - No Graphviz required
|
|
37
|
+
- 🎯 **Orthogonal routing** - Clean, professional arrow paths
|
|
38
|
+
- 📝 **Smart text wrapping** - Long table names split across lines
|
|
39
|
+
|
|
40
|
+
## Installation
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
pip install dataform-dependency-visualizer
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## Quick Start
|
|
47
|
+
|
|
48
|
+
### 1. Generate dependency report
|
|
49
|
+
|
|
50
|
+
In your Dataform project directory:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
dataform compile --json > dependencies_report.txt
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### 2. Generate SVG diagrams
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
# Generate for specific schema
|
|
60
|
+
dataform-deps generate dashboard_wwim
|
|
61
|
+
|
|
62
|
+
# Generate for all schemas (excluding refined_*)
|
|
63
|
+
dataform-deps generate-all
|
|
64
|
+
|
|
65
|
+
# Generate master index
|
|
66
|
+
dataform-deps index
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### 3. View diagrams
|
|
70
|
+
|
|
71
|
+
Open `output/dependencies_master_index.html` in your browser to browse all tables and their dependencies.
|
|
72
|
+
|
|
73
|
+
## Usage
|
|
74
|
+
|
|
75
|
+
### Command Line
|
|
76
|
+
|
|
77
|
+
**Generate diagrams for a specific schema:**
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
dataform-deps generate SCHEMA_NAME
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Example:
|
|
84
|
+
```bash
|
|
85
|
+
dataform-deps generate dashboard_wwim
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Generate for all schemas:**
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
dataform-deps generate-all
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
By default, schemas starting with `refined_` are excluded. To include them:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
dataform-deps generate-all --include-refined
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
**Generate master index:**
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
dataform-deps index
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Creates `output/dependencies_master_index.html` with links to all tables.
|
|
107
|
+
|
|
108
|
+
**Check prerequisites:**
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
dataform-deps check
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
Verifies that:
|
|
115
|
+
- You're in a Dataform project directory
|
|
116
|
+
- `dependencies_report.txt` exists
|
|
117
|
+
- The report contains valid dependency data
|
|
118
|
+
|
|
119
|
+
## Example Output
|
|
120
|
+
|
|
121
|
+
### Individual Table Diagram
|
|
122
|
+
|
|
123
|
+
Each table gets its own SVG showing:
|
|
124
|
+
- **Dependencies (upstream)** - Tables this table depends on
|
|
125
|
+
- **Dependents (downstream)** - Tables that depend on this table
|
|
126
|
+
- **Color coding** - Tables (blue), views (green), operations (orange)
|
|
127
|
+
|
|
128
|
+
### Master Index
|
|
129
|
+
|
|
130
|
+
The master index organizes all tables by schema with:
|
|
131
|
+
- Clickable table names that open their SVG
|
|
132
|
+
- Expandable/collapsible schemas
|
|
133
|
+
- Type badges (table/view/incremental)
|
|
134
|
+
- Search functionality
|
|
135
|
+
|
|
136
|
+
## Requirements
|
|
137
|
+
|
|
138
|
+
- Python 3.8+
|
|
139
|
+
- Dataform project with compiled dependencies
|
|
140
|
+
|
|
141
|
+
## How It Works
|
|
142
|
+
|
|
143
|
+
1. Parse `dependencies_report.txt` generated by Dataform
|
|
144
|
+
2. Extract table dependencies and metadata
|
|
145
|
+
3. Generate SVG diagrams with orthogonal routing
|
|
146
|
+
4. Create master index HTML for easy navigation
|
|
147
|
+
|
|
148
|
+
## Project Structure
|
|
149
|
+
|
|
150
|
+
```
|
|
151
|
+
output/
|
|
152
|
+
├── dependencies_master_index.html # Main entry point
|
|
153
|
+
└── dependencies/
|
|
154
|
+
├── schema1_table1.svg
|
|
155
|
+
├── schema1_table2.svg
|
|
156
|
+
└── ...
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
## License
|
|
160
|
+
|
|
161
|
+
MIT License - see LICENSE file for details.
|
|
162
|
+
|
|
163
|
+
## Contributing
|
|
164
|
+
|
|
165
|
+
Contributions welcome! Please open an issue or PR on GitHub.
|
|
166
|
+
|
|
167
|
+
## Links
|
|
168
|
+
|
|
169
|
+
- [GitHub Repository](https://github.com/yourusername/dataform-dependency-visualizer)
|
|
170
|
+
- [Issue Tracker](https://github.com/yourusername/dataform-dependency-visualizer/issues)
|
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
README_PACKAGE.md
|
|
4
|
+
pyproject.toml
|
|
5
|
+
src/dataform_dependency_visualizer.egg-info/PKG-INFO
|
|
6
|
+
src/dataform_dependency_visualizer.egg-info/SOURCES.txt
|
|
7
|
+
src/dataform_dependency_visualizer.egg-info/dependency_links.txt
|
|
8
|
+
src/dataform_dependency_visualizer.egg-info/entry_points.txt
|
|
9
|
+
src/dataform_dependency_visualizer.egg-info/top_level.txt
|
|
10
|
+
src/dataform_viz/__init__.py
|
|
11
|
+
src/dataform_viz/cli.py
|
|
12
|
+
src/dataform_viz/dataform_check.py
|
|
13
|
+
src/dataform_viz/master_index.py
|
|
14
|
+
src/dataform_viz/parser.py
|
|
15
|
+
src/dataform_viz/svg_generator.py
|
|
16
|
+
src/dataform_viz/visualizer.py
|