dbt-lineage-cli 0.15.0__py3-none-win_amd64.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- dbt_lineage_cli-0.15.0.data/scripts/dbt-lineage.exe +0 -0
- dbt_lineage_cli-0.15.0.dist-info/METADATA +581 -0
- dbt_lineage_cli-0.15.0.dist-info/RECORD +6 -0
- dbt_lineage_cli-0.15.0.dist-info/WHEEL +4 -0
- dbt_lineage_cli-0.15.0.dist-info/licenses/LICENSE +21 -0
- dbt_lineage_cli-0.15.0.dist-info/sboms/dbt-lineage.cyclonedx.json +6699 -0
|
Binary file
|
|
@@ -0,0 +1,581 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: dbt-lineage-cli
|
|
3
|
+
Version: 0.15.0
|
|
4
|
+
Classifier: Environment :: Console
|
|
5
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
6
|
+
Classifier: Programming Language :: Rust
|
|
7
|
+
Classifier: Topic :: Database
|
|
8
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Summary: PR-time dbt lineage, impact analysis and rebuild plans without a warehouse: CLI, TUI, CI report and MCP server
|
|
11
|
+
Keywords: dbt,lineage,data-lineage,sql,ci,mcp
|
|
12
|
+
Home-Page: https://github.com/sipemu/dbt-lineage-viewer
|
|
13
|
+
License: MIT
|
|
14
|
+
Requires-Python: >=3.8
|
|
15
|
+
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
16
|
+
Project-URL: Changelog, https://github.com/sipemu/dbt-lineage-viewer/blob/master/CHANGELOG.md
|
|
17
|
+
Project-URL: Demo, https://github.com/sipemu/dbt-lineage-demo/pull/1
|
|
18
|
+
Project-URL: Homepage, https://github.com/sipemu/dbt-lineage-viewer
|
|
19
|
+
|
|
20
|
+
# dbt-lineage
|
|
21
|
+
|
|
22
|
+
[](https://github.com/sipemu/dbt-lineage-viewer/actions/workflows/ci.yml)
|
|
23
|
+
[](https://crates.io/crates/dbt-lineage)
|
|
24
|
+
[](https://docs.rs/dbt-lineage)
|
|
25
|
+
[](https://codecov.io/gh/sipemu/dbt-lineage-viewer)
|
|
26
|
+
|
|
27
|
+

|
|
28
|
+
|
|
29
|
+
A fast Rust CLI tool for visualizing [dbt](https://www.getdbt.com/) model lineage. Parses SQL files directly to extract `ref()` and `source()` dependencies, builds a DAG, and renders it as ASCII art, Graphviz DOT, SVG, interactive HTML, or a terminal UI.
|
|
30
|
+
|
|
31
|
+
Supports both direct SQL parsing (no dbt compilation or Python runtime needed) and `manifest.json` for full-fidelity graphs.
|
|
32
|
+
|
|
33
|
+
## Features
|
|
34
|
+
|
|
35
|
+
### Source modes
|
|
36
|
+
|
|
37
|
+
- **Direct SQL parsing** — extracts `ref()` and `source()` via a Jinja-aware lexer, no `dbt compile` needed
|
|
38
|
+
- **Manifest support** — optionally read `manifest.json` for column metadata, materializations, and full graph fidelity
|
|
39
|
+
- **Macro-aware ref()** — SQL-parse mode follows simple `{% macro smart_ref(name) %}{{ ref(name) }}{% endmacro %}` wrappers automatically
|
|
40
|
+
- **Parallel parsing + on-disk cache** — [`rayon`](https://crates.io/crates/rayon)-parallelized parsing and a content-hash cache at `.dbt-lineage/cache.bin`; warm runs are near-instant on large monorepos. Bypass with `--no-cache`. See [`docs/CACHE.md`](docs/CACHE.md)
|
|
41
|
+
|
|
42
|
+
### Analysis subcommands
|
|
43
|
+
|
|
44
|
+
- **`summary`** — one-shot project overview (counts, tags, top fan-out, orphans)
|
|
45
|
+
- **`impact <model>`** — downstream impact with severity scoring (Critical / High / Medium / Low)
|
|
46
|
+
- **`diff --base <ref>`** — compares lineage between git refs, showing added / removed / modified nodes and edges
|
|
47
|
+
- **`plan --base <ref>`** — emits a dbt selector for the minimal rebuild set; pipes into `dbt run -s "$(…)"`
|
|
48
|
+
- **`coverage`** — tests-per-model + per-column map; text, JSON, or SARIF (PR annotations)
|
|
49
|
+
- **`lint`** — flags unused sources, undefined refs, dead-end models, missing descriptions, models with only unresolved upstream, circular refs
|
|
50
|
+
- **`site`** — static lineage site (search, per-node pages with lineage diagram, column-level lineage, SQL) for GitHub/GitLab Pages
|
|
51
|
+
- **`docs`** — generates per-model Markdown with description, upstream / downstream, columns, embedded Mermaid lineage
|
|
52
|
+
- **`perf`** — joins `run_results.json` with the DAG: slowest models, critical paths
|
|
53
|
+
- **`check-manifest`** — flags drift between `manifest.json` and current SQL files using dbt's recorded SHA-256 (mtime fallback); exits non-zero for CI gating
|
|
54
|
+
|
|
55
|
+
### Column-level lineage
|
|
56
|
+
|
|
57
|
+
- AST-backed analyzer via [`sqlparser`](https://crates.io/crates/sqlparser) (0.6.0+) handles aliases, JOINs, aggregations, and CTEs with proper transformation classification (`Direct`, `Aliased`, `Derived`)
|
|
58
|
+
- Regex-based analyzer remains as a fallback for dialect-specific syntax sqlparser can't yet parse
|
|
59
|
+
|
|
60
|
+
### MCP server (AI-agent integration)
|
|
61
|
+
|
|
62
|
+
- **`dbt-lineage mcp`** — stdio Model Context Protocol server. See [`docs/MCP.md`](docs/MCP.md) for setup and cookbook.
|
|
63
|
+
- **10 tools**: `summary`, `search_models`, `lineage`, `impact`, `get_model_details`, `read_model_sql`, `column_upstream`, `column_downstream`, `lineage_bundle`, `propose_test`
|
|
64
|
+
- **Resources**: model SQL, focused lineage diagrams, source/exposure metadata, always-fresh summary
|
|
65
|
+
- **Prompts**: `review-impact`, `propose-tests`, `explain-column`, `find-bottleneck`, `document-model`, `onboard-project`
|
|
66
|
+
|
|
67
|
+
### Output
|
|
68
|
+
|
|
69
|
+
- **6 formats** — ASCII, Graphviz DOT, JSON, Mermaid, self-contained SVG, interactive HTML (pan / zoom / search)
|
|
70
|
+
- **Mermaid niceties** — `--show-columns` inlines column names in node labels; `--group-by directory` emits Mermaid subgraphs
|
|
71
|
+
- **Graph collapse** — `--collapse` / `--collapse=focal` drops intermediate nodes and labels transitive paths with `(via N)` edges
|
|
72
|
+
- **Structured errors** — `--error-format json` writes `{level, what, why, hint}` to stderr for deterministic agent / CI consumers
|
|
73
|
+
|
|
74
|
+
### Editor integration
|
|
75
|
+
|
|
76
|
+
- **`dbt-lineage lsp`** — language server: hover and go-to-definition on `ref()` / `source()`, lineage code lens per model, lint diagnostics. VS Code extension in [`editors/vscode`](editors/vscode); Neovim and Helix setup in [`docs/EDITORS.md`](docs/EDITORS.md)
|
|
77
|
+
|
|
78
|
+
### Interactive TUI
|
|
79
|
+
|
|
80
|
+
- Navigate, search, and explore lineage in a terminal UI ([`ratatui`](https://crates.io/crates/ratatui)) with Unicode box-drawing nodes, orthogonal edge routing, and full mouse support
|
|
81
|
+
- **Run dbt from TUI** — execute `dbt run` / `dbt test` on selected models with scope control (`+upstream`, `downstream+`, `+all+`) via keyboard menu or right-click context menu
|
|
82
|
+
- **Run status tracking** — color-coded nodes show success (green), error (red), outdated (yellow), or never-run (default)
|
|
83
|
+
- **Path highlighting** — trace upstream / downstream paths with impact analysis
|
|
84
|
+
|
|
85
|
+
### Filtering and node types
|
|
86
|
+
|
|
87
|
+
- **Selector expressions** — `-s tag:finance,path:marts` filters by tag, path, or model name (comma-separated)
|
|
88
|
+
- **Node-type filters** — `--include-tests`, `--include-seeds`, `--include-snapshots`, `--include-exposures`
|
|
89
|
+
- **Node-type support** — models, sources, seeds, snapshots, tests, exposures
|
|
90
|
+
|
|
91
|
+
## Installation
|
|
92
|
+
|
|
93
|
+
### From crates.io
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
cargo install dbt-lineage
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### Homebrew (macOS, Linux)
|
|
100
|
+
|
|
101
|
+
```sh
|
|
102
|
+
brew install sipemu/tap/dbt-lineage
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
### With cargo-binstall (no compile)
|
|
106
|
+
|
|
107
|
+
```sh
|
|
108
|
+
cargo binstall dbt-lineage
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
### Pre-built binaries
|
|
112
|
+
|
|
113
|
+
Every tagged release ships pre-built binaries for Linux (x86_64 + aarch64), macOS (Intel + Apple Silicon), and Windows (x86_64) as downloadable assets on the [Releases page](https://github.com/sipemu/dbt-lineage-viewer/releases). Download the archive for your platform, extract, and put the binary on your `PATH`.
|
|
114
|
+
|
|
115
|
+
### As a GitHub Action
|
|
116
|
+
|
|
117
|
+
**Live demo:** [sipemu/dbt-lineage-demo#1](https://github.com/sipemu/dbt-lineage-demo/pull/1) shows the PR comment (lineage diff with SQL body changes, column impact, Mermaid impact diagram, rebuild selector, lint); [sipemu.github.io/dbt-lineage-demo](https://sipemu.github.io/dbt-lineage-demo/) is the same project as a `dbt-lineage site`.
|
|
118
|
+
|
|
119
|
+
```yaml
|
|
120
|
+
on: pull_request
|
|
121
|
+
|
|
122
|
+
permissions:
|
|
123
|
+
contents: read
|
|
124
|
+
pull-requests: write # sticky PR comment
|
|
125
|
+
security-events: write # only needed with `sarif:`
|
|
126
|
+
|
|
127
|
+
jobs:
|
|
128
|
+
lineage:
|
|
129
|
+
runs-on: ubuntu-latest
|
|
130
|
+
steps:
|
|
131
|
+
- uses: actions/checkout@v5
|
|
132
|
+
with:
|
|
133
|
+
fetch-depth: 0 # `diff` / `plan` need the base branch history
|
|
134
|
+
- uses: sipemu/dbt-lineage-viewer@v0
|
|
135
|
+
with:
|
|
136
|
+
base: ${{ github.base_ref }} # e.g. main; resolved to origin/main if needed
|
|
137
|
+
project-dir: dbt/
|
|
138
|
+
comment: true # post a Markdown summary as a PR comment
|
|
139
|
+
sarif: lint,coverage # also surface findings as PR annotations
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
`@v0` tracks the latest `0.x` release; pin an exact tag (e.g. `@v0.9.1`) if you prefer. The action installs the matching `dbt-lineage` binary from this repo's release artifacts, runs `summary`, `diff`, `plan`, and optionally `lint`/`coverage` SARIF, then posts a single sticky comment on the PR with the rendered report. See [`action.yml`](action.yml) for the full input/output reference.
|
|
143
|
+
|
|
144
|
+
### Container image
|
|
145
|
+
|
|
146
|
+
```sh
|
|
147
|
+
docker run --rm -v "$PWD:/workspace" ghcr.io/sipemu/dbt-lineage:latest summary
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Published to `ghcr.io/sipemu/dbt-lineage` on every release (`latest`, `v0`, `0.x`, `0.x.y`; amd64 + arm64), with `git` included for `diff` / `plan` / `ci report`. In GitLab CI, override the entrypoint:
|
|
151
|
+
|
|
152
|
+
```yaml
|
|
153
|
+
dbt-lineage:
|
|
154
|
+
image: { name: ghcr.io/sipemu/dbt-lineage:v0, entrypoint: [""] }
|
|
155
|
+
script: dbt-lineage ci report --out report.md
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### As a pre-commit hook
|
|
159
|
+
|
|
160
|
+
```yaml
|
|
161
|
+
# .pre-commit-config.yaml in your dbt repo
|
|
162
|
+
repos:
|
|
163
|
+
- repo: https://github.com/sipemu/dbt-lineage-viewer
|
|
164
|
+
rev: v0.10.0
|
|
165
|
+
hooks:
|
|
166
|
+
- id: dbt-lineage-lint
|
|
167
|
+
- id: dbt-lineage-check-manifest # needs a compiled target/manifest.json
|
|
168
|
+
args: [-p, path/to/dbt] # only if the project isn't at the repo root
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
### From source
|
|
172
|
+
|
|
173
|
+
```sh
|
|
174
|
+
git clone https://github.com/sipemu/dbt-lineage-viewer.git
|
|
175
|
+
cd dbt-lineage-viewer
|
|
176
|
+
cargo install --path .
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
The binary is installed to `~/.cargo/bin/dbt-lineage`.
|
|
180
|
+
|
|
181
|
+
## Usage
|
|
182
|
+
|
|
183
|
+
### Static output
|
|
184
|
+
|
|
185
|
+
```sh
|
|
186
|
+
# Full lineage of current directory's dbt project
|
|
187
|
+
dbt-lineage
|
|
188
|
+
|
|
189
|
+
# Focus on a specific model
|
|
190
|
+
dbt-lineage stg_orders
|
|
191
|
+
|
|
192
|
+
# Point at a different project directory
|
|
193
|
+
dbt-lineage -p path/to/dbt/project
|
|
194
|
+
|
|
195
|
+
# Show 2 levels upstream, 1 downstream
|
|
196
|
+
dbt-lineage stg_orders -u 2 -d 1
|
|
197
|
+
|
|
198
|
+
# Include seeds, tests, snapshots, exposures
|
|
199
|
+
dbt-lineage --include-seeds --include-tests --include-snapshots --include-exposures
|
|
200
|
+
|
|
201
|
+
# Selector expressions
|
|
202
|
+
dbt-lineage -s tag:finance,path:marts
|
|
203
|
+
|
|
204
|
+
# Use manifest.json instead of parsing SQL
|
|
205
|
+
dbt-lineage --manifest target/manifest.json
|
|
206
|
+
|
|
207
|
+
# Output formats
|
|
208
|
+
dbt-lineage -o dot > lineage.dot # Graphviz DOT
|
|
209
|
+
dbt-lineage -o json # JSON graph
|
|
210
|
+
dbt-lineage -o mermaid # Mermaid diagram
|
|
211
|
+
dbt-lineage -o svg > lineage.svg # Self-contained SVG
|
|
212
|
+
dbt-lineage -o html > lineage.html # Interactive HTML (pan/zoom/search)
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
### Interactive TUI
|
|
216
|
+
|
|
217
|
+
```sh
|
|
218
|
+
dbt-lineage -i
|
|
219
|
+
dbt-lineage -i -p path/to/dbt/project
|
|
220
|
+
dbt-lineage -i stg_orders -u 3 -d 3
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
### Breaking changes to governed models
|
|
224
|
+
|
|
225
|
+
`diff`, `ci report` and the PR comment flag changes that break consumers of governed models (dbt 1.5+ model governance in schema YAML):
|
|
226
|
+
|
|
227
|
+
- contracted models (`config: {contract: {enforced: true}}`): a contract column removed or its `data_type` changed, or the contract dropped;
|
|
228
|
+
- `access` narrowed (`public` → `protected` → `private`);
|
|
229
|
+
- a model version (`versions:`) or a governed model removed.
|
|
230
|
+
|
|
231
|
+
`ci report --fail-on breaking-changes` fails the check. `lint` adds a `private-ref` error for refs to `access: private` models from outside their `group`, which dbt rejects at parse time.
|
|
232
|
+
|
|
233
|
+
### Impact analysis
|
|
234
|
+
|
|
235
|
+
Compute downstream impact for a model with severity scoring:
|
|
236
|
+
|
|
237
|
+
```sh
|
|
238
|
+
dbt-lineage impact orders -p path/to/project # text report
|
|
239
|
+
dbt-lineage impact orders -o json # JSON for CI
|
|
240
|
+
dbt-lineage impact orders --manifest target/manifest.json
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Severity levels:
|
|
244
|
+
- **Critical** — impacts exposures (dashboards, reports)
|
|
245
|
+
- **High** — impacts table/incremental materializations or mart models
|
|
246
|
+
- **Medium** — impacts staging or intermediate models
|
|
247
|
+
- **Low** — impacts tests only
|
|
248
|
+
|
|
249
|
+
### Lineage diff
|
|
250
|
+
|
|
251
|
+
Compare lineage between git refs to see what changed:
|
|
252
|
+
|
|
253
|
+
```sh
|
|
254
|
+
dbt-lineage diff --base main # compare main to working tree
|
|
255
|
+
dbt-lineage diff --base main --head feature-branch # compare two branches
|
|
256
|
+
dbt-lineage diff --base HEAD~1 -o json # JSON for CI integration
|
|
257
|
+
```
|
|
258
|
+
|
|
259
|
+
Shows added, removed, and modified nodes and edges with a summary of changes.
|
|
260
|
+
|
|
261
|
+
### Static lineage site
|
|
262
|
+
|
|
263
|
+
A self-contained static site — search, one page per model/source/seed/exposure with its lineage diagram, columns with column-level lineage, and highlighted SQL — ready for GitHub or GitLab Pages:
|
|
264
|
+
|
|
265
|
+
```sh
|
|
266
|
+
dbt-lineage site -p path/to/project --out site
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
Light/dark, phone-friendly, no server or CDN. See [`docs/SITE.md`](docs/SITE.md) for Pages workflows.
|
|
270
|
+
|
|
271
|
+
### Project summary
|
|
272
|
+
|
|
273
|
+
One-shot project overview, designed as the first thing an agent (or human) runs against an unfamiliar dbt project:
|
|
274
|
+
|
|
275
|
+
```sh
|
|
276
|
+
dbt-lineage summary -p path/to/project # text
|
|
277
|
+
dbt-lineage summary --manifest target/manifest.json # manifest mode
|
|
278
|
+
dbt-lineage summary -o json # structured for piping into jq
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
Reports counts per node type, tag distribution, top downstream-reach models, and orphan models (no downstream consumers).
|
|
282
|
+
|
|
283
|
+
### BI dashboards as exposures
|
|
284
|
+
|
|
285
|
+
Generate dbt exposures from Metabase, anofox-visualization, Tableau, Looker, Superset or Power BI dashboards, so impact analysis knows which dashboards a change reaches:
|
|
286
|
+
|
|
287
|
+
```sh
|
|
288
|
+
METABASE_API_KEY=mb_… dbt-lineage exposures metabase --url https://metabase.example.com \
|
|
289
|
+
--write models/exposures_metabase.yml
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
From a LookML project, offline (LookML dashboards → explores → views → tables):
|
|
293
|
+
|
|
294
|
+
```sh
|
|
295
|
+
dbt-lineage exposures looker --lookml path/to/lookml-project --write models/exposures_looker.yml
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
From [anofox-visualization](https://github.com/DataZooDE/anofox-visualization) dashboards (DuckDB `.sql` files with `::ROLE` panels):
|
|
299
|
+
|
|
300
|
+
```sh
|
|
301
|
+
dbt-lineage exposures anofox --dashboards dashboards/ --url 'https://viz.example.com/d/{name}' \
|
|
302
|
+
--write models/exposures_anofox.yml
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
From Tableau workbooks, offline (`.twb` / `.twbx`):
|
|
306
|
+
|
|
307
|
+
```sh
|
|
308
|
+
dbt-lineage exposures tableau --workbooks workbooks/ --write models/exposures_tableau.yml
|
|
309
|
+
```
|
|
310
|
+
|
|
311
|
+
From Superset dashboard exports, offline (ZIPs from `superset export-dashboards` or the UI; dashboard → charts → datasets → tables, virtual-dataset SQL included):
|
|
312
|
+
|
|
313
|
+
```sh
|
|
314
|
+
dbt-lineage exposures superset --exports exports/ \
|
|
315
|
+
--url 'https://superset.example.com/superset/dashboard/{name}/' --write models/exposures_superset.yml
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
From Power BI, offline: PBIP projects (`*.Report` + `*.SemanticModel`, TMDL or `model.bim`) and `.pbit` templates; tables come from the partitions' Power Query (M) sources and native SQL:
|
|
319
|
+
|
|
320
|
+
```sh
|
|
321
|
+
dbt-lineage exposures powerbi --reports powerbi/ --url 'https://app.powerbi.com/groups/me/reports/{name}' \
|
|
322
|
+
--write models/exposures_powerbi.yml
|
|
323
|
+
```
|
|
324
|
+
|
|
325
|
+
Tables the dashboards read are mapped onto models and sources; unmatched tables are listed on stderr. See [`docs/BI.md`](docs/BI.md).
|
|
326
|
+
|
|
327
|
+
### CI report
|
|
328
|
+
|
|
329
|
+
One Markdown report for PR/MR comments — lineage diff, rebuild selector, manifest freshness, lint, and project summary — plus exit codes for gating:
|
|
330
|
+
|
|
331
|
+
```sh
|
|
332
|
+
dbt-lineage ci report # base = PR/MR target branch from CI env
|
|
333
|
+
dbt-lineage ci report --base main --out report.md # explicit base, write to file
|
|
334
|
+
dbt-lineage ci report --fail-on lint-errors,plan-empty --outputs "$GITHUB_OUTPUT"
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
`--base auto` (default) reads the target branch from GitHub Actions, GitLab CI, Bitbucket Pipelines or Azure Pipelines, and resolves it as `origin/<branch>` (fetching it if the checkout is shallow). `--outputs` appends `selector=…` and `failed=…` lines, usable as `$GITHUB_OUTPUT` or a GitLab dotenv report. `dbt-lineage ci comment --file report.md` posts it as one sticky PR/MR comment, and `dbt-lineage ci annotations --format sarif|codeclimate|bitbucket|azure` turns lint findings into inline annotations (Bitbucket Code Insights with `--post`). The GitHub Action, the GitLab component, the Bitbucket Pipe and the Azure Pipelines template are thin wrappers around these commands; see [`docs/CI.md`](docs/CI.md).
|
|
338
|
+
|
|
339
|
+
### Manifest freshness
|
|
340
|
+
|
|
341
|
+
CI-friendly drift check between a distributed `manifest.json` and current SQL files:
|
|
342
|
+
|
|
343
|
+
```sh
|
|
344
|
+
dbt-lineage check-manifest -p path/to/project # text; exit 1 if stale
|
|
345
|
+
dbt-lineage check-manifest -o json # JSON for tooling
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
Flags three kinds of drift: SQL files referenced by the manifest that are now missing, files modified since the manifest was generated, and SQL files added but not yet compiled into the manifest.
|
|
349
|
+
|
|
350
|
+
### Graph collapse
|
|
351
|
+
|
|
352
|
+
Collapse intermediate nodes so big projects render as just their endpoints, with transitive paths labeled `(via N)`:
|
|
353
|
+
|
|
354
|
+
```sh
|
|
355
|
+
dbt-lineage --collapse -o mermaid # keep endpoints + focus model
|
|
356
|
+
dbt-lineage orders --collapse=focal -u 3 -o mermaid # focal: only sources/exposures/focus
|
|
357
|
+
dbt-lineage --collapse -o json # JSON edges carry a via_hops field
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
Positional focus models (e.g. `dbt-lineage orders --collapse`) are always preserved even when they would otherwise be classified as intermediate.
|
|
361
|
+
|
|
362
|
+
### Mermaid options
|
|
363
|
+
|
|
364
|
+
```sh
|
|
365
|
+
dbt-lineage -o mermaid --show-columns # inline column names in node labels
|
|
366
|
+
dbt-lineage -o mermaid --group-by directory # group nodes by directory via subgraph
|
|
367
|
+
dbt-lineage -o mermaid --show-columns --collapse # combine: endpoint nodes with full column lists
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
### Structured errors for agents and CI
|
|
371
|
+
|
|
372
|
+
```sh
|
|
373
|
+
dbt-lineage --error-format json some_unknown_model
|
|
374
|
+
# stderr: {"level":"error","what":"model not found","why":"...","hint":"..."}
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
`--error-format json` writes one JSON object per line to stderr. `stdout` is unaffected, so piping into `jq` still works.
|
|
378
|
+
|
|
379
|
+
### Editor integration (language server)
|
|
380
|
+
|
|
381
|
+
`dbt-lineage lsp` serves the Language Server Protocol over stdio: hover and go-to-definition on `ref()` / `source()`, a code lens with upstream/downstream counts and impact on every model, and `lint` findings as diagnostics. Use the VS Code extension in [`editors/vscode`](editors/vscode), or point Neovim, Helix or any LSP client at `dbt-lineage lsp`. See [`docs/EDITORS.md`](docs/EDITORS.md).
|
|
382
|
+
|
|
383
|
+
### MCP server
|
|
384
|
+
|
|
385
|
+
Expose lineage as stdio MCP tools, resources, and prompts that Claude Code, Claude Desktop, and other MCP-aware clients can call directly:
|
|
386
|
+
|
|
387
|
+
```sh
|
|
388
|
+
claude mcp add dbt-lineage -- dbt-lineage mcp --project-dir /abs/path/to/project # SQL-parse mode
|
|
389
|
+
claude mcp add dbt-lineage -- dbt-lineage mcp --manifest /abs/path/to/manifest.json # manifest mode
|
|
390
|
+
```
|
|
391
|
+
|
|
392
|
+
Works in both SQL-parse and manifest mode (`target/manifest.json` is picked up automatically when present). Tools: `summary`, `search_models`, `lineage`, `impact`, `get_model_details`, `read_model_sql`, `column_upstream`, `column_downstream`, `lineage_bundle`, `propose_test`; plus model resources and predefined prompts. MCP Registry name: `mcp-name: io.github.sipemu/dbt-lineage`. See [`docs/MCP.md`](docs/MCP.md) for client setup, prompt walkthroughs, and composition recipes.
|
|
393
|
+
|
|
394
|
+
## CLI Reference
|
|
395
|
+
|
|
396
|
+
Run `dbt-lineage <command> --help` for per-subcommand options.
|
|
397
|
+
|
|
398
|
+
```
|
|
399
|
+
Usage: dbt-lineage [OPTIONS] [MODEL] [COMMAND]
|
|
400
|
+
|
|
401
|
+
Commands:
|
|
402
|
+
impact Compute downstream impact analysis for a model
|
|
403
|
+
summary Print a one-shot project overview: counts, tags, top fan-out, orphans
|
|
404
|
+
mcp Run an MCP (Model Context Protocol) stdio server exposing lineage tools to AI agents. Reads JSON-RPC requests on stdin, writes responses to stdout
|
|
405
|
+
lsp Run a Language Server (stdio) for editors: hover and go-to-definition on ref()/source(), a lineage code lens per model, and lint diagnostics
|
|
406
|
+
perf Critical-path and slowest-model performance report from target/run_results.json
|
|
407
|
+
macros Show the macro call graph (macros calling other macros)
|
|
408
|
+
site Generate a static lineage site (search, one page per node with columns, column lineage, SQL and a lineage diagram) for GitHub/GitLab Pages
|
|
409
|
+
docs Generate Markdown documentation per model (or one consolidated file)
|
|
410
|
+
lint Structural lint: dead-end models, unused sources, undefined refs, missing docs. Exit code 1 if any error-severity finding is reported
|
|
411
|
+
coverage Test coverage map: which models have which tests, column coverage gaps
|
|
412
|
+
check-manifest Verify a manifest.json is still in sync with the project's SQL files. Exits with code 1 on staleness, making it directly usable in CI pipelines
|
|
413
|
+
plan Emit a dbt selector expression covering models added or modified since `--base`. Designed to pipe into `dbt run -s "$(dbt-lineage plan --base main)"`
|
|
414
|
+
ci CI integration: one Markdown report for PR/MR comments, plus exit codes for gating. Works on GitHub Actions, GitLab CI, Bitbucket Pipelines and Azure Pipelines
|
|
415
|
+
exposures Import BI dashboards as dbt exposures (YAML), mapping the tables they read onto the project's models and sources
|
|
416
|
+
diff Compare lineage between git refs
|
|
417
|
+
help Print this message or the help of the given subcommand(s)
|
|
418
|
+
|
|
419
|
+
Arguments:
|
|
420
|
+
[MODEL] Model name to focus on (shows full lineage if omitted)
|
|
421
|
+
|
|
422
|
+
Options:
|
|
423
|
+
-p, --project-dir <PROJECT_DIR> Path to dbt project directory [default: .]
|
|
424
|
+
-u, --upstream <UPSTREAM> Upstream levels to show (default: all)
|
|
425
|
+
-d, --downstream <DOWNSTREAM> Downstream levels to show (default: all)
|
|
426
|
+
-i, --interactive Launch interactive TUI mode
|
|
427
|
+
-o, --output <OUTPUT> Output format: ascii (default), dot, json, mermaid, svg, html [default: ascii] [possible values: ascii, dot, json, mermaid, svg, html]
|
|
428
|
+
--include-tests Include test nodes
|
|
429
|
+
--include-seeds Include seed nodes
|
|
430
|
+
--include-snapshots Include snapshot nodes
|
|
431
|
+
--include-exposures Include exposure nodes
|
|
432
|
+
-s, --select <SELECT> Selector expression: tag:X, path:Y, or model name (comma-separated)
|
|
433
|
+
--manifest <MANIFEST> Use manifest.json instead of parsing SQL (path to manifest file or directory containing target/manifest.json)
|
|
434
|
+
--show-columns Mermaid only: inline column names inside each node's label
|
|
435
|
+
--group-by <GROUP_BY> Mermaid only: group nodes by directory using Mermaid subgraphs [default: none] [possible values: none, directory]
|
|
436
|
+
--error-format <ERROR_FORMAT> How errors are written to stderr: human-readable text (default) or one JSON object per line (`{level, what, why, hint}`) for agent/CI consumers [default: plain] [possible values: plain, json]
|
|
437
|
+
--collapse [<COLLAPSE>] Drop intermediate nodes and replace transitive paths with `(via N)` edges. Bare `--collapse` is equivalent to `--collapse auto` (keep endpoints + focus); `--collapse focal` keeps only sources, exposures, and the focus model [default: none] [possible values: none, auto, focal]
|
|
438
|
+
--no-cache Bypass the on-disk parse cache (.dbt-lineage/cache.bin). Useful when debugging parser changes or you want a clean baseline benchmark
|
|
439
|
+
-h, --help Print help (see more with '--help')
|
|
440
|
+
-V, --version Print version
|
|
441
|
+
```
|
|
442
|
+
|
|
443
|
+
## TUI Keybindings
|
|
444
|
+
|
|
445
|
+
### Navigation
|
|
446
|
+
|
|
447
|
+
| Key | Action |
|
|
448
|
+
|-----|--------|
|
|
449
|
+
| `h` `j` `k` `l` / arrow keys | Navigate between nodes (left/down/up/right) |
|
|
450
|
+
| `H` `J` `K` `L` | Pan the viewport |
|
|
451
|
+
| `+` / `-` | Zoom in / out (adjusts spacing) |
|
|
452
|
+
| `Tab` / `Shift+Tab` | Cycle through nodes sequentially |
|
|
453
|
+
| `r` | Reset view (center + zoom) |
|
|
454
|
+
|
|
455
|
+
### Mouse
|
|
456
|
+
|
|
457
|
+
| Action | Target | Effect |
|
|
458
|
+
|--------|--------|--------|
|
|
459
|
+
| Left click | Node on graph | Select node (no viewport jump) |
|
|
460
|
+
| Left click | Empty graph area | Begin drag to pan |
|
|
461
|
+
| Drag | Graph area | Pan the viewport |
|
|
462
|
+
| Scroll up / down | Graph area | Zoom in / out |
|
|
463
|
+
| Left click | Node list entry | Select node and center viewport |
|
|
464
|
+
| Left click | Group header | Collapse / expand group |
|
|
465
|
+
| Right click | Node on graph | Open context menu (run options) |
|
|
466
|
+
|
|
467
|
+
### Search
|
|
468
|
+
|
|
469
|
+
| Key | Action |
|
|
470
|
+
|-----|--------|
|
|
471
|
+
| `/` | Open search (fuzzy, powered by [nucleo](https://crates.io/crates/nucleo-matcher) — type any few characters in any order) |
|
|
472
|
+
| `Tab` | Next search result |
|
|
473
|
+
| `Esc` / `Enter` | Close search |
|
|
474
|
+
|
|
475
|
+
### Analysis
|
|
476
|
+
|
|
477
|
+
| Key | Action |
|
|
478
|
+
|-----|--------|
|
|
479
|
+
| `p` | Toggle path highlighting (upstream/downstream trace with impact analysis) |
|
|
480
|
+
| `C` (Shift+C) | Toggle column-level lineage in detail panel |
|
|
481
|
+
| `v` | Toggle in-app SQL viewer (syntax-highlighted SQL of the selected node) |
|
|
482
|
+
| `m` | Toggle minimap overlay (top-right corner of the graph area) |
|
|
483
|
+
|
|
484
|
+
### Node list panel
|
|
485
|
+
|
|
486
|
+
| Key | Action |
|
|
487
|
+
|-----|--------|
|
|
488
|
+
| `n` | Toggle node list sidebar |
|
|
489
|
+
| `c` | Collapse/expand directory group |
|
|
490
|
+
|
|
491
|
+
### Running dbt
|
|
492
|
+
|
|
493
|
+
| Key | Action |
|
|
494
|
+
|-----|--------|
|
|
495
|
+
| `x` | Open run menu for selected node |
|
|
496
|
+
| Right click | Open context menu on a node (same run options) |
|
|
497
|
+
| `o` | View last run output |
|
|
498
|
+
|
|
499
|
+
Run menu / context menu options:
|
|
500
|
+
|
|
501
|
+
| Key | Command |
|
|
502
|
+
|-----|---------|
|
|
503
|
+
| `r` | `dbt run` (this model) |
|
|
504
|
+
| `u` | `dbt run` +upstream |
|
|
505
|
+
| `d` | `dbt run` downstream+ |
|
|
506
|
+
| `a` | `dbt run` +all+ |
|
|
507
|
+
| `t` | `dbt test` |
|
|
508
|
+
|
|
509
|
+
### General
|
|
510
|
+
|
|
511
|
+
| Key | Action |
|
|
512
|
+
|-----|--------|
|
|
513
|
+
| `q` | Quit |
|
|
514
|
+
| `Ctrl+C` | Quit (any mode) |
|
|
515
|
+
|
|
516
|
+
## Node colors in TUI
|
|
517
|
+
|
|
518
|
+
**By run status** (when `target/run_results.json` exists):
|
|
519
|
+
|
|
520
|
+
| Color | Meaning |
|
|
521
|
+
|-------|---------|
|
|
522
|
+
| Green | Last run succeeded |
|
|
523
|
+
| Red | Last run failed |
|
|
524
|
+
| Yellow | Outdated (source file modified after last run) |
|
|
525
|
+
| DarkGray | Skipped |
|
|
526
|
+
|
|
527
|
+
**By node type** (when never run):
|
|
528
|
+
|
|
529
|
+
| Color | Type |
|
|
530
|
+
|-------|------|
|
|
531
|
+
| Blue | Model |
|
|
532
|
+
| Green | Source |
|
|
533
|
+
| Yellow | Seed |
|
|
534
|
+
| Magenta | Snapshot |
|
|
535
|
+
| Cyan | Test |
|
|
536
|
+
| Red | Exposure |
|
|
537
|
+
| DarkGray | Phantom (unresolved ref) |
|
|
538
|
+
|
|
539
|
+
## How it works
|
|
540
|
+
|
|
541
|
+
1. **Parse** `dbt_project.yml` to find model/seed/snapshot paths (or read `manifest.json`)
|
|
542
|
+
2. **Walk** those directories, collecting `.sql` and `.yml` files
|
|
543
|
+
3. **Extract** `ref('model')` and `source('schema', 'table')` from SQL via regex
|
|
544
|
+
4. **Parse** YAML schema files for sources, model descriptions, and exposures
|
|
545
|
+
5. **Build** a directed acyclic graph (petgraph) where edges flow from dependency to dependent
|
|
546
|
+
6. **Resolve** column-level lineage by tracing SELECT/FROM/JOIN through the graph
|
|
547
|
+
7. **Filter** by focus model, depth, selectors, and node type
|
|
548
|
+
8. **Layout** using a Sugiyama-style layered algorithm (longest-path layering + barycenter ordering)
|
|
549
|
+
9. **Render** as ASCII, DOT, JSON, Mermaid, SVG, HTML, or interactive TUI
|
|
550
|
+
|
|
551
|
+
## uv / virtualenv support
|
|
552
|
+
|
|
553
|
+
When running dbt from the TUI, the tool auto-detects whether to use `uv run dbt` or plain `dbt`:
|
|
554
|
+
|
|
555
|
+
- If `uv.lock` or `pyproject.toml` exists in the dbt project directory, uses `uv run dbt`
|
|
556
|
+
- Otherwise, if `dbt` is on PATH, uses it directly
|
|
557
|
+
- If `dbt` is not on PATH but `uv` is, falls back to `uv run dbt`
|
|
558
|
+
|
|
559
|
+
## Building without TUI
|
|
560
|
+
|
|
561
|
+
The TUI is enabled by default. To build a minimal binary with only static output:
|
|
562
|
+
|
|
563
|
+
```sh
|
|
564
|
+
cargo build --release --no-default-features
|
|
565
|
+
```
|
|
566
|
+
|
|
567
|
+
## Further reading
|
|
568
|
+
|
|
569
|
+
- [`CHANGELOG.md`](CHANGELOG.md) — per-release notes from 0.1.0 onward.
|
|
570
|
+
- [`docs/MCP.md`](docs/MCP.md) — cookbook for the MCP server: setup, tool reference, prompt walkthroughs, composition recipes.
|
|
571
|
+
- [`docs/CACHE.md`](docs/CACHE.md) — parse-cache contract, file format, when it helps.
|
|
572
|
+
- [`BENCHMARKS.md`](BENCHMARKS.md) — baseline numbers from the bench harness.
|
|
573
|
+
|
|
574
|
+
## Credits
|
|
575
|
+
|
|
576
|
+
The agent-oriented additions in 0.3.0 — `summary`, `check-manifest`, `mcp`, `--collapse`, `--error-format json`, and the Mermaid `--show-columns` / `--group-by directory` options — were inspired by [eitsupi/dlin](https://github.com/eitsupi/dlin), a hard fork that focuses on non-interactive (CI/agent) workflows. dbt-lineage retains its TUI focus and brings these ideas back upstream.
|
|
577
|
+
|
|
578
|
+
## License
|
|
579
|
+
|
|
580
|
+
MIT
|
|
581
|
+
|
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
dbt_lineage_cli-0.15.0.data/scripts/dbt-lineage.exe,sha256=jgkpPihRlR9RU-AE-XP7Xmc4EzahYqdRwVpW_XmXA-g,13384192
|
|
2
|
+
dbt_lineage_cli-0.15.0.dist-info/METADATA,sha256=gxeC4o5BkucK33ta8A23I9XR3wYRmI-eXS2uo00yXi8,27973
|
|
3
|
+
dbt_lineage_cli-0.15.0.dist-info/WHEEL,sha256=Uvw7WSmy7Kj2FBk8Va4L_dq4MDW8X5Ynby-pOX3ZNS0,94
|
|
4
|
+
dbt_lineage_cli-0.15.0.dist-info/licenses/LICENSE,sha256=MsV1O2lnM3q-M_CwnZezEmuZEB-77OcUeBAtJ2AfL9s,1091
|
|
5
|
+
dbt_lineage_cli-0.15.0.dist-info/sboms/dbt-lineage.cyclonedx.json,sha256=RxDU5wqEQOkYd-OoEy8b_YjJvxBF4UnAY-8HDoHxuPQ,211612
|
|
6
|
+
dbt_lineage_cli-0.15.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Simon Müller
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|