sqlquality 0.2.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- sqlquality/__init__.py +3 -0
- sqlquality/adapters/__init__.py +20 -0
- sqlquality/adapters/base.py +21 -0
- sqlquality/adapters/postgres.py +60 -0
- sqlquality/adapters/redshift.py +46 -0
- sqlquality/antipatterns.py +174 -0
- sqlquality/changeset.py +93 -0
- sqlquality/cli.py +540 -0
- sqlquality/complexity.py +41 -0
- sqlquality/config.py +86 -0
- sqlquality/dbtproject.py +126 -0
- sqlquality/delta.py +76 -0
- sqlquality/dialects.py +23 -0
- sqlquality/gate.py +37 -0
- sqlquality/keys.py +106 -0
- sqlquality/linter.py +70 -0
- sqlquality/llm.py +101 -0
- sqlquality/models.py +56 -0
- sqlquality/py.typed +0 -0
- sqlquality/report.py +124 -0
- sqlquality/sqlast.py +128 -0
- sqlquality-0.2.0.dist-info/METADATA +519 -0
- sqlquality-0.2.0.dist-info/RECORD +26 -0
- sqlquality-0.2.0.dist-info/WHEEL +4 -0
- sqlquality-0.2.0.dist-info/entry_points.txt +2 -0
- sqlquality-0.2.0.dist-info/licenses/LICENSE +21 -0
|
@@ -0,0 +1,519 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: sqlquality
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Measure dbt model performance and complexity, and gate changes on the delta.
|
|
5
|
+
Project-URL: Homepage, https://github.com/hanslemm/sqlquality
|
|
6
|
+
Project-URL: Issues, https://github.com/hanslemm/sqlquality/issues
|
|
7
|
+
Author: Hans Lemm
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: complexity,data-engineering,dbt,lint,sql
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
18
|
+
Classifier: Topic :: Database
|
|
19
|
+
Requires-Python: >=3.11
|
|
20
|
+
Requires-Dist: pyyaml>=6
|
|
21
|
+
Requires-Dist: rich>=13
|
|
22
|
+
Requires-Dist: sqlfluff<5,>=4
|
|
23
|
+
Requires-Dist: sqlglot<31,>=30.12
|
|
24
|
+
Requires-Dist: typer>=0.12
|
|
25
|
+
Provides-Extra: llm
|
|
26
|
+
Requires-Dist: anthropic>=0.40; extra == 'llm'
|
|
27
|
+
Description-Content-Type: text/markdown
|
|
28
|
+
|
|
29
|
+
# sqlquality
|
|
30
|
+
|
|
31
|
+
Measure the **structural complexity** of dbt models' SQL and **gate pull requests
|
|
32
|
+
on the complexity delta** between two dbt manifests. Alongside the gate, `sqlquality`
|
|
33
|
+
runs per-engine static **performance anti-pattern** checks (with optional
|
|
34
|
+
captured-`EXPLAIN` analysis), sqlfluff-backed **linting**, and optional, advisory
|
|
35
|
+
**LLM suggestions**.
|
|
36
|
+
|
|
37
|
+
It is a static tool. It does not connect to your warehouse or execute queries:
|
|
38
|
+
|
|
39
|
+
- **Complexity** is computed from the SQL AST (via [sqlglot](https://github.com/tobymao/sqlglot)).
|
|
40
|
+
- **Performance** is static anti-pattern detection plus ingestion of an `EXPLAIN`
|
|
41
|
+
plan you captured yourself — no query is ever run.
|
|
42
|
+
- **Neighbors** (a changed model's direct upstream/downstream models) are *reported*
|
|
43
|
+
for context; they are not scored or gated.
|
|
44
|
+
|
|
45
|
+
Requires Python 3.11+.
|
|
46
|
+
|
|
47
|
+
## Contents
|
|
48
|
+
|
|
49
|
+
- [Install](#install)
|
|
50
|
+
- [Commands](#commands)
|
|
51
|
+
- [complexity](#complexity)
|
|
52
|
+
- [lint](#lint)
|
|
53
|
+
- [perf](#perf)
|
|
54
|
+
- [check](#check-the-ci-gate)
|
|
55
|
+
- [Configuration](#configuration-sqlqualityyml)
|
|
56
|
+
- [Exit codes](#exit-codes)
|
|
57
|
+
- [CI recipe (a gate that actually gates)](#ci-recipe-a-gate-that-actually-gates)
|
|
58
|
+
- [Pre-commit hook](#pre-commit-hook)
|
|
59
|
+
- [LLM suggestions](#llm-suggestions-optional-advisory)
|
|
60
|
+
- [Limitations](#limitations)
|
|
61
|
+
|
|
62
|
+
## Install
|
|
63
|
+
|
|
64
|
+
Once published to PyPI:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
pip install sqlquality
|
|
68
|
+
# or
|
|
69
|
+
uv add sqlquality
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Until then, install from git:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
pip install "sqlquality @ git+https://github.com/hanslemm/sqlquality"
|
|
76
|
+
# or
|
|
77
|
+
uv add "git+https://github.com/hanslemm/sqlquality"
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
The optional LLM suggestions feature needs the `llm` extra (pulls in the Anthropic
|
|
81
|
+
SDK):
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
pip install "sqlquality[llm]"
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`--version` prints the installed version:
|
|
88
|
+
|
|
89
|
+
```console
|
|
90
|
+
$ sqlquality --version
|
|
91
|
+
0.2.0
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Commands
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
sqlquality complexity Score the structural complexity of a single SQL file.
|
|
98
|
+
sqlquality check Gate a dbt change on the complexity delta of its changed models.
|
|
99
|
+
sqlquality lint Lint SQL files for best-practice violations (SQLFluff); --fix rewrites them.
|
|
100
|
+
sqlquality perf Analyze a SQL file for performance anti-patterns (+ optional EXPLAIN plan).
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
The `--dialect` / `-d` flag is validated against sqlglot's dialect registry on every
|
|
104
|
+
command; an unknown value fails fast with exit 2 and a suggestion. `complexity` and
|
|
105
|
+
`lint` also accept `-` to read SQL from stdin.
|
|
106
|
+
|
|
107
|
+
### complexity
|
|
108
|
+
|
|
109
|
+
Scores one SQL file and prints a per-metric contribution breakdown plus a composite.
|
|
110
|
+
|
|
111
|
+
```console
|
|
112
|
+
$ sqlquality complexity model.sql
|
|
113
|
+
Complexity — model.sql (composite 18.4)
|
|
114
|
+
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━┓
|
|
115
|
+
┃ metric ┃ value ┃ contribution ┃
|
|
116
|
+
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━┩
|
|
117
|
+
│ join_count │ 0 │ 0.0 │
|
|
118
|
+
│ cte_count │ 2 │ 4.0 │
|
|
119
|
+
│ subquery_count │ 0 │ 0.0 │
|
|
120
|
+
│ window_count │ 1 │ 4.0 │
|
|
121
|
+
│ case_count │ 0 │ 0.0 │
|
|
122
|
+
│ union_count │ 0 │ 0.0 │
|
|
123
|
+
│ distinct_count │ 0 │ 0.0 │
|
|
124
|
+
│ max_select_depth │ 2 │ 10.0 │
|
|
125
|
+
│ projected_columns │ 2 │ 0.4 │
|
|
126
|
+
└───────────────────┴───────┴──────────────┘
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
Read SQL from stdin with `-`:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
cat model.sql | sqlquality complexity -
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
`--json` emits a machine-readable payload (composite, per-metric contributions, and
|
|
136
|
+
the raw metrics):
|
|
137
|
+
|
|
138
|
+
```console
|
|
139
|
+
$ sqlquality complexity model.sql --json
|
|
140
|
+
{
|
|
141
|
+
"components": {
|
|
142
|
+
"case_count": 0.0,
|
|
143
|
+
"cte_count": 4.0,
|
|
144
|
+
"distinct_count": 0.0,
|
|
145
|
+
"join_count": 0.0,
|
|
146
|
+
"max_select_depth": 10.0,
|
|
147
|
+
"projected_columns": 0.4,
|
|
148
|
+
"subquery_count": 0.0,
|
|
149
|
+
"union_count": 0.0,
|
|
150
|
+
"window_count": 4.0
|
|
151
|
+
},
|
|
152
|
+
"composite": 18.4,
|
|
153
|
+
"dialect": "postgres",
|
|
154
|
+
"metrics": {
|
|
155
|
+
"case_count": 0,
|
|
156
|
+
"cte_count": 2,
|
|
157
|
+
"distinct_count": 0,
|
|
158
|
+
"join_count": 0,
|
|
159
|
+
"max_select_depth": 2,
|
|
160
|
+
"projected_columns": 2,
|
|
161
|
+
"select_count": 3,
|
|
162
|
+
"subquery_count": 0,
|
|
163
|
+
"union_count": 0,
|
|
164
|
+
"window_count": 1
|
|
165
|
+
},
|
|
166
|
+
"path": "model.sql"
|
|
167
|
+
}
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
**dbt / Jinja models:** if the file contains Jinja (`{{ ... }}`, `{% ... %}`),
|
|
171
|
+
`sqlquality` first tries to parse it as-is; on failure it retries with Jinja
|
|
172
|
+
markers stripped to placeholders and prints a notice to **stderr**:
|
|
173
|
+
|
|
174
|
+
```
|
|
175
|
+
analyzed with Jinja placeholders — results are approximate; prefer compiled SQL from target/compiled/
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
For accurate scores, point `complexity` at compiled SQL from `target/compiled/`
|
|
179
|
+
after `dbt compile`. The composite is a real, comparable score in both cases — but
|
|
180
|
+
placeholder-stripped results are approximate.
|
|
181
|
+
|
|
182
|
+
### lint
|
|
183
|
+
|
|
184
|
+
Lints SQL with sqlfluff and prints findings per file.
|
|
185
|
+
|
|
186
|
+
```console
|
|
187
|
+
$ sqlquality lint messy.sql
|
|
188
|
+
Lint —
|
|
189
|
+
messy.sql (5 findings)
|
|
190
|
+
┏━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
|
191
|
+
┃ line ┃ code ┃ severity ┃ fix? ┃ message ┃
|
|
192
|
+
┡━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
|
193
|
+
│ 1 │ AM04 │ warning │ │ Query produces an unknown number of result │
|
|
194
|
+
│ │ │ │ │ columns. │
|
|
195
|
+
│ 1 │ RF02 │ warning │ │ Unqualified reference '*' found in select … │
|
|
196
|
+
│ 2 │ AL01 │ warning │ ✓ │ Implicit/explicit aliasing of table. │
|
|
197
|
+
│ 2 │ AL01 │ warning │ ✓ │ Implicit/explicit aliasing of table. │
|
|
198
|
+
│ 2 │ AL05 │ warning │ ✓ │ Alias 'o' is never used in SELECT statement. │
|
|
199
|
+
└──────┴──────┴──────────┴──────┴──────────────────────────────────────────────┘
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
**Exit-code semantics:** `lint` exits **1** when any `WARNING`/`ERROR` finding is
|
|
203
|
+
present, so it gates CI and pre-commit by default. `--warn-only` prints/emits
|
|
204
|
+
findings but always exits 0. Findings from unresolved Jinja are demoted to `info`
|
|
205
|
+
severity and **never** gate.
|
|
206
|
+
|
|
207
|
+
Useful flags:
|
|
208
|
+
|
|
209
|
+
| Flag | Effect |
|
|
210
|
+
|---|---|
|
|
211
|
+
| `--fix` | Rewrite the file with auto-fixes. The exit code still reflects *pre-fix* findings (a fully-fixed file still exits 1). Cannot rewrite stdin. |
|
|
212
|
+
| `--warn-only` | Always exit 0. |
|
|
213
|
+
| `--sqlfluff-config <file>` | Apply a custom sqlfluff config (e.g. `.sqlfluff`). |
|
|
214
|
+
| `--exclude-rules <codes>` | Comma-separated rule codes to skip. |
|
|
215
|
+
| `--json` | Emit machine-readable JSON. |
|
|
216
|
+
|
|
217
|
+
`lint` accepts multiple files (and `-` for stdin), which is what the pre-commit hook
|
|
218
|
+
relies on.
|
|
219
|
+
|
|
220
|
+
### perf
|
|
221
|
+
|
|
222
|
+
Detects static performance anti-patterns for a given engine, and optionally folds in
|
|
223
|
+
findings parsed from a captured `EXPLAIN` plan.
|
|
224
|
+
|
|
225
|
+
```console
|
|
226
|
+
$ sqlquality perf messy.sql
|
|
227
|
+
Perf — messy.sql (postgres, 3 findings)
|
|
228
|
+
┏━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
|
229
|
+
┃ code ┃ severity ┃ message ┃
|
|
230
|
+
┡━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
|
231
|
+
│ SQ001 │ warning │ SELECT * projects an unknown/wide column set; list columns │
|
|
232
|
+
│ │ │ explicitly. │
|
|
233
|
+
│ SQ002 │ warning │ Cartesian/cross join without an ON/USING condition. │
|
|
234
|
+
│ SQ003 │ warning │ Leading-wildcard LIKE ('%...') is non-sargable and cannot use an │
|
|
235
|
+
│ │ │ index. │
|
|
236
|
+
└───────┴──────────┴─────────────────────────────────────────────────────────────────────┘
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
Supported engines: **`postgres`** and **`redshift`** (Redshift additionally infers
|
|
240
|
+
`DISTKEY`/`SORTKEY` advice). Any other valid sqlglot dialect is accepted for
|
|
241
|
+
`complexity`/`lint` but has no perf adapter, so `perf` exits 2 for it.
|
|
242
|
+
|
|
243
|
+
**Exit code:** `perf` exits **1 only when a finding is `ERROR` severity** — which in
|
|
244
|
+
practice means the SQL was unparseable (`SQ000`). Anti-pattern findings are `warning`
|
|
245
|
+
severity and exit **0**, so `perf` surfaces advice without blocking a build. Bad
|
|
246
|
+
input (missing file, unreadable `--explain`) exits 2.
|
|
247
|
+
|
|
248
|
+
**Captured EXPLAIN.** `--explain <file>` takes a plan you captured yourself:
|
|
249
|
+
|
|
250
|
+
- **Postgres:** `EXPLAIN (FORMAT JSON) <query>` output (JSON).
|
|
251
|
+
- **Redshift:** the plan text from `EXPLAIN <query>`.
|
|
252
|
+
|
|
253
|
+
```console
|
|
254
|
+
$ sqlquality perf messy.sql --explain plan.json
|
|
255
|
+
Perf — messy.sql (postgres, 4 findings)
|
|
256
|
+
┏━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
|
|
257
|
+
┃ code ┃ severity ┃ message ┃
|
|
258
|
+
┡━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
|
|
259
|
+
│ SQ001 │ warning │ SELECT * projects an unknown/wide column set; list columns … │
|
|
260
|
+
│ SQ002 │ warning │ Cartesian/cross join without an ON/USING condition. │
|
|
261
|
+
│ SQ003 │ warning │ Leading-wildcard LIKE ('%...') is non-sargable … │
|
|
262
|
+
│ PG001 │ warning │ Seq Scan on orders — consider an index if the filter is selective. │
|
|
263
|
+
└───────┴──────────┴─────────────────────────────────────────────────────────────────────┘
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
`--json` emits findings and any LLM suggestions. `--suggest` enriches findings with
|
|
267
|
+
advisory LLM suggestions — see [LLM suggestions](#llm-suggestions-optional-advisory).
|
|
268
|
+
|
|
269
|
+
### check (the CI gate)
|
|
270
|
+
|
|
271
|
+
Scores each changed model on both a candidate and a baseline dbt manifest, and gates
|
|
272
|
+
the change on the per-model **complexity delta**.
|
|
273
|
+
|
|
274
|
+
Requirements:
|
|
275
|
+
|
|
276
|
+
- **dbt >= 1.5 on `PATH`** (override the executable with `--dbt`). `check` shells out
|
|
277
|
+
to `dbt ls --select state:modified` to discover changed models.
|
|
278
|
+
- A **compiled candidate manifest** at `<project-dir>/target/manifest.json` — run
|
|
279
|
+
`dbt compile` first (the gate scores compiled SQL; uncompiled models are skipped).
|
|
280
|
+
- A **baseline artifacts directory** (`--state`) containing the prior
|
|
281
|
+
`manifest.json` to diff against.
|
|
282
|
+
|
|
283
|
+
The dialect is auto-resolved from the manifest's `adapter_type` (falling back to
|
|
284
|
+
`postgres`), and printed to stderr; pass `--dialect` to override. `--state` and
|
|
285
|
+
`--project-dir` are resolved to absolute paths, so `check` works from a monorepo root.
|
|
286
|
+
|
|
287
|
+
```console
|
|
288
|
+
$ sqlquality check --project-dir . --state prod-artifacts/
|
|
289
|
+
dialect: postgres (from manifest adapter_type)
|
|
290
|
+
sqlquality: ❌ FAIL (changed 1, neighbors 2)
|
|
291
|
+
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┳━━━━┓
|
|
292
|
+
┃ model ┃ baseline ┃ candidate ┃ delta ┃ ┃
|
|
293
|
+
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━╇━━━━┩
|
|
294
|
+
│ model.demo.customer_orders │ 11.2 │ 17.6 │ +6.4 │ ⚠️ │
|
|
295
|
+
└────────────────────────────┴──────────┴───────────┴───────┴────┘
|
|
296
|
+
```
|
|
297
|
+
|
|
298
|
+
Whether a regression **fails** the build depends on `gate.mode` (see
|
|
299
|
+
[Configuration](#configuration-sqlqualityyml)). In the default `warn` mode the same
|
|
300
|
+
change reports the regression but exits 0:
|
|
301
|
+
|
|
302
|
+
```console
|
|
303
|
+
$ sqlquality check --project-dir . --state prod-artifacts/
|
|
304
|
+
sqlquality: ⚠️ WARN (1 regression, gate mode: warn) (changed 1, neighbors 2)
|
|
305
|
+
...
|
|
306
|
+
```
|
|
307
|
+
|
|
308
|
+
`--json` emits the full gate report (verdict, per-model deltas, neighbors, skipped
|
|
309
|
+
models):
|
|
310
|
+
|
|
311
|
+
```console
|
|
312
|
+
$ sqlquality check --project-dir . --state prod-artifacts/ --json
|
|
313
|
+
{
|
|
314
|
+
"mode": "fail",
|
|
315
|
+
"models": [
|
|
316
|
+
{
|
|
317
|
+
"baseline": 11.2,
|
|
318
|
+
"candidate": 17.6,
|
|
319
|
+
"delta": 6.4,
|
|
320
|
+
"is_new": false,
|
|
321
|
+
"unique_id": "model.demo.customer_orders"
|
|
322
|
+
}
|
|
323
|
+
],
|
|
324
|
+
"neighbors": [
|
|
325
|
+
"model.demo.orders",
|
|
326
|
+
"model.demo.stg_orders"
|
|
327
|
+
],
|
|
328
|
+
"passed": false,
|
|
329
|
+
"regressions": [
|
|
330
|
+
"model.demo.customer_orders"
|
|
331
|
+
],
|
|
332
|
+
"skipped": [],
|
|
333
|
+
"warned": false
|
|
334
|
+
}
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
`--markdown <path>` writes a report suitable for a PR comment, and `--html <path>`
|
|
338
|
+
writes a self-contained HTML report. The markdown looks like:
|
|
339
|
+
|
|
340
|
+
```markdown
|
|
341
|
+
# sqlquality: ❌ FAIL
|
|
342
|
+
|
|
343
|
+
| model | baseline | candidate | delta | |
|
|
344
|
+
|---|---:|---:|---:|:--:|
|
|
345
|
+
| model.demo.customer_orders | 11.2 | 17.6 | +6.4 | ⚠️ |
|
|
346
|
+
```
|
|
347
|
+
|
|
348
|
+
## Configuration (`sqlquality.yml`)
|
|
349
|
+
|
|
350
|
+
`check` reads `<project-dir>/sqlquality.yml` by default, or the path given to
|
|
351
|
+
`--config`. All keys are optional; absent files use the defaults below.
|
|
352
|
+
|
|
353
|
+
| Key | Type | Default | Meaning |
|
|
354
|
+
|---|---|---|---|
|
|
355
|
+
| `gate.mode` | `warn` \| `fail` | `warn` | `warn` reports regressions but exits 0; `fail` exits 1 on any regression. **The default `warn` does not fail CI** — set `fail` to actually gate. An invalid value is rejected with exit 2. |
|
|
356
|
+
| `gate.max_complexity_increase` | float | `10.0` | A model is a regression when its delta exceeds this threshold. New models (no baseline) are never counted as regressions. |
|
|
357
|
+
| `waivers` | list of strings | `[]` | Model `unique_id`s exempt from the gate. |
|
|
358
|
+
|
|
359
|
+
A complete example:
|
|
360
|
+
|
|
361
|
+
```yaml
|
|
362
|
+
gate:
|
|
363
|
+
mode: fail
|
|
364
|
+
max_complexity_increase: 10.0
|
|
365
|
+
waivers:
|
|
366
|
+
- model.my_project.legacy_wide_fact
|
|
367
|
+
- model.my_project.known_gnarly_rollup
|
|
368
|
+
```
|
|
369
|
+
|
|
370
|
+
## Exit codes
|
|
371
|
+
|
|
372
|
+
Every command follows the same contract:
|
|
373
|
+
|
|
374
|
+
| Code | Meaning |
|
|
375
|
+
|---|---|
|
|
376
|
+
| `0` | Pass / no findings. |
|
|
377
|
+
| `1` | Findings present, or the gate failed. |
|
|
378
|
+
| `2` | Usage, config, or input error (bad flag, unknown dialect, unparseable SQL, unreadable file, malformed `sqlquality.yml`, dbt invocation failure). |
|
|
379
|
+
|
|
380
|
+
Per-command nuances of code `1`:
|
|
381
|
+
|
|
382
|
+
- **`complexity`** never gates — it always exits 0 unless the input errors (2).
|
|
383
|
+
- **`lint`** exits 1 on any `WARNING`/`ERROR` finding; `info`-level (unresolved-Jinja)
|
|
384
|
+
findings never gate; `--warn-only` forces 0.
|
|
385
|
+
- **`perf`** exits 1 only on an `ERROR`-severity finding (unparseable SQL). Anti-pattern
|
|
386
|
+
warnings exit 0.
|
|
387
|
+
- **`check`** exits 1 only when `gate.mode: fail` and a regression is present; `warn`
|
|
388
|
+
mode exits 0 even with regressions.
|
|
389
|
+
|
|
390
|
+
## CI recipe (a gate that actually gates)
|
|
391
|
+
|
|
392
|
+
To make CI fail on a complexity regression you must (1) set `gate.mode: fail` in
|
|
393
|
+
`sqlquality.yml`, (2) `dbt compile` so the candidate manifest exists, and (3) provide
|
|
394
|
+
a baseline (`--state`) produced from your production `dbt compile` artifacts.
|
|
395
|
+
|
|
396
|
+
```yaml
|
|
397
|
+
# sqlquality.yml (committed to the repo)
|
|
398
|
+
gate:
|
|
399
|
+
mode: fail
|
|
400
|
+
max_complexity_increase: 10.0
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
```yaml
|
|
404
|
+
# .github/workflows/sqlquality.yml
|
|
405
|
+
name: sqlquality
|
|
406
|
+
on: pull_request
|
|
407
|
+
|
|
408
|
+
jobs:
|
|
409
|
+
gate:
|
|
410
|
+
runs-on: ubuntu-latest
|
|
411
|
+
steps:
|
|
412
|
+
- uses: actions/checkout@v4
|
|
413
|
+
- uses: astral-sh/setup-uv@v5
|
|
414
|
+
|
|
415
|
+
# Fetch the baseline artifacts your production run published.
|
|
416
|
+
# These must come from `dbt compile` (a compiled manifest.json), not a bare parse.
|
|
417
|
+
- name: Download baseline artifacts
|
|
418
|
+
run: ./scripts/download-prod-artifacts.sh prod-artifacts/
|
|
419
|
+
|
|
420
|
+
# Produce the candidate manifest for the PR.
|
|
421
|
+
- name: dbt compile
|
|
422
|
+
run: uv run dbt compile
|
|
423
|
+
|
|
424
|
+
- name: sqlquality gate
|
|
425
|
+
run: >
|
|
426
|
+
uv run sqlquality check
|
|
427
|
+
--project-dir .
|
|
428
|
+
--state prod-artifacts/
|
|
429
|
+
--markdown report.md
|
|
430
|
+
|
|
431
|
+
- name: Comment report on the PR
|
|
432
|
+
uses: actions/github-script@v7
|
|
433
|
+
if: always() # post the report even when the gate fails
|
|
434
|
+
with:
|
|
435
|
+
script: |
|
|
436
|
+
const body = require('fs').readFileSync('report.md', 'utf8')
|
|
437
|
+
github.rest.issues.createComment({
|
|
438
|
+
...context.repo,
|
|
439
|
+
issue_number: context.issue.number,
|
|
440
|
+
body,
|
|
441
|
+
})
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
Baseline hygiene: the baseline `manifest.json` must be a **compiled** artifact
|
|
445
|
+
(`dbt compile` output). A parse-only manifest lacks `compiled_code`, so those models
|
|
446
|
+
are skipped rather than scored.
|
|
447
|
+
|
|
448
|
+
## Pre-commit hook
|
|
449
|
+
|
|
450
|
+
`sqlquality` ships a [pre-commit](https://pre-commit.com) hook that lints staged SQL:
|
|
451
|
+
|
|
452
|
+
```yaml
|
|
453
|
+
# .pre-commit-config.yaml
|
|
454
|
+
repos:
|
|
455
|
+
- repo: https://github.com/hanslemm/sqlquality
|
|
456
|
+
rev: v0.2.0
|
|
457
|
+
hooks:
|
|
458
|
+
- id: sqlquality-lint
|
|
459
|
+
```
|
|
460
|
+
|
|
461
|
+
The hook runs `sqlquality lint` on staged `.sql` files and excludes `target/`. It
|
|
462
|
+
lints **raw model files** (not compiled SQL), so unresolved-Jinja findings are demoted
|
|
463
|
+
to `info` and don't block the commit — only real `WARNING`/`ERROR` findings do.
|
|
464
|
+
|
|
465
|
+
To make the hook non-blocking (report only), pass `--warn-only`:
|
|
466
|
+
|
|
467
|
+
```yaml
|
|
468
|
+
- id: sqlquality-lint
|
|
469
|
+
args: [--warn-only]
|
|
470
|
+
```
|
|
471
|
+
|
|
472
|
+
## LLM suggestions (optional, advisory)
|
|
473
|
+
|
|
474
|
+
`perf --suggest` can attach a short, concrete rewrite suggestion to each finding using
|
|
475
|
+
an LLM. It is **off by default** and **advisory only** — suggestions never change
|
|
476
|
+
findings, severities, exit codes, or the gate.
|
|
477
|
+
|
|
478
|
+
Setup:
|
|
479
|
+
|
|
480
|
+
1. Install the extra: `pip install "sqlquality[llm]"`.
|
|
481
|
+
2. Set `SQLQUALITY_LLM=anthropic` (also accepts `1` or `true`).
|
|
482
|
+
3. Provide `ANTHROPIC_API_KEY` (read by the Anthropic SDK).
|
|
483
|
+
4. Optionally set `SQLQUALITY_LLM_MODEL` to override the model (the built-in default is
|
|
484
|
+
`claude-opus-4-8`).
|
|
485
|
+
|
|
486
|
+
```bash
|
|
487
|
+
export SQLQUALITY_LLM=anthropic
|
|
488
|
+
export ANTHROPIC_API_KEY=sk-ant-...
|
|
489
|
+
sqlquality perf model.sql --suggest
|
|
490
|
+
```
|
|
491
|
+
|
|
492
|
+
If `--suggest` is passed without `SQLQUALITY_LLM` set, `perf` prints a note to stderr
|
|
493
|
+
and continues without suggestions. If the extra or credentials are missing, it
|
|
494
|
+
degrades gracefully (findings still print, exit code unchanged):
|
|
495
|
+
|
|
496
|
+
```
|
|
497
|
+
LLM suggestions unavailable: The 'anthropic' package is required for AnthropicProvider. Install it with: pip install 'sqlquality[llm]'
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
> **⚠️ Data egress warning.** `perf --suggest` sends the analyzed SQL (up to 20,000
|
|
501
|
+
> characters per finding) to the Anthropic API. Do **not** enable it on proprietary or
|
|
502
|
+
> sensitive SQL without clearance. API cost scales with the number of findings (one
|
|
503
|
+
> call per finding).
|
|
504
|
+
|
|
505
|
+
## Limitations
|
|
506
|
+
|
|
507
|
+
- **Complexity is structural.** The composite is an open-ended, weighted score of
|
|
508
|
+
AST features (joins, CTEs, subqueries, windows, select depth, …); it is not capped,
|
|
509
|
+
so a large model can exceed 100. As a rough guide, ~100 is very complex. It measures
|
|
510
|
+
shape, not runtime cost or correctness.
|
|
511
|
+
- **Performance is static.** Anti-patterns and captured-`EXPLAIN` ingestion only —
|
|
512
|
+
`sqlquality` never runs your queries. Perf adapters exist for **postgres** and
|
|
513
|
+
**redshift** only today.
|
|
514
|
+
- **Jinja analysis is approximate.** Raw dbt models are analyzed by stripping Jinja to
|
|
515
|
+
placeholders (with a stderr notice). Prefer compiled SQL from `target/compiled/` for
|
|
516
|
+
accurate results.
|
|
517
|
+
- **Neighbors are reported, not scored.** A changed model's direct upstream/downstream
|
|
518
|
+
models are surfaced for context; the gate only evaluates the changed models
|
|
519
|
+
themselves.
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
sqlquality/__init__.py,sha256=DI55pgLNfxIJxAhVKp0D0gQ0nBsxsT1nhX5vz_5uHcQ,90
|
|
2
|
+
sqlquality/antipatterns.py,sha256=JnAH28WOTzoOuKcdluEllih_LN-o3Y6Kx6vSu7m25dA,6307
|
|
3
|
+
sqlquality/changeset.py,sha256=9Efd2b844Ij5xSgXh53BfniLR8ErHAkx792V67P8v3I,3244
|
|
4
|
+
sqlquality/cli.py,sha256=9l9bwO6Zon2HppNKK2qYBKp5DSvuwyZwTI9hUf6lNQA,20295
|
|
5
|
+
sqlquality/complexity.py,sha256=55X0TdqfD7xtEhe3qehnEaedNDSLLdJ6WVenxhQyNm4,1391
|
|
6
|
+
sqlquality/config.py,sha256=7k2wf3UsnWjCbJ9gF-XD5PUUB0g3XFMO0TIYXvbpa-g,3038
|
|
7
|
+
sqlquality/dbtproject.py,sha256=jbNq3RgylpyPOEqGb2sg3WV7crWP2HASd1R9Fc80ws8,4488
|
|
8
|
+
sqlquality/delta.py,sha256=Rz1dCBXZ2CM8EAQUgnW7NqNcBauVrdhgcURky6F2tkU,2589
|
|
9
|
+
sqlquality/dialects.py,sha256=iuhn1z9I7pPXa_jpskbj8rtvQ5LU-rAUISAiNz3x0w0,902
|
|
10
|
+
sqlquality/gate.py,sha256=J280RC7ISLL9wgd45YBi2UTP_1NunoEq8jNW05Cpbi0,1324
|
|
11
|
+
sqlquality/keys.py,sha256=VrafO3xGbBusALzAKU3DcYhD0J5jldk-Jo97yH0fG8w,3954
|
|
12
|
+
sqlquality/linter.py,sha256=-h9u1krhxF1HSjAjp4xtQxMkXfU69I2s6z3F8TurmcA,2585
|
|
13
|
+
sqlquality/llm.py,sha256=E3z7Rt9-FwtvDP53ykjH6--IAyscz1mWxejWiWn92zE,3561
|
|
14
|
+
sqlquality/models.py,sha256=Rtt9i2TgdRH1pQudAID1vI5HUMAkeKMTF1sRL9tch4I,1127
|
|
15
|
+
sqlquality/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
16
|
+
sqlquality/report.py,sha256=35u86jA67MqhMp3T56LecjsdYIxsS5nd5h-zDkszONk,4503
|
|
17
|
+
sqlquality/sqlast.py,sha256=wLdj1ULAGi6u4hgdKzAS_mfuaoiDgxVj19y_eeuSOao,5218
|
|
18
|
+
sqlquality/adapters/__init__.py,sha256=PzqxArCSpF3vwamc7A8VuP_zbAqV72OG9cXGQXSH2Ik,592
|
|
19
|
+
sqlquality/adapters/base.py,sha256=af3uunKbmo0nKZWEhz8KC0c08REDdqIOwHVvfpxLHlU,599
|
|
20
|
+
sqlquality/adapters/postgres.py,sha256=xnA5OobWzPJIvjCGBHSp5sSzBuH9CajqXkn8CjPN2Eo,2006
|
|
21
|
+
sqlquality/adapters/redshift.py,sha256=CrbEeKumyVstKztCzb0vh_89ratPRmmJOELMSzKNXg4,1600
|
|
22
|
+
sqlquality-0.2.0.dist-info/METADATA,sha256=EeXSXUGuQx2PgTsumwg590GPDBvBEtqklKM_yZfg8mQ,21644
|
|
23
|
+
sqlquality-0.2.0.dist-info/WHEEL,sha256=lCkmxWfQsSc9CfIClYeavTdQeEX2toPqufh9gI35EQA,87
|
|
24
|
+
sqlquality-0.2.0.dist-info/entry_points.txt,sha256=0RvZcG-MkuYEMW5B0TXwmleHJaRloJzn54AjUfpIjf0,51
|
|
25
|
+
sqlquality-0.2.0.dist-info/licenses/LICENSE,sha256=s22d3_96yxaU8cALgDugiEOVKu5lBIjTkNY79WpaSnE,1077
|
|
26
|
+
sqlquality-0.2.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Hans C. Ravache Lemm
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|