datawright-cli 0.1.0__tar.gz → 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/PKG-INFO +5 -3
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/README.md +4 -2
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/pyproject.toml +1 -1
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/scripts/datawright.py +25 -2
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/.gitignore +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/LICENSE +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/benchmarks/README.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/benchmarks/meaning/README.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/docs/demo.svg +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/docs/logo.svg +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/examples/shop/README.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/build-pipeline/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/change-dataset/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/bi_export.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/config.yml +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/dagster.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/dbt_yaml.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/kafka_connect.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/lineage_yaml.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/lookml.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/metabase.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/mybatis.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/odcs.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/superset.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/references/connectors.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/references/warehouse-export.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-tests/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/debug-data-issue/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/on-call/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/on-call/assets/datawright_checks_dag.py +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/optimize-pipeline/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/pipeline-review/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/assets/AGENTS.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/assets/ci/github-actions.yml +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/assets/ci/gitlab-ci.yml +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/references/connections.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/sql-review/SKILL.md +0 -0
- {datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/work-ticket/SKILL.md +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: datawright-cli
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.1
|
|
4
4
|
Summary: Lineage, impact reports and join paths compiled from your data repo's code, for AI agents and reviewers
|
|
5
5
|
Project-URL: Homepage, https://github.com/clementlemon02/datawright
|
|
6
6
|
Project-URL: Issues, https://github.com/clementlemon02/datawright/issues
|
|
@@ -31,6 +31,8 @@ Description-Content-Type: text/markdown
|
|
|
31
31
|
|
|
32
32
|
[Quick start](#quick-start) · [What you get](#what-you-get) · [Benchmarks](#how-well-does-it-work) · [FAQ](#faq) · [Reference](docs/reference.md) · [Roadmap](ROADMAP.md)
|
|
33
33
|
|
|
34
|
+
<img src="docs/demo.gif" alt="dw review on a branch that renames a status value: changed table, downstream marts, readers" width="860">
|
|
35
|
+
|
|
34
36
|
<img src="docs/demo.svg" alt="datawright impact, join and card output on the shop example" width="860">
|
|
35
37
|
|
|
36
38
|
</div>
|
|
@@ -64,12 +66,12 @@ You need [uv](https://docs.astral.sh/uv/) and git.
|
|
|
64
66
|
**1. Try the tool.** One minute, no agent needed. From your data repo's root:
|
|
65
67
|
|
|
66
68
|
```bash
|
|
67
|
-
|
|
69
|
+
uvx datawright-cli init
|
|
68
70
|
.data-context/dw card <a table you know>
|
|
69
71
|
.data-context/dw impact <that table>
|
|
70
72
|
```
|
|
71
73
|
|
|
72
|
-
On a feature branch, `.data-context/dw review --base main` prints the merge-request report. `.data-context/dw coverage` lists what it doesn't read in your repo yet.
|
|
74
|
+
Want `dw` on your PATH everywhere? `uv tool install datawright-cli`. On a feature branch, `.data-context/dw review --base main` prints the merge-request report. `.data-context/dw coverage` lists what it doesn't read in your repo yet.
|
|
73
75
|
|
|
74
76
|
**2. Give your agent the skills.** Claude Code:
|
|
75
77
|
```
|
|
@@ -13,6 +13,8 @@
|
|
|
13
13
|
|
|
14
14
|
[Quick start](#quick-start) · [What you get](#what-you-get) · [Benchmarks](#how-well-does-it-work) · [FAQ](#faq) · [Reference](docs/reference.md) · [Roadmap](ROADMAP.md)
|
|
15
15
|
|
|
16
|
+
<img src="docs/demo.gif" alt="dw review on a branch that renames a status value: changed table, downstream marts, readers" width="860">
|
|
17
|
+
|
|
16
18
|
<img src="docs/demo.svg" alt="datawright impact, join and card output on the shop example" width="860">
|
|
17
19
|
|
|
18
20
|
</div>
|
|
@@ -46,12 +48,12 @@ You need [uv](https://docs.astral.sh/uv/) and git.
|
|
|
46
48
|
**1. Try the tool.** One minute, no agent needed. From your data repo's root:
|
|
47
49
|
|
|
48
50
|
```bash
|
|
49
|
-
|
|
51
|
+
uvx datawright-cli init
|
|
50
52
|
.data-context/dw card <a table you know>
|
|
51
53
|
.data-context/dw impact <that table>
|
|
52
54
|
```
|
|
53
55
|
|
|
54
|
-
On a feature branch, `.data-context/dw review --base main` prints the merge-request report. `.data-context/dw coverage` lists what it doesn't read in your repo yet.
|
|
56
|
+
Want `dw` on your PATH everywhere? `uv tool install datawright-cli`. On a feature branch, `.data-context/dw review --base main` prints the merge-request report. `.data-context/dw coverage` lists what it doesn't read in your repo yet.
|
|
55
57
|
|
|
56
58
|
**2. Give your agent the skills.** Claude Code:
|
|
57
59
|
```
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[project]
|
|
2
2
|
name = "datawright-cli"
|
|
3
|
-
version = "0.1.
|
|
3
|
+
version = "0.1.1" # keep equal to VERSION in skills/data-context/scripts/datawright.py
|
|
4
4
|
description = "Lineage, impact reports and join paths compiled from your data repo's code, for AI agents and reviewers"
|
|
5
5
|
readme = "README.md"
|
|
6
6
|
requires-python = ">=3.10"
|
|
@@ -44,7 +44,7 @@ from fnmatch import fnmatch
|
|
|
44
44
|
from pathlib import Path
|
|
45
45
|
|
|
46
46
|
logging.getLogger("sqlglot").setLevel(logging.CRITICAL) # its warnings quote SQL, which can contain secrets
|
|
47
|
-
VERSION = "0.1.
|
|
47
|
+
VERSION = "0.1.1"
|
|
48
48
|
SKIP = {".git", ".venv", "venv", "node_modules", "target", "dbt_packages", "dbt_modules", "__pycache__", ".data-context"}
|
|
49
49
|
SQL_START = re.compile(r"\s*(?:--[^\n]*\n\s*|/\*.*?\*/\s*|jinja_expr\s*)*(with|select|insert|merge|create|update)\b", re.I | re.S) # dbt models open with {{ config(...) }}
|
|
50
50
|
KINDS = ("correction", "gotcha", "caveat", "decision") # also the order notes appear in cards
|
|
@@ -3410,6 +3410,29 @@ def data_diff(con, tables, where=""):
|
|
|
3410
3410
|
return out + [""], blocking
|
|
3411
3411
|
|
|
3412
3412
|
|
|
3413
|
+
def terminal_text(out):
|
|
3414
|
+
"""Markdown for merge request comments; on a terminal, plain text: tables become one block per row."""
|
|
3415
|
+
if not sys.stdout.isatty() or os.environ.get("DW_MARKDOWN"):
|
|
3416
|
+
return "\n".join(out)
|
|
3417
|
+
res, head, i = [], None, 0
|
|
3418
|
+
while i < len(out):
|
|
3419
|
+
l = out[i]
|
|
3420
|
+
if l.startswith("|") and i + 1 < len(out) and set(out[i + 1]) <= set("|-: "):
|
|
3421
|
+
head, i = [c.strip() for c in l.strip("|").split("|")], i + 2
|
|
3422
|
+
while i < len(out) and out[i].startswith("|"):
|
|
3423
|
+
cells = [c.strip() for c in out[i].strip("|").split("|")]
|
|
3424
|
+
res.append(cells[0])
|
|
3425
|
+
res += [f" {h:<{max(map(len, head[1:]))}} {c}" for h, c in zip(head[1:], cells[1:]) if c != "-"]
|
|
3426
|
+
res.append("")
|
|
3427
|
+
i += 1
|
|
3428
|
+
continue
|
|
3429
|
+
res.append(l)
|
|
3430
|
+
i += 1
|
|
3431
|
+
t = re.sub(r"\n{3,}", "\n\n", "\n".join(res))
|
|
3432
|
+
t = re.sub(r"^##(.*)$", lambda m: "\033[1m" + m[1] + "\033[0m", t, flags=re.M)
|
|
3433
|
+
return re.sub(r"</?sub>|`|\*\*", "", t)
|
|
3434
|
+
|
|
3435
|
+
|
|
3413
3436
|
def review(con, base=None, strict=False, diff=False, where=""):
|
|
3414
3437
|
"""Markdown impact report for this branch's changes against base: what the changed code produces and reads, what's
|
|
3415
3438
|
downstream, recorded gotchas, SQL that stopped parsing, and the ticket keys; with diff, the data before and after.
|
|
@@ -3501,7 +3524,7 @@ def review(con, base=None, strict=False, diff=False, where=""):
|
|
|
3501
3524
|
out.append(f"Ticket: {', '.join(sorted(keys_branch | keys_commits))} (check it's the intended ticket).")
|
|
3502
3525
|
out += ["", f"<sub>{len(changed)} changed files. Generated by datawright {VERSION} from the repo's code; "
|
|
3503
3526
|
+ ("the data diff ran read-only SELECTs.</sub>" if diff and produced else "nothing was queried.</sub>")]
|
|
3504
|
-
print(
|
|
3527
|
+
print(terminal_text(out))
|
|
3505
3528
|
return blocking if strict else 0
|
|
3506
3529
|
|
|
3507
3530
|
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/dagster.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/dbt_yaml.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/kafka_connect.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/lineage_yaml.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/lookml.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/metabase.py
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/mybatis.py
RENAMED
|
File without changes
|
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/assets/connectors/superset.py
RENAMED
|
File without changes
|
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/data-context/references/warehouse-export.md
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/on-call/assets/datawright_checks_dag.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/assets/ci/github-actions.yml
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/assets/ci/gitlab-ci.yml
RENAMED
|
File without changes
|
{datawright_cli-0.1.0 → datawright_cli-0.1.1}/skills/setup-data-project/references/connections.md
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|