dataform-sqlx-lint 0.1.0__tar.gz

dataform_sqlx_lint-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Acuantia
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dataform_sqlx_lint-0.1.0/PKG-INFO

@@ -0,0 +1,124 @@
+Metadata-Version: 2.4
+Name: dataform-sqlx-lint
+Version: 0.1.0
+Summary: Convention linter for Dataform .sqlx files — checks the config-block and project conventions that SQL linters cannot see
+Author-email: Ivan Histand <ihistand@rotoplas.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/acuantia/dataform-sqlx-lint
+Keywords: dataform,sqlx,lint,bigquery,pre-commit
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# dataform-sqlx-lint
+
+A convention linter for [Dataform](https://cloud.google.com/dataform) `.sqlx`
+files. SQL linters (sqlfluff) check the SQL body; `dataform compile` checks
+syntax. Neither sees the **config-block and project conventions** that keep a
+Dataform repo healthy — this tool does.
+
+Zero dependencies (Python ≥ 3.11 standard library only). Designed for
+[pre-commit](https://pre-commit.com).
+
+```bash
+pip install dataform-sqlx-lint
+```
+
+## Rules
+
+| Code | Default | Checks |
+|------|---------|--------|
+| E001 | on | `config {}` block present and balanced |
+| E002 | on | non-empty `columns: {}` documentation on tables/views/incrementals/declarations |
+| E003 | on | `schema:` must not hardcode an environment suffix (`_prod`/`_dev` by default) — `--schema-suffix` appends it, so a literal doubles up (`looker_prod_prod`) |
+| E004 | on | `name:` matching the filename is redundant (declarations exempt) |
+| E005 | opt-in | operations/assertions must not set `schema:` (`hasOutput: true` operations exempt — schema+name define `${self()}`) |
+| E006 | on | hardcoded `` `project.dataset.table` `` paths instead of `${ref()}` — these silently break Dataform's dependency graph |
+| E007 | on* | configurable per-directory naming/type policies (*no-op until policies are configured) |
+| W008 | opt-in | `post_operations {}` placed before the main SELECT (style preference; Dataform accepts either) |
+| E010 | on | every determinable output column appears in `columns: {}` — parses the main SELECT conservatively (unparseable expressions are skipped, never false-flagged) and follows `select *` through a single plain `${ref()}` into the upstream file |
+
+Why E010 matters: `columns: {}` is what Dataform writes to BigQuery column
+descriptions — the metadata data catalogs, BI tools, and AI/conversational
+analytics agents read. Partial blocks leave silent gaps.
+
+## Usage
+
+```bash
+dataform-sqlx-lint definitions/output/my_table.sqlx [...]
+# exit 0 = clean or warnings only; 1 = errors
+```
+
+Run from the repo root so `--definitions-root` (default `./definitions`) can
+index `${ref()}` targets for E010's star-resolution.
+
+### pre-commit
+
+```yaml
+repos:
+  - repo: https://github.com/acuantia/dataform-sqlx-lint
+    rev: v0.1.0
+    hooks:
+      - id: dataform-sqlx-lint
+```
+
+### Configuration
+
+`.sqlx-lint.toml` in the repo root, or a `[tool.sqlx-lint]` table in
+`pyproject.toml` (the standalone file wins). All keys optional:
+
+```toml
+schema_suffixes = ["_prod", "_dev"]        # E003 suffix list ([] disables)
+documented_types = ["table", "view", "incremental", "declaration"]  # E002
+coverage_paths = ["definitions/output/"]   # E010 scope; empty = everywhere
+enable = ["E005", "W008"]                  # switch on opt-in rules
+disable = ["E004"]                         # switch off default rules
+
+[[dir_policies]]                           # E007 (repeatable)
+path_contains = "definitions/output/looker/"
+require_prefix = "looker_"
+require_types = ["table", "incremental"]
+severity = "error"                         # or "warning"
+```
+
+See `examples/acuantia.sqlx-lint.toml` for a complete real-world config.
+
+### Suppressing findings
+
+```sql
+from `proj.raw_api.events` -- sqlx-lint: disable=E006 (declaration repoints at cutover)
+```
+
+or file-wide, anywhere in the file:
+
+```sql
+-- sqlx-lint: disable-file=E006
+```
+
+Suppress with a reason, sparingly — the convention is usually the fix.
+
+## Design notes
+
+- **Conservative by construction**: the SQL projection parser only claims
+  column names it can determine (aliases, simple identifiers, resolvable
+  `select *`); anything ambiguous is skipped, so E010 never false-flags.
+- **Declarations are exempt** from E003/E004 deliberately: raw source datasets
+  legitimately carry environment-suffixed names, and Dataform requires `name:`
+  on declarations.
+- Rule codes are stable; gaps in the numbering are historical.
+
+## Development
+
+```bash
+python3 -m venv .venv && .venv/bin/pip install -e . pytest
+.venv/bin/pytest        # 53 tests
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

dataform_sqlx_lint-0.1.0/README.md

@@ -0,0 +1,107 @@
+# dataform-sqlx-lint
+
+A convention linter for [Dataform](https://cloud.google.com/dataform) `.sqlx`
+files. SQL linters (sqlfluff) check the SQL body; `dataform compile` checks
+syntax. Neither sees the **config-block and project conventions** that keep a
+Dataform repo healthy — this tool does.
+
+Zero dependencies (Python ≥ 3.11 standard library only). Designed for
+[pre-commit](https://pre-commit.com).
+
+```bash
+pip install dataform-sqlx-lint
+```
+
+## Rules
+
+| Code | Default | Checks |
+|------|---------|--------|
+| E001 | on | `config {}` block present and balanced |
+| E002 | on | non-empty `columns: {}` documentation on tables/views/incrementals/declarations |
+| E003 | on | `schema:` must not hardcode an environment suffix (`_prod`/`_dev` by default) — `--schema-suffix` appends it, so a literal doubles up (`looker_prod_prod`) |
+| E004 | on | `name:` matching the filename is redundant (declarations exempt) |
+| E005 | opt-in | operations/assertions must not set `schema:` (`hasOutput: true` operations exempt — schema+name define `${self()}`) |
+| E006 | on | hardcoded `` `project.dataset.table` `` paths instead of `${ref()}` — these silently break Dataform's dependency graph |
+| E007 | on* | configurable per-directory naming/type policies (*no-op until policies are configured) |
+| W008 | opt-in | `post_operations {}` placed before the main SELECT (style preference; Dataform accepts either) |
+| E010 | on | every determinable output column appears in `columns: {}` — parses the main SELECT conservatively (unparseable expressions are skipped, never false-flagged) and follows `select *` through a single plain `${ref()}` into the upstream file |
+
+Why E010 matters: `columns: {}` is what Dataform writes to BigQuery column
+descriptions — the metadata data catalogs, BI tools, and AI/conversational
+analytics agents read. Partial blocks leave silent gaps.
+
+## Usage
+
+```bash
+dataform-sqlx-lint definitions/output/my_table.sqlx [...]
+# exit 0 = clean or warnings only; 1 = errors
+```
+
+Run from the repo root so `--definitions-root` (default `./definitions`) can
+index `${ref()}` targets for E010's star-resolution.
+
+### pre-commit
+
+```yaml
+repos:
+  - repo: https://github.com/acuantia/dataform-sqlx-lint
+    rev: v0.1.0
+    hooks:
+      - id: dataform-sqlx-lint
+```
+
+### Configuration
+
+`.sqlx-lint.toml` in the repo root, or a `[tool.sqlx-lint]` table in
+`pyproject.toml` (the standalone file wins). All keys optional:
+
+```toml
+schema_suffixes = ["_prod", "_dev"]        # E003 suffix list ([] disables)
+documented_types = ["table", "view", "incremental", "declaration"]  # E002
+coverage_paths = ["definitions/output/"]   # E010 scope; empty = everywhere
+enable = ["E005", "W008"]                  # switch on opt-in rules
+disable = ["E004"]                         # switch off default rules
+
+[[dir_policies]]                           # E007 (repeatable)
+path_contains = "definitions/output/looker/"
+require_prefix = "looker_"
+require_types = ["table", "incremental"]
+severity = "error"                         # or "warning"
+```
+
+See `examples/acuantia.sqlx-lint.toml` for a complete real-world config.
+
+### Suppressing findings
+
+```sql
+from `proj.raw_api.events` -- sqlx-lint: disable=E006 (declaration repoints at cutover)
+```
+
+or file-wide, anywhere in the file:
+
+```sql
+-- sqlx-lint: disable-file=E006
+```
+
+Suppress with a reason, sparingly — the convention is usually the fix.
+
+## Design notes
+
+- **Conservative by construction**: the SQL projection parser only claims
+  column names it can determine (aliases, simple identifiers, resolvable
+  `select *`); anything ambiguous is skipped, so E010 never false-flags.
+- **Declarations are exempt** from E003/E004 deliberately: raw source datasets
+  legitimately carry environment-suffixed names, and Dataform requires `name:`
+  on declarations.
+- Rule codes are stable; gaps in the numbering are historical.
+
+## Development
+
+```bash
+python3 -m venv .venv && .venv/bin/pip install -e . pytest
+.venv/bin/pytest        # 53 tests
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE).

dataform_sqlx_lint-0.1.0/pyproject.toml

@@ -0,0 +1,32 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dataform-sqlx-lint"
+version = "0.1.0"
+description = "Convention linter for Dataform .sqlx files — checks the config-block and project conventions that SQL linters cannot see"
+readme = "README.md"
+requires-python = ">=3.11"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Ivan Histand", email = "ihistand@rotoplas.com" }]
+keywords = ["dataform", "sqlx", "lint", "bigquery", "pre-commit"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Environment :: Console",
+  "Programming Language :: Python :: 3",
+  "Topic :: Software Development :: Quality Assurance",
+]
+
+[project.scripts]
+dataform-sqlx-lint = "dataform_sqlx_lint.cli:entrypoint"
+
+[project.urls]
+Homepage = "https://github.com/acuantia/dataform-sqlx-lint"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

dataform_sqlx_lint-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

dataform_sqlx_lint-0.1.0/src/dataform_sqlx_lint/__init__.py

@@ -0,0 +1,7 @@
+"""dataform-sqlx-lint: convention linter for Dataform .sqlx files."""
+
+from .config import Config, DirPolicy, load_config
+from .linter import Finding, lint_file, lint_text
+
+__all__ = ["Config", "DirPolicy", "Finding", "lint_file", "lint_text", "load_config"]
+__version__ = "0.1.0"

dataform_sqlx_lint-0.1.0/src/dataform_sqlx_lint/cli.py

@@ -0,0 +1,81 @@
+"""Command-line interface.
+
+Usage: dataform-sqlx-lint [--config PATH] [--definitions-root DIR] FILE [FILE ...]
+Exit codes: 0 = clean or warnings only, 1 = errors found, 2 = usage error.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+import tomllib
+
+from .config import Config, _from_dict, load_config
+from .linter import lint_file
+
+
+def _repo_resolver(root: Path):
+    """Resolve an action name to the text of <root>/**/<name>.sqlx.
+    Used to follow `select *` through ${ref()} for E010."""
+    if not root.is_dir():
+        return lambda name: None
+    index = {p.stem: p for p in root.rglob("*.sqlx")}
+
+    def resolve(name):
+        p = index.get(name)
+        try:
+            return p.read_text(encoding="utf-8") if p else None
+        except OSError:
+            return None
+
+    return resolve
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="dataform-sqlx-lint",
+        description="Convention linter for Dataform .sqlx files",
+    )
+    parser.add_argument("files", nargs="+", help=".sqlx files to lint")
+    parser.add_argument(
+        "--config",
+        help="path to a TOML config file (default: .sqlx-lint.toml or "
+        "[tool.sqlx-lint] in ./pyproject.toml)",
+    )
+    parser.add_argument(
+        "--definitions-root",
+        default="definitions",
+        help="directory indexed to resolve ${ref()} targets for the E010 "
+        "coverage rule (default: ./definitions)",
+    )
+    args = parser.parse_args(argv)
+
+    if args.config:
+        cfg: Config = _from_dict(tomllib.loads(Path(args.config).read_text()))
+    else:
+        cfg = load_config(".")
+    resolver = _repo_resolver(Path(args.definitions_root))
+
+    errors = warnings = 0
+    for path in args.files:
+        try:
+            findings = lint_file(path, config=cfg, resolver=resolver)
+        except OSError as exc:
+            print(f"{path}: cannot read: {exc}")
+            errors += 1
+            continue
+        for f in sorted(findings, key=lambda f: f.line):
+            print(f"{path}:{f.line}: {f.code} [{f.severity}] {f.message}")
+            if f.severity == "error":
+                errors += 1
+            else:
+                warnings += 1
+    if errors or warnings:
+        print(f"sqlx-lint: {errors} error(s), {warnings} warning(s)")
+    return 1 if errors else 0
+
+
+def entrypoint() -> None:
+    sys.exit(main())

dataform_sqlx_lint-0.1.0/src/dataform_sqlx_lint/config.py

@@ -0,0 +1,110 @@
+"""Configuration model and TOML loading.
+
+Precedence: .sqlx-lint.toml in the working directory, else the
+[tool.sqlx-lint] table of pyproject.toml, else built-in defaults.
+"""
+
+from __future__ import annotations
+
+import tomllib
+from dataclasses import dataclass, field
+from pathlib import Path
+
+#: Rules that run unless disabled.
+DEFAULT_ENABLED = {"E001", "E002", "E003", "E004", "E006", "E007", "E010"}
+#: Opt-in rules (house-style checks): enable via `enable = [...]`.
+OPT_IN = {"E005", "W008"}
+
+
+@dataclass(frozen=True)
+class DirPolicy:
+    """Naming/type policy applied to files whose path contains a substring."""
+
+    path_contains: str
+    require_prefix: str | None = None
+    require_types: tuple[str, ...] | None = None
+    severity: str = "error"
+
+
+@dataclass(frozen=True)
+class Config:
+    schema_suffixes: list[str] = field(default_factory=lambda: ["_prod", "_dev"])
+    documented_types: set[str] = field(
+        default_factory=lambda: {"table", "view", "incremental", "declaration"}
+    )
+    #: E010 applies only to files whose path contains one of these; empty = all.
+    coverage_paths: list[str] = field(default_factory=list)
+    dir_policies: list[DirPolicy] = field(default_factory=list)
+    enabled_extra: set[str] = field(default_factory=set)
+    disabled: set[str] = field(default_factory=set)
+
+    def rule_on(self, code: str) -> bool:
+        if code in self.disabled:
+            return False
+        return code in DEFAULT_ENABLED or code in self.enabled_extra
+
+    def __eq__(self, other):
+        if not isinstance(other, Config):
+            return NotImplemented
+        return (
+            self.schema_suffixes == other.schema_suffixes
+            and self.documented_types == other.documented_types
+            and self.coverage_paths == other.coverage_paths
+            and self.dir_policies == other.dir_policies
+            and self.enabled_extra == other.enabled_extra
+            and self.disabled == other.disabled
+        )
+
+
+_KNOWN_KEYS = {
+    "schema_suffixes",
+    "documented_types",
+    "coverage_paths",
+    "dir_policies",
+    "enable",
+    "disable",
+}
+
+
+def _from_dict(raw: dict) -> Config:
+    unknown = set(raw) - _KNOWN_KEYS
+    if unknown:
+        raise ValueError(
+            f"unknown sqlx-lint config key(s): {', '.join(sorted(unknown))}"
+        )
+    policies = [
+        DirPolicy(
+            path_contains=p["path_contains"],
+            require_prefix=p.get("require_prefix"),
+            require_types=tuple(p["require_types"]) if p.get("require_types") else None,
+            severity=p.get("severity", "error"),
+        )
+        for p in raw.get("dir_policies", [])
+    ]
+    kwargs = {}
+    if "schema_suffixes" in raw:
+        kwargs["schema_suffixes"] = list(raw["schema_suffixes"])
+    if "documented_types" in raw:
+        kwargs["documented_types"] = set(raw["documented_types"])
+    if "coverage_paths" in raw:
+        kwargs["coverage_paths"] = list(raw["coverage_paths"])
+    return Config(
+        dir_policies=policies,
+        enabled_extra=set(raw.get("enable", [])),
+        disabled=set(raw.get("disable", [])),
+        **kwargs,
+    )
+
+
+def load_config(root: str | Path = ".") -> Config:
+    root = Path(root)
+    standalone = root / ".sqlx-lint.toml"
+    if standalone.is_file():
+        return _from_dict(tomllib.loads(standalone.read_text(encoding="utf-8")))
+    pyproject = root / "pyproject.toml"
+    if pyproject.is_file():
+        data = tomllib.loads(pyproject.read_text(encoding="utf-8"))
+        tool = data.get("tool", {}).get("sqlx-lint")
+        if tool is not None:
+            return _from_dict(tool)
+    return Config()