trino-sql-validator 0.1.0__tar.gz → 0.2.0__tar.gz

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/AGENTS.md

@@ -10,10 +10,12 @@ Built with PyO3 + maturin. Distributed as wheels + sdist on PyPI.
 Layout:
 - `src/` — Rust crate (the compiled `_native` extension). `src/lib.rs` holds the
   `#[pymodule] fn _native`. `src/dialects/` holds the `TrinoDialect`/`SqlDialect`.
+  `src/functions.rs` is the **generated** Trino function catalog (`@generated`).
 - `python/trino_sql_validator/` — pure-Python public API (`__init__.py`), type
   stubs (`_native.pyi`), `py.typed`.
 - `tests/` — pytest suite for the public API (+ `.sql` fixtures).
-- `plan/` — planning docs (`plan.md`, `roadmap.md`).
+- `plan/` — planning docs (`plan.md`, `roadmap.md`, `functions-validation.md`).
+- `tools/extract_functions.py` — regenerates `src/functions.rs` from Trino docs.
 
 ## Source of truth
 
@@ -47,6 +49,10 @@ mypy python/trino_sql_validator
 # Produce distributable artifacts
 maturin build --release          # wheels
 maturin sdist                    # source distribution
+
+# Regenerate the Trino function catalog from upstream docs
+python tools/extract_functions.py            # fetch from GitHub
+python tools/extract_functions.py --docs-path /path/to/trino/docs/src/main/sphinx/functions  # local checkout
 ```
 
 ## Conventions / rules
@@ -86,4 +92,10 @@ maturin sdist                    # source distribution
 `sqlparser-rs` does syntax, not semantics. It accepts some SQL Trino rejects at
 semantic analysis and can reject exotic Trino-specific DDL. This is a documented,
 accepted limitation (see README + plan/roadmap.md). Do not "fix" by loosening the
-dialect to Generic by default for `dialect="trino"`.
+dialect to Generic by default for `dialect="trino"`.
+
+Function validation (`ValidationResult.warnings`) checks only **name existence**
+against the documented Trino catalog (`src/functions.rs`); it does not check
+arity or argument types — that is semantic analysis, out of scope for a syntax
+validator. False positives are possible if a deployed Trino has plugin functions
+beyond the docs; warnings are non-fatal by design.

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/CHANGELOG.md

@@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.2.0] - 2026-09-06
+
+### Added
+
+- Trino **function-name validation**: for `dialect="trino"`, `validate()` and
+  `validate_file()` now walk the parsed AST and flag calls to functions that are
+  not in the documented Trino catalog (e.g. a misspelled `round`).
+- New `ValidationResult.warnings` (`FunctionWarning`) and convenience property
+  `ValidationResult.unknown_functions`. Unknown functions are non-fatal warnings
+  — `valid` stays `True` because the syntax is fine.
+- `tools/extract_functions.py` — regenerates the embedded catalog
+  (`src/functions.rs`, 459 canonical names) from the Trino docs; committed so
+  builds stay offline and deterministic.
+
 ## [0.1.0] - 2026-09-06
 
 Initial release.

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/Cargo.lock

@@ -235,6 +235,18 @@ checksum = "13c6d1b651dc4edf07eead2a0c6c78016ce971bc2c10da5266861b13f25e7cec"
 dependencies = [
  "log",
  "recursive",
+ "sqlparser_derive",
+]
+
+[[package]]
+name = "sqlparser_derive"
+version = "0.5.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a6dd45d8fc1c79299bfbb7190e42ccbbdf6a5f52e4a6ad98d92357ea965bd289"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
 ]
 
 [[package]]
@@ -269,7 +281,7 @@ checksum = "adb6935a6f5c20170eeceb1a3835a49e12e19d792f6dd344ccc76a985ca5a6ca"
 
 [[package]]
 name = "trino-sql-validator"
-version = "0.1.0"
+version = "0.2.0"
 dependencies = [
  "pyo3",
  "regex",

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "trino-sql-validator"
-version = "0.1.0"
+version = "0.2.0"
 edition = "2021"
 description = "Fast Trino SQL syntax validator — Python library backed by Rust"
 license = "MIT"
@@ -16,7 +16,7 @@ crate-type = ["cdylib"]
 
 [dependencies]
 pyo3 = { version = "0.29", features = ["abi3-py310"] }
-sqlparser = { version = "0.62", default-features = true }
+sqlparser = { version = "0.62", default-features = true, features = ["visitor"] }
 regex = "1"
 
 [dev-dependencies]

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: trino-sql-validator
-Version: 0.1.0
+Version: 0.2.0
 Classifier: Development Status :: 3 - Alpha
 Classifier: Intended Audience :: Developers
 Classifier: License :: OSI Approved :: MIT License
@@ -60,6 +60,23 @@ Invalid SQL (and files containing it) is returned as a `ValidationResult`;
 it is **not** raised as an exception. Only genuine misuse (unknown dialect,
 unreadable file) raises.
 
+### Function-name warnings
+
+For `dialect="trino"`, `validate()` also checks that every function called in
+the SQL exists in the documented Trino catalog. Unknown names are reported as
+non-fatal `warnings` — `valid` stays `True` because syntax is fine:
+
+```python
+result = validate("SELECT marh(1.5)")      # round() misspelled
+assert result.valid
+print(result.warnings)                     # (FunctionWarning(name='marh', line=1, column=8),)
+print(result.unknown_functions)            # ['marh']
+```
+
+The catalog is auto-generated from the Trino docs and only checks *name
+existence*, not argument count or types. `hive`/`generic` dialects skip this
+check.
+
 ### Dialects
 
 - `"trino"` (default) — Trino-flavored with a custom override tuned for

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/README.md

@@ -33,6 +33,23 @@ Invalid SQL (and files containing it) is returned as a `ValidationResult`;
 it is **not** raised as an exception. Only genuine misuse (unknown dialect,
 unreadable file) raises.
 
+### Function-name warnings
+
+For `dialect="trino"`, `validate()` also checks that every function called in
+the SQL exists in the documented Trino catalog. Unknown names are reported as
+non-fatal `warnings` — `valid` stays `True` because syntax is fine:
+
+```python
+result = validate("SELECT marh(1.5)")      # round() misspelled
+assert result.valid
+print(result.warnings)                     # (FunctionWarning(name='marh', line=1, column=8),)
+print(result.unknown_functions)            # ['marh']
+```
+
+The catalog is auto-generated from the Trino docs and only checks *name
+existence*, not argument count or types. `hive`/`generic` dialects skip this
+check.
+
 ### Dialects
 
 - `"trino"` (default) — Trino-flavored with a custom override tuned for

trino_sql_validator-0.2.0/plan/functions-validation.md

@@ -0,0 +1,109 @@
+# Function-name validation for Trino
+
+Status: implemented for v0.2.0.
+Source of truth for the catalog: Trino docs at
+`docs/src/main/sphinx/functions/` in `trinodb/trino` (MyST, current format).
+
+## Goal
+
+Catch a common class of error today's validator ships past: a syntactically
+well-formed SQL statement that calls a function Trino does not provide (typo,
+wrong engine, e.g. `SELECT marh(x)` vs `round(x)`). `sqlparser-rs` treats any
+`name(...)` as a function call, so it never flags unknown names.
+
+## Where the catalog comes from
+
+The `functions/` tree has 35 topic files (`aggregate.md`, `string.md`, ...) plus
+`list.md`. A function name appears in two machine-greppable forms:
+
+- `:::{function} name(args) -> type` directives (the authoritative signatures,
+  one per overload).
+- `{func}\`name\`` cross-references in `list.md` (the alphabetical catalog of
+  every documented function).
+
+Our catalog = **union** of directive names across all topic files AND `{func}`
+names in `list.md`. Current Trino docs yield **459 canonical names**
+(measured 2026-09-06). This catches all of `count/sum/avg/min/max/...` plus
+Trino-only functions.
+
+Cast/keywords that sqlparser models specially (`cast`, `try`, `if`, `coalesce`)
+never reach the function walker, so they need no special-casing.
+
+## Algorithm (time-optimal)
+
+1. **No extra cost on invalid SQL.** Parse first. If `Parser::parse_sql` fails,
+   return the existing error today and skip function walking entirely.
+2. **Single linear AST walk.** For a successful parse, walk every statement once
+   with `sqlparser`'s `visit_expressions` (`feature = "visitor"`). For each
+   `Expr::Function`, look up the *last* `Ident` of its `ObjectName` in a static
+   `HashSet<&'static str>` of canonical names. O(functions-in-query), O(1) lookup.
+3. **Exact line/column for free.** `Ident.span` gives byte-exact start
+   `Location` — no regex, no string scanning (unlike the error-location path).
+4. **Static, lock-free data.** The name list is committed to `src/functions.rs`
+   as a `LazyLock<HashSet>`. Built once by `tools/extract_functions.py` (a
+   dev-only script that pulls the docs). No runtime/build-time network.
+
+The name list only changes when Trino adds/removes functions (a release-scale
+event). Committing the generated list keeps the crate offline-buildable and
+deterministic; regenerating is opt-in (`python tools/extract_functions.py`).
+
+## Behavior / semantics
+
+- Only `dialect == "trino"` checks functions. `hive`/`generic` stay as-is
+  (permissive escapes hatch; their function surface differs from Trino's).
+- A call to an unknown function does **not** make the statement "invalid" in the
+  existing `valid: bool` sense — that stays strictly about syntax. It is
+  reported as a `warning` alongside the syntax result. This preserves the
+  "bad syntax is a value, not an exception" contract on parse errors and keeps
+  the truly-syntactic check peer-reviewable.
+- Stopping early: the walker records unknown names and their locations and can
+  short-circuit (via `ControlFlow::Break`) to avoid scanning the rest of a large
+  script once already-warned — bounded memory, unchanged O(n) time.
+- Fully-qualified calls (`schema.round(x)`): match on the bare function name;
+  the leading qualifier is a schema reference, not part of the function.
+
+## API shape
+
+`ValidationResult` gains an optional `warnings: tuple[FunctionWarning, ...]`.
+
+```python
+@dataclass(frozen=True)
+class FunctionWarning:
+    name: str
+    line: int | None = None
+    column: int | None = None
+```
+
+Rust `validate`/`validate_file` return a 6-tuple
+`(valid, statement_count, error_message, line, column, warnings)` where
+`warnings` is a list of `(name, line, column)` triples. pyo3 converts the
+plain `Vec<(String, Option<usize>, Option<usize>)>` to a Python list of tuples
+directly — no serialization dependency needed.
+
+## Files
+
+- `tools/extract_functions.py` — dev-only doc parser → `src/functions.rs`.
+- `src/functions.rs` — generated static catalog + `is_known_function()`.
+- `src/lib.rs` — enable `visitor` feature usage; walk AST after successful parse.
+- `python/trino_sql_validator/__init__.py` — `FunctionWarning`, `warnings` field.
+- `python/trino_sql_validator/_native.pyi` — updated tuple type.
+- `tests/test_validator.py` — unknown/known/qualified/trino-only cases.
+- `tests/fixtures/` — sample queries with/without unknown functions.
+
+## Verification
+
+- `cargo test` (Rust unit tests for the walker + catalog sanity).
+- `pytest` — unknown function → warning with correct line/column; known &
+  multi-argument & qualified calls → no warning; `dialect="generic"` → no
+  function checks; invalid SQL → parse error, no warnings.
+- `cargo fmt --check`, `cargo clippy --all-targets -- -D warnings`, `ruff`,
+  `mypy` all stay green.
+
+## Trade-offs / limits (documented, not "fixed" here)
+
+- Catalog is the *documented* function set. Trino deploys may have
+  connector/plugin-specific functions beyond docs → possible false positives for
+  exotic setups. Warnings are non-fatal and per-dialect-trino, so this is safe.
+- We validate name existence, not arity/type (sqlparser does not model Trino's
+  function signatures beyond parse; arity/type checking is semantic and out of
+  scope for a syntax validator). See roadmap for `trino-parser` on the far end.

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/plan/plan.md

@@ -43,12 +43,16 @@ trino-sql-validator/
 ├── AGENTS.md
 ├── plan/
 │   ├── plan.md               # this file
-│   └── roadmap.md            # future work (Trino-exact grammar, etc.)
+│   ├── roadmap.md            # future work (Trino-exact grammar, etc.)
+│   └── functions-validation.md # function-name validation plan (v0.2.0)
 ├── src/
 │   ├── lib.rs                # #[pymodule] entry point
+│   ├── functions.rs          # GENERATED Trino function catalog (459 names)
 │   ├── dialects/
 │   │   └── mod.rs            # TrinoDialect (impl Dialect trait)
 │   └── error.rs              # Mapping ParserError -> Python exception info
+├── tools/
+│   └── extract_functions.py  # regenerates src/functions.rs from Trino docs
 ├── python/trino_sql_validator/
 │   ├── __init__.py           # pure-Python public API (re-exports, __all__)
 │   ├── _validator.pyi        # type stubs referencing the compiled _native module

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/plan/roadmap.md

@@ -6,6 +6,12 @@
 - Wheels for Linux (x86_64, aarch64), macOS (arm64, x86_64), Windows (x86_64).
 - PyPI publishing via GitHub Actions + Trusted Publishing.
 
+## v0.2.0
+- Trino function-name validation: AST walk via `sqlparser::visitor`; unknown
+  functions are surfaced as non-fatal `FunctionWarning` in `ValidationResult.warnings`.
+  Catalog (459 names) auto-generated from Trino docs and committed in `src/functions.rs`.
+  Only active for `dialect="trino"`.
+
 ## v0.2.x — clearer Trino cursor
 - Enrich `TrinoDialect` overrides for commonly-mis-parsed Trino-specific syntax:
   - `EXECUTE IMMEDIATE`, `CALL` signatures

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "trino-sql-validator"
-version = "0.1.0"
+version = "0.2.0"
 description = "Fast Trino SQL syntax validator — Python library backed by Rust"
 readme = "README.md"
 license = { text = "MIT" }

{trino_sql_validator-0.1.0 → trino_sql_validator-0.2.0}/python/trino_sql_validator/__init__.py

@@ -15,15 +15,23 @@ from ._native import validate_file as _native_validate_file
 
 __all__ = [
     "Error",
+    "FunctionWarning",
     "ValidationResult",
     "__version__",
     "validate",
     "validate_file",
 ]
 
-_NativeResult = tuple[bool, int, str | None, int | None, int | None]
-"""Tuple shape returned by the Rust core: (valid, statement_count, error,
-message, line, column)."""
+_NativeResult = tuple[
+    bool,
+    int,
+    str | None,
+    int | None,
+    int | None,
+    tuple[tuple[str, int | None, int | None], ...],
+]
+"""Tuple shape returned by the Rust core: (valid, statement_count, error
+message, line, column, warnings)."""
 
 __version__ = _native.__version__
 
@@ -44,6 +52,25 @@ class Error:
         return self.message
 
 
+@dataclass(frozen=True)
+class FunctionWarning:
+    """A call to a function that is not in the documented Trino catalog.
+
+    ``valid`` stays ``True`` for such statements — name checks are advisory,
+    not syntax errors (a deployed Trino may still offer plugin functions that
+    the docs do not list).
+    """
+
+    name: str
+    line: int | None = None
+    column: int | None = None
+
+    def __str__(self) -> str:
+        if self.line is not None and self.column is not None:
+            return f"unknown function '{self.name}' at line {self.line}, column {self.column}"
+        return f"unknown function '{self.name}'"
+
+
 @dataclass(frozen=True)
 class ValidationResult:
     """Structured outcome of validating one or more SQL statements."""
@@ -51,14 +78,26 @@ class ValidationResult:
     valid: bool
     statement_count: int
     error: Error | None = None
+    warnings: tuple[FunctionWarning, ...] = ()
+
+    @property
+    def unknown_functions(self) -> list[str]:
+        """Function names used in the SQL that have no Trino documentation
+        entry, in order of appearance."""
+        return [warning.name for warning in self.warnings]
 
     def __bool__(self) -> bool:
         return self.valid
 
     def __repr__(self) -> str:
-        if self.valid:
-            return f"<ValidationResult valid=True statements={self.statement_count}>"
-        return f"<ValidationResult valid=False error={self.error!r}>"
+        if not self.valid:
+            return f"<ValidationResult valid=False error={self.error!r}>"
+        if self.warnings:
+            return (
+                f"<ValidationResult valid=True statements={self.statement_count} "
+                f"warnings={len(self.warnings)}>"
+            )
+        return f"<ValidationResult valid=True statements={self.statement_count}>"
 
 
 _SUPPORTED_DIALECTS = ("trino", "hive", "generic")
@@ -69,11 +108,12 @@ def _validate(dialect: Dialect, call: _NativeResult) -> ValidationResult:
         raise ValueError(
             f"unknown dialect {dialect!r}; expected one of {_SUPPORTED_DIALECTS}"
         )
-    valid, statement_count, message, line, column = call
+    valid, statement_count, message, line, column, warnings = call
     return ValidationResult(
         valid=bool(valid),
         statement_count=int(statement_count),
         error=Error(message=message, line=line, column=column) if message else None,
+        warnings=tuple(FunctionWarning(name=name, line=wl, column=wc) for name, wl, wc in warnings),
     )
 
 

trino_sql_validator-0.2.0/python/trino_sql_validator/_native.pyi

@@ -0,0 +1,15 @@
+from typing import TypeAlias
+
+__version__: str
+
+_Warning: TypeAlias = tuple[str, int | None, int | None]
+
+def validate(
+    sql: str,
+    dialect: str = "trino",
+) -> tuple[bool, int, str | None, int | None, int | None, tuple[_Warning, ...]]: ...
+
+def validate_file(
+    path: str,
+    dialect: str = "trino",
+) -> tuple[bool, int, str | None, int | None, int | None, tuple[_Warning, ...]]: ...