bmsdna-devtools 0.34.0__tar.gz → 0.36.0__tar.gz

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: bmsdna-devtools
-Version: 0.34.0
+Version: 0.36.0
 Summary: Shared Azure DevOps / GitHub / git / Azure Monitor developer tooling for BMS projects
 Requires-Python: >=3.14
 Requires-Dist: pgdevkit[db]>=0.7.1
@@ -18,7 +18,7 @@ Shared developer tooling for BMS projects: PR build/check status and a
 one-shot PR summary (`bdt pr info`), PR creation, claiming an issue (`bdt issue take`), issue/work item creation and comments, git worktrees (creation and
 merged-worktree/orphaned-test-DB cleanup), a commit-and-push helper with
 pre-flight checks, Azure log queries, and static checks (`bdt lint`) for
-postgres/psycopg SQL rules, pydantic-model placement, hand-wired HTTP access in
+postgres/psycopg (and arrowbricks) SQL rules, pydantic-model placement, hand-wired HTTP access in
 TypeScript and baseline tooling, plus `bdt dead-code` for `.sql` files nothing loads and
 backend routes nothing calls.
 `bdt pr *` and `bdt issue *` auto-detect whether the current repo's `origin`
@@ -630,12 +630,36 @@ bdt lint backend/db/a.py b.py  # scan only these files -- e.g. from a prek/pre-c
 bdt lint --no-tooling-check    # skip the tooling-config check for this run
 ```
 
-Every `.execute()`/`.executemany()` call -- and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar` and
+Every `.execute()`/`.executemany()` call -- and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar`/`execute` and
 `PostgresJsonResponse`, which only accept a literal string, sqlglot expression, `psycopg.sql` or t-string -- whose SQL argument can be resolved to a
 literal or f-string/concatenation/`%`-format expression is checked (an
 unresolvable argument, e.g. a plain function parameter, is silently skipped --
 this can't false-positive on non-psycopg `.execute()` calls, or on dynamic SQL
-it can't see through):
+it can't see through).
+
+**arrowbricks** (the Databricks SQL client): `Cursor.execute`/`execute_streamed`, `DatabricksClient.query_agent`/`stream_query_json`
+and the free `arrowbricks.stream_query_json(client, sql)` are checked as well. `execute_streamed`/`query_agent`/`stream_query_json`
+are matched by name (the free function's SQL is its *second* argument; the method's is the first; `sql=` as keyword works too).
+Only the injection rules apply there -- an f-string/concatenation/`%`/`.format()` is flagged and the fix suggests an
+arrowbricks t-string (`{value}` binds a named parameter, `{name:i}` quotes an identifier) or literal SQL with
+`parameters=[...]` (`params=[...]` on `query_agent`/`stream_query_json`) for `:name` markers. The postgres house style (`sql-inline-too-complex`, `sql-positional-param`, `sql-forbidden-join`) is not applied to
+them, since Databricks SQL isn't postgres. The SQL is also parsed as Databricks SQL (backticks, `QUALIFY`, ...), so such
+statements are no longer missed; a plain `.execute()` (incl. `execute(sql=...)`) gets that extra dialect only in a file that
+imports `arrowbricks`, and keeps the postgres rules there (it may be a psycopg cursor too). The free function is recognised
+under `from arrowbricks import stream_query_json as x` and `arrowbricks[._streaming].stream_query_json(...)`.
+Like pgdevkit's helpers the three names match by name wherever they are called (a client injected from another module is the
+common case), so an unrelated `agent.query_agent(f"select ...")` whose text parses as SQL is flagged too.
+arrowbricks' `unsafe_sql(x)` -- its typing-only escape hatch -- is treated like `cast(LiteralString, x)` (in any file):
+only as safe as `x`.
+
+pgdevkit's helpers are matched by name (`fetch_all(...)`, `db.fetch_all(...)`, `pgdevkit.db.execute(...)`). The one
+exception is a *bare* `execute(...)`, a far too generic name: it is only checked in a file that imports it from
+`pgdevkit`/`pgdevkit.db` (`from pgdevkit.db import execute`, also aliased: `... import execute as run`; the import is
+recognised file-wide), so an unrelated `execute()` from another library, or one defined locally in a file without that
+import, is never flagged. Aliased imports of the other helpers (`fetch_all as fa`) are recognised too. Not covered: a re-export through the
+project's own module (`from app.db import execute`; call it as `db.execute(...)` or import it from `pgdevkit.db`
+instead), a subclass of `PostgresJsonResponse` under another name, and CCMT2's pre-pgdevkit
+`PostgresJsonResponse("postgres", sql)`, whose first argument is a connection source, not SQL.
 
 - **`sql-inline-too-complex`** — more than a trivial (≤4 line) query, or a
   JOIN/CTE/subquery/aggregation, inline instead of `load_sql()`/a `.sql` file.
@@ -648,6 +672,8 @@ it can't see through):
   with an f-string, `+` concatenation, the `%` operator, or `str.format()`
   instead of a psycopg t-string (3.14+), `psycopg.sql`, or bound params. For an f-string the fix is
   usually just `f"..."` -> `t"..."` (`{value}` is bound, `{name:i}` quotes an identifier).
+  A dynamic fragment may be an identifier/expression, an optional clause, or a (possibly empty) list of
+  assignments -- the statement is flagged if it parses as DML with any of these readings of its fragments.
 - **`sql-sqlglot-string-injection`** — an f-string / concatenation / `%` / `.format()` string passed to a
   sqlglot builder that parses it as SQL (`select(...)`, `.from_()`, `.where()`, `.order_by()`, `parse_one()`, ...).
   The result is a sqlglot expression, which the rules above otherwise trust, so this is the hole they'd miss.
@@ -835,8 +861,8 @@ Findings have two severities: **error** (a definite unsafe pattern, exit code 1)
 (depends on where a value comes from; listed with an instruction for an AI/human to verify, exit
 code 0 unless `--strict`).
 
-- **SQL** (Python `.execute()` / `fetch_all()` / `fetch_one()` / `fetch_scalar()` / `PostgresJsonResponse()`): f-string / `%` / concatenation / `.format()` SQL is an error.
-  SQL from `load_sql()`, `sql.SQL`, sqlglot (`expr.sql()`, `sqlglot.*`), `cast(LiteralString, <sqlglot expr>)` (the cast is only as safe as its argument) or
+- **SQL** (Python `.execute()` / `fetch_all()` / `fetch_one()` / `fetch_scalar()` / `execute()` from pgdevkit / `PostgresJsonResponse()` / arrowbricks `execute_streamed()` / `query_agent()` / `stream_query_json()`): f-string / `%` / concatenation / `.format()` SQL is an error.
+  SQL from `load_sql()`, `sql.SQL`, sqlglot (`expr.sql()`, `sqlglot.*`), `cast(LiteralString, <sqlglot expr>)`/`unsafe_sql(...)` (the cast is only as safe as its argument) or
   a function in the same file annotated `-> LiteralString` is trusted; SQL from any other function
   call is a `sql-unverified-call` review item. (`bdt lint` accepts the same trusted forms but never
   reports unverified calls.)

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/README.md

@@ -4,7 +4,7 @@ Shared developer tooling for BMS projects: PR build/check status and a
 one-shot PR summary (`bdt pr info`), PR creation, claiming an issue (`bdt issue take`), issue/work item creation and comments, git worktrees (creation and
 merged-worktree/orphaned-test-DB cleanup), a commit-and-push helper with
 pre-flight checks, Azure log queries, and static checks (`bdt lint`) for
-postgres/psycopg SQL rules, pydantic-model placement, hand-wired HTTP access in
+postgres/psycopg (and arrowbricks) SQL rules, pydantic-model placement, hand-wired HTTP access in
 TypeScript and baseline tooling, plus `bdt dead-code` for `.sql` files nothing loads and
 backend routes nothing calls.
 `bdt pr *` and `bdt issue *` auto-detect whether the current repo's `origin`
@@ -616,12 +616,36 @@ bdt lint backend/db/a.py b.py  # scan only these files -- e.g. from a prek/pre-c
 bdt lint --no-tooling-check    # skip the tooling-config check for this run
 ```
 
-Every `.execute()`/`.executemany()` call -- and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar` and
+Every `.execute()`/`.executemany()` call -- and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar`/`execute` and
 `PostgresJsonResponse`, which only accept a literal string, sqlglot expression, `psycopg.sql` or t-string -- whose SQL argument can be resolved to a
 literal or f-string/concatenation/`%`-format expression is checked (an
 unresolvable argument, e.g. a plain function parameter, is silently skipped --
 this can't false-positive on non-psycopg `.execute()` calls, or on dynamic SQL
-it can't see through):
+it can't see through).
+
+**arrowbricks** (the Databricks SQL client): `Cursor.execute`/`execute_streamed`, `DatabricksClient.query_agent`/`stream_query_json`
+and the free `arrowbricks.stream_query_json(client, sql)` are checked as well. `execute_streamed`/`query_agent`/`stream_query_json`
+are matched by name (the free function's SQL is its *second* argument; the method's is the first; `sql=` as keyword works too).
+Only the injection rules apply there -- an f-string/concatenation/`%`/`.format()` is flagged and the fix suggests an
+arrowbricks t-string (`{value}` binds a named parameter, `{name:i}` quotes an identifier) or literal SQL with
+`parameters=[...]` (`params=[...]` on `query_agent`/`stream_query_json`) for `:name` markers. The postgres house style (`sql-inline-too-complex`, `sql-positional-param`, `sql-forbidden-join`) is not applied to
+them, since Databricks SQL isn't postgres. The SQL is also parsed as Databricks SQL (backticks, `QUALIFY`, ...), so such
+statements are no longer missed; a plain `.execute()` (incl. `execute(sql=...)`) gets that extra dialect only in a file that
+imports `arrowbricks`, and keeps the postgres rules there (it may be a psycopg cursor too). The free function is recognised
+under `from arrowbricks import stream_query_json as x` and `arrowbricks[._streaming].stream_query_json(...)`.
+Like pgdevkit's helpers the three names match by name wherever they are called (a client injected from another module is the
+common case), so an unrelated `agent.query_agent(f"select ...")` whose text parses as SQL is flagged too.
+arrowbricks' `unsafe_sql(x)` -- its typing-only escape hatch -- is treated like `cast(LiteralString, x)` (in any file):
+only as safe as `x`.
+
+pgdevkit's helpers are matched by name (`fetch_all(...)`, `db.fetch_all(...)`, `pgdevkit.db.execute(...)`). The one
+exception is a *bare* `execute(...)`, a far too generic name: it is only checked in a file that imports it from
+`pgdevkit`/`pgdevkit.db` (`from pgdevkit.db import execute`, also aliased: `... import execute as run`; the import is
+recognised file-wide), so an unrelated `execute()` from another library, or one defined locally in a file without that
+import, is never flagged. Aliased imports of the other helpers (`fetch_all as fa`) are recognised too. Not covered: a re-export through the
+project's own module (`from app.db import execute`; call it as `db.execute(...)` or import it from `pgdevkit.db`
+instead), a subclass of `PostgresJsonResponse` under another name, and CCMT2's pre-pgdevkit
+`PostgresJsonResponse("postgres", sql)`, whose first argument is a connection source, not SQL.
 
 - **`sql-inline-too-complex`** — more than a trivial (≤4 line) query, or a
   JOIN/CTE/subquery/aggregation, inline instead of `load_sql()`/a `.sql` file.
@@ -634,6 +658,8 @@ it can't see through):
   with an f-string, `+` concatenation, the `%` operator, or `str.format()`
   instead of a psycopg t-string (3.14+), `psycopg.sql`, or bound params. For an f-string the fix is
   usually just `f"..."` -> `t"..."` (`{value}` is bound, `{name:i}` quotes an identifier).
+  A dynamic fragment may be an identifier/expression, an optional clause, or a (possibly empty) list of
+  assignments -- the statement is flagged if it parses as DML with any of these readings of its fragments.
 - **`sql-sqlglot-string-injection`** — an f-string / concatenation / `%` / `.format()` string passed to a
   sqlglot builder that parses it as SQL (`select(...)`, `.from_()`, `.where()`, `.order_by()`, `parse_one()`, ...).
   The result is a sqlglot expression, which the rules above otherwise trust, so this is the hole they'd miss.
@@ -821,8 +847,8 @@ Findings have two severities: **error** (a definite unsafe pattern, exit code 1)
 (depends on where a value comes from; listed with an instruction for an AI/human to verify, exit
 code 0 unless `--strict`).
 
-- **SQL** (Python `.execute()` / `fetch_all()` / `fetch_one()` / `fetch_scalar()` / `PostgresJsonResponse()`): f-string / `%` / concatenation / `.format()` SQL is an error.
-  SQL from `load_sql()`, `sql.SQL`, sqlglot (`expr.sql()`, `sqlglot.*`), `cast(LiteralString, <sqlglot expr>)` (the cast is only as safe as its argument) or
+- **SQL** (Python `.execute()` / `fetch_all()` / `fetch_one()` / `fetch_scalar()` / `execute()` from pgdevkit / `PostgresJsonResponse()` / arrowbricks `execute_streamed()` / `query_agent()` / `stream_query_json()`): f-string / `%` / concatenation / `.format()` SQL is an error.
+  SQL from `load_sql()`, `sql.SQL`, sqlglot (`expr.sql()`, `sqlglot.*`), `cast(LiteralString, <sqlglot expr>)`/`unsafe_sql(...)` (the cast is only as safe as its argument) or
   a function in the same file annotated `-> LiteralString` is trusted; SQL from any other function
   call is a `sql-unverified-call` review item. (`bdt lint` accepts the same trusted forms but never
   reports unverified calls.)

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/bmsdna/devtools/cli.py

@@ -1093,7 +1093,8 @@ def lint(
     ),
 ) -> None:
     """Static checks (bmsuisse/skills#52): postgres/psycopg SQL rules on every `.execute()` call and pgdevkit
-    `fetch_all`/`fetch_one`/`fetch_scalar`/`PostgresJsonResponse` call
+    `fetch_all`/`fetch_one`/`fetch_scalar`/`execute`/`PostgresJsonResponse` call (and arrowbricks'
+    `execute_streamed`/`query_agent`/`stream_query_json`, injection rules only)
     (must use load_sql()/a .sql file, a t-string, or psycopg.sql for anything beyond a trivial
     query; never an f-string/concatenation/`%`-formatting), pydantic-model placement under api/
     directories, hand-wired HTTP in TypeScript, and that the repo declares/configures ty, ruff, pytest and prek.

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/bmsdna/devtools/lint_sql.py

@@ -1,6 +1,6 @@
 """AST rule engine for `bdt lint`'s postgres/psycopg checks (bmsuisse/skills#52):
-every `.execute()`/`.executemany()` call (and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar` and
-`PostgresJsonResponse`) is classified by how its SQL argument was
+every `.execute()`/`.executemany()` call (and pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar`/`execute` and
+`PostgresJsonResponse`, and arrowbricks' `execute_streamed`/`query_agent`/`stream_query_json`) is classified by how its SQL argument was
 built, then checked against the `postgres-best-practices` skill's rules
 (../../skills/postgres-best-practices in a checkout of bmsuisse/skills).
 
@@ -24,10 +24,11 @@ than guessed at -- a linter that can't be sure must stay quiet, not noisy.
 from __future__ import annotations
 
 import ast
+import itertools
 import re
 import sys
 from collections.abc import Mapping
-from dataclasses import dataclass, field
+from dataclasses import dataclass, field, replace
 from pathlib import Path
 
 from .lint_findings import Finding
@@ -63,10 +64,18 @@ _POSITIONAL_PARAM_RE = re.compile(r"(?<!%)%s\b")
 # Placeholder substituted for a non-literal fragment (an f-string {expr}, a concatenated
 # non-constant operand, a str.format() {field}) before attempting an sqlglot parse -- padded
 # with spaces so it never merges into a neighboring identifier/keyword, and looks like an
-# ordinary identifier so the parse doesn't fail on the substitution itself.
+# ordinary identifier so the parse doesn't fail on the substitution itself (see
+# `_PROBE_SLOT_VARIANTS`/`_parse_probe_text` for the fallbacks when an identifier doesn't fit).
 _PROBE_PLACEHOLDER = " __X__ "
 _FORMAT_FIELD_RE = re.compile(r"\{[^{}]*\}")
 
+# A dynamic fragment isn't always an identifier/expression: `SET {assignments}, {extra}` where `extra` is
+# empty or `"is_approved = false,"`, or an optional `{where}` clause. When the probe with every fragment as an
+# identifier doesn't parse, each fragment is retried as "nothing" and as an assignment, all combinations
+# (only up to _MAX_PROBE_SLOTS fragments, so the number of parses stays small).
+_PROBE_SLOT_VARIANTS = (_PROBE_PLACEHOLDER, " ", " __X__ = 1 ")
+_MAX_PROBE_SLOTS = 4
+
 _MAX_RESOLVE_HOPS = 5
 
 
@@ -127,6 +136,15 @@ class _Trust:
     sqlglot_names: frozenset[str] = frozenset()
     literal_funcs: frozenset[str] = frozenset()
     sqlglot_aliases: Mapping[str, str] = field(default_factory=dict)  # local name -> name it was imported as
+    pgdevkit_sinks: frozenset[str] = frozenset()  # local names bound by `from pgdevkit[.x] import <sql-taking helper>`
+    arrowbricks_modules: frozenset[str] = frozenset()  # local names bound by `import arrowbricks[.x] [as y]`
+    arrowbricks_file: bool = False  # the file imports arrowbricks (`import arrowbricks` / `from arrowbricks[.x] import ...`)
+    arrowbricks_stream_funcs: frozenset[str] = frozenset()  # local names of the free `stream_query_json(client, sql)`
+    # Per-call, set by the visitor (see `_call_trust`): this sink's SQL is Databricks SQL, so a probe that does not parse
+    # as postgres is retried as Databricks (backticks, `:name` markers, QUALIFY, ...) ...
+    databricks: bool = False
+    # ... and the postgres house style (inline-complexity, `%s` params, forbidden joins) is not applied to it.
+    postgres_rules: bool = True
 
 
 _NO_TRUST = _Trust()
@@ -166,10 +184,13 @@ def _is_trusted_sql_call(node: ast.AST, trust: _Trust = _NO_TRUST) -> bool:
 
 
 def _literal_string_cast_inner(node: ast.Call) -> ast.expr | None:
-    """`cast(LiteralString, x)` -> `x`. The cast itself proves nothing: it is only as safe as `x`."""
+    """`cast(LiteralString, x)` -> `x`, and arrowbricks' `unsafe_sql(x)` (its no-op spelling of the same cast) -> `x`.
+    The cast itself proves nothing: it is only as safe as `x`."""
     func = node.func
-    is_cast = (isinstance(func, ast.Name) and func.id == "cast") or (isinstance(func, ast.Attribute) and func.attr == "cast")
-    if is_cast and len(node.args) == 2 and _is_literal_string_ref(node.args[0]):
+    name = func.id if isinstance(func, ast.Name) else func.attr if isinstance(func, ast.Attribute) else None
+    if name == "unsafe_sql" and len(node.args) == 1 and not node.keywords:
+        return node.args[0]
+    if name == "cast" and len(node.args) == 2 and _is_literal_string_ref(node.args[0]):
         return node.args[1]
     return None
 
@@ -185,19 +206,52 @@ def _unwrap_str_call(node: ast.Call) -> ast.expr | None:
     return None
 
 
+def _is_pgdevkit_module(module: str | None) -> bool:
+    return module is not None and (module == "pgdevkit" or module.startswith("pgdevkit."))
+
+
+def _is_arrowbricks_module(module: str | None) -> bool:
+    return module is not None and (module == "arrowbricks" or module.startswith("arrowbricks."))
+
+
 def _file_trust(tree: ast.AST) -> _Trust:
     literal_funcs: set[str] = set()
     sqlglot_names: set[str] = set()
     sqlglot_aliases: dict[str, str] = {}
+    pgdevkit_sinks: set[str] = set()
+    arrowbricks_modules: set[str] = set()
+    arrowbricks_stream_funcs: set[str] = set()
+    arrowbricks_file = False
     for node in ast.walk(tree):
         if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.returns is not None and _is_literal_string_ref(node.returns):
             literal_funcs.add(node.name)
         elif isinstance(node, ast.Import):
             sqlglot_names.update((a.asname or a.name).split(".")[0] for a in node.names if a.name.split(".")[0] == "sqlglot")
+            for alias in node.names:
+                if _is_arrowbricks_module(alias.name):
+                    arrowbricks_file = True
+                    arrowbricks_modules.add(alias.asname or alias.name.split(".")[0])
         elif isinstance(node, ast.ImportFrom) and (node.module or "").split(".")[0] == "sqlglot":
             sqlglot_names.update(a.asname or a.name for a in node.names)
             sqlglot_aliases.update({a.asname or a.name: a.name for a in node.names})
-    return _Trust(frozenset(sqlglot_names), frozenset(literal_funcs), sqlglot_aliases)
+        elif isinstance(node, ast.ImportFrom) and node.level == 0 and _is_pgdevkit_module(node.module):
+            pgdevkit_sinks.update(a.asname or a.name for a in node.names if a.name in _PGDEVKIT_SQL_CALLEES)
+        elif isinstance(node, ast.ImportFrom) and node.level == 0 and _is_arrowbricks_module(node.module):
+            arrowbricks_file = True
+            for alias in node.names:
+                if alias.name == "stream_query_json":
+                    arrowbricks_stream_funcs.add(alias.asname or alias.name)
+                elif alias.name == "_streaming":  # the module that defines the free function
+                    arrowbricks_modules.add(alias.asname or alias.name)
+    return _Trust(
+        frozenset(sqlglot_names),
+        frozenset(literal_funcs),
+        sqlglot_aliases,
+        frozenset(pgdevkit_sinks),
+        frozenset(arrowbricks_modules),
+        arrowbricks_file,
+        frozenset(arrowbricks_stream_funcs),
+    )
 
 
 def _resolve_single(expr: ast.expr, lookup) -> ast.expr:
@@ -276,12 +330,77 @@ _FSTRING_FIX = (
     "`{name:i}` quotes a table/column identifier. Or use load_sql()/a .sql file for static SQL."
 )
 _GENERIC_FIX = "Use a psycopg t-string (Python 3.14+), psycopg.sql for dynamic SQL, or load_sql()/a .sql file for static SQL."
+_ARROWBRICKS_FSTRING_FIX = (
+    "Change the `f` prefix to `t` (arrowbricks t-string, Python 3.14+): `{value}` is bound as a named parameter and "
+    "`{name:i}` quotes a table/column identifier. Or keep the SQL literal and pass its `:name` markers' values as "
+    "`parameters=[...]` (`params=[...]` for `query_agent`/`stream_query_json`)."
+)
+_ARROWBRICKS_GENERIC_FIX = (
+    "Use an arrowbricks t-string (Python 3.14+: `{value}` binds a named parameter, `{name:i}` quotes an identifier), or a "
+    "literal SQL string with `parameters=[...]` (`params=[...]` for `query_agent`/`stream_query_json`) for its `:name` markers."
+)
+# A plain `.execute()` in a file that imports arrowbricks may be either driver's cursor: name neither.
+_MIXED_FIX = (
+    "Change the `f` prefix to `t` (psycopg/arrowbricks t-string, Python 3.14+): `{value}` becomes a bound parameter and "
+    "`{name:i}` quotes a table/column identifier. Or use load_sql()/a .sql file for static SQL."
+)
+
+
+# Keyword that must literally be in the text before a variant reading (a fragment as nothing/an assignment) may
+# count: sqlglot is lenient enough to read prose like "Update to version {v}" as an UPDATE with the fragment gone.
+_VARIANT_REQUIRED_KEYWORD: dict[type, re.Pattern[str]] = {}
+if exp is not None:
+    _VARIANT_REQUIRED_KEYWORD = {
+        exp.Select: re.compile(r"(?i)\bfrom\b"),
+        exp.Union: re.compile(r"(?i)\bfrom\b"),
+        exp.Insert: re.compile(r"(?i)\binto\b"),
+        exp.Update: re.compile(r"(?i)\bset\b"),
+        exp.Delete: re.compile(r"(?i)\bfrom\b"),
+        exp.Merge: re.compile(r"(?i)\busing\b"),
+    }
+
+
+def _parse_any_dialect(text: str, databricks: bool) -> "exp.Expression | None":
+    parsed = parse_sql_text(text)
+    if parsed is None and databricks:
+        parsed = parse_sql_text(text, "databricks")
+    return parsed
+
+
+def _parse_probe_text(text: str, databricks: bool = False) -> "exp.Expression | None":
+    """`parse_sql_text` for text with `_PROBE_PLACEHOLDER` standing in for dynamic fragments: the placeholder
+    as an identifier first, then (see `_PROBE_SLOT_VARIANTS`) every fragment as nothing/an assignment -- the
+    latter only when the literal text carries the statement's own keyword (`_VARIANT_REQUIRED_KEYWORD`)."""
+    parsed = _parse_any_dialect(text, databricks)
+    if parsed is not None:
+        return parsed
+    parts = text.split(_PROBE_PLACEHOLDER)
+    slots = len(parts) - 1
+    if not 1 <= slots <= _MAX_PROBE_SLOTS:
+        return None
+    literal_text = " ".join(parts)
+    for combination in itertools.product(_PROBE_SLOT_VARIANTS, repeat=slots):
+        if all(c == _PROBE_PLACEHOLDER for c in combination):
+            continue  # that's `text` itself, already tried
+        candidate = "".join(a + b for a, b in itertools.zip_longest(parts, combination, fillvalue=""))
+        parsed = _parse_any_dialect(candidate, databricks)
+        if parsed is None:
+            continue
+        keyword = next((rx for t, rx in _VARIANT_REQUIRED_KEYWORD.items() if isinstance(parsed, t)), None)
+        if keyword is not None and keyword.search(literal_text):
+            return parsed
+    return None
 
 
-def _injection_finding(text: str, path: Path, lineno: int, rule: str, how: str) -> list[Finding]:
-    if parse_sql_text(text) is None:
+def _injection_finding(text: str, path: Path, lineno: int, rule: str, how: str, trust: _Trust = _NO_TRUST) -> list[Finding]:
+    if _parse_probe_text(text, trust.databricks) is None:
         return []
-    fix = _FSTRING_FIX if rule == "sql-fstring-injection" else _GENERIC_FIX
+    if trust.databricks and not trust.postgres_rules:
+        fix = _ARROWBRICKS_FSTRING_FIX if rule == "sql-fstring-injection" else _ARROWBRICKS_GENERIC_FIX
+    elif trust.databricks:
+        fix = _MIXED_FIX
+    else:
+        fix = _FSTRING_FIX if rule == "sql-fstring-injection" else _GENERIC_FIX
     return [Finding(path, lineno, rule, f"SQL built with {how} -- injection risk. {fix}")]
 
 
@@ -360,30 +479,30 @@ def _check_resolved_candidate(
     if _is_load_sql_call(resolved) or _is_sql_composed_call(resolved) or _is_trusted_sql_call(resolved, trust):
         return []
     if isinstance(resolved, getattr(ast, "TemplateStr", ())):
-        return []  # psycopg t-string: params are always bound, never interpolated
+        return []  # psycopg/arrowbricks t-string: params are always bound, never interpolated
 
     if isinstance(resolved, ast.JoinedStr):
-        return _injection_finding(_fstring_probe_text(resolved), path, lineno, "sql-fstring-injection", "an f-string")
+        return _injection_finding(_fstring_probe_text(resolved), path, lineno, "sql-fstring-injection", "an f-string", trust)
 
     if isinstance(resolved, ast.BinOp) and isinstance(resolved.op, ast.Mod):
         left = _resolve_single(resolved.left, lookup)
         if isinstance(left, ast.Constant) and isinstance(left.value, str):
-            return _injection_finding(left.value, path, lineno, "sql-percent-format-injection", "the `%` string-formatting operator")
+            return _injection_finding(left.value, path, lineno, "sql-percent-format-injection", "the `%` string-formatting operator", trust)
         return []
 
     if isinstance(resolved, ast.BinOp) and isinstance(resolved.op, ast.Add):
         text = _concat_probe_text(resolved, lookup)
-        return _injection_finding(text, path, lineno, "sql-concat-injection", "string concatenation") if text is not None else []
+        return _injection_finding(text, path, lineno, "sql-concat-injection", "string concatenation", trust) if text is not None else []
 
     if isinstance(resolved, ast.Call) and isinstance(resolved.func, ast.Attribute) and resolved.func.attr == "format":
         template = _resolve_single(resolved.func.value, lookup)
         if isinstance(template, ast.Constant) and isinstance(template.value, str):
             text = _FORMAT_FIELD_RE.sub(_PROBE_PLACEHOLDER, template.value)
-            return _injection_finding(text, path, lineno, "sql-format-injection", "`str.format()`")
+            return _injection_finding(text, path, lineno, "sql-format-injection", "`str.format()`", trust)
         return []
 
     if isinstance(resolved, ast.Constant) and isinstance(resolved.value, str):
-        return _literal_findings(resolved.value, path, lineno)
+        return _literal_findings(resolved.value, path, lineno) if trust.postgres_rules else []
 
     if isinstance(resolved, ast.Call):
         cast_inner = _literal_string_cast_inner(resolved)
@@ -418,7 +537,7 @@ def _unverified_cast_finding(path: Path, lineno: int) -> Finding:
         path,
         lineno,
         "sql-unverified-cast",
-        "`cast(LiteralString, ...)` wraps a value that isn't visibly a sqlglot expression or a literal. AI reviewer: verify it "
+        "`cast(LiteralString, ...)`/`unsafe_sql(...)` wraps a value that isn't visibly a sqlglot expression or a literal. AI reviewer: verify it "
         "cannot carry user-controlled text (the cast doesn't make it safe); pass a sqlglot expression, or add "
         "`# bdt-lint: ignore sql-unverified-cast`.",
         severity="review",
@@ -455,26 +574,37 @@ def _is_sql_named(name: str) -> bool:
     return lowered == "sql" or lowered.endswith("_sql")
 
 
-# pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar` and `pgdevkit.fastapi.PostgresJsonResponse` run their first
-# argument as SQL, exactly like `.execute()`, so they get the same rules (plain functions / a class, so matched by
-# bare name too). Matched by name only, so not covered: an alias (`as PJR`), a subclass under another name (the
-# documented `class AppJson(PostgresJsonResponse): expose_errors = ...`), and the pre-pgdevkit
+# pgdevkit's `fetch_all`/`fetch_one`/`fetch_scalar`/`execute` and `pgdevkit.fastapi.PostgresJsonResponse` run their
+# first argument as SQL, exactly like `.execute()`, so they get the same rules. As an attribute (`db.fetch_all(...)`,
+# `pgdevkit.db.execute(...)`) they are matched by name only, like `.execute()` itself.
+_PGDEVKIT_SQL_CALLEES = frozenset({"fetch_all", "fetch_one", "fetch_scalar", "execute", "PostgresJsonResponse"})
+# A bare `execute(...)` is far too generic a name (sqlite helpers, subprocess wrappers, ...): it counts only in a file
+# that imports it from `pgdevkit`/`pgdevkit.db` (also under an alias: `from pgdevkit.db import execute as run`; see
+# `_file_trust`). The import is tracked file-wide, so a local def/parameter named like it is not told apart. The sqlglot
+# DML gate still applies on top. The other callees are matched by bare name too, and by their pgdevkit import alias.
+# Not covered: a re-export through the project's own module (`from app.db import execute`), a subclass of
+# PostgresJsonResponse under another name (the documented `class AppJson(PostgresJsonResponse)`), and the pre-pgdevkit
 # `PostgresJsonResponse("postgres", sql, ...)` copy in CCMT2, whose first argument is a connection source -- its SQL
 # only gets checked once that call site moves to pgdevkit.
-_PGDEVKIT_SQL_CALLEES = frozenset({"fetch_all", "fetch_one", "fetch_scalar", "PostgresJsonResponse"})
+_IMPORT_GATED_BARE_CALLEES = frozenset({"execute"})
+_BARE_NAME_SQL_CALLEES = _PGDEVKIT_SQL_CALLEES - _IMPORT_GATED_BARE_CALLEES
+
+
+def _is_pgdevkit_sink_call(func: ast.expr, trust: _Trust) -> bool:
+    if isinstance(func, ast.Name):
+        return func.id in _BARE_NAME_SQL_CALLEES or func.id in trust.pgdevkit_sinks
+    return isinstance(func, ast.Attribute) and func.attr in _PGDEVKIT_SQL_CALLEES
 
 
-def _execute_query_arg(call: ast.Call) -> ast.expr | None:
+def _execute_query_arg(call: ast.Call, trust: _Trust = _NO_TRUST) -> ast.expr | None:
     func = call.func
     is_execute = isinstance(func, ast.Attribute) and func.attr in ("execute", "executemany")
-    is_fetch_helper = (isinstance(func, ast.Name) and func.id in _PGDEVKIT_SQL_CALLEES) or (
-        isinstance(func, ast.Attribute) and func.attr in _PGDEVKIT_SQL_CALLEES
-    )
-    if not (is_execute or is_fetch_helper):
+    if not (is_execute or _is_pgdevkit_sink_call(func, trust)):
         return None
     if call.args:
         return call.args[0]
-    return next((kw.value for kw in call.keywords if kw.arg == "query"), None)
+    keywords = ("query", "sql") if trust.arrowbricks_file else ("query",)  # arrowbricks' Cursor.execute(sql=...)
+    return next((kw.value for kw in call.keywords if kw.arg in keywords), None)
 
 
 # sqlglot's builders parse plain strings as SQL (`select(f"a, {x}")`, `.where(f"id = {x}")`), so text
@@ -568,6 +698,52 @@ def _sqlglot_builder_findings(
     return []
 
 
+# arrowbricks' (Databricks SQL client) calls that take SQL: `Cursor.execute_streamed(sql, ...)`,
+# `DatabricksClient.query_agent(sql, ...)`, `DatabricksClient.stream_query_json(sql, ...)` and the free function
+# `arrowbricks.stream_query_json(client, sql, ...)`. (`Cursor.execute` is already matched as any `.execute()`.) The names are
+# specific enough to match anywhere, like pgdevkit's `fetch_all`. Their SQL is Databricks SQL, so only the injection rules apply
+# (`_call_trust`); a t-string or `unsafe_sql(...)` is handled like psycopg's t-string / `cast(LiteralString, ...)`.
+_ARROWBRICKS_SQL_CALLEES = frozenset({"execute_streamed", "query_agent", "stream_query_json"})
+
+
+def _attribute_root(node: ast.expr) -> str | None:
+    while isinstance(node, ast.Attribute):
+        node = node.value
+    return node.id if isinstance(node, ast.Name) else None
+
+
+def _arrowbricks_sink_arg(call: ast.Call, trust: _Trust) -> ast.expr | None:
+    func = call.func
+    if isinstance(func, ast.Name):
+        # a bare name is the free function (also under an import alias) or, for the other two, the user's own helper
+        name = "stream_query_json" if func.id in trust.arrowbricks_stream_funcs else func.id
+        free_function = True
+    elif isinstance(func, ast.Attribute):
+        name = func.attr
+        free_function = _attribute_root(func.value) in trust.arrowbricks_modules  # `arrowbricks.stream_query_json(client, sql)`
+    else:
+        return None
+    if name not in _ARROWBRICKS_SQL_CALLEES:
+        return None
+    index = 1 if name == "stream_query_json" and free_function else 0  # the free function takes the client first
+    if len(call.args) > index and not isinstance(call.args[index], ast.Starred):
+        return call.args[index]
+    return next((kw.value for kw in call.keywords if kw.arg == "sql"), None)
+
+
+def _call_trust(call: ast.Call, trust: _Trust) -> tuple[ast.expr | None, _Trust]:
+    """The SQL argument of `call` (None if it isn't a SQL sink) and the trust to check it with: an arrowbricks call is
+    Databricks SQL without the postgres house style; a plain `.execute()` in a file that imports arrowbricks keeps the
+    postgres rules but may also be Databricks SQL (arrowbricks' `Cursor.execute`)."""
+    arrowbricks_arg = _arrowbricks_sink_arg(call, trust)
+    if arrowbricks_arg is not None:
+        return arrowbricks_arg, replace(trust, databricks=True, postgres_rules=False)
+    query_arg = _execute_query_arg(call, trust)
+    if query_arg is not None and trust.arrowbricks_file:
+        return query_arg, replace(trust, databricks=True)
+    return query_arg, trust
+
+
 class _ExecuteCallVisitor(ast.NodeVisitor):
     """Walks a module, tracking a stack of (function/class/module) local-variable scopes so a
     `cur.execute(query, ...)` call can resolve `query` back to every assignment it had in that
@@ -575,10 +751,11 @@ class _ExecuteCallVisitor(ast.NodeVisitor):
     branch-shadowed unsafe assignment is still seen), then applies the SQL rules to every
     `.execute()`/`.executemany()` call found."""
 
-    def __init__(self, path: Path, trust: _Trust = _NO_TRUST, review: bool = False) -> None:
+    def __init__(self, path: Path, trust: _Trust = _NO_TRUST, review: bool = False, arrowbricks_names: frozenset[str] = frozenset()) -> None:
         self.path = path
         self.trust = trust
         self.review = review
+        self._arrowbricks_names = arrowbricks_names  # `*_sql` variables passed to an arrowbricks call (no postgres house style)
         self.findings: list[Finding] = []
         self._scopes: list[dict[str, list[ast.expr]]] = [{}]
         self._sqlglot_vars: set[str] = set()  # names bound from a sqlglot expression chain
@@ -622,6 +799,8 @@ class _ExecuteCallVisitor(ast.NodeVisitor):
         `.execute()` call in this file (e.g. it's passed to a helper) -- apply the complexity rule."""
         if not (_is_sql_named(name) and isinstance(value, ast.Constant) and isinstance(value.value, str)):
             return
+        if name in self._arrowbricks_names:
+            return
         parsed = parse_sql_text(value.value)
         if parsed is not None:
             self.findings.extend(_complexity_findings(parsed, value.value, self.path, lineno))
@@ -634,9 +813,9 @@ class _ExecuteCallVisitor(ast.NodeVisitor):
 
     def visit_Call(self, node: ast.Call) -> None:
         self.findings.extend(_sqlglot_builder_findings(node, self.path, self.trust, self._lookup, self._sqlglot_vars))
-        query_arg = _execute_query_arg(node)
+        query_arg, trust = _call_trust(node, self.trust)
         if query_arg is not None:
-            found = _check_query_arg(query_arg, self._lookup, self.path, node.lineno, self.trust, self.review)
+            found = _check_query_arg(query_arg, self._lookup, self.path, node.lineno, trust, self.review)
             if isinstance(query_arg, ast.Name) and _is_sql_named(query_arg.id) and self._all_bare_strings(query_arg.id):
                 found = [f for f in found if f.rule != "sql-inline-too-complex"]  # already reported at the assignment
             self.findings.extend(found)
@@ -660,6 +839,16 @@ def check_sql_file(path: Path, source: str | None = None, *, review: bool = Fals
         tree = ast.parse(text, filename=str(path))
     except SyntaxError:
         return []
-    visitor = _ExecuteCallVisitor(path, _file_trust(tree), review)
+    trust = _file_trust(tree)
+    arrowbricks_names: set[str] = set()  # `*_sql` variables handed to an arrowbricks call ...
+    other_names: set[str] = set()  # ... minus any name that is also handed to some other sink (it could be a psycopg query)
+    for call in ast.walk(tree):
+        if not isinstance(call, ast.Call):
+            continue
+        if isinstance(arg := _arrowbricks_sink_arg(call, trust), ast.Name):
+            arrowbricks_names.add(arg.id)
+        elif isinstance(arg := _execute_query_arg(call, trust), ast.Name):
+            other_names.add(arg.id)
+    visitor = _ExecuteCallVisitor(path, trust, review, frozenset(arrowbricks_names - other_names))
     visitor.visit(tree)
     return visitor.findings

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/pyproject.toml

@@ -10,7 +10,7 @@ packages = ["bmsdna"]
 
 [project]
 name = "bmsdna-devtools"
-version = "0.34.0"
+version = "0.36.0"
 description = "Shared Azure DevOps / GitHub / git / Azure Monitor developer tooling for BMS projects"
 readme = "README.md"
 requires-python = ">=3.14"  # pgdevkit>=0.7.1 requires 3.14; was >=3.11 before adding it as a dependency

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/skills/bmsdna-devtools/SKILL.md

@@ -5,7 +5,7 @@ description: >
   commands or repo-local scripts for: checking PR build/check status, creating
   a PR, creating a git worktree, committing and pushing files (with pre-flight
   checks), claiming an issue / work item (`bdt issue take`), a one-shot PR summary
-  (`bdt pr info`: link, build state, closed issues), querying Azure logs, and static checks (`bdt lint`: SQL/psycopg rules, pydantic placement, tooling; `bdt dead-code`:
+  (`bdt pr info`: link, build state, closed issues), querying Azure logs, and static checks (`bdt lint`: SQL/psycopg/arrowbricks rules, pydantic placement, tooling; `bdt dead-code`:
   unreferenced .sql files and backend routes nothing calls), listing which env vars a repo defines (`bdt env get-keys`, names only). `bdt pr *` works against both Azure DevOps
   and GitHub — it auto-detects which one from the `origin` remote. Trigger
   whenever the user asks to check a build/PR status, create a PR, make a
@@ -123,7 +123,7 @@ vendor a git submodule under that path.
 
 ## Static checks
 
-- `bdt lint [paths]` -- SQL/psycopg rules (`.execute()`, pgdevkit `fetch_*`, `PostgresJsonResponse`), pydantic-model placement, hand-wired HTTP in TypeScript, baseline tooling.
+- `bdt lint [paths]` -- SQL/psycopg rules (`.execute()`, pgdevkit `fetch_*`/`execute`, `PostgresJsonResponse`, arrowbricks `execute_streamed`/`query_agent`/`stream_query_json`), pydantic-model placement, hand-wired HTTP in TypeScript, baseline tooling.
 - `uv run bdt dead-code` -- dead code a linter can't see, configured under `[tool.bdt.dead_code]` in pyproject.toml:
   `.sql` files no Python code loads (`sql_roots`) and FastAPI routes that neither non-generated frontend code nor a
   `url_for(...)` call uses (`[[tool.bdt.dead_code.apps]]`: app or openapi file, frontends, excludes, optional `baseline`;

{bmsdna_devtools-0.34.0 → bmsdna_devtools-0.36.0}/tests/test_find_injection.py

@@ -151,6 +151,32 @@ def test_sql_injection_is_error_and_unverified_call_is_review(tmp_path: Path) ->
     assert [f.rule for f in result.reviews] == ["sql-unverified-call"]
 
 
+def test_pgdevkit_execute_is_a_sql_sink_for_find_injection(tmp_path: Path) -> None:
+    (tmp_path / "db.py").write_text(
+        "from pgdevkit.db import execute\n\n"
+        'async def a(x):\n    await execute(f"update t set a = {x}")\n\n'
+        "async def b(x):\n    await execute(build(x))\n"
+    )
+    (tmp_path / "other.py").write_text('from lib import execute\n\ndef a(x):\n    execute(f"update t set a = {x}")\n')
+    result = _run(tmp_path)
+    assert [(f.path.name, f.rule) for f in result.errors] == [("db.py", "sql-fstring-injection")]
+    assert [(f.path.name, f.rule) for f in result.reviews] == [("db.py", "sql-unverified-call")]
+
+
+def test_arrowbricks_calls_are_sql_sinks_for_find_injection(tmp_path: Path) -> None:
+    (tmp_path / "ab.py").write_text(
+        "from arrowbricks import unsafe_sql\n\n"
+        'async def a(client, x):\n    await client.query_agent(f"select * from t where id = {x}")\n\n'
+        "async def b(client, x):\n    await client.query_agent(build(x))\n\n"
+        "async def c(client, request_sql):\n    await client.query_agent(unsafe_sql(request_sql))\n\n"
+        'async def d(client, x):\n    await client.query_agent(unsafe_sql(f"select * from t where id = {x}"))\n\n'
+        'async def e(client):\n    await client.query_agent(unsafe_sql("select 1"))\n    await client.query_agent(t"select {1}")\n'
+    )
+    result = _run(tmp_path)
+    assert [(f.rule, f.line) for f in result.errors] == [("sql-fstring-injection", 4), ("sql-fstring-injection", 13)]
+    assert [(f.rule, f.line) for f in result.reviews] == [("sql-unverified-call", 7), ("sql-unverified-cast", 10)]
+
+
 def test_tests_and_missing_paths(tmp_path: Path) -> None:
     (tmp_path / "tests").mkdir()
     (tmp_path / "tests" / "test_x.py").write_text("eval(x)\n")