execsql2 2.27.0__tar.gz → 2.28.0__tar.gz

{execsql2-2.27.0 → execsql2-2.28.0}/CHANGELOG.md

@@ -13,6 +13,29 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.28.0] - 2026-10-13
+
+### Added
+
+- **`execsql lsp --check`** reports whether the language server can start (exit 0) or what is missing (exit 1).
+
+### Changed
+
+- **Missing-package messages** (database drivers, spreadsheet, YAML and template exports, `PG_UPSERT`, the formatter and the language server) name the execsql2 extra and show how to add it while keeping the extras already installed.
+
+### Fixed
+
+- **`execsql format` keeps optimizer hints and MySQL executable comments:** a statement holding `/*+ ... */` or `/*! ... */` is left as written in both modes. It was rewritten to `/* + ... */` and the hint moved off its keyword, so MySQL skipped the `/*! */` SQL (for example a `WHERE` or `PARTITION BY`) and Oracle/MySQL ignored the hint; re-check files the formatter or its pre-commit hook rewrote.
+- **`execsql format` no longer ends a final statement with a `;` the script did not have:** a script execsql refused for an unfinished last statement became one that ran it.
+- **`execsql format`:** a statement kept as written after another on the same line no longer starts with a space.
+- **`execsql format` without sqlglot** printed its `pip install` hint without the `[formatter]` extra.
+- **`execsql lint` and the language server read a `SUB_INI` file once** however many lines name it; a script naming a large INI file on many lines took minutes to lint and froze the editor. V002 for a variable a `SUB_INI` defines is reported at the `SUB_INI` line, not line 0.
+- **`execsql lsp` with an incompatible pygls** names the installed pygls version instead of saying the `lsp` extra is missing.
+- **`COPY` keeps binary columns** (SQLite/DuckDB BLOB, PostgreSQL `bytea`) byte for byte, and `COPY ... TO NEW` creates them as binary columns. They were copied as escaped text (`\x89PNG...`), and into PostgreSQL as text.
+- **DuckDB `TO NEW`** is no longer refused because a table of the same name exists in an `ATTACH`ed database; only the current database counts.
+
+______________________________________________________________________
+
 ## [2.27.0] - 2026-10-12
 
 ### Added

{execsql2-2.27.0 → execsql2-2.28.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: execsql2
-Version: 2.27.0
+Version: 2.28.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Homepage, https://execsql2.readthedocs.io
 Project-URL: Repository, https://github.com/geocoug/execsql
@@ -95,8 +95,10 @@ Requires-Dist: openpyxl>=3.1; extra == 'dev'
 Requires-Dist: polars>=1.0; extra == 'dev'
 Requires-Dist: pre-commit>=3.5.0; extra == 'dev'
 Requires-Dist: pygls<3,>=2.1; extra == 'dev'
+Requires-Dist: pytest-asyncio>=1.0; extra == 'dev'
 Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
 Requires-Dist: pytest-lsp>=1.0; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
 Requires-Dist: pyyaml>=6.0; extra == 'dev'
 Requires-Dist: ruff>=0.4; extra == 'dev'
 Requires-Dist: sqlglot<30.22,>=30.21.0; extra == 'dev'
@@ -464,7 +466,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.27.0
+    rev: v2.28.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.27.0 → execsql2-2.28.0}/README.md

@@ -321,7 +321,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.27.0
+    rev: v2.28.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.27.0 → execsql2-2.28.0}/docs/about/divergence.md

@@ -286,6 +286,7 @@ The legacy flat command-list engine (`_parse_script_lines` / `runscripts` / `Com
 - **Exception hierarchy** — All custom exceptions inherit from `ExecSqlError`, enabling `except ExecSqlError` to catch any execsql-originated error.
 - **Exception chaining** — All `raise` statements inside `except` blocks preserve the original traceback via `from`.
 - **Database and driver error messages** — End with the error itself (`OperationalError: table t already exists`). Upstream appended "in <file> on line N of execsql", a line of its own source; in the package the file was often a driver's and the line number another file's. Where in execsql the error was raised goes to `execsql.log` as `Raised at: execsql/<module>.py:<line>`.
+- **Missing optional package messages** — Name the execsql2 extra that installs the package and how to add it to a `uv tool` or `pipx` install without dropping the extras already there (`uv tool install "execsql2[...,postgres]"`). Upstream pointed at the package's web page.
 - **Config file errors** — An invalid value in `execsql.conf` stops a run with `Configuration error: <file>: <problem>` (and the line, for an unreadable file) plus a pointer to `execsql config --validate`. Upstream printed `Configuration error on line N of execsql.py`, where N was a line of its own source. A duplicate key, a missing section header or a bare `%` is reported the same way; upstream failed with an uncaught exception. Errors raised by `ConfigData` for a file are `ConfigFileError`, a subclass of `ConfigError`.
 - **ASSERT error type** — `ASSERT` failures now use a dedicated `"assert"` error type that produces `**** Assertion failed.` instead of `**** Error in metacommand.`. This distinguishes intentional script-level checks from actual metacommand errors. Upstream did not have `ASSERT`.
 
@@ -360,10 +361,11 @@ These are behavioral changes driven by security or correctness issues in the ups
 | `DT_Timestamp` claims time-only values                             | `dateutil.parser.parse()` silently fills in today's date for bare time strings like `"13:15:45"`, so `DT_Timestamp` matched before `DT_Time` in the inference order. `parse_datetime()` now rejects time-only strings.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
 | Partial dates inferred as timestamps                               | `dateutil` fills fields missing from a value with today's date, so `1-2`, `5/6`, `March 5`, `Jan` and `Monday` were inferred as timestamps and, on DuckDB, SQL Server, MySQL, Oracle, Firebird, Access and every ODS/XLSX/JSON/Parquet/Feather import, stored as dates in the current year. Inference now requires a four-digit year (or a month-first `m/d/yy` or `m-d-yy`) and a month and day, which restores upstream's behaviour for these values. `m-d-yy` is accepted in addition to upstream's `m/d/yy`.                                                                                                                                                                                                                                                                                                   |
 | Offset values stored as plain timestamps                           | With `dateutil`, a value with a UTC offset (`2024-01-01 10:00+05:00`) also counted as a plain timestamp, so a column mixing offset and plain values was created as `timestamp` and the driver converted the offset values with the client's time zone: the stored value depended on the machine that ran the import. A value with an offset is again not a plain timestamp, as with upstream's offset-free format list, so a mixed column is text and keeps each value as written. A column where every value has an offset is still `timestamptz`.                                                                                                                                                                                                                                                                |
+| Binary columns in `COPY`                                           | Every value a source cursor returned as bytes was decoded to text (undecodable bytes as `\xNN` escapes) before `COPY` wrote it, and `TO NEW` typed the column as text. Upstream did the same for SQLite BLOBs; with psycopg2 a PostgreSQL `bytea` arrived as a `memoryview` and escaped the decode, so for PostgreSQL this was an execsql2 regression. `COPY` now moves bytes unchanged and creates binary columns. Exports still write the decoded text.                                                                                                                                                                                                                                                                                                                                                          |
 | Short data lines on import                                         | A line with fewer values than the header was padded with NULLs only on PostgreSQL and MySQL. SQLite stopped with "Too few values", DuckDB and the other DBMSs with a driver parameter-count error. With `TO NEW` / `REPLACEMENT`, type inference skipped the missing values, so the column could be created `NOT NULL` and then reject the padded line, on PostgreSQL too. Missing trailing values are now NULL on every DBMS, and inference counts them as NULL.                                                                                                                                                                                                                                                                                                                                                  |
 | `COPY` re-typed text values                                        | Upstream described the destination table by re-reading the source's text values as if they came from a file, so a text column of `1`/`0` became `boolean` and one of dates became `timestamp` with `NEW` / `REPLACEMENT`, and values copied into an existing text column were converted on the way in (`1` stored as `true`). Text from the source database now stays text. Upstream also ran the source query twice, once to describe the table and once to copy it; execsql2 runs it once and holds the rows in a temporary file in between.                                                                                                                                                                                                                                                                     |
 | `REPLACEMENT` of a quoted or schema-qualified table                | The SQLite, SQL Server, Oracle, Firebird and Access adapters quoted the table name a second time before dropping it, so `IMPORT` / `COPY ... TO REPLACEMENT "my tbl"` dropped a table named `"my tbl"` (quotes included) that did not exist and then failed to create the table, and on SQL Server, Oracle and Firebird `REPLACEMENT staging.t` tried to drop one table named `staging.t`. The name is now quoted once.                                                                                                                                                                                                                                                                                                                                                                                            |
-| `TO NEW` onto an existing table                                    | `COPY`'s checks that the source exists and that a `NEW` target does not were dead code: each raised its error inside a `try` that discarded it. `IMPORT ... TO NEW` had no check. Both now stop with `Table <name> already exists` before any DDL (when the database cannot say whether the table exists, the command goes ahead). A name without a schema counts only in the schema the table would be created in (MySQL: the current database). The source check was removed instead of revived: PostgreSQL reads an unquoted `COPY Orders` as `orders`, which the check would have reported missing.                                                                                                                                                                                                            |
+| `TO NEW` onto an existing table                                    | `COPY`'s checks that the source exists and that a `NEW` target does not were dead code: each raised its error inside a `try` that discarded it. `IMPORT ... TO NEW` had no check. Both now stop with `Table <name> already exists` before any DDL (when the database cannot say whether the table exists, the command goes ahead). A name without a schema counts only in the schema the table would be created in (MySQL: the current database), and on DuckDB only in the current database, not an `ATTACH`ed one. The source check was removed instead of revived: PostgreSQL reads an unquoted `COPY Orders` as `orders`, which the check would have reported missing.                                                                                                                                         |
 | No `exit` log record on error or halt                              | The log was closed before the `exit` record was written, so a run that ended with an error, a `HALT` or a cancelled prompt left no `exit` line, and Ctrl-C was recorded as `unknown`. Every run now writes exactly one `exit` record; Ctrl-C is `interrupted`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
 | SIGTERM lost the run's log                                         | Python's default SIGTERM action ends the process without running exit handlers, so the buffered `execsql.log` (every record of the run) was never written, the transaction was not rolled back explicitly and a running `SYSTEM_CMD` kept going. The `execsql` command now turns SIGTERM into a normal exit with status 143 and an `exit` record of type `terminated`. `execsql.run()` does not install a handler.                                                                                                                                                                                                                                                                                                                                                                                                 |
 | `access_use_numeric` ignored                                       | The config option was read but never applied, so MS-Access tables created by `IMPORT` / `COPY` always got `DOUBLE` for decimal data. With `Yes` they get `NUMERIC(precision,scale)` again, as upstream. The setting is read when each table is created, so it applies per run (upstream changed the shared type map once at startup).                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |

{execsql2-2.27.0 → execsql2-2.28.0}/docs/getting-started/syntax.md

@@ -300,8 +300,9 @@ The options below belong to `run`: `execsql run -tl script.sql mydb.sqlite`.
 `--version` and `-o`/`--online-help` work on their own (`execsql --version`)
 and anywhere on a `run` command line.
 `format` has its own options, described in the [formatter guide](../guides/formatter.md);
-`lint`'s are in the [lint rules reference](../reference/lint.md); `config`, `list` and `init`
-are described [above](#config_command).
+`lint`'s are in the [lint rules reference](../reference/lint.md); `lsp`'s (`--check`) in
+[Editor Support](../guides/editors.md#install); `config`, `list` and `init` are described
+[above](#config_command).
 
 ### Connection options
 

{execsql2-2.27.0 → execsql2-2.28.0}/docs/guides/editors.md

@@ -123,9 +123,8 @@ pipx install --force "execsql2[postgres,lsp]"
 ```
 
 See [Adding an extra later](../getting-started/installation.md#adding-an-extra-later).
-Check it with `execsql lsp < /dev/null` (on Windows, `execsql lsp < NUL`): it
-exits at once without output when the server is installed, and says the `lsp`
-extra is needed when it is not.
+Check it with `execsql lsp --check`: it prints `execsql lsp is ready (pygls 2.x.y)` and exits 0 when the server can start, and otherwise says what is
+missing and exits 1.
 
 ## Settings { #settings }
 
@@ -163,7 +162,8 @@ execsql is the metacommand [plugins](../dev/architecture.md#plugin-system) insta
 environment, so lint knows their keywords, as `execsql lint` does. Besides the
 open script and the configuration files, it reads the files the script names
 in `INCLUDE` (to find the variables they define: up to 200 files of at most
-10 MiB each) and in `SUB_INI` (at most 1 MiB). It reads only regular files,
+10 MiB each and 20 MiB together) and in `SUB_INI` (at most 1 MiB, read once
+however many lines name it). It reads only regular files,
 never a FIFO or a device.
 
 ## VS Code { #vscode }

{execsql2-2.27.0 → execsql2-2.28.0}/docs/guides/formatter.md

@@ -130,7 +130,7 @@ Conditions joined by `AND` / `OR` go on one line when their combined text is at
 
 #### Layout only, or rewrite { #layout-only }
 
-sqlglot does more than lay SQL out: it writes the SQL it parsed in its own preferred form. By default the formatter changes only layout and keyword case. A statement sqlglot would also rewrite is kept exactly as written, while the statements around it are still formatted.
+sqlglot does more than lay SQL out: it writes the SQL it parsed in its own preferred form. By default the formatter changes only layout and the case of keywords and unquoted names (a function written `myFunc(x)` becomes `MYFUNC(x)`; quoted names and strings keep their case). A statement sqlglot would also rewrite is kept exactly as written, while the statements around it are still formatted.
 
 | You wrote                   | sqlglot writes                        |
 | --------------------------- | ------------------------------------- |
@@ -149,6 +149,15 @@ rewrite_sql = Yes
 
 With it on, the rewrites a statement gets can change when the pinned sqlglot changes (see above). Turning it off later does not undo rewrites already written; it only stops new ones.
 
+In either mode, a statement holding an optimizer hint (`/*+ ... */`, Oracle and MySQL) or a MySQL executable comment (`/*! ... */`) is kept exactly as written: the server reads them as part of the statement, and sqlglot would turn them into ordinary comments or move a hint away from its keyword.
+
+```sql
+INSERT /*+ APPEND */ INTO sales_hist SELECT * FROM sales_stage;
+SELECT a FROM t /*!50001 WHERE a > 1 */;
+```
+
+The formatter never adds a `;` to a final statement that has none: execsql refuses to run a script that ends with an unfinished statement, and formatting leaves that for you to fix.
+
 #### Comment handling
 
 Comments interleaved within SQL statements (e.g. `--` comments between SELECT columns, or inside CASE expressions) are preserved through formatting using a marker-based round-trip:
@@ -261,7 +270,7 @@ select id,name,created_at from users where active = true order by name;
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.27.0
+    rev: v2.28.0
     hooks:
       - id: execsql-format
       - id: execsql-lint # optional: fail the commit on lint errors

{execsql2-2.27.0 → execsql2-2.28.0}/docs/reference/lint.md

@@ -158,7 +158,7 @@ The `execsql-lint` [pre-commit](https://pre-commit.com/) hook runs `execsql lint
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.27.0
+    rev: v2.28.0
     hooks:
       - id: execsql-lint
         args: [--ignore, V002] # optional: any execsql lint option

{execsql2-2.27.0 → execsql2-2.28.0}/extras/vscode-execsql/README.md

@@ -196,9 +196,8 @@ ______________________________________________________________________
 
 **No lint findings or completion**
 
-- Run `execsql lsp < /dev/null` in a terminal (on Windows: `execsql lsp < NUL`). It exits at once
-    without output when the server is installed; otherwise it says the `lsp` extra is needed. Add the
-    extra with the extras you already have, e.g. `uv tool install "execsql2[postgres,lsp]"`
+- Run `execsql lsp --check` in a terminal. It says `execsql lsp is ready` when the server can start;
+    otherwise it says what is missing. Add the extra with the extras you already have, e.g. `uv tool install "execsql2[postgres,lsp]"`
     ([Adding an extra later](https://execsql2.readthedocs.io/en/latest/getting-started/installation/#adding-an-extra-later)).
 - If `execsql` works in a terminal but not in VS Code, set `execsql.server.path` to its full path
     (`which execsql`).

{execsql2-2.27.0 → execsql2-2.28.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "execsql2"
-version = "2.27.0"
+version = "2.28.0"
 description = "Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables."
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { file = "LICENSE.txt" }
@@ -103,6 +103,8 @@ dev = [
     "mkdocstrings-python>=2.0.3",
     "mypy>=1.10",
     "pre-commit>=3.5.0",
+    "pytest>=8",
+    "pytest-asyncio>=1.0",
     "pytest-cov>=5.0.0",
     "pytest-lsp>=1.0",
     "ruff>=0.4",
@@ -187,7 +189,7 @@ line-ending = "auto"
 exclude = ["*.md"]
 
 [tool.bumpversion]
-current_version = "2.27.0"
+current_version = "2.28.0"
 commit = true
 tag = true
 tag_name = "v{new_version}"

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/cli/commands/format.py

@@ -5,6 +5,7 @@ from __future__ import annotations
 from pathlib import Path
 
 import typer
+from rich.markup import escape
 
 from execsql.cli.application import ExecsqlCommand, app
 from execsql.cli.help import _err_console
@@ -90,7 +91,7 @@ def format_cmd(
         try:
             _require_sqlglot()
         except ImportError as exc:
-            _err_console.print(f"[bold red]Error:[/bold red] {exc}", highlight=False)
+            _err_console.print(f"[bold red]Error:[/bold red] {escape(str(exc))}", highlight=False)
             raise typer.Exit(code=1) from exc
     raise typer.Exit(
         code=run_formatter(

execsql2-2.28.0/src/execsql/cli/commands/lsp.py

@@ -0,0 +1,64 @@
+"""The ``lsp`` command: the language server editors start for ``.sql`` files."""
+
+from __future__ import annotations
+
+import typer
+from rich.markup import escape
+
+from execsql.cli.application import ExecsqlCommand, app
+from execsql.cli.help import _err_console
+from execsql.utils.errors import install_hint
+
+__all__ = ["lsp_cmd"]
+
+
+def _lsp_libraries_installed() -> bool:
+    """Whether pygls and lsprotocol are installed (whether or not this execsql can use them)."""
+    from importlib.util import find_spec
+
+    return all(find_spec(name) is not None for name in ("pygls", "lsprotocol"))
+
+
+def _pygls_version() -> str:
+    from importlib.metadata import PackageNotFoundError, version
+
+    try:
+        return version("pygls")
+    except PackageNotFoundError:
+        return "unknown"
+
+
+@app.command(
+    cls=ExecsqlCommand,
+    name="lsp",
+    help=(
+        "Run the execsql language server on stdin/stdout, for an editor to start. "
+        "Shows execsql lint findings as you type, completes metacommands, "
+        "conditional tests and variables, explains them on hover, and finds where variables, "
+        "SCRIPTs and included files are defined, offers quick fixes, and formats documents. Requires the lsp "
+        "extra; --check reports whether it is installed."
+    ),
+)
+def lsp_cmd(
+    # Editors' language clients pass --stdio (VS Code always does); stdio is the only transport.
+    stdio: bool = typer.Option(False, "--stdio", hidden=True),
+    check: bool = typer.Option(False, "--check", help="Report whether the language server can start, and exit."),
+) -> None:
+    """The ``lsp`` command; its user-facing help is the decorator's ``help``."""
+    try:
+        from execsql.lsp.server import create_server
+    except ImportError as exc:
+        if _lsp_libraries_installed():
+            # pygls is there but this execsql cannot use it (another major version, a broken install).
+            message = (
+                f"execsql lsp cannot use the installed pygls {_pygls_version()} ({exc}). "
+                f"It needs pygls 2.x, which the lsp extra installs. {install_hint('lsp')}"
+            )
+        else:
+            message = f"execsql lsp needs the lsp extra. {install_hint('lsp')}"
+        _err_console.print(f"[bold red]Error:[/bold red] {escape(message)}", highlight=False)
+        raise typer.Exit(code=1) from exc
+    if check:
+        typer.echo(f"execsql lsp is ready (pygls {_pygls_version()}).")
+        return
+    create_server().start_io()

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/cli/lint.py

@@ -48,6 +48,7 @@ Public surface:
 
 from __future__ import annotations
 
+import functools
 import json
 import re
 from collections import Counter
@@ -460,23 +461,40 @@ def _read_ini_vars(
     language server; a file that is unreadable or not valid INI defines
     nothing.
     """
-    from configparser import ConfigParser
-    from configparser import Error as ConfigError
-
     p = Path(ini_file)
     if not p.is_absolute() and script_dir is not None:
         p = script_dir / p
+    try:
+        if not p.is_file():
+            return
+        stat = p.stat()
+        if stat.st_size > _MAX_INI_BYTES:
+            return
+        resolved = str(p.resolve())
+    except OSError:
+        return
+    defined_vars.update(_ini_section_keys(resolved, stat.st_size, stat.st_mtime_ns, section))
+
+
+@functools.lru_cache(maxsize=64)
+def _ini_section_keys(path: str, size: int, mtime_ns: int, section: str) -> frozenset[str]:
+    """The upper-cased keys of *section* in the INI file at *path*.
+
+    Cached by the file's size and modification time: a script that names
+    the same file on many SUB_INI lines, and the language server re-linting
+    on every edit, read it once.
+    """
+    from configparser import ConfigParser
+    from configparser import Error as ConfigError
 
     cp = ConfigParser()
     try:
-        if not p.is_file() or p.stat().st_size > _MAX_INI_BYTES:
-            return
-        cp.read_string(p.read_text(encoding="utf-8"), source=str(p))
+        cp.read_string(Path(path).read_text(encoding="utf-8"), source=path)
     except (OSError, UnicodeDecodeError, ConfigError):
-        return
-    if cp.has_section(section):
-        for key, _value in cp.items(section):
-            defined_vars.add(key.upper())
+        return frozenset()
+    if not cp.has_section(section):
+        return frozenset()
+    return frozenset(key.upper() for key, _value in cp.items(section))
 
 
 def _is_known_var(raw_name: str, defined_vars: set[str]) -> bool:
@@ -834,7 +852,12 @@ def _referencing_text(node: Node) -> list[str]:
     return texts
 
 
-def _check_unused_variables(script: Script, defined: set[str], issues: list[_Issue]) -> None:
+def _check_unused_variables(
+    script: Script,
+    defined: set[str],
+    issues: list[_Issue],
+    script_dir: Path | None = None,
+) -> None:
     """Report variables a SUB defines that nothing ever reads.
 
     Almost always a typo in either the definition or the reference — the two
@@ -845,8 +868,11 @@ def _check_unused_variables(script: Script, defined: set[str], issues: list[_Iss
     referenced: set[str] = set()
     _collect_var_references(script.body, referenced)
     unused = sorted(defined - referenced)
+    lines: dict[str, int] = {}
+    if unused:
+        _definition_lines(script.body, script_dir, lines)
     for name in unused:
-        line = _definition_line(script.body, name)
+        line = lines.get(name, 0)
         issues.append(
             _issue(
                 "V002",
@@ -857,18 +883,15 @@ def _check_unused_variables(script: Script, defined: set[str], issues: list[_Iss
         )
 
 
-def _definition_line(nodes: list[Node], name: str) -> int:
-    """Line of the SUB that defines *name*, or 0 when it cannot be located."""
+def _definition_lines(nodes: list[Node], script_dir: Path | None, lines: dict[str, int]) -> None:
+    """Add to *lines* the line of the first metacommand that defines each variable, in script order."""
     for node in nodes:
         if isinstance(node, MetaCommandStatement):
             probe: set[str] = set()
-            _extract_var_definition(node.command, None, probe)
-            if name in probe:
-                return node.span.start_line
-        found = _definition_line(list(node.children()), name)
-        if found:
-            return found
-    return 0
+            _extract_var_definition(node.command, script_dir, probe)
+            for name in probe:
+                lines.setdefault(name, node.span.start_line)
+        _definition_lines(list(node.children()), script_dir, lines)
 
 
 # ---------------------------------------------------------------------------
@@ -1008,7 +1031,7 @@ def lint(
     )
 
     # Pass 3: whole-script rules, which need every reference collected first
-    _check_unused_variables(script, all_defined, issues)
+    _check_unused_variables(script, all_defined, issues, script_dir)
 
     return issues
 

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/access.py

@@ -17,7 +17,7 @@ from typing import Any, cast
 from execsql.db.base import Database, _fetched
 from execsql.db.tiers import SupportTier
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 from execsql.utils.auth import (
     begin_login,
     clear_stored_password,
@@ -68,7 +68,7 @@ class AccessDatabase(Database):
         try:
             import pyodbc  # noqa: F401
         except Exception:
-            fatal_error("The pyodbc module is required.  See http://github.com/mkleehammer/pyodbc")
+            fatal_error(f"The pyodbc module is required to connect to Access. {install_hint('odbc')}")
         from execsql.types import dbt_access
 
         super().__init__(
@@ -327,8 +327,8 @@ class AccessDatabase(Database):
             rows = curs.fetchall()
             return [d[0] for d in curs.description], rows
 
-    def select_rowsource(self, sql: str) -> tuple[list[str], Any]:
-        """Return column names and an iterable that yields rows one at a time."""
+    def select_rowsource(self, sql: str, *, decode_bytes: bool = True) -> tuple[list[str], Any]:  # noqa: ARG002
+        """Return column names and an iterable that yields rows one at a time (values as the driver returns them)."""
         # Return 1) a list of column names, and 2) an iterable that yields rows.
         self.dao_flush_check()
         curs = self.cursor()

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/base.py

@@ -405,8 +405,13 @@ class Database(ABC):
             rows = curs.fetchall()
             return [d[0] for d in curs.description], rows
 
-    def select_rowsource(self, sql: str) -> tuple[list[str], Generator]:
-        """Execute *sql* and return ``(column_names, row_generator)`` for streaming large result sets."""
+    def select_rowsource(self, sql: str, *, decode_bytes: bool = True) -> tuple[list[str], Generator]:
+        """Execute *sql* and return ``(column_names, row_generator)`` for streaming large result sets.
+
+        A binary value (``bytes``) is decoded to text with the connection's
+        encoding, as the exporters expect, unless *decode_bytes* is false:
+        COPY passes false so binary columns reach the target unchanged.
+        """
         # Return 1) a list of column names, and 2) an iterable that yields rows.
         curs = self.cursor()
         try:
@@ -433,7 +438,7 @@ class Database(ABC):
                         break
                     else:
                         for row in rows:
-                            if self.encoding:
+                            if self.encoding and decode_bytes:
                                 yield [
                                     c.decode(self.encoding, "backslashreplace") if isinstance(c, bytes) else c
                                     for c in row
@@ -501,6 +506,15 @@ class Database(ABC):
         """The schema (MySQL: database) an unqualified table name is created in, or ``None`` if unknown."""
         return None
 
+    def table_exists_where_created(self, table_name: str, schema_name: str | None = None) -> bool:
+        """Whether *table_name* exists where a ``CREATE TABLE`` on this connection would put it.
+
+        The same as :meth:`table_exists`, except on a connection that sees
+        tables of other databases (DuckDB ``ATTACH``): only the current
+        database counts there.
+        """
+        return self.table_exists(table_name, schema_name)
+
     def _query_current_schema(self, sql: str) -> str | None:
         with self._cursor() as curs:
             curs.execute(sql)

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/dsn.py

@@ -12,7 +12,7 @@ the CLI.
 from execsql.db.base import Database
 from execsql.db.tiers import SupportTier
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 from execsql.utils.auth import (
     begin_login,
     clear_stored_password,
@@ -51,7 +51,7 @@ class DsnDatabase(Database):
         try:
             import pyodbc  # noqa: F401
         except Exception:
-            fatal_error("The pyodbc module is required.  See http://github.com/mkleehammer/pyodbc")
+            fatal_error(f"The pyodbc module is required to connect through an ODBC DSN. {install_hint('odbc')}")
         from execsql.types import dbt_dsn
 
         super().__init__(

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/duckdb.py

@@ -14,7 +14,7 @@ from typing import Any
 
 from execsql.db.base import Database, statement_start
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 import execsql.state as _state
 
 __all__ = ["DuckDBDatabase"]
@@ -62,7 +62,7 @@ class DuckDBDatabase(Database):
         try:
             import duckdb  # noqa: F401
         except Exception:
-            fatal_error("The duckdb module is required.")
+            fatal_error(f"The duckdb module is required to use DuckDB. {install_hint('duckdb')}")
         from execsql.types import dbt_duckdb
 
         super().__init__(
@@ -167,6 +167,18 @@ class DuckDBDatabase(Database):
                 self.rollback()
                 raise
 
+    def table_exists_where_created(self, table_name: str, schema_name: str | None = None) -> bool:
+        """Whether *table_name* exists in the current database (not an ``ATTACH``ed one)."""
+        # information_schema.tables lists the tables of every attached database.
+        sql = "select table_name from information_schema.tables where table_name = ? and table_catalog = current_database()"
+        params: list = [table_name]
+        if schema_name:
+            sql += " and table_schema = ?"
+            params.append(schema_name)
+        with self._cursor() as curs:
+            curs.execute(sql, params)
+            return len(curs.fetchall()) > 0
+
     def view_exists(self, view_name: str, schema_name: str | None = None) -> bool:
         """Return True if the named view exists in the DuckDB database."""
         # DuckDB information_schema has no 'views' table; views are listed in 'tables'

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/firebird.py

@@ -11,7 +11,7 @@ via the ``firebird-driver`` package.  Corresponds to ``-t f`` on the CLI.
 from execsql.db.base import Database
 from execsql.db.tiers import SupportTier
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 from execsql.utils.auth import (
     begin_login,
     clear_stored_password,
@@ -46,8 +46,7 @@ class FirebirdDatabase(Database):
             from firebird import driver as firebird_lib  # noqa: F401
         except Exception:
             fatal_error(
-                "The firebird-driver module is required to connect to Firebird.   "
-                "See https://pypi.org/project/firebird-driver/",
+                f"The firebird-driver module is required to connect to Firebird. {install_hint('firebird')}",
             )
         from execsql.types import dbt_firebird
 

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/mysql.py

@@ -12,7 +12,7 @@ from typing import Any, cast
 
 from execsql.db.base import Database
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 from execsql.utils.auth import (
     begin_login,
     clear_stored_password,
@@ -81,7 +81,7 @@ class MySQLDatabase(Database):
             import pymysql as mysql_lib  # noqa: F401
         except Exception:
             fatal_error(
-                "The pymysql module is required to connect to MySQL.   See https://pypi.python.org/pypi/PyMySQL",
+                f"The pymysql module is required to connect to MySQL. {install_hint('mysql')}",
             )
         from execsql.types import dbt_mysql
 

{execsql2-2.27.0 → execsql2-2.28.0}/src/execsql/db/oracle.py

@@ -14,7 +14,7 @@ from typing import Any
 from execsql.db.base import Database
 from execsql.db.tiers import SupportTier
 from execsql.exceptions import ErrInfo
-from execsql.utils.errors import exception_desc, fatal_error
+from execsql.utils.errors import exception_desc, fatal_error, install_hint
 from execsql.utils.auth import (
     begin_login,
     clear_stored_password,
@@ -76,7 +76,7 @@ class OracleDatabase(Database):
     ) -> None:
         if _oracle_driver() is None:
             fatal_error(
-                'The oracledb module is required to connect to Oracle. Install it with: pip install "execsql2[oracle]"',
+                f"The oracledb module is required to connect to Oracle. {install_hint('oracle')}",
             )
         from execsql.types import dbt_oracle
 
@@ -164,12 +164,12 @@ class OracleDatabase(Database):
         else:
             return super().select_data(sql)
 
-    def select_rowsource(self, sql: str) -> Any:
+    def select_rowsource(self, sql: str, *, decode_bytes: bool = True) -> Any:
         """Return column names and an iterable that yields rows one at a time."""
         if sql[-1:] == ";":
-            return super().select_rowsource(sql[:-1])
+            return super().select_rowsource(sql[:-1], decode_bytes=decode_bytes)
         else:
-            return super().select_rowsource(sql)
+            return super().select_rowsource(sql, decode_bytes=decode_bytes)
 
     def select_rowdict(self, sql: str) -> Any:
         """Return column names and an iterable that yields rows as dictionaries."""