execsql2 2.34.0__tar.gz → 2.37.0__tar.gz

execsql2-2.37.0/.gitattributes

@@ -0,0 +1,4 @@
+# Corpus inputs are compared byte for byte with upstream's recorded results
+# (IMPORT_FILE stores a file's raw bytes), so Git must not convert their line
+# endings on checkout.
+tests/corpus/cases/** -text

{execsql2-2.34.0 → execsql2-2.37.0}/CHANGELOG.md

@@ -13,6 +13,68 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.37.0] - 2026-10-23
+
+### Changed
+
+- **A failed commit at `END BATCH`** is an SQL error on the `END BATCH` line, which `ERROR_HALT` governs; it was reported as an uncaught exception on a line of execsql's own source and always stopped the run. With `ERROR_HALT OFF` the script now carries on: test `SQL_ERROR()` right after `END BATCH`.
+- **`SQL_ERROR()` right after `END BATCH`** says whether the batch's commit failed: `END BATCH` clears it first, as an SQL statement does. It kept the value left by the last statement run before it.
+- **Anything that fails inside `BEGIN BATCH`**, an SQL statement, a metacommand or a metacommand's SQL, keeps the batch from being committed: `END BATCH` rolls it back and reports an SQL error, which stops the run if `ERROR_HALT` is `ON` there. The batch's statements after a failure were committed, even when a failed statement had already rolled back the ones before it. Put a statement or metacommand whose failure the script handles outside the batch.
+- **A batch around one whose commit failed** is not committed at its own `END BATCH`, nor is any batch opened inside it afterwards, and `ROLLBACK BATCH` does not undo this; put work that must survive after the outermost `END BATCH`.
+- **`DISCONNECT` inside `BEGIN BATCH`** refuses a database the batch has used, as documented: the run halts and the batch is rolled back, or with `METACOMMAND_ERROR_HALT OFF` the connection stays open and `END BATCH` rolls the batch back; it closed the connection and the batch's statements were lost. Put the `DISCONNECT` after `END BATCH`.
+- **`DISCONNECT FROM <alias>`** matches the alias in any case, as `USE` does; `DISCONNECT FROM B` left a connection made `AS b` open, so a later `USE b` worked. Reconnect with `CONNECT` before using an alias you disconnected.
+- **`DISCONNECT FROM` an alias that is not connected** is a metacommand error; it did nothing, so a misspelled alias left the real connection open. To disconnect only if connected, write `IF (ALIAS_DEFINED(staging)) { DISCONNECT FROM staging }`, with the alias in lowercase inside `ALIAS_DEFINED`.
+- **`BREAK` in a `SCRIPT` or `INCLUDE`d file, outside a `LOOP`,** returns from that script or file, as the `BREAK` documentation says and upstream execsql did; it stopped the run. A `BREAK` in an `INCLUDE`d file inside a `LOOP` now leaves only the file, and the loop carries on; it ended the loop.
+- **`BREAK` in the main script, outside any `LOOP`, `SCRIPT` or `INCLUDE`d file,** is a metacommand error: with `METACOMMAND_ERROR_HALT OFF` the script carries on and `METACOMMAND_ERROR()` is true. It stopped the run whatever that setting said.
+- **A `BEGIN BATCH` block left before its `END BATCH`** by `BREAK`, an error or `HALT` is rolled back; it committed the statements run so far. To keep a batch's work and then `BREAK`, put the `BREAK` after `END BATCH`. A nested batch left by `BREAK` while the outer batch carries on is now committed or rolled back with the outer batch; it was committed at once. Under `AUTOCOMMIT OFF` your own `COMMIT` or `ROLLBACK` still decides, as before.
+- **An `ON ERROR_HALT` or `ON CANCEL_HALT` script after a halt inside `BEGIN BATCH`** runs after the batch is rolled back and still commits its own SQL, such as a row it writes to a log table (unless `AUTOCOMMIT` is `OFF`).
+- **`--manifest`** counts each `INCLUDE` and `EXECUTE SCRIPT` in `statements.metacommands`, including one that fails or that `IF EXISTS` skips.
+
+### Fixed
+
+- **A commit that fails** (a deferred constraint checked at COMMIT) rolls back what it was committing, so none of it is stored: an SQL statement, `IMPORT`, `COPY`, `EXECUTE`, or `AUTOCOMMIT ON` committing what was held under `AUTOCOMMIT OFF`. On SQLite, when the script carried on (`ERROR_HALT OFF` for an SQL statement, `METACOMMAND_ERROR_HALT OFF` for a metacommand, or an `ON ERROR_HALT` script), the next statement's commit stored it, including the rows reported as failed.
+- **SIGTERM while `SYSTEM_CMD` runs a command** stops the command just as it starts, as its `system_cmd_timeout` runs out, and under the Textual console (`--gui-framework textual`), where the script also stops at that `SYSTEM_CMD`; the command was left running after execsql exited.
+- **`system_cmd_timeout` running out on a command *execsql* cannot stop** (one run as another user, through `sudo` for example) fails the `SYSTEM_CMD` at once, with `$SYSTEM_CMD_EXIT_STATUS` 124 and an error giving the command's process ID; execsql waited for the command to finish and then stopped with a permission error.
+- **`END BATCH` whose commit fails on one database** rolls back the batch's databases not yet committed; a later statement on them, such as an `ON ERROR_HALT` script's, committed the failed batch's rows.
+- **`BREAK` in the main script, outside any `LOOP`, `SCRIPT` or `INCLUDE`d file,** is reported on its own line; it named the statement before it.
+- **`INCLUDE` and `EXECUTE SCRIPT` inside an inline `IF (…) { … }`** run again; they stopped the run with "should be handled by the AST executor".
+- **`INCLUDE`, `EXECUTE SCRIPT` and an inline `IF` added with `EXTEND SCRIPT … WITH METACOMMAND`** run when the script does; they stopped the run the same way.
+- **An `INCLUDE` or `EXECUTE SCRIPT` that a substitution variable supplies** (`-- !x! !!cmd!!`) says it cannot run and to write it in the script; it reported "should be handled by the AST executor".
+- **`execsql lint` and the language server** check and link `INCLUDE` and `EXECUTE SCRIPT` inside an inline `IF`.
+- **`execsql run --dry-run`** lists `INCLUDE` and `EXECUTE SCRIPT`.
+- **`execsql lint`** checks the variables in an `INCLUDE` file name and in `EXECUTE SCRIPT` arguments and loop conditions (V001, V002), and no longer reports an `INCLUDE` inside `IF (FILE_EXISTS(file))` as a missing file (I001).
+- **`INCLUDE` of a file that does not exist, `EXECUTE SCRIPT` of a script that does not exist, and `EXECUTE SCRIPT` without a required argument** are metacommand errors: with `METACOMMAND_ERROR_HALT OFF` the script carries on and `METACOMMAND_ERROR()`, `$LAST_ERROR` and `$ERROR_MESSAGE` say what failed. They stopped the run whatever that setting said.
+- **Error reports name the metacommand as written** (`Metacommand: INCLUDE missing.sql`), without `-- !x!`; a failed `INCLUDE`, `EXECUTE SCRIPT`, `IF`, `ELSEIF` or `LOOP` was reported with an empty `Metacommand:` line.
+- **An `IF`, `ELSEIF`, `ANDIF`, `ORIF` or `LOOP` condition that cannot be evaluated** is reported on its own line with its text (`Line 3 … Metacommand: ANDIF (NO_SUCH_FN(x))`), not as an "Uncaught exception" on another statement's line. It still stops the run.
+- **`WRITE SCRIPT`** writes an `EXECUTE SCRIPT … WHILE` / `UNTIL` as written; it wrote `LOOP WHILE`, which does not read back as `EXECUTE SCRIPT`.
+- **`--profile`** shows an `EXECUTE SCRIPT` or `INCLUDE` as written, with its `IF EXISTS`, arguments and `WHILE` / `UNTIL`.
+
+______________________________________________________________________
+
+## [2.36.0] - 2026-10-19
+
+### Changed
+
+- **`execsql.run()` from a notebook, the REPL or `python -c`** runs `IMPORT` and `COPY` about four times as fast: reading execsql's settings there cost about 12 µs each time, now 0.1 µs (a 200,000-row column scan went from 3.5 s to 0.8 s). The `execsql` command gets a smaller gain.
+
+______________________________________________________________________
+
+## [2.35.0] - 2026-10-18
+
+### Added
+
+- **`exact_decimals`** (`[input]`) and **`CONFIG EXACT_DECIMALS YES|NO`**: with Yes, `IMPORT` and `COPY … TO NEW` give a column of decimals with different numbers of places an exact `DECIMAL(p,s)` type instead of a floating-point one.
+
+### Changed
+
+- **A column of decimals with different numbers of places** (`1.50`, `2`, `-3.25`) gets a floating-point type again when `IMPORT` or `COPY … TO NEW` creates a table (it was `DECIMAL(p,s)` from 2.22.0). A column with a value of more than 15 significant digits stays `DECIMAL(p,s)`, so no digit is lost. Set `exact_decimals` to keep every such column exact.
+
+### Fixed
+
+- **The "potential un-substituted variable" warning** is printed only when `write_warnings` is on, to stderr as `**** Warning …`, as documented; it was printed to stdout for every command with an undefined `!!var!!`. It is always written to `execsql.log`.
+
+______________________________________________________________________
+
 ## [2.34.0] - 2026-10-18
 
 ### Added

{execsql2-2.34.0 → execsql2-2.37.0}/CONTRIBUTING.md

@@ -64,6 +64,19 @@ just test-templates-down   # removes the containers
 
 SQL Server is included only when an ODBC driver for it is installed. A template bug the tests have found but not yet fixed is marked with `known_bug(...)`, a strict expected failure: the fix makes the test pass, which fails the run until the marker is removed with it.
 
+### Script behavior corpus
+
+`tests/corpus/` checks that whole scripts do under execsql2 what they did under upstream execsql 1.130.1: the same exit status, console output, files written and table contents. Each case is a directory in `tests/corpus/cases/` holding `script.sql` and the files it reads. Upstream's results are recorded in `tests/corpus/results/`, so the normal test run compares against them without upstream installed. A variable a case checks is `WRITE`n to a file, because upstream has no way to read variables after a run.
+
+To add a case, create its directory, then record what upstream does (this installs execsql 1.130.1 from PyPI under Python 3.12 into `~/.cache/execsql-corpus/` on first use):
+
+```bash
+uv run python -m tests.corpus.record my_new_case
+uv run pytest tests/corpus
+```
+
+Use only syntax upstream accepts; read `_execsql/execsql.py` when unsure. A case must not open a GUI dialog (`HALT "message"`, `PROMPT`) or loop forever under upstream: both hang the recording. Nor may it read a file it has just written (`FILE_EXISTS`, `ZIP`, `IMPORT` right after a `WRITE`): both programs write files in the background, so the result depends on timing; put such a file among the case's inputs instead. A difference from upstream fails the test unless `tests/corpus/changes.json` lists it with both values, a `status` (`accepted` for an upstream bug execsql2 fixed on purpose, `open` for one not yet decided) and the reason.
+
 ## Building the Docs
 
 ```bash

{execsql2-2.34.0 → execsql2-2.37.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: execsql2
-Version: 2.34.0
+Version: 2.37.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Homepage, https://execsql2.readthedocs.io/en/stable/
 Project-URL: Repository, https://github.com/geocoug/execsql
@@ -441,7 +441,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.34.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.34.0 → execsql2-2.37.0}/README.md

@@ -321,7 +321,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.34.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.34.0 → execsql2-2.37.0}/docs/about/divergence.md

@@ -88,24 +88,25 @@ ______________________________________________________________________
 
 New options in `execsql.conf`:
 
-| Option                           | Section       | Description                                                                                                                                                                                                      |
-| -------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `use_keyring`                    | `[connect]`   | Use the OS keyring for credential storage (default: `yes`).                                                                                                                                                      |
-| `trust_server_certificate`       | `[connect]`   | SQL Server: accept a self-signed server certificate (ODBC Driver 18 verifies it by default); also `?trust_server_certificate=yes` on an `mssql://` URL. Default: `no`.                                           |
-| `show_progress`                  | `[input]`     | Enable Rich progress bar for IMPORT (default: `no`).                                                                                                                                                             |
-| `import_progress_interval`       | `[input]`     | Log a status line every N rows during IMPORT (default: `0`).                                                                                                                                                     |
-| `gui_framework`                  | `[interface]` | GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                                                                     |
-| `log_sql`                        | `[config]`    | Enable SQL audit logging (default: `no`).                                                                                                                                                                        |
-| `max_log_size_mb`                | `[config]`    | Rotate the log file at this size in MB (default: `0` = disabled).                                                                                                                                                |
-| `macos_config_file`              | `[config]`    | Additional config file path, active only on macOS.                                                                                                                                                               |
-| `allow_rm_file`                  | `[config]`    | Disable the `RM_FILE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-rm-file` on the CLI.                                                                                           |
-| `allow_serve`                    | `[config]`    | Disable the `SERVE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-serve` on the CLI.                                                                                               |
-| `include_root`                   | `[config]`    | Path-containment root for `INCLUDE` / `EXECUTE SCRIPT` (default: none). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error. |
-| `serve_root`                     | `[config]`    | Path-containment root for `SERVE` (default: none). Same semantics as `include_root`.                                                                                                                             |
-| `template_root`                  | `[config]`    | Path-containment root for Jinja2 / `string.Template` loaders (default: none). Same semantics as `include_root`.                                                                                                  |
-| `max_substitution_bytes`         | `[config]`    | Byte ceiling on a single substitution-variable expansion (default: `10485760` = 10 MB) to defeat exponential-expansion bombs.                                                                                    |
-| `indent`, `leading_comma`, `sql` | `[format]`    | Layout defaults for `execsql format`; a flag on the command line wins.                                                                                                                                           |
-| `select`, `ignore`, `strict`     | `[lint]`      | Rule selection and strictness for `execsql lint`; `--select` / `--ignore` replace the lists, `--strict` / `--no-strict` override `strict`.                                                                       |
+| Option                           | Section       | Description                                                                                                                                                                                                                                                    |
+| -------------------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `use_keyring`                    | `[connect]`   | Use the OS keyring for credential storage (default: `yes`).                                                                                                                                                                                                    |
+| `trust_server_certificate`       | `[connect]`   | SQL Server: accept a self-signed server certificate (ODBC Driver 18 verifies it by default); also `?trust_server_certificate=yes` on an `mssql://` URL. Default: `no`.                                                                                         |
+| `exact_decimals`                 | `[input]`     | Give a column of decimals with different numbers of places an exact `DECIMAL(p,s)` type instead of a float (default: `no`); also `CONFIG EXACT_DECIMALS`.                                                                                                      |
+| `show_progress`                  | `[input]`     | Enable Rich progress bar for IMPORT (default: `no`).                                                                                                                                                                                                           |
+| `import_progress_interval`       | `[input]`     | Log a status line every N rows during IMPORT (default: `0`).                                                                                                                                                                                                   |
+| `gui_framework`                  | `[interface]` | GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                                                                                                                   |
+| `log_sql`                        | `[config]`    | Enable SQL audit logging (default: `no`).                                                                                                                                                                                                                      |
+| `max_log_size_mb`                | `[config]`    | Rotate the log file at this size in MB (default: `0` = disabled).                                                                                                                                                                                              |
+| `macos_config_file`              | `[config]`    | Additional config file path, active only on macOS.                                                                                                                                                                                                             |
+| `allow_rm_file`                  | `[config]`    | Disable the `RM_FILE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-rm-file` on the CLI.                                                                                                                                         |
+| `allow_serve`                    | `[config]`    | Disable the `SERVE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-serve` on the CLI.                                                                                                                                             |
+| `include_root`                   | `[config]`    | Path-containment root for `INCLUDE` files (default: none). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error that stops the run, even with `METACOMMAND_ERROR_HALT OFF`. |
+| `serve_root`                     | `[config]`    | Path-containment root for `SERVE` (default: none). The same containment checks as `include_root`; a refused path is an ordinary metacommand error.                                                                                                             |
+| `template_root`                  | `[config]`    | Path-containment root for Jinja2 / `string.Template` loaders (default: none). The same containment checks as `include_root`; a refused path is an ordinary metacommand error.                                                                                  |
+| `max_substitution_bytes`         | `[config]`    | Byte ceiling on a single substitution-variable expansion (default: `10485760` = 10 MB) to defeat exponential-expansion bombs.                                                                                                                                  |
+| `indent`, `leading_comma`, `sql` | `[format]`    | Layout defaults for `execsql format`; a flag on the command line wins.                                                                                                                                                                                         |
+| `select`, `ignore`, `strict`     | `[lint]`      | Rule selection and strictness for `execsql lint`; `--select` / `--ignore` replace the lists, `--strict` / `--no-strict` override `strict`.                                                                                                                     |
 
 ### Tools
 
@@ -189,7 +190,7 @@ Formatter correctness notes added in the 2.19.x line:
 
 ### Debugging { #debugging }
 
-execsql2 adds an interactive debug REPL (no upstream equivalent), triggered either by the `BREAKPOINT` metacommand or the `--debug` CLI flag (step-through mode). The REPL prompts with `execsql debug>`, and input runs as if it were the next lines of the script where it paused, so SQL (ending with `;`, results shown as tables), metacommands and blocks all work, in the paused script's variable scope and transaction. A `SUB` typed there is still set when the script resumes, a `BREAK` inside a loop leaves the loop, and `HALT` ends the run. An error ends only the input that caused it, without running `ON ERROR_HALT` actions. Input starting with `.` is a REPL command (`.continue`, `.quit`, `.next`, `.vars`, `.vars VAR`, `.where`, `.stack`, `.set`, `.scripts`, `.cancel`, `.help`). `BREAKPOINT` is silently skipped when stdin is not a TTY so it does not hang CI. Full command list and behavior: [Debugging guide](../guides/debugging.md).
+execsql2 adds an interactive debug REPL (no upstream equivalent), triggered either by the `BREAKPOINT` metacommand or the `--debug` CLI flag (step-through mode). The REPL prompts with `execsql debug>`, and input runs as if it were the next lines of the script where it paused, so SQL (ending with `;`, results shown as tables), metacommands and blocks all work, in the paused script's variable scope and transaction. A `SUB` typed there is still set when the script resumes, a `BREAK` leaves the `LOOP`, `SCRIPT` or included file the script paused in, and `HALT` ends the run. An error ends only the input that caused it, without running `ON ERROR_HALT` actions. Input starting with `.` is a REPL command (`.continue`, `.quit`, `.next`, `.vars`, `.vars VAR`, `.where`, `.stack`, `.set`, `.scripts`, `.cancel`, `.help`). `BREAKPOINT` is silently skipped when stdin is not a TTY so it does not hang CI. Full command list and behavior: [Debugging guide](../guides/debugging.md).
 
 ### Library API
 
@@ -257,6 +258,7 @@ All 33 mutable runtime globals in `state.py` have been consolidated into a `Runt
 
 - **`$HOSTNAME`** — Network name of the machine running execsql (`platform.node()`). Useful for log messages and environment detection.
 - **`$SYSTEM_CMD_PID`** — New system variable set to the PID of the background process when `SHELL … CONTINUE` is used.
+- **`!'!var!'!` adds the quotes** — `!'!name!'!` becomes a complete string literal: the value is wrapped in single quotes and its apostrophes doubled (`O'Brien` → `'O''Brien'`), so a script writes `where author = !'!name!'!`. Upstream only doubled the apostrophes, so its scripts wrote the quotes themselves (`'!'!name!'!'`); in execsql2 that form has two sets of quotes and fails. Remove the outer quotes when moving such a script.
 - **Cycle detection** — `substitute_vars()` raises an error after 100 iterations to prevent infinite loops when variables reference each other cyclically. Upstream had no protection.
 - **O(1) substitution** — Variable substitution uses a single combined regex and dict lookup instead of O(V) per-variable regex passes. Behavior is identical; performance is improved.
 - **Lazy `$RANDOM`/`$UUID`** — These system variables are now computed on first access rather than generated unconditionally for every statement. Behavior is identical when referenced; scripts that never reference them skip the computation entirely.
@@ -288,7 +290,13 @@ The legacy flat command-list engine (`_parse_script_lines` / `runscripts` / `Com
 
 - **Native INCLUDE handling** — The AST executor parses INCLUDE'd files with the AST parser and executes them through the tree-walking executor. Control flow structures (IF/LOOP/BATCH/SCRIPT) in included files are fully tree-driven, with correct deferred variable handling and source-span error reporting.
 - **Circular INCLUDE detection** — The executor tracks the full include chain and detects circular references (e.g. A includes B includes A), reporting the full chain in the error message. Upstream has no such detection.
-- **BREAK outside LOOP is an error** — `BREAK` outside a loop block now raises an error (exit 1) instead of being silently ignored. This catches script bugs that upstream would not report.
+- **LOOP conditions are read on every pass** — The variables in a `LOOP WHILE` / `LOOP UNTIL` condition are substituted afresh before each pass, so `LOOP WHILE (IS_GT(10, !!n!!))` with `SUB_ADD n 1` in the body stops when `n` reaches 10. Upstream substituted them once, when the `LOOP` line ran, so a condition on a variable already defined never changed: the loop ran until a `BREAK`, or forever. Upstream scripts got per-pass conditions only from variables not yet defined when the loop started ("phantom variables") or from deferred `!{var}!` references, and both still work.
+- **Control flow from a substitution variable** — A metacommand line whose text a substitution variable supplies (`-- !x! !!cmd!!`) cannot be an `INCLUDE`, `EXECUTE SCRIPT` (or `EXEC SCRIPT` / `RUN SCRIPT`) or a block metacommand (`IF`, `ELSEIF`, `ANDIF`, `ORIF`, `ELSE`, `ENDIF`, `LOOP`, `END LOOP`, `BEGIN BATCH`, `END BATCH`, `BEGIN SCRIPT` (or `CREATE SCRIPT`), `END SCRIPT`, `BEGIN SQL`, `END SQL`). execsql2 decides which lines are includes and blocks when it parses the script, before any variable is substituted, so such a line does not run. Most of these are a metacommand error, which `METACOMMAND_ERROR_HALT OFF` skips without printing anything (test for it with `METACOMMAND_ERROR()`; `execsql.run()` still lists it in `result.errors`); `END LOOP`, `BEGIN` / `CREATE` / `END SCRIPT` and `BEGIN` / `END SQL` stop the run whatever that setting says. Upstream substituted the line first and ran it. Write the metacommand in the script and put the variable in its argument instead: `INCLUDE !!file!!`, `EXECUTE SCRIPT !!name!!` and `IF (!!condition!!)` all work (decided in #113).
+- **BREAK leaves the innermost LOOP, SCRIPT or included file** — As upstream documents and did: a `BREAK` in a `SCRIPT` or `INCLUDE`d file outside a `LOOP` returns from that script or file, and never ends a loop that called the script or included the file. Two differences remain. A `BREAK` in the main script, outside all of these, is a metacommand error reported on the `BREAK`'s own line (the run stops unless `METACOMMAND_ERROR_HALT` is `OFF`); upstream logged a warning and carried on, so a misplaced `BREAK` went unnoticed. And a `BREAK` that is the body of an inline `IF (…) { BREAK }` leaves the loop; upstream ignored it and the loop ran on (decided in #115).
+- **A batch left before END BATCH is rolled back** — A `BREAK`, an error or a `HALT` inside `BEGIN BATCH` rolls the batch back, and the script's later statements are committed as usual. Upstream left the batch open after a `BREAK`, so nothing after it was committed either and the run ended by rolling it all back. An error or `HALT` rolled the batch back in both, but upstream ran the `ON ERROR_HALT` / `ON CANCEL_HALT` script inside the still-open batch and so lost what that script wrote; execsql2 rolls back first, and the script's SQL is committed (unless `AUTOCOMMIT` is `OFF`). A nested batch that a `BREAK` leaves while the batch around it carries on is not rolled back on its own: the outer batch commits or rolls back its statements, so the result matches upstream there. Under `AUTOCOMMIT OFF`, leaving a batch early rolls nothing back, as `END BATCH` commits nothing. execsql2 2.36.0 and earlier committed the half-finished batch on a `BREAK`, an error or `HALT`, and committed a nested batch left by `BREAK` at once instead of with the outer batch (#121).
+- **END BATCH whose commit fails** — When a commit at `END BATCH` fails on one database (a deferred constraint, for example), execsql2 rolls back that database and the batch's databases not yet committed, then reports an SQL error on the `END BATCH` line, which `ERROR_HALT` governs. An enclosing batch then rolls back at its own `END BATCH` instead of committing, and no batch opened inside it afterwards commits either (`ROLLBACK BATCH` does not undo this; `IMPORT`, `COPY`, `EXECUTE` and `AUTOCOMMIT ON WITH COMMIT` still commit unless `AUTOCOMMIT` is `OFF`, as in any batch). A database under `AUTOCOMMIT OFF` is left to the script. `END BATCH` clears `SQL_ERROR()` before its commit, as an SQL statement does, so right after `END BATCH` it says whether the batch was committed; upstream kept the value left by the last statement run before it. Upstream left the uncommitted databases' statements pending, so the next statement that committed on them (an `ON ERROR_HALT` script's, say) committed part of the failed batch; it reported the failure as a metacommand error. execsql2 2.36.0 and earlier did the same and reported it as an uncaught exception on a line of execsql's source (#125).
+- **A failed commit is rolled back** — When a commit fails (a deferred foreign key, for example), execsql2 rolls the transaction back before it reports the error, so nothing it was committing is stored: an SQL statement, `IMPORT`, `COPY`, `EXECUTE`, or the commit of what was held under `AUTOCOMMIT OFF` (`AUTOCOMMIT ON WITH COMMIT`, or the next statement after a plain `AUTOCOMMIT ON`). Upstream left the transaction open where the DBMS does (SQLite), and when the script carried on (`ERROR_HALT OFF` for an SQL statement, `METACOMMAND_ERROR_HALT OFF` for a metacommand, or an `ON ERROR_HALT` script), the next statement that committed stored what it had reported as failed (#135).
+- **Anything that fails inside a batch** — An SQL statement, a metacommand, or SQL a metacommand runs (`EXECUTE`, `EXPORT QUERY`, `SUBDATA`, `SELECT_SUB`, `IMPORT`, …) that fails inside `BEGIN BATCH` keeps the batch, and every batch around it, from being committed: `END BATCH` rolls back what the batch ran after the failure and reports an SQL error ("The batch was not committed: an error on line …."), so `SQL_ERROR()` right after `END BATCH` is true. The script reaches `END BATCH` only with `ERROR_HALT OFF` (for an SQL error) or `METACOMMAND_ERROR_HALT OFF` (for a metacommand error); if `ERROR_HALT` is `ON` at `END BATCH`, the run stops there, where upstream committed and carried on. A failed statement had already rolled back the batch's earlier statements on its database, in upstream too. Upstream committed the statements after the failure at `END BATCH`, a partial batch, and `SQL_ERROR()` was false if the last one succeeded. `ROLLBACK BATCH` does not undo the failure. Under `AUTOCOMMIT OFF` the script's own transaction is left to it (#125).
 - **File writer is a thread, one per run** — `WRITE`, `TEE` and the other text-file metacommands hand their output to a background thread owned by the run's `RuntimeContext`, which `execsql.run()` stops before returning. Upstream uses one `multiprocessing` subprocess for the whole program; under the `spawn` start method (macOS, Windows) that re-imported the calling program, so a library caller without an `if __name__ == "__main__":` guard ran its script twice.
 - **Instance-scoped script registry** — Named SCRIPT blocks are stored on the `RuntimeContext` instance instead of a module-level dict, preventing cross-execution contamination.
 
@@ -299,6 +307,7 @@ The legacy flat command-list engine (`_parse_script_lines` / `runscripts` / `Com
 - **Database and driver error messages** — End with the error itself (`OperationalError: table t already exists`). Upstream appended "in <file> on line N of execsql", a line of its own source; in the package the file was often a driver's and the line number another file's. Where in execsql the error was raised goes to `execsql.log` as `Raised at: execsql/<module>.py:<line>`.
 - **Missing optional package messages** — Name the execsql2 extra that installs the package and how to add it to a `uv tool` or `pipx` install without dropping the extras already there (`uv tool install "execsql2[...,postgres]"`). Upstream pointed at the package's web page.
 - **Config file errors** — An invalid value in `execsql.conf` stops a run with `Configuration error: <file>: <problem>` (and the line, for an unreadable file) plus a pointer to `execsql config --validate`. Upstream printed `Configuration error on line N of execsql.py`, where N was a line of its own source. A duplicate key, a missing section header or a bare `%` is reported the same way; upstream failed with an uncaught exception. Errors raised by `ConfigData` for a file are `ConfigFileError`, a subclass of `ConfigError`.
+- **A condition that cannot be evaluated stops the run** — An `IF`, `ELSEIF`, `ANDIF`, `ORIF`, `LOOP` or `EXECUTE SCRIPT … WHILE` / `UNTIL` condition that cannot be evaluated stops the run, even with `METACOMMAND_ERROR_HALT OFF` and `ERROR_HALT OFF`, and the error report names the line and the condition. That covers an unknown test or malformed condition (`IF (NO_SUCH_FN(x))`) and a test that fails in the database (`HASROWS(no_such_table)`). Upstream carried on: it skipped a bad inline `IF (…) { … }` as a metacommand error, hiding a failed query along with a typo; a top-level block `IF` ran both its body and its `ELSE` branch, and one inside another `IF` ran its body, skipped its `ELSE`, and closed the enclosing `IF` early, so that `IF`'s `ELSE` branch ran too. Upstream also stopped on a bad `LOOP` or `EXECUTE SCRIPT … WHILE` condition, but reported a line inside the loop. A condition that cannot be evaluated is a script bug, and carrying on would run the script down a branch nobody chose (decided in #112).
 - **ASSERT error type** — `ASSERT` failures now use a dedicated `"assert"` error type that produces `**** Assertion failed.` instead of `**** Error in metacommand.`. This distinguishes intentional script-level checks from actual metacommand errors. Upstream did not have `ASSERT`.
 
 ______________________________________________________________________
@@ -435,6 +444,12 @@ These are behavioral changes driven by security or correctness issues in the ups
 | DuckDB float columns were single precision                         | `dbt_duckdb` mapped `DT_Float` to `REAL`. `DT_Float` is documented as IEEE double precision and a Python float always is one, so every float written to DuckDB was narrowed: `1234.567` was stored and returned as `1234.5670166015625`. PostgreSQL, SQLite, SQL Server, MySQL, Access, Firebird and DSN all map `DT_Float` to a double; DuckDB was the only backend that did not, and is now `DOUBLE`.                                                                                                                                                                                                                                                                                                                                                                                                            |
 | VALUES export wrote dates and timestamps unquoted                  | The literal renderer quoted `str` and emitted `NULL` for `None`, but sent everything else through `str()` bare. A `date` became `2026-09-24` in the generated `INSERT`, which is not a syntax error — it is a valid SQL expression evaluating to 1993, so PostgreSQL inserted that and reported nothing. A `timestamp` contains a space and failed outright. Quoting is now the default and only numbers are emitted bare, so a type the renderer has not been taught about is safe rather than silently wrong.                                                                                                                                                                                                                                                                                                    |
 | LaTeX export escaped only the underscore                           | Cells were written as `str(c).replace("_", r"\_")`, leaving every other LaTeX special raw. `&` is the column separator this exporter itself emits, so an ampersand in a value added a column and the `.tex` no longer compiled; `%` commented out the rest of the row. All of `\ & % $ # _ { } ~ ^` are now escaped, in both the header and the body.                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+| `ON ERROR_HALT WRITE … TO` a file wrote nothing                    | Upstream passed the `(text, flag)` pair that variable substitution returns on as the text, so writing the message failed after the file was opened and the file stayed empty. The message is written.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+| Mixed-scale decimals keep every digit                              | `IMPORT` / `COPY … TO NEW` gave a column of decimals with different numbers of places (`1.50`, `2`, `-3.25`) a floating-point type, which also held values a double cannot store exactly: `1234567890.1234567891` was stored as `1234567890.1234567`. Such a column is still a float, unless a value has more than 15 significant digits; then it is `DECIMAL(p,s)`. The `exact_decimals` setting makes every such column exact.                                                                                                                                                                                                                                                                                                                                                                                   |
+| XML export wrote NULL as the literal "None"                        | `str()` was applied to every value, so a NULL appeared in the XML as the text None. It is now an empty element.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+| `WRITE SCRIPT` output                                              | The `BEGIN SCRIPT` and `END SCRIPT` lines were written without `-- !x!`, so a script that `INCLUDE`d the file ran them as SQL. They are written as metacommands.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| Line break inside a quoted value                                   | Delimiter detection read the file line by line, so a quoted value with a line break in it made every line look different and no delimiter was found: the whole file became one column named after the header line (`id,note`). The file is read as CSV, so the line break stays inside the value.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| `EXPORT` after `CD`                                                | `EXPORT` failed after `CD` with `No such file or directory: …/<new dir>/script.sql`, looking for the script in the new directory. The output file is written relative to the current directory.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
 | LaTeX export wrote NULL as the literal "None"                      | `str(c)` was applied to every value including `None`, so a NULL appeared in the typeset table as the word None. It is now an empty cell, matching every other exporter.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
 | HTML export rendered zero as an empty cell                         | Cells were written as `html.escape(str(v)) if v else ''`. `0`, `False`, and `""` are all falsy, so a zero measurement rendered exactly like a NULL and the two could not be told apart in the output. The test is now `if v is not None`. Both `export_html` and `export_cgi_html` were affected.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Markdown export broke the table on an embedded newline             | `_cell()` escaped pipes but not newlines, and a GFM table row is a single line: a value containing one ended the row mid-cell and left the rest of that row's values on a stray line outside the table entirely. Newlines are now written as `<br>`, which is how GFM expresses a line break inside a cell.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
@@ -446,6 +461,7 @@ These are behavioral changes driven by security or correctness issues in the ups
 | Values loaded into an existing table                               | `IMPORT` and `COPY` converted every value to the type guessed from the source data, even when the table was already there with its own column types, so a text column got a different string than the source held (`yes` and `1` stored as `true`, `01/05/2024` as `2024-01-05 00:00:00`, `1e3` as `1000`), and on PostgreSQL a column of 0s and 1s copied into an integer column failed (the guess was boolean). For a table that exists, the target column's type now decides: a text column gets the value as written, a number column gets true/false values as 1 and 0, and other columns are converted as before. Each adapter reads the column types from its catalog (`pg_attribute`, `SHOW COLUMNS`, `sys.columns`) or its driver's description of the table. `TO NEW` / `REPLACEMENT` are unchanged.     |
 | Missing dates loaded into MySQL                                    | `IMPORT` and `COPY` sent a missing value in a column scanned as a date or timestamp as `''`, so MySQL stored `0000-00-00` (or `0` in a number column) instead of NULL. It is sent as NULL.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
 | Fractions loaded into an integer column                            | A value with a fraction going into an integer column (including `DECIMAL(p,0)`, `NUMERIC(p)` and Oracle `NUMBER(p)`) of an existing table is an error naming the value and the column. Upstream sent the number and the database rounded it (`1.5` → `2`; SQL Server truncated it to `1`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+| `DISCONNECT`                                                       | Refuses a database that an open `BEGIN BATCH` has used, as documented; upstream's check compared the alias to connection objects and never fired, so the connection closed and the batch's statements were lost. The refusal is an error inside the batch, so `END BATCH` does not commit it. The alias matches in any case, as `USE` does (upstream's `DISCONNECT FROM B` left a connection made `AS b` open), and an alias that is not connected is a metacommand error (upstream ignored it) (#124).                                                                                                                                                                                                                                                                                                            |
 
 ______________________________________________________________________
 

{execsql2-2.34.0 → execsql2-2.37.0}/docs/guides/debugging.md

@@ -61,18 +61,18 @@ execsql debug>
 
 Because input runs inside the paused script, it changes what the script sees after `.continue`:
 
-| You type at the breakpoint        | Effect on the script                                                                  |
-| --------------------------------- | ------------------------------------------------------------------------------------- |
-| `!x! SUB threshold 50`            | `!!threshold!!` is 50 for the rest of the run                                         |
-| `select '!!~counter!!';`          | Reads the paused SCRIPT's `~local` and `#param` variables                             |
-| `insert into staging values (1);` | Committed as the script's own SQL would be: now, unless `AUTOCOMMIT OFF` or a batch   |
-| `!x! USE other_db`                | The script continues against `other_db`                                               |
-| `!x! BREAK`                       | Inside a `LOOP` (or a looping `EXECUTE SCRIPT`), leaves that loop, as a `BREAK` would |
-| `!x! HALT`                        | Ends the run                                                                          |
+| You type at the breakpoint        | Effect on the script                                                                    |
+| --------------------------------- | --------------------------------------------------------------------------------------- |
+| `!x! SUB threshold 50`            | `!!threshold!!` is 50 for the rest of the run                                           |
+| `select '!!~counter!!';`          | Reads the paused SCRIPT's `~local` and `#param` variables                               |
+| `insert into staging values (1);` | Committed as the script's own SQL would be: now, unless `AUTOCOMMIT OFF` or a batch     |
+| `!x! USE other_db`                | The script continues against `other_db`                                                 |
+| `!x! BREAK`                       | Leaves the `LOOP`, `SCRIPT` or `INCLUDE`d file the script paused in, as a `BREAK` would |
+| `!x! HALT`                        | Ends the run                                                                            |
 
 **Errors end only the input that caused them.** A SQL or metacommand error that would halt the script is printed, and the prompt reads the next input; `ON ERROR_HALT` actions do not run. `HALT` and `.quit` still end the run.
 
-**Transactions follow the script's rules.** Each SQL statement is committed as it runs unless `AUTOCOMMIT OFF` is in effect or the breakpoint is inside `BEGIN BATCH`. The prompt then reads `execsql debug*>`, so you can see that what you type is not committed yet.
+**Transactions follow the script's rules.** Each SQL statement is committed as it runs unless `AUTOCOMMIT OFF` is in effect or the breakpoint is inside `BEGIN BATCH`. The prompt then reads `execsql debug*>`, so you can see that what you type is not committed yet. Inside `BEGIN BATCH`, a statement or metacommand you type that fails keeps the batch from being committed, as one in the script does: `END BATCH` rolls it back and reports an error.
 
 **Commands to the REPL** start with `.`:
 

{execsql2-2.34.0 → execsql2-2.37.0}/docs/guides/formatter.md

@@ -272,7 +272,7 @@ select id,name,created_at from users where active = true order by name;
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.34.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint # optional: fail the commit on lint errors

{execsql2-2.34.0 → execsql2-2.37.0}/docs/reference/configuration.md

@@ -141,6 +141,10 @@ The section and property names that may be used in a configuration file are list
 
 :   Determines whether empty strings in the input are preserved or, alternatively, will be replaced by NULL. The property value should be either "Yes" or "No". The default, "Yes", indicates that empty strings are allowed. A value of "No" will cause all empty strings to be replaced by NULL. When this is set to "No", a string value consisting of a sequence of zero or more space characters will be considered to be an empty string. There is no command-line option corresponding to this configuration parameter, but the metacommand [CONFIG EMPTY_STRINGS](metacommands.md#empty_strings) can also be used to change this configuration item.
 
+`exact_decimals`
+
+:   Determines the data type of a column of decimal numbers with different numbers of decimal places, such as `1.50`, `2` and `-3.25`, when the [IMPORT](metacommands.md#import) or [COPY](metacommands.md#copy) metacommand creates a table. The default, "No", gives such a column a floating-point type (`double precision` on PostgreSQL, `double` on MySQL and DuckDB, `float` on SQL Server), unless a value has more than 15 significant digits, which a floating-point column cannot hold exactly: then the column is an exact decimal, so no digit is lost. "Yes" always gives such a column an exact decimal type wide enough for every value (`numeric(3,2)` for those values), for values such as money amounts that should be stored and compared exactly. A column whose values all have the same number of decimal places is an exact decimal either way. The metacommand [CONFIG EXACT_DECIMALS](metacommands.md#exact_decimals) can also be used to change this configuration item.
+
 `fold_column_headers` { #setting_fold_column_headers }
 
 :   Whether or not to fold (convert) the case of all column headers to lowercase or uppercase, or to leave them unchanged when data are [IMPORTed](metacommands.md#import). Valid values are "No" (the default), "Lower", and "Upper". Case does not matter in the specification.
@@ -372,7 +376,7 @@ The section and property names that may be used in a configuration file are list
 :   When set to "No", the `SYSTEM_CMD` (SHELL) metacommand is disabled. Any script that attempts to execute an OS command will fail with an error. The default is "Yes". This can also be set via the `--no-system-cmd` CLI flag or `allow_system_cmd=False` in the library API. See [Security — Disabling SYSTEM_CMD](security.md#disable_system_cmd) for details.
 
 `system_cmd_timeout` { #system_cmd_timeout }
-:   Maximum number of seconds a foreground `SYSTEM_CMD` may run before *execsql* stops it. The default is `0`, which means no timeout. When a foreground command times out, `$SYSTEM_CMD_EXIT_STATUS` is set to `124` and the metacommand fails. `SYSTEM_CMD ... CONTINUE` starts a detached background process and is not timeout-managed by this setting.
+:   Maximum number of seconds a foreground `SYSTEM_CMD` may run before *execsql* stops it. The default is `0`, which means no timeout. When a foreground command times out, `$SYSTEM_CMD_EXIT_STATUS` is set to `124` and the metacommand fails. If the operating system will not let *execsql* stop the command (it runs as another user, through `sudo` for example), the error says so and gives its process ID. `SYSTEM_CMD ... CONTINUE` starts a detached background process and is not timeout-managed by this setting.
 
 `allow_rm_file` { #allow_rm_file }
 :   When set to "No", the `RM_FILE` metacommand (which deletes a file) is disabled. Any script that attempts to remove a file will fail with an error. The default is "Yes". This can also be set via the `--no-rm-file` CLI flag or `allow_rm_file=False` in the library API. Symmetric with `allow_system_cmd`.
@@ -381,13 +385,13 @@ The section and property names that may be used in a configuration file are list
 :   When set to "No", the [`SERVE`](metacommands.md#serve) metacommand (which copies a file to stdout with Content-Type and Content-Disposition headers, for use as a CGI script) is disabled. The default is "Yes". This can also be set via the `--no-serve` CLI flag or `allow_serve=False` in the library API. Symmetric with `allow_system_cmd`.
 
 `include_root` { #include_root }
-:   Root directory under which `INCLUDE` and `EXECUTE SCRIPT` targets must resolve. When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error. Default: no containment (any readable path is permitted). See [Security — Path containment roots](security.md#path-containment-roots) for the full per-handler matrix.
+:   Root directory under which the files `INCLUDE` reads must resolve (`EXECUTE SCRIPT` runs a `SCRIPT` block, not a file). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error that stops the run, even with `METACOMMAND_ERROR_HALT OFF`. Default: no containment (any readable path is permitted). See [Security — Path containment roots](security.md#path-containment-roots) for the full per-handler matrix.
 
 `serve_root` { #serve_root }
-:   Root directory under which `SERVE` targets must resolve. Same containment semantics as `include_root` — paths that escape via `../` or absolute references are rejected. Default: no containment.
+:   Root directory under which `SERVE` targets must resolve. The same containment checks as `include_root`, but a refused path is an ordinary metacommand error — paths that escape via `../` or absolute references are rejected. Default: no containment.
 
 `template_root` { #template_root }
-:   Root directory under which Jinja2 / `string.Template` loader paths must resolve. Same containment semantics as `include_root`. Default: no containment.
+:   Root directory under which Jinja2 / `string.Template` loader paths must resolve. The same containment checks as `include_root`, but a refused path is an ordinary metacommand error. Default: no containment.
 
 `max_substitution_bytes` { #max_substitution_bytes }
 :   Maximum byte size of any single substitution-variable expansion, enforced by the `substitute_vars()` engine to defeat exponential-expansion bombs (a script that nests `!!a!!` referring to `!!b!!` referring to `!!a!!` and so on can otherwise blow past memory). When set to a positive integer, expansions exceeding the threshold raise an error. Default: `10485760` (10 MB).

{execsql2-2.34.0 → execsql2-2.37.0}/docs/reference/lint.md

@@ -158,7 +158,7 @@ The `execsql-lint` [pre-commit](https://pre-commit.com/) hook runs `execsql lint
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.34.0
+    rev: v2.37.0
     hooks:
       - id: execsql-lint
         args: [--ignore, V002] # optional: any execsql lint option
@@ -340,6 +340,14 @@ An `INCLUDE` names a file that does not exist. Lint resolves a relative path aga
 
 Not reported for `INCLUDE IF EXISTS`, or when the path contains a `!!variable!!`, since the file then depends on values known only at run time.
 
+Not reported either inside an `IF` that runs only when the file exists, block or inline:
+
+```sql
+-- !x! IF (FILE_EXISTS(local_settings.sql)) { INCLUDE local_settings.sql }
+```
+
+An `ANDIF (FILE_EXISTS(…))` guards its file too; an `ORIF` lets the branch run without the file, so its `IF` guards nothing.
+
 Also reported, as "INCLUDE path cannot be checked", for a name the operating system rejects outright (longer than it allows, for example). The other scripts in the same `execsql lint` run are still checked; a script lint cannot check at all is reported as one P001 error naming it.
 
 ### I002 `missing-script` { #i002 }