execsql2 2.35.0__tar.gz → 2.37.0__tar.gz

execsql2-2.37.0/.gitattributes

@@ -0,0 +1,4 @@
+# Corpus inputs are compared byte for byte with upstream's recorded results
+# (IMPORT_FILE stores a file's raw bytes), so Git must not convert their line
+# endings on checkout.
+tests/corpus/cases/** -text

{execsql2-2.35.0 → execsql2-2.37.0}/CHANGELOG.md

@@ -13,6 +13,52 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.37.0] - 2026-10-23
+
+### Changed
+
+- **A failed commit at `END BATCH`** is an SQL error on the `END BATCH` line, which `ERROR_HALT` governs; it was reported as an uncaught exception on a line of execsql's own source and always stopped the run. With `ERROR_HALT OFF` the script now carries on: test `SQL_ERROR()` right after `END BATCH`.
+- **`SQL_ERROR()` right after `END BATCH`** says whether the batch's commit failed: `END BATCH` clears it first, as an SQL statement does. It kept the value left by the last statement run before it.
+- **Anything that fails inside `BEGIN BATCH`**, an SQL statement, a metacommand or a metacommand's SQL, keeps the batch from being committed: `END BATCH` rolls it back and reports an SQL error, which stops the run if `ERROR_HALT` is `ON` there. The batch's statements after a failure were committed, even when a failed statement had already rolled back the ones before it. Put a statement or metacommand whose failure the script handles outside the batch.
+- **A batch around one whose commit failed** is not committed at its own `END BATCH`, nor is any batch opened inside it afterwards, and `ROLLBACK BATCH` does not undo this; put work that must survive after the outermost `END BATCH`.
+- **`DISCONNECT` inside `BEGIN BATCH`** refuses a database the batch has used, as documented: the run halts and the batch is rolled back, or with `METACOMMAND_ERROR_HALT OFF` the connection stays open and `END BATCH` rolls the batch back; it closed the connection and the batch's statements were lost. Put the `DISCONNECT` after `END BATCH`.
+- **`DISCONNECT FROM <alias>`** matches the alias in any case, as `USE` does; `DISCONNECT FROM B` left a connection made `AS b` open, so a later `USE b` worked. Reconnect with `CONNECT` before using an alias you disconnected.
+- **`DISCONNECT FROM` an alias that is not connected** is a metacommand error; it did nothing, so a misspelled alias left the real connection open. To disconnect only if connected, write `IF (ALIAS_DEFINED(staging)) { DISCONNECT FROM staging }`, with the alias in lowercase inside `ALIAS_DEFINED`.
+- **`BREAK` in a `SCRIPT` or `INCLUDE`d file, outside a `LOOP`,** returns from that script or file, as the `BREAK` documentation says and upstream execsql did; it stopped the run. A `BREAK` in an `INCLUDE`d file inside a `LOOP` now leaves only the file, and the loop carries on; it ended the loop.
+- **`BREAK` in the main script, outside any `LOOP`, `SCRIPT` or `INCLUDE`d file,** is a metacommand error: with `METACOMMAND_ERROR_HALT OFF` the script carries on and `METACOMMAND_ERROR()` is true. It stopped the run whatever that setting said.
+- **A `BEGIN BATCH` block left before its `END BATCH`** by `BREAK`, an error or `HALT` is rolled back; it committed the statements run so far. To keep a batch's work and then `BREAK`, put the `BREAK` after `END BATCH`. A nested batch left by `BREAK` while the outer batch carries on is now committed or rolled back with the outer batch; it was committed at once. Under `AUTOCOMMIT OFF` your own `COMMIT` or `ROLLBACK` still decides, as before.
+- **An `ON ERROR_HALT` or `ON CANCEL_HALT` script after a halt inside `BEGIN BATCH`** runs after the batch is rolled back and still commits its own SQL, such as a row it writes to a log table (unless `AUTOCOMMIT` is `OFF`).
+- **`--manifest`** counts each `INCLUDE` and `EXECUTE SCRIPT` in `statements.metacommands`, including one that fails or that `IF EXISTS` skips.
+
+### Fixed
+
+- **A commit that fails** (a deferred constraint checked at COMMIT) rolls back what it was committing, so none of it is stored: an SQL statement, `IMPORT`, `COPY`, `EXECUTE`, or `AUTOCOMMIT ON` committing what was held under `AUTOCOMMIT OFF`. On SQLite, when the script carried on (`ERROR_HALT OFF` for an SQL statement, `METACOMMAND_ERROR_HALT OFF` for a metacommand, or an `ON ERROR_HALT` script), the next statement's commit stored it, including the rows reported as failed.
+- **SIGTERM while `SYSTEM_CMD` runs a command** stops the command just as it starts, as its `system_cmd_timeout` runs out, and under the Textual console (`--gui-framework textual`), where the script also stops at that `SYSTEM_CMD`; the command was left running after execsql exited.
+- **`system_cmd_timeout` running out on a command *execsql* cannot stop** (one run as another user, through `sudo` for example) fails the `SYSTEM_CMD` at once, with `$SYSTEM_CMD_EXIT_STATUS` 124 and an error giving the command's process ID; execsql waited for the command to finish and then stopped with a permission error.
+- **`END BATCH` whose commit fails on one database** rolls back the batch's databases not yet committed; a later statement on them, such as an `ON ERROR_HALT` script's, committed the failed batch's rows.
+- **`BREAK` in the main script, outside any `LOOP`, `SCRIPT` or `INCLUDE`d file,** is reported on its own line; it named the statement before it.
+- **`INCLUDE` and `EXECUTE SCRIPT` inside an inline `IF (…) { … }`** run again; they stopped the run with "should be handled by the AST executor".
+- **`INCLUDE`, `EXECUTE SCRIPT` and an inline `IF` added with `EXTEND SCRIPT … WITH METACOMMAND`** run when the script does; they stopped the run the same way.
+- **An `INCLUDE` or `EXECUTE SCRIPT` that a substitution variable supplies** (`-- !x! !!cmd!!`) says it cannot run and to write it in the script; it reported "should be handled by the AST executor".
+- **`execsql lint` and the language server** check and link `INCLUDE` and `EXECUTE SCRIPT` inside an inline `IF`.
+- **`execsql run --dry-run`** lists `INCLUDE` and `EXECUTE SCRIPT`.
+- **`execsql lint`** checks the variables in an `INCLUDE` file name and in `EXECUTE SCRIPT` arguments and loop conditions (V001, V002), and no longer reports an `INCLUDE` inside `IF (FILE_EXISTS(file))` as a missing file (I001).
+- **`INCLUDE` of a file that does not exist, `EXECUTE SCRIPT` of a script that does not exist, and `EXECUTE SCRIPT` without a required argument** are metacommand errors: with `METACOMMAND_ERROR_HALT OFF` the script carries on and `METACOMMAND_ERROR()`, `$LAST_ERROR` and `$ERROR_MESSAGE` say what failed. They stopped the run whatever that setting said.
+- **Error reports name the metacommand as written** (`Metacommand: INCLUDE missing.sql`), without `-- !x!`; a failed `INCLUDE`, `EXECUTE SCRIPT`, `IF`, `ELSEIF` or `LOOP` was reported with an empty `Metacommand:` line.
+- **An `IF`, `ELSEIF`, `ANDIF`, `ORIF` or `LOOP` condition that cannot be evaluated** is reported on its own line with its text (`Line 3 … Metacommand: ANDIF (NO_SUCH_FN(x))`), not as an "Uncaught exception" on another statement's line. It still stops the run.
+- **`WRITE SCRIPT`** writes an `EXECUTE SCRIPT … WHILE` / `UNTIL` as written; it wrote `LOOP WHILE`, which does not read back as `EXECUTE SCRIPT`.
+- **`--profile`** shows an `EXECUTE SCRIPT` or `INCLUDE` as written, with its `IF EXISTS`, arguments and `WHILE` / `UNTIL`.
+
+______________________________________________________________________
+
+## [2.36.0] - 2026-10-19
+
+### Changed
+
+- **`execsql.run()` from a notebook, the REPL or `python -c`** runs `IMPORT` and `COPY` about four times as fast: reading execsql's settings there cost about 12 µs each time, now 0.1 µs (a 200,000-row column scan went from 3.5 s to 0.8 s). The `execsql` command gets a smaller gain.
+
+______________________________________________________________________
+
 ## [2.35.0] - 2026-10-18
 
 ### Added

{execsql2-2.35.0 → execsql2-2.37.0}/CONTRIBUTING.md

@@ -75,7 +75,7 @@ uv run python -m tests.corpus.record my_new_case
 uv run pytest tests/corpus
 ```
 
-Use only syntax upstream accepts; read `_execsql/execsql.py` when unsure. A case must not open a GUI dialog (`HALT "message"`, `PROMPT`) or loop forever under upstream: both hang the recording. A difference from upstream fails the test unless `tests/corpus/changes.json` lists it with both values, a `status` (`accepted` for an upstream bug execsql2 fixed on purpose, `open` for one not yet decided) and the reason.
+Use only syntax upstream accepts; read `_execsql/execsql.py` when unsure. A case must not open a GUI dialog (`HALT "message"`, `PROMPT`) or loop forever under upstream: both hang the recording. Nor may it read a file it has just written (`FILE_EXISTS`, `ZIP`, `IMPORT` right after a `WRITE`): both programs write files in the background, so the result depends on timing; put such a file among the case's inputs instead. A difference from upstream fails the test unless `tests/corpus/changes.json` lists it with both values, a `status` (`accepted` for an upstream bug execsql2 fixed on purpose, `open` for one not yet decided) and the reason.
 
 ## Building the Docs
 

{execsql2-2.35.0 → execsql2-2.37.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: execsql2
-Version: 2.35.0
+Version: 2.37.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Homepage, https://execsql2.readthedocs.io/en/stable/
 Project-URL: Repository, https://github.com/geocoug/execsql
@@ -441,7 +441,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.35.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.35.0 → execsql2-2.37.0}/README.md

@@ -321,7 +321,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.35.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.35.0 → execsql2-2.37.0}/docs/about/divergence.md

@@ -88,25 +88,25 @@ ______________________________________________________________________
 
 New options in `execsql.conf`:
 
-| Option                           | Section       | Description                                                                                                                                                                                                      |
-| -------------------------------- | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `use_keyring`                    | `[connect]`   | Use the OS keyring for credential storage (default: `yes`).                                                                                                                                                      |
-| `trust_server_certificate`       | `[connect]`   | SQL Server: accept a self-signed server certificate (ODBC Driver 18 verifies it by default); also `?trust_server_certificate=yes` on an `mssql://` URL. Default: `no`.                                           |
-| `exact_decimals`                 | `[input]`     | Give a column of decimals with different numbers of places an exact `DECIMAL(p,s)` type instead of a float (default: `no`); also `CONFIG EXACT_DECIMALS`.                                                        |
-| `show_progress`                  | `[input]`     | Enable Rich progress bar for IMPORT (default: `no`).                                                                                                                                                             |
-| `import_progress_interval`       | `[input]`     | Log a status line every N rows during IMPORT (default: `0`).                                                                                                                                                     |
-| `gui_framework`                  | `[interface]` | GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                                                                     |
-| `log_sql`                        | `[config]`    | Enable SQL audit logging (default: `no`).                                                                                                                                                                        |
-| `max_log_size_mb`                | `[config]`    | Rotate the log file at this size in MB (default: `0` = disabled).                                                                                                                                                |
-| `macos_config_file`              | `[config]`    | Additional config file path, active only on macOS.                                                                                                                                                               |
-| `allow_rm_file`                  | `[config]`    | Disable the `RM_FILE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-rm-file` on the CLI.                                                                                           |
-| `allow_serve`                    | `[config]`    | Disable the `SERVE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-serve` on the CLI.                                                                                               |
-| `include_root`                   | `[config]`    | Path-containment root for `INCLUDE` / `EXECUTE SCRIPT` (default: none). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error. |
-| `serve_root`                     | `[config]`    | Path-containment root for `SERVE` (default: none). Same semantics as `include_root`.                                                                                                                             |
-| `template_root`                  | `[config]`    | Path-containment root for Jinja2 / `string.Template` loaders (default: none). Same semantics as `include_root`.                                                                                                  |
-| `max_substitution_bytes`         | `[config]`    | Byte ceiling on a single substitution-variable expansion (default: `10485760` = 10 MB) to defeat exponential-expansion bombs.                                                                                    |
-| `indent`, `leading_comma`, `sql` | `[format]`    | Layout defaults for `execsql format`; a flag on the command line wins.                                                                                                                                           |
-| `select`, `ignore`, `strict`     | `[lint]`      | Rule selection and strictness for `execsql lint`; `--select` / `--ignore` replace the lists, `--strict` / `--no-strict` override `strict`.                                                                       |
+| Option                           | Section       | Description                                                                                                                                                                                                                                                    |
+| -------------------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `use_keyring`                    | `[connect]`   | Use the OS keyring for credential storage (default: `yes`).                                                                                                                                                                                                    |
+| `trust_server_certificate`       | `[connect]`   | SQL Server: accept a self-signed server certificate (ODBC Driver 18 verifies it by default); also `?trust_server_certificate=yes` on an `mssql://` URL. Default: `no`.                                                                                         |
+| `exact_decimals`                 | `[input]`     | Give a column of decimals with different numbers of places an exact `DECIMAL(p,s)` type instead of a float (default: `no`); also `CONFIG EXACT_DECIMALS`.                                                                                                      |
+| `show_progress`                  | `[input]`     | Enable Rich progress bar for IMPORT (default: `no`).                                                                                                                                                                                                           |
+| `import_progress_interval`       | `[input]`     | Log a status line every N rows during IMPORT (default: `0`).                                                                                                                                                                                                   |
+| `gui_framework`                  | `[interface]` | GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                                                                                                                   |
+| `log_sql`                        | `[config]`    | Enable SQL audit logging (default: `no`).                                                                                                                                                                                                                      |
+| `max_log_size_mb`                | `[config]`    | Rotate the log file at this size in MB (default: `0` = disabled).                                                                                                                                                                                              |
+| `macos_config_file`              | `[config]`    | Additional config file path, active only on macOS.                                                                                                                                                                                                             |
+| `allow_rm_file`                  | `[config]`    | Disable the `RM_FILE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-rm-file` on the CLI.                                                                                                                                         |
+| `allow_serve`                    | `[config]`    | Disable the `SERVE` metacommand (default: `yes`). Symmetric with `allow_system_cmd`; also `--no-serve` on the CLI.                                                                                                                                             |
+| `include_root`                   | `[config]`    | Path-containment root for `INCLUDE` files (default: none). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error that stops the run, even with `METACOMMAND_ERROR_HALT OFF`. |
+| `serve_root`                     | `[config]`    | Path-containment root for `SERVE` (default: none). The same containment checks as `include_root`; a refused path is an ordinary metacommand error.                                                                                                             |
+| `template_root`                  | `[config]`    | Path-containment root for Jinja2 / `string.Template` loaders (default: none). The same containment checks as `include_root`; a refused path is an ordinary metacommand error.                                                                                  |
+| `max_substitution_bytes`         | `[config]`    | Byte ceiling on a single substitution-variable expansion (default: `10485760` = 10 MB) to defeat exponential-expansion bombs.                                                                                                                                  |
+| `indent`, `leading_comma`, `sql` | `[format]`    | Layout defaults for `execsql format`; a flag on the command line wins.                                                                                                                                                                                         |
+| `select`, `ignore`, `strict`     | `[lint]`      | Rule selection and strictness for `execsql lint`; `--select` / `--ignore` replace the lists, `--strict` / `--no-strict` override `strict`.                                                                                                                     |
 
 ### Tools
 
@@ -190,7 +190,7 @@ Formatter correctness notes added in the 2.19.x line:
 
 ### Debugging { #debugging }
 
-execsql2 adds an interactive debug REPL (no upstream equivalent), triggered either by the `BREAKPOINT` metacommand or the `--debug` CLI flag (step-through mode). The REPL prompts with `execsql debug>`, and input runs as if it were the next lines of the script where it paused, so SQL (ending with `;`, results shown as tables), metacommands and blocks all work, in the paused script's variable scope and transaction. A `SUB` typed there is still set when the script resumes, a `BREAK` inside a loop leaves the loop, and `HALT` ends the run. An error ends only the input that caused it, without running `ON ERROR_HALT` actions. Input starting with `.` is a REPL command (`.continue`, `.quit`, `.next`, `.vars`, `.vars VAR`, `.where`, `.stack`, `.set`, `.scripts`, `.cancel`, `.help`). `BREAKPOINT` is silently skipped when stdin is not a TTY so it does not hang CI. Full command list and behavior: [Debugging guide](../guides/debugging.md).
+execsql2 adds an interactive debug REPL (no upstream equivalent), triggered either by the `BREAKPOINT` metacommand or the `--debug` CLI flag (step-through mode). The REPL prompts with `execsql debug>`, and input runs as if it were the next lines of the script where it paused, so SQL (ending with `;`, results shown as tables), metacommands and blocks all work, in the paused script's variable scope and transaction. A `SUB` typed there is still set when the script resumes, a `BREAK` leaves the `LOOP`, `SCRIPT` or included file the script paused in, and `HALT` ends the run. An error ends only the input that caused it, without running `ON ERROR_HALT` actions. Input starting with `.` is a REPL command (`.continue`, `.quit`, `.next`, `.vars`, `.vars VAR`, `.where`, `.stack`, `.set`, `.scripts`, `.cancel`, `.help`). `BREAKPOINT` is silently skipped when stdin is not a TTY so it does not hang CI. Full command list and behavior: [Debugging guide](../guides/debugging.md).
 
 ### Library API
 
@@ -291,7 +291,12 @@ The legacy flat command-list engine (`_parse_script_lines` / `runscripts` / `Com
 - **Native INCLUDE handling** — The AST executor parses INCLUDE'd files with the AST parser and executes them through the tree-walking executor. Control flow structures (IF/LOOP/BATCH/SCRIPT) in included files are fully tree-driven, with correct deferred variable handling and source-span error reporting.
 - **Circular INCLUDE detection** — The executor tracks the full include chain and detects circular references (e.g. A includes B includes A), reporting the full chain in the error message. Upstream has no such detection.
 - **LOOP conditions are read on every pass** — The variables in a `LOOP WHILE` / `LOOP UNTIL` condition are substituted afresh before each pass, so `LOOP WHILE (IS_GT(10, !!n!!))` with `SUB_ADD n 1` in the body stops when `n` reaches 10. Upstream substituted them once, when the `LOOP` line ran, so a condition on a variable already defined never changed: the loop ran until a `BREAK`, or forever. Upstream scripts got per-pass conditions only from variables not yet defined when the loop started ("phantom variables") or from deferred `!{var}!` references, and both still work.
-- **BREAK outside LOOP is an error** — `BREAK` outside a loop block now raises an error (exit 1) instead of being silently ignored. This catches script bugs that upstream would not report.
+- **Control flow from a substitution variable** — A metacommand line whose text a substitution variable supplies (`-- !x! !!cmd!!`) cannot be an `INCLUDE`, `EXECUTE SCRIPT` (or `EXEC SCRIPT` / `RUN SCRIPT`) or a block metacommand (`IF`, `ELSEIF`, `ANDIF`, `ORIF`, `ELSE`, `ENDIF`, `LOOP`, `END LOOP`, `BEGIN BATCH`, `END BATCH`, `BEGIN SCRIPT` (or `CREATE SCRIPT`), `END SCRIPT`, `BEGIN SQL`, `END SQL`). execsql2 decides which lines are includes and blocks when it parses the script, before any variable is substituted, so such a line does not run. Most of these are a metacommand error, which `METACOMMAND_ERROR_HALT OFF` skips without printing anything (test for it with `METACOMMAND_ERROR()`; `execsql.run()` still lists it in `result.errors`); `END LOOP`, `BEGIN` / `CREATE` / `END SCRIPT` and `BEGIN` / `END SQL` stop the run whatever that setting says. Upstream substituted the line first and ran it. Write the metacommand in the script and put the variable in its argument instead: `INCLUDE !!file!!`, `EXECUTE SCRIPT !!name!!` and `IF (!!condition!!)` all work (decided in #113).
+- **BREAK leaves the innermost LOOP, SCRIPT or included file** — As upstream documents and did: a `BREAK` in a `SCRIPT` or `INCLUDE`d file outside a `LOOP` returns from that script or file, and never ends a loop that called the script or included the file. Two differences remain. A `BREAK` in the main script, outside all of these, is a metacommand error reported on the `BREAK`'s own line (the run stops unless `METACOMMAND_ERROR_HALT` is `OFF`); upstream logged a warning and carried on, so a misplaced `BREAK` went unnoticed. And a `BREAK` that is the body of an inline `IF (…) { BREAK }` leaves the loop; upstream ignored it and the loop ran on (decided in #115).
+- **A batch left before END BATCH is rolled back** — A `BREAK`, an error or a `HALT` inside `BEGIN BATCH` rolls the batch back, and the script's later statements are committed as usual. Upstream left the batch open after a `BREAK`, so nothing after it was committed either and the run ended by rolling it all back. An error or `HALT` rolled the batch back in both, but upstream ran the `ON ERROR_HALT` / `ON CANCEL_HALT` script inside the still-open batch and so lost what that script wrote; execsql2 rolls back first, and the script's SQL is committed (unless `AUTOCOMMIT` is `OFF`). A nested batch that a `BREAK` leaves while the batch around it carries on is not rolled back on its own: the outer batch commits or rolls back its statements, so the result matches upstream there. Under `AUTOCOMMIT OFF`, leaving a batch early rolls nothing back, as `END BATCH` commits nothing. execsql2 2.36.0 and earlier committed the half-finished batch on a `BREAK`, an error or `HALT`, and committed a nested batch left by `BREAK` at once instead of with the outer batch (#121).
+- **END BATCH whose commit fails** — When a commit at `END BATCH` fails on one database (a deferred constraint, for example), execsql2 rolls back that database and the batch's databases not yet committed, then reports an SQL error on the `END BATCH` line, which `ERROR_HALT` governs. An enclosing batch then rolls back at its own `END BATCH` instead of committing, and no batch opened inside it afterwards commits either (`ROLLBACK BATCH` does not undo this; `IMPORT`, `COPY`, `EXECUTE` and `AUTOCOMMIT ON WITH COMMIT` still commit unless `AUTOCOMMIT` is `OFF`, as in any batch). A database under `AUTOCOMMIT OFF` is left to the script. `END BATCH` clears `SQL_ERROR()` before its commit, as an SQL statement does, so right after `END BATCH` it says whether the batch was committed; upstream kept the value left by the last statement run before it. Upstream left the uncommitted databases' statements pending, so the next statement that committed on them (an `ON ERROR_HALT` script's, say) committed part of the failed batch; it reported the failure as a metacommand error. execsql2 2.36.0 and earlier did the same and reported it as an uncaught exception on a line of execsql's source (#125).
+- **A failed commit is rolled back** — When a commit fails (a deferred foreign key, for example), execsql2 rolls the transaction back before it reports the error, so nothing it was committing is stored: an SQL statement, `IMPORT`, `COPY`, `EXECUTE`, or the commit of what was held under `AUTOCOMMIT OFF` (`AUTOCOMMIT ON WITH COMMIT`, or the next statement after a plain `AUTOCOMMIT ON`). Upstream left the transaction open where the DBMS does (SQLite), and when the script carried on (`ERROR_HALT OFF` for an SQL statement, `METACOMMAND_ERROR_HALT OFF` for a metacommand, or an `ON ERROR_HALT` script), the next statement that committed stored what it had reported as failed (#135).
+- **Anything that fails inside a batch** — An SQL statement, a metacommand, or SQL a metacommand runs (`EXECUTE`, `EXPORT QUERY`, `SUBDATA`, `SELECT_SUB`, `IMPORT`, …) that fails inside `BEGIN BATCH` keeps the batch, and every batch around it, from being committed: `END BATCH` rolls back what the batch ran after the failure and reports an SQL error ("The batch was not committed: an error on line …."), so `SQL_ERROR()` right after `END BATCH` is true. The script reaches `END BATCH` only with `ERROR_HALT OFF` (for an SQL error) or `METACOMMAND_ERROR_HALT OFF` (for a metacommand error); if `ERROR_HALT` is `ON` at `END BATCH`, the run stops there, where upstream committed and carried on. A failed statement had already rolled back the batch's earlier statements on its database, in upstream too. Upstream committed the statements after the failure at `END BATCH`, a partial batch, and `SQL_ERROR()` was false if the last one succeeded. `ROLLBACK BATCH` does not undo the failure. Under `AUTOCOMMIT OFF` the script's own transaction is left to it (#125).
 - **File writer is a thread, one per run** — `WRITE`, `TEE` and the other text-file metacommands hand their output to a background thread owned by the run's `RuntimeContext`, which `execsql.run()` stops before returning. Upstream uses one `multiprocessing` subprocess for the whole program; under the `spawn` start method (macOS, Windows) that re-imported the calling program, so a library caller without an `if __name__ == "__main__":` guard ran its script twice.
 - **Instance-scoped script registry** — Named SCRIPT blocks are stored on the `RuntimeContext` instance instead of a module-level dict, preventing cross-execution contamination.
 
@@ -302,6 +307,7 @@ The legacy flat command-list engine (`_parse_script_lines` / `runscripts` / `Com
 - **Database and driver error messages** — End with the error itself (`OperationalError: table t already exists`). Upstream appended "in <file> on line N of execsql", a line of its own source; in the package the file was often a driver's and the line number another file's. Where in execsql the error was raised goes to `execsql.log` as `Raised at: execsql/<module>.py:<line>`.
 - **Missing optional package messages** — Name the execsql2 extra that installs the package and how to add it to a `uv tool` or `pipx` install without dropping the extras already there (`uv tool install "execsql2[...,postgres]"`). Upstream pointed at the package's web page.
 - **Config file errors** — An invalid value in `execsql.conf` stops a run with `Configuration error: <file>: <problem>` (and the line, for an unreadable file) plus a pointer to `execsql config --validate`. Upstream printed `Configuration error on line N of execsql.py`, where N was a line of its own source. A duplicate key, a missing section header or a bare `%` is reported the same way; upstream failed with an uncaught exception. Errors raised by `ConfigData` for a file are `ConfigFileError`, a subclass of `ConfigError`.
+- **A condition that cannot be evaluated stops the run** — An `IF`, `ELSEIF`, `ANDIF`, `ORIF`, `LOOP` or `EXECUTE SCRIPT … WHILE` / `UNTIL` condition that cannot be evaluated stops the run, even with `METACOMMAND_ERROR_HALT OFF` and `ERROR_HALT OFF`, and the error report names the line and the condition. That covers an unknown test or malformed condition (`IF (NO_SUCH_FN(x))`) and a test that fails in the database (`HASROWS(no_such_table)`). Upstream carried on: it skipped a bad inline `IF (…) { … }` as a metacommand error, hiding a failed query along with a typo; a top-level block `IF` ran both its body and its `ELSE` branch, and one inside another `IF` ran its body, skipped its `ELSE`, and closed the enclosing `IF` early, so that `IF`'s `ELSE` branch ran too. Upstream also stopped on a bad `LOOP` or `EXECUTE SCRIPT … WHILE` condition, but reported a line inside the loop. A condition that cannot be evaluated is a script bug, and carrying on would run the script down a branch nobody chose (decided in #112).
 - **ASSERT error type** — `ASSERT` failures now use a dedicated `"assert"` error type that produces `**** Assertion failed.` instead of `**** Error in metacommand.`. This distinguishes intentional script-level checks from actual metacommand errors. Upstream did not have `ASSERT`.
 
 ______________________________________________________________________
@@ -440,6 +446,10 @@ These are behavioral changes driven by security or correctness issues in the ups
 | LaTeX export escaped only the underscore                           | Cells were written as `str(c).replace("_", r"\_")`, leaving every other LaTeX special raw. `&` is the column separator this exporter itself emits, so an ampersand in a value added a column and the `.tex` no longer compiled; `%` commented out the rest of the row. All of `\ & % $ # _ { } ~ ^` are now escaped, in both the header and the body.                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
 | `ON ERROR_HALT WRITE … TO` a file wrote nothing                    | Upstream passed the `(text, flag)` pair that variable substitution returns on as the text, so writing the message failed after the file was opened and the file stayed empty. The message is written.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
 | Mixed-scale decimals keep every digit                              | `IMPORT` / `COPY … TO NEW` gave a column of decimals with different numbers of places (`1.50`, `2`, `-3.25`) a floating-point type, which also held values a double cannot store exactly: `1234567890.1234567891` was stored as `1234567890.1234567`. Such a column is still a float, unless a value has more than 15 significant digits; then it is `DECIMAL(p,s)`. The `exact_decimals` setting makes every such column exact.                                                                                                                                                                                                                                                                                                                                                                                   |
+| XML export wrote NULL as the literal "None"                        | `str()` was applied to every value, so a NULL appeared in the XML as the text None. It is now an empty element.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
+| `WRITE SCRIPT` output                                              | The `BEGIN SCRIPT` and `END SCRIPT` lines were written without `-- !x!`, so a script that `INCLUDE`d the file ran them as SQL. They are written as metacommands.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+| Line break inside a quoted value                                   | Delimiter detection read the file line by line, so a quoted value with a line break in it made every line look different and no delimiter was found: the whole file became one column named after the header line (`id,note`). The file is read as CSV, so the line break stays inside the value.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
+| `EXPORT` after `CD`                                                | `EXPORT` failed after `CD` with `No such file or directory: …/<new dir>/script.sql`, looking for the script in the new directory. The output file is written relative to the current directory.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
 | LaTeX export wrote NULL as the literal "None"                      | `str(c)` was applied to every value including `None`, so a NULL appeared in the typeset table as the word None. It is now an empty cell, matching every other exporter.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
 | HTML export rendered zero as an empty cell                         | Cells were written as `html.escape(str(v)) if v else ''`. `0`, `False`, and `""` are all falsy, so a zero measurement rendered exactly like a NULL and the two could not be told apart in the output. The test is now `if v is not None`. Both `export_html` and `export_cgi_html` were affected.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Markdown export broke the table on an embedded newline             | `_cell()` escaped pipes but not newlines, and a GFM table row is a single line: a value containing one ended the row mid-cell and left the rest of that row's values on a stray line outside the table entirely. Newlines are now written as `<br>`, which is how GFM expresses a line break inside a cell.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
@@ -451,6 +461,7 @@ These are behavioral changes driven by security or correctness issues in the ups
 | Values loaded into an existing table                               | `IMPORT` and `COPY` converted every value to the type guessed from the source data, even when the table was already there with its own column types, so a text column got a different string than the source held (`yes` and `1` stored as `true`, `01/05/2024` as `2024-01-05 00:00:00`, `1e3` as `1000`), and on PostgreSQL a column of 0s and 1s copied into an integer column failed (the guess was boolean). For a table that exists, the target column's type now decides: a text column gets the value as written, a number column gets true/false values as 1 and 0, and other columns are converted as before. Each adapter reads the column types from its catalog (`pg_attribute`, `SHOW COLUMNS`, `sys.columns`) or its driver's description of the table. `TO NEW` / `REPLACEMENT` are unchanged.     |
 | Missing dates loaded into MySQL                                    | `IMPORT` and `COPY` sent a missing value in a column scanned as a date or timestamp as `''`, so MySQL stored `0000-00-00` (or `0` in a number column) instead of NULL. It is sent as NULL.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
 | Fractions loaded into an integer column                            | A value with a fraction going into an integer column (including `DECIMAL(p,0)`, `NUMERIC(p)` and Oracle `NUMBER(p)`) of an existing table is an error naming the value and the column. Upstream sent the number and the database rounded it (`1.5` → `2`; SQL Server truncated it to `1`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+| `DISCONNECT`                                                       | Refuses a database that an open `BEGIN BATCH` has used, as documented; upstream's check compared the alias to connection objects and never fired, so the connection closed and the batch's statements were lost. The refusal is an error inside the batch, so `END BATCH` does not commit it. The alias matches in any case, as `USE` does (upstream's `DISCONNECT FROM B` left a connection made `AS b` open), and an alias that is not connected is a metacommand error (upstream ignored it) (#124).                                                                                                                                                                                                                                                                                                            |
 
 ______________________________________________________________________
 

{execsql2-2.35.0 → execsql2-2.37.0}/docs/guides/debugging.md

@@ -61,18 +61,18 @@ execsql debug>
 
 Because input runs inside the paused script, it changes what the script sees after `.continue`:
 
-| You type at the breakpoint        | Effect on the script                                                                  |
-| --------------------------------- | ------------------------------------------------------------------------------------- |
-| `!x! SUB threshold 50`            | `!!threshold!!` is 50 for the rest of the run                                         |
-| `select '!!~counter!!';`          | Reads the paused SCRIPT's `~local` and `#param` variables                             |
-| `insert into staging values (1);` | Committed as the script's own SQL would be: now, unless `AUTOCOMMIT OFF` or a batch   |
-| `!x! USE other_db`                | The script continues against `other_db`                                               |
-| `!x! BREAK`                       | Inside a `LOOP` (or a looping `EXECUTE SCRIPT`), leaves that loop, as a `BREAK` would |
-| `!x! HALT`                        | Ends the run                                                                          |
+| You type at the breakpoint        | Effect on the script                                                                    |
+| --------------------------------- | --------------------------------------------------------------------------------------- |
+| `!x! SUB threshold 50`            | `!!threshold!!` is 50 for the rest of the run                                           |
+| `select '!!~counter!!';`          | Reads the paused SCRIPT's `~local` and `#param` variables                               |
+| `insert into staging values (1);` | Committed as the script's own SQL would be: now, unless `AUTOCOMMIT OFF` or a batch     |
+| `!x! USE other_db`                | The script continues against `other_db`                                                 |
+| `!x! BREAK`                       | Leaves the `LOOP`, `SCRIPT` or `INCLUDE`d file the script paused in, as a `BREAK` would |
+| `!x! HALT`                        | Ends the run                                                                            |
 
 **Errors end only the input that caused them.** A SQL or metacommand error that would halt the script is printed, and the prompt reads the next input; `ON ERROR_HALT` actions do not run. `HALT` and `.quit` still end the run.
 
-**Transactions follow the script's rules.** Each SQL statement is committed as it runs unless `AUTOCOMMIT OFF` is in effect or the breakpoint is inside `BEGIN BATCH`. The prompt then reads `execsql debug*>`, so you can see that what you type is not committed yet.
+**Transactions follow the script's rules.** Each SQL statement is committed as it runs unless `AUTOCOMMIT OFF` is in effect or the breakpoint is inside `BEGIN BATCH`. The prompt then reads `execsql debug*>`, so you can see that what you type is not committed yet. Inside `BEGIN BATCH`, a statement or metacommand you type that fails keeps the batch from being committed, as one in the script does: `END BATCH` rolls it back and reports an error.
 
 **Commands to the REPL** start with `.`:
 

{execsql2-2.35.0 → execsql2-2.37.0}/docs/guides/formatter.md

@@ -272,7 +272,7 @@ select id,name,created_at from users where active = true order by name;
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.35.0
+    rev: v2.37.0
     hooks:
       - id: execsql-format
       - id: execsql-lint # optional: fail the commit on lint errors

{execsql2-2.35.0 → execsql2-2.37.0}/docs/reference/configuration.md

@@ -376,7 +376,7 @@ The section and property names that may be used in a configuration file are list
 :   When set to "No", the `SYSTEM_CMD` (SHELL) metacommand is disabled. Any script that attempts to execute an OS command will fail with an error. The default is "Yes". This can also be set via the `--no-system-cmd` CLI flag or `allow_system_cmd=False` in the library API. See [Security — Disabling SYSTEM_CMD](security.md#disable_system_cmd) for details.
 
 `system_cmd_timeout` { #system_cmd_timeout }
-:   Maximum number of seconds a foreground `SYSTEM_CMD` may run before *execsql* stops it. The default is `0`, which means no timeout. When a foreground command times out, `$SYSTEM_CMD_EXIT_STATUS` is set to `124` and the metacommand fails. `SYSTEM_CMD ... CONTINUE` starts a detached background process and is not timeout-managed by this setting.
+:   Maximum number of seconds a foreground `SYSTEM_CMD` may run before *execsql* stops it. The default is `0`, which means no timeout. When a foreground command times out, `$SYSTEM_CMD_EXIT_STATUS` is set to `124` and the metacommand fails. If the operating system will not let *execsql* stop the command (it runs as another user, through `sudo` for example), the error says so and gives its process ID. `SYSTEM_CMD ... CONTINUE` starts a detached background process and is not timeout-managed by this setting.
 
 `allow_rm_file` { #allow_rm_file }
 :   When set to "No", the `RM_FILE` metacommand (which deletes a file) is disabled. Any script that attempts to remove a file will fail with an error. The default is "Yes". This can also be set via the `--no-rm-file` CLI flag or `allow_rm_file=False` in the library API. Symmetric with `allow_system_cmd`.
@@ -385,13 +385,13 @@ The section and property names that may be used in a configuration file are list
 :   When set to "No", the [`SERVE`](metacommands.md#serve) metacommand (which copies a file to stdout with Content-Type and Content-Disposition headers, for use as a CGI script) is disabled. The default is "Yes". This can also be set via the `--no-serve` CLI flag or `allow_serve=False` in the library API. Symmetric with `allow_system_cmd`.
 
 `include_root` { #include_root }
-:   Root directory under which `INCLUDE` and `EXECUTE SCRIPT` targets must resolve. When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error. Default: no containment (any readable path is permitted). See [Security — Path containment roots](security.md#path-containment-roots) for the full per-handler matrix.
+:   Root directory under which the files `INCLUDE` reads must resolve (`EXECUTE SCRIPT` runs a `SCRIPT` block, not a file). When set, attempts to include files outside this root via `../`, absolute paths, drive letters, or UNC paths are rejected with an error that stops the run, even with `METACOMMAND_ERROR_HALT OFF`. Default: no containment (any readable path is permitted). See [Security — Path containment roots](security.md#path-containment-roots) for the full per-handler matrix.
 
 `serve_root` { #serve_root }
-:   Root directory under which `SERVE` targets must resolve. Same containment semantics as `include_root` — paths that escape via `../` or absolute references are rejected. Default: no containment.
+:   Root directory under which `SERVE` targets must resolve. The same containment checks as `include_root`, but a refused path is an ordinary metacommand error — paths that escape via `../` or absolute references are rejected. Default: no containment.
 
 `template_root` { #template_root }
-:   Root directory under which Jinja2 / `string.Template` loader paths must resolve. Same containment semantics as `include_root`. Default: no containment.
+:   Root directory under which Jinja2 / `string.Template` loader paths must resolve. The same containment checks as `include_root`, but a refused path is an ordinary metacommand error. Default: no containment.
 
 `max_substitution_bytes` { #max_substitution_bytes }
 :   Maximum byte size of any single substitution-variable expansion, enforced by the `substitute_vars()` engine to defeat exponential-expansion bombs (a script that nests `!!a!!` referring to `!!b!!` referring to `!!a!!` and so on can otherwise blow past memory). When set to a positive integer, expansions exceeding the threshold raise an error. Default: `10485760` (10 MB).

{execsql2-2.35.0 → execsql2-2.37.0}/docs/reference/lint.md

@@ -158,7 +158,7 @@ The `execsql-lint` [pre-commit](https://pre-commit.com/) hook runs `execsql lint
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.35.0
+    rev: v2.37.0
     hooks:
       - id: execsql-lint
         args: [--ignore, V002] # optional: any execsql lint option
@@ -340,6 +340,14 @@ An `INCLUDE` names a file that does not exist. Lint resolves a relative path aga
 
 Not reported for `INCLUDE IF EXISTS`, or when the path contains a `!!variable!!`, since the file then depends on values known only at run time.
 
+Not reported either inside an `IF` that runs only when the file exists, block or inline:
+
+```sql
+-- !x! IF (FILE_EXISTS(local_settings.sql)) { INCLUDE local_settings.sql }
+```
+
+An `ANDIF (FILE_EXISTS(…))` guards its file too; an `ORIF` lets the branch run without the file, so its `IF` guards nothing.
+
 Also reported, as "INCLUDE path cannot be checked", for a name the operating system rejects outright (longer than it allows, for example). The other scripts in the same `execsql lint` run are still checked; a script lint cannot check at all is reported as one P001 error naming it.
 
 ### I002 `missing-script` { #i002 }

{execsql2-2.35.0 → execsql2-2.37.0}/docs/reference/metacommands.md

@@ -110,6 +110,8 @@ If AUTOCOMMIT is set to ON without using the WITH clause, the next SQL statement
 
 If the WITH ROLLBACK clause is used when AUTOCOMMIT is turned back on, all SQL statements issued since AUTOCOMMIT was turned OFF will be rolled back.
 
+If that commit fails (for example, on a deferred foreign key that is checked at COMMIT), everything issued since AUTOCOMMIT was turned OFF is rolled back, together with whatever made the commit, so none of it is stored by a later commit. The failure is reported as the error of whatever made the commit: the AUTOCOMMIT ON WITH COMMIT metacommand itself or, without the WITH clause, the next statement or metacommand that commits (or END BATCH, inside a [BATCH](#batch)). [ERROR_HALT](#error_halt) governs an SQL statement's error and [METACOMMAND_ERROR_HALT](#metacommanderrorhalt) a metacommand's.
+
 
 SQL statements that were issued while AUTOCOMMIT is OFF can also be committed using a SQL "COMMIT;" statement. This approach should be avoided with SQL Server, however, because *execsql* connects with SQL Server using ODBC, and Microsoft warns that SQL statements to manage transactions should not be used ["because this can cause indeterminate behavior in the driver"](https://docs.microsoft.com/en-us/sql/relational-databases/native-client/odbc/performing-transactions-in-odbc?view=sql-server-2017). For example, two "COMMIT;" statements may be needed the first time that SQL statements are committed this way.
 
@@ -211,7 +213,16 @@ Metacommands may be included inside a batch, but note that the [IMPORT](#import)
 
 What a batch can hold back depends on the DBMS; see the table under [AUTOCOMMIT](#autocommit). MySQL and Oracle commit DDL inside a batch at once, with everything before it, and *execsql* warns when that happens.
 
-When the END BATCH metacommand is processed by *execsql*, a commit command is sent to all databases that have been used inside the batch. Multiple databases may be used inside a batch if the [USE](#use) metacommand is used inside the batch. The BATCH metacommands therefore provide a limited sort of cross-database transaction control — note, however, that the commit statements are issued sequentially, and if a later commit statement fails, this will not roll back changes to databases for which the commit has already succeeded.
+When the END BATCH metacommand is processed by *execsql*, a commit command is sent to all databases that have been used inside the batch. Multiple databases may be used inside a batch if the [USE](#use) metacommand is used inside the batch. The BATCH metacommands therefore provide a limited sort of cross-database transaction control — note, however, that the commit statements are issued sequentially, and if a later commit statement fails, this will not roll back changes to databases for which the commit has already succeeded. When a commit fails (for example, on a deferred foreign key that is checked at COMMIT), that database and the batch's databases not yet committed are rolled back, so nothing else of the batch is left for a later statement to commit; a database under [AUTOCOMMIT](#autocommit) OFF is left to the script's own COMMIT or ROLLBACK. The failure is an SQL error on the END BATCH line, reported with `Metacommand: END BATCH`: [ERROR_HALT](#error_halt), not METACOMMAND_ERROR_HALT, decides whether it halts the script. With ERROR_HALT OFF, the script carries on, and `$LAST_ERROR` is `END BATCH`. Like an SQL statement, END BATCH clears `SQL_ERROR()` before its commit and sets it if the commit fails, so test it right after END BATCH:
+
+```sql
+-- !x! END BATCH
+-- !x! IF (SQL_ERROR()) { WRITE "The batch was not committed." }
+```
+
+Anything that fails inside a batch has the same result: an SQL statement, a metacommand, or SQL that a metacommand runs (EXECUTE, EXPORT QUERY, SUBDATA, SELECT_SUB, IMPORT and the like). If anything in a batch fails, the batch is not committed. A failed SQL statement has already rolled back its connection, taking the batch's earlier statements on it with it, and a metacommand's SQL may have done the same. END BATCH rolls back what the batch ran after the failure and reports an SQL error, "The batch was not committed: an error on line ….", which the test above catches. The script reaches END BATCH only if the error did not stop it: ERROR_HALT OFF for an SQL error, METACOMMAND_ERROR_HALT OFF for a metacommand error. If ERROR_HALT is ON at END BATCH (the default), the run stops there. Put a statement or metacommand that may fail, and that the script handles, outside the batch. Under AUTOCOMMIT OFF, END BATCH still reports the error, but the script's own transaction is left to its COMMIT or ROLLBACK. On a connection whose driver commits every statement (an ODBC DSN without transactions), the batch's statements were committed as they ran, whatever END BATCH reports.
+
+If the failing batch is nested in another, or the failure is in a nested batch, the rollback can take the enclosing batch's earlier statements with it, so the enclosing batch is rolled back at its own END BATCH instead of committing, with an error of its own ("The batch was not committed: …", naming the failure). Once anything in a batch has failed, no END BATCH in that stack of batches commits: a batch opened inside the enclosing batch afterwards is not committed either, and ROLLBACK BATCH does not undo the failure. IMPORT, COPY, EXECUTE and AUTOCOMMIT ON WITH COMMIT, which commit on their own, still do unless AUTOCOMMIT is OFF, as they do in any batch. Statements after the outermost END BATCH run and commit as usual.
 
 The BEGIN/END BATCH metacommands can be nested. However, the inner END BATCH metacommand will ordinarily commit all changes to the databases that have been used, which may include databases used in the outer batch as well. (This behavior may be DBMS-specific, and also depend on whether any additional explicit transaction control is being used.) Therefore completion of a nested batch may result in premature commitment of some or all SQL statements in the outer batch. Similarly, a ROLLBACK BATCH metacommand within the inner batch will also roll back any SQL commands sent to the same databases in the outer batch. Thus, although the BATCH commands can be nested, database transactions may not be, depending on the DBMS in use. Nesting of BATCH metacommands allows a script file or a [SCRIPT](#beginscript) containing a batch to be [INCLUDEEd](#include) or [EXECUTEEd](#executescript), respectively, within another batch.
 
@@ -226,7 +237,23 @@ Alternatives to using batches to control the execution time of SQL statements ar
 
 The END BATCH metacommand is analogous to the "GO" command of the T-SQL language used with SQL Server utilities such as *sqlcmd*. There is no explicit equivalent to BEGIN BATCH in *sqlcmd* or other SQL Server utilities. In *sqlcmd* a new batch is automatically begun at the beginning of the script or immediately after a GO statement. execsql only starts a new batch when a BEGIN BATCH statement is encountered.
 
-If the end of the script file is encountered while a batch of statements is being compiled, but there is no END BATCH metacommand, the SQL statements in that incomplete batch will not be committed.
+A batch is committed when it reaches END BATCH, unless something in it failed (see below). A batch left any other way is rolled back: a [BREAK](#break) out of a LOOP, SCRIPT or included file the batch is in, a [HALT](#halt), or an error that stops the run. The rollback cannot undo what was already committed inside the batch: by an IMPORT or COPY, by a nested batch's END BATCH, or by the DBMS itself, as MySQL and Oracle do for DDL (see above). Like ROLLBACK BATCH, it rolls back the connection's whole transaction, so statements held from before the batch under AUTOCOMMIT OFF, and not committed when AUTOCOMMIT was turned back ON, are rolled back with it. A nested batch left this way, while the batch around it carries on, is not rolled back on its own: its statements stay with the outer batch, which commits them at its END BATCH or rolls them back if it is left early too. With [AUTOCOMMIT](#autocommit) OFF, END BATCH commits nothing, and leaving a batch early rolls nothing back; the script's own COMMIT or ROLLBACK decides. A BEGIN BATCH with no END BATCH is an error before the script runs. To keep a batch's work and then leave a loop, end the batch first:
+
+```sql
+create table passes (n integer);
+-- !x! SUB n 0
+-- !x! LOOP WHILE (TRUE)
+-- !x! SUB_ADD n 1
+-- !x! BEGIN BATCH
+insert into passes values (!!n!!);
+-- !x! END BATCH
+-- !x! IF (EQUALS(!!n!!, 3)) { BREAK }
+-- !x! END LOOP
+```
+
+`passes` ends up with 1, 2 and 3. With the `IF` before `END BATCH`, the third pass's row would be rolled back.
+
+When an error or a HALT stops the run inside a batch, the batch is rolled back before an [ON ERROR_HALT EXECUTE SCRIPT](#error_halt_exec) or [ON CANCEL_HALT EXECUTE SCRIPT](#cancel_halt_exec) script runs, so that script's own SQL is committed, such as a row it writes to a log table (unless AUTOCOMMIT is OFF).
 
 
 ## BEGIN SCRIPT and END SCRIPT { #beginscript }
@@ -374,9 +401,32 @@ Metacommands that appear within a BEGIN/END SQL block will be treated as comment
 BREAK
 ```
 
-This metacommand will cause an immediate exit from any [LOOP](#loop), [SCRIPT](#beginscript) or [INCLUDEEd](#include) file. If there are multiple levels of these structures active, only the latest level will be exited.
+This metacommand will cause an immediate exit from any [LOOP](#loop), [SCRIPT](#beginscript) or [INCLUDEEd](#include) file. If there are multiple levels of these structures active, only the latest level will be exited:
+
+- In a `LOOP`, it ends the loop, and the script carries on after `END LOOP`.
+- In a `SCRIPT`, outside any `LOOP` of its own, it returns from the script. A `SCRIPT` that [EXECUTE SCRIPT](#executescript) repeats with `WHILE` or `UNTIL` stops repeating.
+- In an `INCLUDE`d file, outside any `LOOP` in that file, it leaves the file, and the script that included it carries on, even when the `INCLUDE` is inside a `LOOP`.
 
-If this metacommand is used when none of these structures are active, a warning will be issued.
+A `BREAK` therefore never ends a loop that called the `SCRIPT` or included the file it is in:
+
+```
+-- !x! BEGIN SCRIPT check_one
+-- !x! IF (NOT HASROWS(staging))
+-- !x!   BREAK
+-- !x! ENDIF
+-- !x! WRITE "staging has rows"
+-- !x! END SCRIPT
+
+-- !x! SUB n 0
+-- !x! LOOP WHILE (IS_GT(3, !{n}!))
+-- !x! SUB_ADD n 1
+-- !x! EXECUTE SCRIPT check_one
+-- !x! END LOOP
+```
+
+Here a `BREAK` in `check_one` skips the rest of that call, and the loop still runs all its passes.
+
+A `BREAK` in the main script, outside any of these structures, has nothing to leave. It is a metacommand error: the run stops unless [METACOMMAND_ERROR_HALT](#metacommanderrorhalt) is `OFF`, in which case the script carries on and `METACOMMAND_ERROR()` is true.
 
 
 ## CANCEL_HALT
@@ -953,11 +1003,19 @@ Writes internal state to the log file (`LOG` form) or to the terminal or a file
 DISCONNECT [[FROM] <alias>]
 ```
 
-Closes the current database connection, or the connection associated with the alias *alias*, if specified. Aliases are defined by the [CONNECT](#connect) metacommand.
+Closes the current database connection, or the connection associated with the alias *alias*, if specified. Aliases are defined by the [CONNECT](#connect) metacommand and, as with [USE](#use), match in any case: `DISCONNECT FROM B` closes a connection made `AS b`.
+
+Disconnecting from an alias that is not connected is an error, so a misspelled alias does not leave the real connection open. Like other metacommand errors, it halts the script unless [METACOMMAND_ERROR_HALT](#metacommanderrorhalt) is OFF. To disconnect only when the alias is connected, test it first:
+
+```sql
+-- !x! IF (ALIAS_DEFINED(staging)) { DISCONNECT FROM staging }
+```
+
+Write the alias in lowercase inside `ALIAS_DEFINED`: aliases are stored in lowercase, and `ALIAS_DEFINED` compares them as written, so `ALIAS_DEFINED(Staging)` is false for a connection made `AS Staging`.
 
-Attempting to disconnect from the initial database connection will result in an error, and *execsql* will halt.
+Attempting to disconnect from the initial database connection is also an error.
 
-Attempting to disconnect from a database that is still in use in a [BATCH](#batch) will result in an error, and *execsql* will halt.
+So is attempting to disconnect from a database that is still in use in a [BATCH](#batch), that is, one the batch, or a batch around it, has run a statement on. The connection stays open. If the error halts the script, the batch is rolled back; with METACOMMAND_ERROR_HALT OFF the script carries on, but the refusal is an error inside the batch, so END BATCH rolls the batch back instead of committing it (see [BATCH](#batch)). A database the batch has not used can be disconnected inside it.
 
 Explicitly disconnecting from a database is not necessary before re-using that alias. The [CONNECT](#connect) metacommand will automatically disconnect from a database if its alias is re-used.
 
@@ -989,9 +1047,11 @@ ERROR_HALT ON|OFF
 ```
 
 
-When ERROR_HALT is set to ON, which is the default, any errors that occur as a result of executing a SQL statement will cause an error message to be displayed immediately, and *execsql* will exit. When ERROR_HALT is set to OFF, then SQL errors will be ignored, but can be evaluated with the [IF SQL_ERROR](#sqlerror) conditional.
+When ERROR_HALT is set to ON, which is the default, any errors that occur as a result of executing a SQL statement, or committing a [BATCH](#batch) at END BATCH, will cause an error message to be displayed immediately, and *execsql* will exit. When ERROR_HALT is set to OFF, then SQL errors will be ignored, but can be evaluated with the [IF SQL_ERROR](#sqlerror) conditional. A test in an `IF`, `LOOP` or other condition that fails in the database (`HASROWS(no_such_table)`) is not ignored: it stops the run, because no branch can be chosen (see [METACOMMAND_ERROR_HALT](#metacommanderrorhalt)).
+
+A statement whose commit fails (for example, on a deferred foreign key that is checked at COMMIT) is an SQL error like any other, and its transaction is rolled back, so no later commit stores it.
 
-When ERROR_HALT is set to OFF inside a transaction, any SQL error will ordinarily cause the entire transaction to fail.
+When ERROR_HALT is set to OFF inside a transaction, any SQL error will ordinarily cause the entire transaction to fail. Inside a [BATCH](#batch), an error means END BATCH rolls the batch back instead of committing it, and reports an SQL error of its own; if ERROR_HALT is ON at END BATCH, the run stops there.
 
 
 ## EXECUTE { #execute }
@@ -1850,7 +1910,7 @@ SQL_ERROR()
 
 Evaluates whether the previous SQL statement generated an error. Errors will result from badly-formed SQL, reference to non-existent database objects, lack of permissions, or database locks. A query (e.g., an update query) that does not do exactly what you expect it to will not necessarily cause an error to occur that can be identified with this statement. This test for SQL errors will only be effective if the [ERROR_HALT OFF](#error_halt) metacommand has previously been issued.
 
-Errors in metacommands and some other errors encountered by *execsql* will cause the program to halt immediately, regardless of the setting of [ERROR_HALT](#error_halt) or the use of the IF(SQL_ERROR()) test.
+Errors in metacommands and some other errors encountered by *execsql* will cause the program to halt immediately, regardless of the setting of [ERROR_HALT](#error_halt) or the use of the IF(SQL_ERROR()) test. A commit that fails at END BATCH is an SQL error, not a metacommand error: ERROR_HALT governs it. END BATCH clears SQL_ERROR() before its commit, as an SQL statement does, so right after END BATCH it says whether the batch was committed: it is set when the commit fails and when an error earlier in the batch, from an SQL statement or a metacommand, kept it from being committed.
 
 
 ### *STARTS_WITH*
@@ -2143,7 +2203,9 @@ The [BREAK](#break) metacommand can be used to exit from a loop at any time.
 METACOMMAND_ERROR_HALT ON|OFF
 ```
 
-When METACOMMAND_ERROR_HALT is set to ON, which is the default, any errors that occur during execution of a metacommand will cause an error message to be displayed immediately, and *execsql* to halt. When METACOMMAND_ERROR_HALT is set to OFF, then metacommand errors will be ignored, but can be evaluated with the [IF METACOMMAND_ERROR](#metacommanderror) conditional; `$LAST_ERROR` and `$ERROR_MESSAGE` hold the failed metacommand and its message.
+When METACOMMAND_ERROR_HALT is set to ON, which is the default, any errors that occur during execution of a metacommand will cause an error message to be displayed immediately, and *execsql* to halt. When METACOMMAND_ERROR_HALT is set to OFF, then metacommand errors will be ignored, but can be evaluated with the [IF METACOMMAND_ERROR](#metacommanderror) conditional; `$LAST_ERROR` and `$ERROR_MESSAGE` hold the failed metacommand and its message. Inside a [BATCH](#batch), a metacommand error that the script carries on from still keeps the batch from being committed: END BATCH rolls it back and reports an SQL error.
+
+A condition that cannot be evaluated is not covered by this setting: an `IF`, `ELSEIF`, `ANDIF`, `ORIF`, `LOOP` or `EXECUTE SCRIPT … WHILE` / `UNTIL` whose condition names an unknown test or is malformed (`IF (NO_SUCH_FN(x))`), or whose test fails in the database (`HASROWS(no_such_table)`), stops the run even with METACOMMAND_ERROR_HALT OFF, because no branch can be chosen. `execsql lint` reports an unknown test or malformed condition before a run (P004); a test that fails in the database shows up only when the script runs.
 
 The METACOMMAND_ERROR_HALT metacommand does not itself set or reset the internal flag that indicates whether a metacommand has encountered an error. This is so that a "METACOMMAND_ERROR_HALT Off" command can be used immediately after a potentially-failing metacommand and not alter the error flag that is set by the previous command. Specifically, constructions like this:
 
@@ -2200,6 +2262,8 @@ If the specified script runs after the user cancels the main script, it will cle
 
 The intended use of this metacommand is to allow cleanup of staging tables and any other database objects that should be removed when the main script terminates.
 
+If the user cancels inside a [BATCH](#batch), the batch is rolled back before the script runs, and the script's own SQL is committed as usual (not while [AUTOCOMMIT](#autocommit) is OFF).
+
 
 ## ON CANCEL_HALT WRITE { #cancel_halt_write }
 
@@ -2267,6 +2331,8 @@ If the specified script runs after an error occurs, it will clear its own specif
 
 The intended use of this metacommand is to allow cleanup of staging tables and any other database objects that would be removed if the main script terminated normally.
 
+If the error happens inside a [BATCH](#batch), the batch is rolled back before the script runs, and the script's own SQL is committed as usual (not while [AUTOCOMMIT](#autocommit) is OFF).
+
 
 ## ON ERROR_HALT WRITE { #error_halt_write }
 

{execsql2-2.35.0 → execsql2-2.37.0}/docs/reference/security.md

@@ -161,9 +161,11 @@ choice you make.
 
 Containment roots are opt-in. When configured, paths must resolve under the configured root; attempts to escape with `../`, absolute paths, drive letters, or UNC paths are rejected.
 
+An `INCLUDE` refused by `include_root` stops the run even with [`METACOMMAND_ERROR_HALT OFF`](metacommands.md#metacommanderrorhalt): the refusal is a security boundary, not a script error to carry on from, so the statements after it never run without the file they expected.
+
 | Setting         | Applies to                                | Default      |
 | --------------- | ----------------------------------------- | ------------ |
-| `include_root`  | `INCLUDE` and `EXECUTE SCRIPT` targets    | unrestricted |
+| `include_root`  | `INCLUDE` files                           | unrestricted |
 | `serve_root`    | `SERVE` targets                           | unrestricted |
 | `template_root` | Jinja2 and `string.Template` loader paths | unrestricted |
 

{execsql2-2.35.0 → execsql2-2.37.0}/docs/reference/substitution_vars.md

@@ -14,6 +14,8 @@ The \<match_string> identifies the substitution variable; the \<replacement_stri
 - `!'!name!'!` — wrap the value in single quotes, doubling any embedded apostrophe; on a MySQL or MariaDB connection, which reads `\` in a string literal as an escape, backslashes are doubled too. The value is stored exactly as written (`C:\data\new` stays `C:\data\new`). Suitable for SQL string literals containing user data. On PostgreSQL, `E!'!name!'!` (an escape string) doubles backslashes too, so it stores the same value as `!'!name!'!`; the `E` is not needed.
 - `!"!name!"!` — wrap the value in double quotes. Suitable for SQL quoted identifiers, or for metacommand arguments that must be quoted.
 
+A variable can supply a whole metacommand (`-- !x! !!cmd!!` runs whatever metacommand `cmd` holds), except `INCLUDE`, `EXECUTE SCRIPT` (or `EXEC SCRIPT` / `RUN SCRIPT`) and the block metacommands (`IF`, `ELSEIF`, `ANDIF`, `ORIF`, `ELSE`, `ENDIF`, `LOOP`, `END LOOP`, `BEGIN BATCH`, `END BATCH`, `BEGIN SCRIPT` (or `CREATE SCRIPT`), `END SCRIPT`, `BEGIN SQL`, `END SQL`): execsql2 reads those when it parses the script, before substitution, so one supplied by a variable does not run. Most of these are a metacommand error, which `METACOMMAND_ERROR_HALT OFF` skips without printing anything (test for it with `METACOMMAND_ERROR()`; `execsql.run()` still lists it in `result.errors`); `END LOOP`, `BEGIN` / `CREATE` / `END SCRIPT` and `BEGIN` / `END SQL` stop the run whatever that setting says. Write the metacommand in the script and put the variable in its argument: `INCLUDE !!file!!`, `EXECUTE SCRIPT !!name!!`, `IF (!!condition!!)`.
+
 Variables can also be set before the script starts: with `--var NAME=VALUE` on the [command line](../getting-started/syntax.md#options), or in the `[variables]` section of a [config file](configuration.md#configuration). `--var` wins over the config file, and a `SUB` in the script can reassign either.
 
 For example, to use a data variable that may contain apostrophes in a SQL string literal:

{execsql2-2.35.0 → execsql2-2.37.0}/extras/vscode-execsql/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "execsql-syntax",
-  "version": "2.35.0",
+  "version": "2.37.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "execsql-syntax",
-      "version": "2.35.0",
+      "version": "2.37.0",
       "license": "GPL-3.0-or-later",
       "dependencies": {
         "vscode-languageclient": "10.1.2"

{execsql2-2.35.0 → execsql2-2.37.0}/extras/vscode-execsql/package.json

@@ -2,7 +2,7 @@
   "name": "execsql-syntax",
   "displayName": "execsql",
   "description": "Syntax highlighting, lint findings, completion, hover, navigation, quick fixes and formatting for execsql scripts.",
-  "version": "2.35.0",
+  "version": "2.37.0",
   "publisher": "geocoug",
   "license": "GPL-3.0-or-later",
   "icon": "icon.png",