execsql2 2.13.2__tar.gz → 2.15.0__tar.gz

{execsql2-2.13.2 → execsql2-2.15.0}/.claude/project_context.md

@@ -153,7 +153,7 @@ ______________________________________________________________________
 | `pre-commit`        | Git hooks                | gitleaks, uv-lock, ruff, mdformat, markdownlint, typos, validate-pyproject                              |
 | `zensical`          | Docs builder             | MkDocs-compatible, Material theme, configured via `zensical.toml`                                       |
 | `mkdocstrings`      | API docs from docstrings | Wired up — `docs/api/` pages auto-generate from source                                                  |
-| `pytest-cov`        | Coverage                 | `--cov-fail-under=80` enforced                                                                          |
+| `pytest-cov`        | Coverage                 | `--cov-fail-under=90` enforced (raised from 80% in v2.12)                                               |
 
 ## Package Layout Decision
 
@@ -192,7 +192,7 @@ Triggered on: push to `main`, any tag `v*.*.*`, pull requests.
 
 ## Versioning
 
-`bump-my-version` manages versions. Current: `2.11.0`. Bump commands:
+`bump-my-version` manages versions. Current: `2.13.2`. Bump commands:
 
 - `just bump-patch` → 2.11.0 → 2.11.1
 - `just bump-minor` → 2.11.0 → 2.12.0
@@ -271,6 +271,16 @@ the foreseeable future.
 | `BREAKPOINT` debug REPL with step mode             | 2.10.0         | 2026-04 |
 | Error messages restored (script location, command) | 2.11.0         | 2026-04 |
 | 16-item codebase analysis fix sweep                | 2.11.x         | 2026-04 |
+| pg-upsert integration (`PG_UPSERT` metacommand)   | 2.12.0         | 2026-04 |
+| Coverage floor raised to 90% (3,600+ tests)        | 2.12.x         | 2026-04 |
+| `--debug` step-through mode                        | 2.12.x         | 2026-04 |
+| `--lint` two-pass variables, EXECUTE SCRIPT flow   | 2.13.x         | 2026-04 |
+| GUI: row counts, help URL button, compare diffs    | unreleased     | 2026-04 |
+| GUI: PROMPT ENTRY_FORM all entry_type values       | unreleased     | 2026-04 |
+| GUI: PROMPT COMPARE AND/BESIDE orientation fix     | unreleased     | 2026-04 |
+| GUI: highlight diffs toggle (Tkinter + Textual)    | unreleased     | 2026-04 |
+| Metacommand audit: regex/doc/handler bug fixes     | unreleased     | 2026-04 |
+| Removed FREE keyword from PROMPT DISPLAY           | unreleased     | 2026-04 |
 
 ______________________________________________________________________
 
@@ -298,6 +308,11 @@ ______________________________________________________________________
 
 ### Candidate Features (unscheduled — pick and assign to milestones)
 
+#### GUI — Essential
+
+- [ ] **Textual console (`CONSOLE ON`)** — The Textual TUI backend has a stub `console_on()` that sets a flag but provides no actual GUI console. Tkinter has a full console window (ConsoleWindow) with output area, status bar, and progress bar. Textual needs an equivalent: a persistent RichLog-based panel that receives WRITE output, shows status, and progress. **This is a gap — users on `--gui-framework textual` with `CONFIG GUI_LEVEL 3` or `CONSOLE ON` get no console.**
+- [ ] **Form validation** — `EntrySpec` has `validation_regex` and `validation_key_regex` fields but no backend enforces them. Tkinter should validate on focus-loss (validation_regex) and per-keystroke (validation_key_regex). Textual should validate on submit. Console should validate before accepting.
+
 #### Quick Wins
 
 - [x] **`--ping`** — test database connectivity and exit with a status message. Useful for CI health checks and connection debugging. (shipped v2.8.x)
@@ -359,6 +374,7 @@ ______________________________________________________________________
 ### Ongoing / No-milestone
 
 - Textual TUI polish
+- Coverage target: raise from 90% → 93% (in progress)
 
 ______________________________________________________________________
 

{execsql2-2.13.2 → execsql2-2.15.0}/CHANGELOG.md

@@ -13,6 +13,66 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.15.0] - 2026-04-09
+
+### Added
+
+- `PG_UPSERT` metacommand: new `EXPORT_FAILURES <dir>`, `EXPORT_FORMAT csv|json|xlsx`, and `EXPORT_MAX_ROWS <n>` keywords that write a "fix sheet" of failing QA rows — one row per unique violating staging row with a consolidated `_issues` column — to CSV, JSON, or XLSX. Works in all three modes (full pipeline, QA-only, schema check) and runs even when QA fails. New `$PG_UPSERT_EXPORT_PATH` substitution variable holds the directory written. A user-visible message reporting the export directory and format is emitted to both the console and the execsql log after every export.
+
+### Changed
+
+- `[upsert]` extra now requires `pg-upsert>=1.21.0` (up from `>=1.20.0`) for the fix-sheet export feature.
+
+### Fixed
+
+- `PROMPT MESSAGE ... CREDENTIALS <user_var> <pw_var>` no longer crashes in console-fallback mode with `TypeError: get_password() missing 2 required positional arguments: 'database_name' and 'user_name'`. The fallback now uses `getpass.getpass()` to read the password, matching the intent (keyring-aware `auth.get_password()` is for CONNECT, not for bare credential prompts).
+
+______________________________________________________________________
+
+## [2.14.1] - 2026-04-07
+
+### Fixed
+
+- Fix Windows CI: use `zf.namelist()[0]` instead of path for zip entry lookup.
+
+______________________________________________________________________
+
+## [2.14.0] - 2026-04-07
+
+### Added
+
+- Row count footer displayed below every table in GUI dialogs (Textual TUI, Tkinter desktop, and console fallback). Shows format like "3 rows" or "1 row" with comma-separated thousands for large counts.
+- Help URL button in all GUI dialogs that support the `HELP` keyword. Clicking the button opens the URL in the system browser. Console fallback prints the URL.
+- Diff summary line in compare dialogs showing matching, differing, and table-exclusive row counts (e.g., "3 matching | 1 differing | 2 only in Table 1").
+- `PROMPT ENTRY_FORM` now enforces `validation_regex` (on submit) and `validation_key_regex` (per-keystroke) validation across all GUI backends. Required fields are also validated on submit. Tkinter shows a messagebox on validation failure; Textual shows a notification; Console re-prompts.
+- "Highlight Diffs" toggle button in compare dialogs (Textual and Tkinter) that color-codes rows: green for matching, yellow for changed, red for rows only in one table.
+
+### Fixed
+
+- `CONFIG GUI_LEVEL` now accepts value `3` (open GUI console on start), matching the `gui_level` configuration file setting.
+- `PROMPT COMPARE` now respects the `AND` vs `BESIDE` keyword: `AND` stacks tables vertically, `BESIDE` displays them side-by-side. Previously both orientations displayed side-by-side.
+- `PROMPT ENTRY_FORM` now renders all documented `entry_type` values: `listbox` (multi-select list), `radiobuttons` (radio button group), `textarea` (multi-line text area), `inputfile` and `outputfile` (text field with file browser button). Previously only `checkbox` and `dropdown`/`select` were implemented; all others fell through to a plain text input.
+- `PROMPT ENTER_SUB` HELP URL regex typo: quoted HELP URLs containing `+` characters now match correctly (was using `[^+]` instead of `[^"]`).
+- PostgreSQL and DSN `CONNECT` handlers now unquote the PASSWORD parameter consistently with all other database handlers.
+- SQL Server `CONNECT` handler now uses consistent keyword argument `user_name=` in all code paths.
+
+### Changed
+
+- Documentation: added `CONFIG LOG_SQL` and `CONFIG SHOW_PROGRESS` sections to metacommands reference (were implemented but undocumented).
+- Documentation: DuckDB `CONNECT` syntax now shows the `NEW` keyword (was supported but undocumented).
+- Documentation: `EXPORT QUERY` format list now explicitly mentions PARQUET, FEATHER, YAML, MARKDOWN support.
+- Documentation: added alias notes for `EXEC SCRIPT` / `RUN SCRIPT` and `APPEND SCRIPT`.
+- Documentation: `RM_SUB` now documents `~` prefix for deleting local variables.
+- Documentation: fixed missing bracket in `WRITE CREATE_TABLE FROM EXCEL` syntax.
+
+### Removed
+
+- `FREE` keyword from `PROMPT DISPLAY` metacommand. The non-blocking display behavior was only implemented in the console backend; Textual and Tkinter backends ignored it.
+- Tkinter dialog buttons are now right-aligned (matching the Textual TUI layout) instead of centered.
+- Tkinter dialog message text is now left-aligned instead of center-justified.
+
+______________________________________________________________________
+
 ## [2.13.2] - 2026-04-06
 
 ### Changed

{execsql2-2.13.2 → execsql2-2.15.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: execsql2
-Version: 2.13.2
+Version: 2.15.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Homepage, https://execsql2.readthedocs.io
 Project-URL: Repository, https://github.com/geocoug/execsql
@@ -51,7 +51,7 @@ Requires-Dist: keyring; extra == 'all'
 Requires-Dist: odfpy; extra == 'all'
 Requires-Dist: openpyxl; extra == 'all'
 Requires-Dist: oracledb; extra == 'all'
-Requires-Dist: pg-upsert>=1.20.0; extra == 'all'
+Requires-Dist: pg-upsert>=1.21.0; extra == 'all'
 Requires-Dist: polars; extra == 'all'
 Requires-Dist: psycopg2-binary; extra == 'all'
 Requires-Dist: pymysql; extra == 'all'
@@ -109,7 +109,7 @@ Requires-Dist: oracledb; extra == 'oracle'
 Provides-Extra: postgres
 Requires-Dist: psycopg2-binary; extra == 'postgres'
 Provides-Extra: upsert
-Requires-Dist: pg-upsert>=1.20.0; extra == 'upsert'
+Requires-Dist: pg-upsert>=1.21.0; extra == 'upsert'
 Description-Content-Type: text/markdown
 
 > [!NOTE]

{execsql2-2.13.2 → execsql2-2.15.0}/docs/about/divergence.md

@@ -41,14 +41,14 @@ ______________________________________________________________________
 
 ### Metacommands
 
-| Metacommand            | Description                                                                                                                                                                                                                      |
-| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `ASSERT`               | Evaluate a condition and raise an error (halting the script) if it is false. Supports all IF conditions. Optional quoted failure message. Skipped in false IF blocks.                                                            |
-| `BREAKPOINT`           | Pause script execution and drop into an interactive debug REPL. See [Debugging](#debugging) below for full details.                                                                                                              |
-| `CONFIG SHOW_PROGRESS` | Enable the Rich progress bar for IMPORT operations at runtime.                                                                                                                                                                   |
-| `CONFIG LOG_SQL`       | Enable SQL query audit logging — writes executed SQL to the log file.                                                                                                                                                            |
-| `PG_UPSERT`            | QA-checked, FK-dependency-ordered upserts from staging to base schema on PostgreSQL. Integrates [pg-upsert](https://pg-upsert.readthedocs.io/) as an optional dependency. Three modes: full pipeline, QA-only, and schema check. |
-| `IMPORT … FROM JSON`   | Import a JSON file (array of objects or NDJSON) into a database table. Nested objects are flattened with dot-separated column names; arrays are stored as JSON strings.                                                          |
+| Metacommand            | Description                                                                                                                                                                                                                                                                                                                                                                                             |
+| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ASSERT`               | Evaluate a condition and raise an error (halting the script) if it is false. Supports all IF conditions. Optional quoted failure message. Skipped in false IF blocks.                                                                                                                                                                                                                                   |
+| `BREAKPOINT`           | Pause script execution and drop into an interactive debug REPL. See [Debugging](#debugging) below for full details.                                                                                                                                                                                                                                                                                     |
+| `CONFIG SHOW_PROGRESS` | Enable the Rich progress bar for IMPORT operations at runtime.                                                                                                                                                                                                                                                                                                                                          |
+| `CONFIG LOG_SQL`       | Enable SQL query audit logging — writes executed SQL to the log file.                                                                                                                                                                                                                                                                                                                                   |
+| `PG_UPSERT`            | QA-checked, FK-dependency-ordered upserts from staging to base schema on PostgreSQL. Integrates [pg-upsert](https://pg-upsert.readthedocs.io/) as an optional dependency. Three modes: full pipeline, QA-only, and schema check. Supports `EXPORT_FAILURES`, `EXPORT_FORMAT`, and `EXPORT_MAX_ROWS` keywords to write a "fix sheet" of failing QA rows to CSV/JSON/XLSX (requires `pg-upsert>=1.21.0`). |
+| `IMPORT … FROM JSON`   | Import a JSON file (array of objects or NDJSON) into a database table. Nested objects are flattened with dot-separated column names; arrays are stored as JSON strings.                                                                                                                                                                                                                                 |
 
 ### Conditional Tests
 
@@ -79,10 +79,14 @@ New options in `execsql.conf`:
 
 ### GUI
 
-| Feature             | Description                                                                                                                                             |
-| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Textual TUI backend | Full terminal-UI backend via the `textual` library. Provides all dialog types (password, pause, message, entry, compare, action, etc.) in the terminal. |
-| Console fallback    | Text-only backend that handles GUI calls in headless environments by printing to stdout.                                                                |
+| Feature              | Description                                                                                                                                             |
+| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Textual TUI backend  | Full terminal-UI backend via the `textual` library. Provides all dialog types (password, pause, message, entry, compare, action, etc.) in the terminal. |
+| Console fallback     | Text-only backend that handles GUI calls in headless environments by printing to stdout.                                                                |
+| Table row counts     | All GUI backends (Tkinter, Textual, Console) display a row count footer below every table widget (e.g. "3 rows", "1 row").                              |
+| Help URL button      | Dialogs that accept the `HELP` keyword display a clickable Help button that opens the URL in the system browser.                                        |
+| Compare diff summary | The compare dialog shows a one-line summary of matching, differing, and table-exclusive rows when key columns are specified.                            |
+| Form validation      | `PROMPT ENTRY_FORM` enforces `validation_regex` (on submit) and `validation_key_regex` (per-keystroke) across all backends. Required fields validated.  |
 
 ### Authentication
 
@@ -235,7 +239,8 @@ ______________________________________________________________________
 
 ## Removed Features
 
-| Feature                     | Reason                                                                                                                                                                                  |
-| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Airspeed template processor | The `airspeed` library (Velocity clone) is unmaintained since ~2018. Use `FORMAT jinja` instead. The `airspeed` value for `template_processor` in `execsql.conf` is no longer accepted. |
-| Python 2 compatibility      | All Python 2 constructs (`stringtypes`, `u""` literals, `optparse`, etc.) have been removed. execsql2 requires Python 3.10+.                                                            |
+| Feature                            | Reason                                                                                                                                                                                  |
+| ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Airspeed template processor        | The `airspeed` library (Velocity clone) is unmaintained since ~2018. Use `FORMAT jinja` instead. The `airspeed` value for `template_processor` in `execsql.conf` is no longer accepted. |
+| Python 2 compatibility             | All Python 2 constructs (`stringtypes`, `u""` literals, `optparse`, etc.) have been removed. execsql2 requires Python 3.10+.                                                            |
+| `FREE` keyword on `PROMPT DISPLAY` | The non-blocking display behavior was only implemented in the console backend; the Textual and Tkinter GUI backends ignored it. Removed rather than partially supported.                |

{execsql2-2.13.2 → execsql2-2.15.0}/docs/reference/metacommands.md

@@ -417,13 +417,14 @@ This setting is also applied to the conversion of spreadsheet names to table nam
 CONFIG GUI_LEVEL <n>
 ```
 
-The level of interaction with the user that should be carried out using GUI dialogs. The numeric value *n* must be 0, 1, or 2. The meanings of these values are:
+The level of interaction with the user that should be carried out using GUI dialogs. The numeric value *n* must be 0, 1, 2, or 3. The meanings of these values are:
 
 > - 0: Do not use any optional GUI dialogs.
 > - 1: Use GUI dialogs for password prompts and for the [PAUSE](#pause) metacommand.
 > - 2: Also use a GUI dialog if a message is included with the [HALT](#halt) metacommand, and prompt for the initial database to use if no database connection parameters are specified in a configuration file or on the command line.
+> - 3: Additionally, open a GUI console when *execsql* starts.
 
-This is equivalent to the `gui_level` [configuration setting](configuration.md#gui_level) except that the corresponding configuration setting also allows a value of 3.
+This is equivalent to the `gui_level` [configuration setting](configuration.md#gui_level).
 
 
 <a id="config_hdf5"></a>
@@ -457,6 +458,13 @@ CONFIG LOG_DATAVARS YES|NO
 Controls whether data variables that are created by the [SELECT_SUB](#select_sub), [PROMPT SELECT_SUB](#prompt_selsub) and [PROMPT ACTION](#prompt_action) metacommands are written to *execsql*'s [log file](../guides/logging.md#logging). By default, all data variable assignments are logged. The performance of scripts that make extensive use of these metacommands (e.g., [Example 27](../guides/examples.md#example27)) can be improved by changing this setting to 'No'.
 
 
+```
+CONFIG LOG_SQL YES|NO
+```
+
+Enable or disable SQL query audit logging at runtime. When enabled, each SQL statement executed against the database is written to *execsql*'s [log file](../guides/logging.md#logging) as a `sql` record containing the database name, source line number, and query text. This is equivalent to the `log_sql` [configuration setting](configuration.md#config_logging). The default value is "No".
+
+
 ```
 CONFIG LOG_WRITE_MESSAGES YES|NO
 ```
@@ -506,6 +514,13 @@ CONFIG SCAN_LINES <n>
 The number of lines of a data file to scan during [IMPORT](#import) to determine the quoting character and delimiter character used. This is equivalent to the "-s" command-line option and the `scan_lines` [configuration setting](configuration.md#scan_lines).
 
 
+```
+CONFIG SHOW_PROGRESS YES|NO
+```
+
+Enable or disable the Rich progress bar for [IMPORT](#import) operations at runtime. When enabled, a progress bar is displayed showing the number of rows imported. This is equivalent to the `show_progress` [configuration setting](configuration.md#config_input) and the `--progress` CLI option. The default value is "No".
+
+
 ```
 CONFIG TRIM_COLUMN_HEADERS NONE|BOTH|LEFT|RIGHT
 ```
@@ -635,7 +650,7 @@ CONNECT USER TO MARIADB(SERVER=<server_name>, DB=<database_name>
 For DuckDB:
 
 ```
-CONNECT TO DUCKDB(FILE=<database_file>) AS <alias_name>
+CONNECT TO DUCKDB(FILE=<database_file> [, NEW]) AS <alias_name>
 ```
 
 For Firebird:
@@ -686,7 +701,7 @@ If the command form with the "USER" keyword is used, the user name that is used
 
 The alias name that is specified in this command can be used to refer to this database in the [USE](#use) and [COPY](#copy) metacommands. Alias names can consist only of letters, digits, and underscores, and must start with a letter. The alias name "initial" is reserved for the database that is used when *execsql* starts script processing, and cannot be used with the [CONNECT](#connect) metacommand. If you re-use an alias name, the connection to the database to which that name was previously assigned will be closed, and the database will no longer be available. Using the same alias for two different databases allows for mistakes wherein script statements are run on the wrong database, and so is not recommended.
 
-If the "NEW" keyword is used with PostgreSQL or SQLite, a new database of the given name will be created. There must be no existing database of that name, and (for Postgres) you must have permissions assigned that allow you to create databases.
+If the "NEW" keyword is used with PostgreSQL, SQLite, or DuckDB, a new database of the given name will be created. There must be no existing database of that name, and (for Postgres) you must have permissions assigned that allow you to create databases.
 
 
 ## CONSOLE
@@ -894,6 +909,8 @@ EXECUTE SCRIPT [IF EXISTS] <script_name>
 ```
 
 
+`EXEC SCRIPT` and `RUN SCRIPT` are accepted as aliases for `EXECUTE SCRIPT`.
+
 This metacommand will execute the set of SQL statements and metacommands that was previously defined and named using the [BEGIN/END SCRIPT](#beginscript) metacommands.
 
 If the IF EXISTS clause is included, the script will be executed only if it has been defined.
@@ -1200,7 +1217,7 @@ EXPORT QUERY <<query>> [TEE] [APPEND] TO <filename>|stdout
 
 Exports data in the same manner as the [EXPORT](#export) metacommand, except that the data source is a SQL query statement that is contained in the metacommand rather than a database table or view. The SQL query statement must be terminated with a semicolon and enclosed in double angle brackets (i.e., literally "`<<`" and "`>>`").
 
-The EXPORT QUERY metacommand does not support export to XML or HDF5 because there is no table name specified. Export to ODS supports only a single query, rather than a list as for the [EXPORT](#export) metacommand.
+The EXPORT QUERY metacommand does not support export to XML, HDF5, SQLITE, or DUCKDB because there is no table name specified. It does support all other formats including PARQUET, FEATHER, YAML, and MARKDOWN. Export to ODS supports only a single query, rather than a list as for the [EXPORT](#export) metacommand.
 
 Like all metacommands, this metacommand must appear on a single line, although the SQL statement may be quite long. To facilitate readability, the SQL statement may be saved in a [substitution
 variable](substitution_vars.md#substitution_vars) and that substitution variable referenced in the EXPORT QUERY metacommand.
@@ -1234,6 +1251,8 @@ EXTEND SCRIPT <script_1> WITH SCRIPT <script_2>
 
 Merges two scripts, appending the lines of *script_2* to the end of *script_1*. Both scripts must have already been defined using the [BEGIN SCRIPT](#beginscript) metacommand. Parameters for *script_2* are also added to *script_1*.
 
+The alternative syntax `APPEND SCRIPT <script_2> TO <script_1>` is also accepted and performs the same operation.
+
 
 ## HALT
 
@@ -2108,6 +2127,9 @@ Keywords can appear in any order after the table list.
 | `COMPACT` | Use compact grid format for QA summary instead of detailed per-table panels. |
 | `LOGFILE <path>` | Append pg-upsert's plain-text log output to the given file. Supports quoted paths for spaces: `LOGFILE "path/to/log.txt"`. |
 | `CLEANUP` | Drop all `ups_*` temporary tables and views after execution. Without it, temp objects persist for inspection (default). |
+| `EXPORT_FAILURES <dir>` | Write a "fix sheet" of failing QA rows into `<dir>` (directory is created if missing). One row per unique violating staging row, with an `_issues` column summarizing every problem on that row. Supports quoted paths for spaces. Export runs even when QA fails — that is the whole point. A confirmation message (`PG_UPSERT: exported QA failures to <dir> (<format>)`) is written to the terminal, the execsql log, and the `LOGFILE` target if one is given. |
+| `EXPORT_FORMAT csv\|json\|xlsx` | Fix sheet format when `EXPORT_FAILURES` is given. `csv` (default) writes one file per table; `json` writes a single nested file; `xlsx` writes a single workbook with one sheet per table (requires `openpyxl`). |
+| `EXPORT_MAX_ROWS <n>` | Maximum rows to capture per check per table for the fix sheet. Default `1000`. Only meaningful with `EXPORT_FAILURES`. |
 
 ### Substitution variables
 
@@ -2131,6 +2153,7 @@ Set after every `PG_UPSERT` execution:
 | `$PG_UPSERT_TABLE_QA_PASSED` | TRUE/FALSE | QA result for the current table (updated per table) |
 | `$PG_UPSERT_TABLE_ROWS_UPDATED` | integer | Rows updated for the current table (updated per table) |
 | `$PG_UPSERT_TABLE_ROWS_INSERTED` | integer | Rows inserted for the current table (updated per table) |
+| `$PG_UPSERT_EXPORT_PATH` | string | Directory path that the QA fix sheet was written to, or empty if `EXPORT_FAILURES` was not given or nothing was exported. |
 
 !!! note "Using `$PG_UPSERT_RESULT_JSON` with WRITE"
     The JSON value is stored as compact single-line JSON. Because it contains double quotes (`"`), square brackets (`[]`), and apostrophes may appear in data, use tilde or backtick delimiters with WRITE:
@@ -2192,6 +2215,13 @@ For the full list of temporary objects and their schemas, see the [pg-upsert Tem
 -- Write the full JSON result using tilde delimiters (JSON contains " and [])
 -- !x! PG_UPSERT FROM staging TO public TABLES books COMMIT
 -- !x! WRITE ~!!$PG_UPSERT_RESULT_JSON!!~
+
+-- Export failing QA rows to an Excel fix sheet (one sheet per table)
+-- !x! PG_UPSERT QA FROM staging TO public TABLES books, authors EXPORT_FAILURES "qa_failures/" EXPORT_FORMAT xlsx
+
+-- Full pipeline with a CSV fix sheet cap of 50 rows per check per table
+-- !x! PG_UPSERT FROM staging TO public TABLES books, authors EXPORT_FAILURES /tmp/fix EXPORT_MAX_ROWS 50 COMMIT
+-- !x! WRITE "Fix sheet: !!$PG_UPSERT_EXPORT_PATH!!"
 ```
 
 
@@ -2367,7 +2397,7 @@ The selection is [logged](../guides/logging.md#logging).
 
 ```
 PROMPT DISPLAY <table_or_view_name> MESSAGE "<text>"
-    [HELP <url>] [FREE]
+    [HELP <url>]
 ```
 
 Displays the selected view or table in a window with the specified message and both 'Continue' and 'Cancel' buttons. If the 'Continue' button is selected, the script will continue to run. If the 'Cancel' button is selected, the script will immediately halt unless [CANCEL_HALT](#cancel_halt) is set to OFF. The Enter key also carries out the action of the 'Continue' button, and the Escape key carries out the action of the 'Cancel' button.
@@ -2384,8 +2414,6 @@ If [CANCEL_HALT](#cancel_halt) is set to OFF, the [DIALOG_CANCELED](#dialog_canc
 
 If a URL is provided with the HELP keyword, the dialog box will include a button that will open that URL when clicked. The URL must be double-quoted if it contains spaces.
 
-if the FREE keyword is used, the script will keep running when the data table is displayed. The display will have only a "Close" button. If the display is not closed manually, it will be closed automatically when *execsql* finishes processing the script.
-
 
 ## PROMPT ENTER_SUB { #prompt_enter }
 
@@ -2627,7 +2655,7 @@ The file name provided may include wildcards to delete multiple files.
 RM_SUB <match_string>
 ```
 
-Deletes the specified user-created substitution variable.
+Deletes the specified user-created substitution variable. If the match string is prefixed with `~`, the local variable with that name is deleted from the current script scope instead of the global substitution variable set.
 
 
 ## SELECT_SUB
@@ -2884,7 +2912,7 @@ For data in an Excel spreadsheet:
 
 ```
 WRITE CREATE_TABLE <table_name> FROM EXCEL <file_name>
-    SHEET <sheet_name> [SKIP <rows>] ENCODING <encoding>]
+    SHEET <sheet_name> [SKIP <rows>] [ENCODING <encoding>]
     [COMMENT "<comment_text>"] [TO <output>]
 ```
 

{execsql2-2.13.2 → execsql2-2.15.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "execsql2"
-version = "2.13.2"
+version = "2.15.0"
 description = "Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables."
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { file = "LICENSE.txt" }
@@ -58,7 +58,7 @@ odbc = ["pyodbc"]
 # Feature bundles
 formats = ["odfpy", "xlrd", "openpyxl", "Jinja2", "polars", "tables", "PyYAML"]
 auth = ["keyring"]
-upsert = ["pg-upsert>=1.20.0"]
+upsert = ["pg-upsert>=1.21.0"]
 # Convenience groups
 all-db = [
     "psycopg2-binary",
@@ -161,7 +161,7 @@ skip-magic-trailing-comma = false
 line-ending = "auto"
 
 [tool.bumpversion]
-current_version = "2.13.2"
+current_version = "2.15.0"
 commit = true
 commit_args = "--no-verify"
 tag = true

{execsql2-2.13.2 → execsql2-2.15.0}/src/execsql/gui/base.py

@@ -8,7 +8,58 @@ from typing import TYPE_CHECKING, Any
 if TYPE_CHECKING:
     pass
 
-__all__ = ["GuiBackend"]
+__all__ = ["GuiBackend", "compare_stats"]
+
+
+def compare_stats(
+    headers1: list,
+    rows1: list,
+    headers2: list,
+    rows2: list,
+    keylist: list,
+) -> str:
+    """Return a one-line diff summary for compare dialogs.
+
+    Computes matching rows, differing rows, and rows only in one table
+    based on the given key columns.  Returns an empty string when *keylist*
+    is empty (stats cannot be computed without keys).
+    """
+    if not keylist:
+        return ""
+    key_idx1 = [i for i, h in enumerate(headers1) if str(h) in keylist]
+    key_idx2 = [i for i, h in enumerate(headers2) if str(h) in keylist]
+    if not key_idx1 or not key_idx2:
+        return ""
+
+    def _kv(row: list | tuple, idxs: list) -> tuple:
+        return tuple(str(row[i]) if row[i] is not None else "" for i in idxs)
+
+    keys1 = {_kv(r, key_idx1) for r in rows1}
+    keys2 = {_kv(r, key_idx2) for r in rows2}
+    only1 = len(keys1 - keys2)
+    only2 = len(keys2 - keys1)
+    common_keys = keys1 & keys2
+    row_map1 = {_kv(r, key_idx1): r for r in rows1}
+    row_map2 = {_kv(r, key_idx2): r for r in rows2}
+    matching = 0
+    differing = 0
+    for k in common_keys:
+        r1 = [str(v) if v is not None else "" for v in row_map1[k]]
+        r2 = [str(v) if v is not None else "" for v in row_map2[k]]
+        if r1 == r2:
+            matching += 1
+        else:
+            differing += 1
+    parts: list[str] = []
+    if matching:
+        parts.append(f"{matching:,} matching")
+    if differing:
+        parts.append(f"{differing:,} differing")
+    if only1:
+        parts.append(f"{only1:,} only in Table 1")
+    if only2:
+        parts.append(f"{only2:,} only in Table 2")
+    return " | ".join(parts) if parts else "Tables are identical"
 
 
 class GuiBackend(ABC):