execsql2 2.6.0__tar.gz → 2.8.0__tar.gz

{execsql2-2.6.0 → execsql2-2.8.0}/.claude/project_context.md

@@ -265,16 +265,16 @@ ______________________________________________________________________
 
 ### v2.7 — New Export/Import Formats
 
-- [ ] **Parquet import** — complement existing Parquet export and Feather import. Natural fit with DuckDB support.
-- [ ] **YAML export** — popular for config-generation workflows; rounds out the format matrix.
-- [ ] **Markdown (GFM) export** — GitHub-flavored markdown tables. Lightweight and useful for docs/reports.
-- [ ] **Excel (XLSX) multi-sheet export** — multiple queries → multiple named sheets in one workbook, with basic formatting.
+- [x] **Parquet import** — already existed as `IMPORT TO table FROM PARQUET file` (verified present).
+- [x] **YAML export** — `FORMAT YAML` via PyYAML, list-of-dicts with native type preservation.
+- [x] **Markdown (GFM) export** — `FORMAT MARKDOWN` / `MD`, pipe tables with alignment and escaping.
+- [x] **Excel (XLSX) multi-sheet export** — `FORMAT XLSX` single + multi-sheet via openpyxl, bold headers, inventory sheet, sheet name deduplication.
 
 ### v2.8 — Scripting Power Features
 
-- [ ] **`ASSERT` metacommand** — `-- !x! ASSERT <condition> "message"`. Data validation for CI pipelines and sanity checks.
-- [ ] **`--dry-run` improvements** — show SQL with substitution variables expanded, not just raw metacommands.
-- [ ] **Script profiling (`--profile`)** — per-statement execution times, summary report at end. Leverages existing `Timer` infrastructure.
+- [x] **`ASSERT` metacommand** — `-- !x! ASSERT <condition> "message"`. Reuses IF condition engine. 13 tests.
+- [x] **`--dry-run` improvements** — expands substitution variables populated at parse time. 4 new tests.
+- [x] **Script profiling (`--profile`)** — per-statement `perf_counter()` timing with sorted summary table. 19 tests.
 - [ ] **Parallel execution blocks** — `PARALLEL BEGIN ... PARALLEL END` for independent statements. See design notes below.
 
 ### v2.9 — Library API & Developer Experience

{execsql2-2.6.0 → execsql2-2.8.0}/CHANGELOG.md

@@ -13,6 +13,37 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.8.0] - 2026-04-01
+
+### Added
+
+- **`--profile` flag** — records wall-clock time for each SQL and metacommand statement and prints a formatted timing summary after the script completes. The summary lists statements sorted by elapsed time descending (top 20 shown), with per-statement percentage of total time, source location, command type, and a preview of the command text.
+- **`ASSERT` metacommand** — evaluates any IF-compatible condition and raises an error (halting the script when `HALT_ON_METACOMMAND_ERROR` is `ON`) if the condition is false. Supports an optional quoted failure message; omitting the message produces `Assertion failed: <condition>`. A passing assertion is logged. ASSERT is silently skipped inside a false IF block.
+
+### Changed
+
+- **`--dry-run` expands substitution variables** — the command list printed by `--dry-run` now shows resolved `!!$VAR!!` / `!!&ENV!!` tokens for variables that are already populated at parse time (environment variables, `--assign-arg` values, config-sourced variables, and built-in start-time variables like `$SCRIPT_START_TIME`). Variables that are set during execution (e.g. `$CURRENT_TIME`, `$DB_NAME`, `$TIMER`) remain unexpanded because the database connection has not yet been established. Local `~`-prefixed script-scope variables are also left unexpanded. If expansion fails (e.g. a cycle is detected), the raw token is displayed instead.
+
+______________________________________________________________________
+
+## [2.7.1] - 2026-04-01
+
+### Fixed
+
+- Fix `AttributeError: module 'execsql.state' has no attribute 'dedup_words'` when importing CSV files with `DEDUP_COL_HDRS` enabled — `dedup_words` is now correctly imported from `execsql.utils.strings` instead of accessed through the state module.
+
+______________________________________________________________________
+
+## [2.7.0] - 2026-04-01
+
+### Added
+
+- **Markdown export** (`FORMAT MARKDOWN` / `FORMAT MD`) — GitHub-flavored pipe tables with column alignment, pipe/backslash escaping, and zip support. No dependencies required.
+- **YAML export** (`FORMAT YAML`) — list-of-dicts output via PyYAML with native type preservation (int, float, null). Requires `PyYAML` (included in `formats` extras).
+- **XLSX export** (`FORMAT XLSX`) — single-sheet and multi-sheet Excel export via openpyxl with bold headers, native type preservation, sheet name deduplication, and a "Datasheets" inventory sheet. Multi-sheet syntax: `EXPORT table1, table2 TO file.xlsx AS XLSX`.
+
+______________________________________________________________________
+
 ## [2.6.0] - 2026-04-01
 
 ### Added

{execsql2-2.6.0 → execsql2-2.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: execsql2
-Version: 2.6.0
+Version: 2.8.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Repository, https://github.com/geocoug/execsql
 Project-URL: Issues, https://github.com/geocoug/execsql/issues
@@ -54,6 +54,7 @@ Requires-Dist: polars; extra == 'all'
 Requires-Dist: psycopg2-binary; extra == 'all'
 Requires-Dist: pymysql; extra == 'all'
 Requires-Dist: pyodbc; extra == 'all'
+Requires-Dist: pyyaml; extra == 'all'
 Requires-Dist: tables; extra == 'all'
 Requires-Dist: xlrd; extra == 'all'
 Provides-Extra: all-db
@@ -76,6 +77,7 @@ Requires-Dist: openpyxl; extra == 'dev'
 Requires-Dist: polars; extra == 'dev'
 Requires-Dist: pre-commit>=3.5.0; extra == 'dev'
 Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
+Requires-Dist: pyyaml; extra == 'dev'
 Requires-Dist: ruff>=0.4; extra == 'dev'
 Requires-Dist: tables; extra == 'dev'
 Requires-Dist: tox-uv>=1.13.1; extra == 'dev'
@@ -91,6 +93,7 @@ Requires-Dist: jinja2; extra == 'formats'
 Requires-Dist: odfpy; extra == 'formats'
 Requires-Dist: openpyxl; extra == 'formats'
 Requires-Dist: polars; extra == 'formats'
+Requires-Dist: pyyaml; extra == 'formats'
 Requires-Dist: tables; extra == 'formats'
 Requires-Dist: xlrd; extra == 'formats'
 Provides-Extra: mssql
@@ -227,9 +230,10 @@ Run `execsql --help` for the full option list, or `execsql -m` to list all metac
 # Features
 
 - Import data from CSV, TSV, Excel, OpenDocument, Feather, or Parquet files into a database table.
-- Export query results in 15+ formats including CSV, TSV, JSON, XML, HTML, LaTeX, OpenDocument, Feather, HDF5, DuckDB, SQLite, plain text, and Jinja2 templates.
+- Export query results in 20+ formats including CSV, TSV, JSON, YAML, XML, HTML, Markdown, LaTeX, XLSX, OpenDocument, Feather, Parquet, HDF5, DuckDB, SQLite, plain text, and Jinja2 templates.
 - Copy data between databases, including across different DBMS types.
 - Conditionally execute SQL and metacommands using `IF`/`ELSE`/`ENDIF` based on data values, DBMS type, or user input.
+- Validate data with `ASSERT` — halt the script with a clear error message if a condition is false (ideal for CI pipelines).
 - Loop over blocks of SQL and metacommands using `LOOP`/`ENDLOOP`.
 - Use substitution variables (`SUB`, `$ARG_x`, built-in variables like `$date_tag`) to parameterize scripts.
 - Include or chain scripts with `INCLUDE` and `SCRIPT`.

{execsql2-2.6.0 → execsql2-2.8.0}/README.md

@@ -120,9 +120,10 @@ Run `execsql --help` for the full option list, or `execsql -m` to list all metac
 # Features
 
 - Import data from CSV, TSV, Excel, OpenDocument, Feather, or Parquet files into a database table.
-- Export query results in 15+ formats including CSV, TSV, JSON, XML, HTML, LaTeX, OpenDocument, Feather, HDF5, DuckDB, SQLite, plain text, and Jinja2 templates.
+- Export query results in 20+ formats including CSV, TSV, JSON, YAML, XML, HTML, Markdown, LaTeX, XLSX, OpenDocument, Feather, Parquet, HDF5, DuckDB, SQLite, plain text, and Jinja2 templates.
 - Copy data between databases, including across different DBMS types.
 - Conditionally execute SQL and metacommands using `IF`/`ELSE`/`ENDIF` based on data values, DBMS type, or user input.
+- Validate data with `ASSERT` — halt the script with a clear error message if a condition is false (ideal for CI pipelines).
 - Loop over blocks of SQL and metacommands using `LOOP`/`ENDLOOP`.
 - Use substitution variables (`SUB`, `$ARG_x`, built-in variables like `$date_tag`) to parameterize scripts.
 - Include or chain scripts with `INCLUDE` and `SCRIPT`.

{execsql2-2.6.0 → execsql2-2.8.0}/docs/about/divergence.md

@@ -13,30 +13,35 @@ ______________________________________________________________________
 
 ### CLI Options
 
-| Flag                            | Description                                                                                                                                                            |
-| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `--version`                     | Print version and exit (Rich-formatted).                                                                                                                               |
-| `-c` / `--command`              | Execute an inline SQL or metacommand string instead of a script file.                                                                                                  |
-| `--dsn` / `--connection-string` | Accept a standard database URL (e.g. `postgresql://user:pass@host/db`). Supports `postgresql`, `mysql`, `mssql`, `oracle`, `firebird`, `sqlite`, and `duckdb` schemes. |
-| `--output-dir`                  | Set a default base directory for export output files.                                                                                                                  |
-| `--progress`                    | Show a Rich progress bar during long-running IMPORT operations.                                                                                                        |
-| `--dump-keywords`               | Emit all metacommand keywords, conditionals, config options, and export formats as structured JSON.                                                                    |
-| `--gui-framework`               | Select GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                    |
-| `--dry-run`                     | Parse the script and print the full command list without connecting to a database or executing anything.                                                               |
+| Flag                            | Description                                                                                                                                                                                                                                                                                                                             |
+| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `--version`                     | Print version and exit (Rich-formatted).                                                                                                                                                                                                                                                                                                |
+| `-c` / `--command`              | Execute an inline SQL or metacommand string instead of a script file.                                                                                                                                                                                                                                                                   |
+| `--dsn` / `--connection-string` | Accept a standard database URL (e.g. `postgresql://user:pass@host/db`). Supports `postgresql`, `mysql`, `mssql`, `oracle`, `firebird`, `sqlite`, and `duckdb` schemes.                                                                                                                                                                  |
+| `--output-dir`                  | Set a default base directory for export output files.                                                                                                                                                                                                                                                                                   |
+| `--progress`                    | Show a Rich progress bar during long-running IMPORT operations.                                                                                                                                                                                                                                                                         |
+| `--dump-keywords`               | Emit all metacommand keywords, conditionals, config options, and export formats as structured JSON.                                                                                                                                                                                                                                     |
+| `--gui-framework`               | Select GUI backend: `tkinter` (default) or `textual` (terminal UI).                                                                                                                                                                                                                                                                     |
+| `--dry-run`                     | Parse the script and print the full command list without connecting to a database or executing anything. Substitution variables already populated at parse time (env vars, `--assign-arg` values, built-in start-time vars) are expanded in the output; execution-time variables (`$DB_NAME`, `$CURRENT_TIME`, etc.) remain unexpanded. |
+| `--profile`                     | Record wall-clock time for each SQL and metacommand statement. After the script completes, print a summary table sorted by elapsed time (descending), showing time, percentage of total, source location, command type, and a command preview. Top 20 slowest statements are shown.                                                     |
 
 ### Export Formats
 
-| Format    | Description                                                                     |
-| --------- | ------------------------------------------------------------------------------- |
-| `PARQUET` | Export query or table results to Apache Parquet via `polars`.                   |
-| `FEATHER` | Export to Apache Feather/IPC via `polars` + `pyarrow` (upstream used `pandas`). |
+| Format            | Description                                                                                                          |
+| ----------------- | -------------------------------------------------------------------------------------------------------------------- |
+| `PARQUET`         | Export query or table results to Apache Parquet via `polars`.                                                        |
+| `FEATHER`         | Export to Apache Feather/IPC via `polars` + `pyarrow` (upstream used `pandas`).                                      |
+| `YAML`            | Export query or table results as a YAML sequence of mappings via `PyYAML`.                                           |
+| `MARKDOWN` / `MD` | Export query or table results as a GitHub-Flavored Markdown (GFM) pipe table. Pure Python, no optional dependencies. |
+| `XLSX`            | Export query or table results to an Excel XLSX workbook via `openpyxl` (single or multi-sheet).                      |
 
 ### Metacommands
 
-| Metacommand            | Description                                                           |
-| ---------------------- | --------------------------------------------------------------------- |
-| `CONFIG SHOW_PROGRESS` | Enable the Rich progress bar for IMPORT operations at runtime.        |
-| `CONFIG LOG_SQL`       | Enable SQL query audit logging — writes executed SQL to the log file. |
+| Metacommand            | Description                                                                                                                                                           |
+| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ASSERT`               | Evaluate a condition and raise an error (halting the script) if it is false. Supports all IF conditions. Optional quoted failure message. Skipped in false IF blocks. |
+| `CONFIG SHOW_PROGRESS` | Enable the Rich progress bar for IMPORT operations at runtime.                                                                                                        |
+| `CONFIG LOG_SQL`       | Enable SQL query audit logging — writes executed SQL to the log file.                                                                                                 |
 
 ### Configuration Options
 

{execsql2-2.6.0 → execsql2-2.8.0}/docs/getting-started/syntax.md

@@ -175,21 +175,33 @@ Valid encoding names can be displayed with the `-y` option. See also [Character
 ### Informational options
 
 `-m`, `--metacommands`
+
 :   List all metacommands and exit.
 
 `-o`, `--online-help`
+
 :   Open the online documentation in the default browser.
 
 `-y`, `--encodings`
+
 :   List all valid character encoding names and exit.
 
 `--dump-keywords`
+
 :   Dump all metacommand keywords, conditional functions, config options, and export formats as JSON and exit. Useful for tooling that consumes execsql's keyword registry (e.g., the VS Code grammar generator).
 
 `--dry-run`
+
 :   Parse the script (or inline `-c` command) and print the full command list — SQL statements and metacommands with source locations — without connecting to a database or executing anything. Useful for validating scripts.
 
+    Substitution variables that are already populated at parse time are expanded in the output: environment variables (`!!&ENV_VAR!!`), `--assign-arg` values (`!!$ARG_1!!`), and built-in start-time variables like `!!$SCRIPT_START_TIME!!` and `!!$USER!!`. Variables that are set during execution — such as `$CURRENT_TIME`, `$DB_NAME`, and `$TIMER` — remain unexpanded because no database connection is established in dry-run mode. Local `~`-prefixed script-scope variables are also left unexpanded.
+
+`--profile`
+
+:   Record the wall-clock execution time of each SQL statement and metacommand. After the script finishes, print a summary table to the console showing elapsed time, percentage of total time, source file and line number, command type, and a preview of the command text. Statements are sorted from slowest to fastest; the top 20 are displayed. Useful for identifying slow queries or metacommands in long-running scripts.
+
 `--version`
+
 :   Show the version number and exit.
 
 ## Configuration File Defaults { #config_defaults }

{execsql2-2.6.0 → execsql2-2.8.0}/docs/reference/metacommands.md

@@ -48,6 +48,42 @@ Double quotes (as shown above), apostrophes, or square brackets can be used to d
 
 See the [PROMPT ASK](#prompt_ask) metacommand for a version of this command that uses a GUI window and that can display a data table with the prompt.
 
+## ASSERT { #assert }
+
+```
+ASSERT <condition>
+ASSERT <condition> "<failure message>"
+ASSERT <condition> '<failure message>'
+```
+
+Evaluates `<condition>` using the same expression engine as [IF](#if_cmd). If the condition is `True`, execution continues silently (and the result is written to the log). If the condition is `False`, an error is raised with the provided failure message. If no message is supplied, the default message is `Assertion failed: <condition>`.
+
+When [HALT_ON_METACOMMAND_ERROR](#config) is `ON` (the default), a failed assertion halts the script. When it is `OFF`, execution continues after the failure is logged.
+
+The `<condition>` supports all conditional tests available to `IF`, including:
+
+- `TABLE_EXISTS <table>` / `TABLE_NOT_EXISTS <table>`
+- `COLUMN_EXISTS <table> <column>` / `COLUMN_NOT_EXISTS <table> <column>`
+- `ROWCOUNT <op> <n>` (e.g. `ROWCOUNT > 0`)
+- Variable comparisons: `$varname = 'value'`, `$varname != 'value'`
+- Numeric comparisons: `$varname > <n>`, `$varname <= <n>`
+- `DATABASE_TYPE <type>` / `DATABASE_TYPE_NOT <type>`
+
+ASSERT is silently skipped inside a `False` [IF](#if_cmd) block.
+
+**Examples:**
+
+```sql
+-- Halt with a custom message if the staging table is missing.
+-- !x! ASSERT TABLE_EXISTS staging "staging table must exist before running this script"
+
+-- Halt with the default message if no rows were returned.
+-- !x! ASSERT ROWCOUNT > 0
+
+-- Verify a substitution variable has the expected value.
+-- !x! ASSERT $env = 'prod' 'expected production environment'
+```
+
 ## AUTOCOMMIT
 
 ```
@@ -932,7 +968,22 @@ JSON_TS or JSON_TABLESCHEMA
 
 LATEX
 
-:   Input for the [LaTeΧ](https://www.latex-project.org/) typesetting system. If the "APPEND" keyword is not used, a complete document (of class article) will be written. If the "APPEND" keyword is used, only the table definition will be written to the output file. If the "APPEND" keyword is used and an existing output file contains an \\end directive, the table will be written before that directive rather than at the physical end of the file. Wide or long tables may exceed LaTeΧ's default page size. If the "DESCRIPTION" keyword is used, the given description will be used as the table's caption. Data exported in LaTeX format cannot be written into a zipfile.
+:   Input for the [LaTeX](https://www.latex-project.org/) typesetting system. If the "APPEND" keyword is not used, a complete document (of class article) will be written. If the "APPEND" keyword is used, only the table definition will be written to the output file. If the "APPEND" keyword is used and an existing output file contains an \\end directive, the table will be written before that directive rather than at the physical end of the file. Wide or long tables may exceed LaTeΧ's default page size. If the "DESCRIPTION" keyword is used, the given description will be used as the table's caption. Data exported in LaTeX format cannot be written into a zipfile.
+
+
+MARKDOWN or MD
+
+:   [GitHub-Flavored Markdown](https://github.github.com/gfm/) pipe table. Column values are aligned and pipe (`|`) and backslash (`\`) characters in data are escaped. If the "DESCRIPTION" keyword is used, the description is written as an HTML comment (`<!-- ... -->`) before the table. If the "APPEND" keyword is used, only the table is appended (no repeated headers). No optional dependencies required.
+
+
+XLSX
+
+:   [Excel](https://www.microsoft.com/en-us/microsoft-365/excel) workbook in the Office Open XML format. One or more tables (or views) can be exported to an XLSX workbook. Each table will be exported to a separate worksheet within the workbook, with the first row containing bold column headers. To export multiple tables, their names must be separated by commas. The "APPEND" keyword can be used to add worksheets to an existing workbook. The name of the view or table exported will be used as the worksheet name; if this conflicts with a sheet already in the workbook, a number will be appended to make the sheet name unique. A "Datasheets" inventory sheet is created with author, date, description, and source information for each data sheet. Data types are preserved natively (integers, floats, dates, datetimes, booleans). The `openpyxl` library must be installed (`pip install execsql2[excel]`). Data exported in XLSX format cannot be written into a zipfile.
+
+
+YAML
+
+:   [YAML](https://yaml.org/) sequence of mappings. Each row is represented as a mapping (dictionary) with column names as keys. Python data types are preserved — integers remain integers, floats remain floats, and `None` becomes YAML `null`. If the "APPEND" keyword is used, a new YAML document is appended to the file (multi-document stream). The `PyYAML` library must be installed (`pip install execsql2[formats]`). No description text is included in the output even if provided.
 
 
 SQLITE

{execsql2-2.6.0 → execsql2-2.8.0}/extras/vscode-execsql/syntaxes/execsql.tmLanguage.json

@@ -93,7 +93,7 @@
     },
     "action-keywords": {
       "comment": "sub, write, execute script, export, etc.",
-      "match": "(?i)\\b(reset\\s+dialog_canceled|write\\s+create_table|export_metadata|sub_querystring|execute\\s+script|append\\s+script|extend\\s+script|reset\\s+counter|export\\s+query|sub_tempfile|write\\s+script|import_file|set\\s+counter|sub_decrypt|sub_encrypt|autocommit|copy\\s+query|disconnect|select_sub|sub_append|system_cmd|pg_vacuum|sub_empty|sub_local|connect|include|max_int|rm_file|subdata|sub_add|sub_ini|export|import|rm_sub|debug|email|serve|write|copy|log|run|sub|use|zip|cd)\\b",
+      "match": "(?i)\\b(reset\\s+dialog_canceled|write\\s+create_table|export_metadata|sub_querystring|execute\\s+script|append\\s+script|extend\\s+script|reset\\s+counter|export\\s+query|sub_tempfile|write\\s+script|import_file|set\\s+counter|sub_decrypt|sub_encrypt|autocommit|copy\\s+query|disconnect|select_sub|sub_append|system_cmd|pg_vacuum|sub_empty|sub_local|connect|include|max_int|rm_file|subdata|sub_add|sub_ini|assert|export|import|rm_sub|debug|email|serve|write|copy|log|run|sub|use|zip|cd)\\b",
       "name": "keyword.other.execsql"
     },
     "config-event-keywords": {
@@ -118,7 +118,7 @@
     },
     "export-formats": {
       "comment": "Export/import format names",
-      "match": "(?i)\\b(json_tableschema|cgi-html|feather|json_ts|txt-and|unitsep|duckdb|sqlite|values|latex|plain|hdf5|html|json|tabq|tsvq|b64|csv|ods|raw|tab|tsv|txt|xml|us)\\b",
+      "match": "(?i)\\b(json_tableschema|cgi-html|markdown|feather|json_ts|txt-and|unitsep|duckdb|sqlite|values|latex|plain|hdf5|html|json|tabq|tsvq|xlsx|yaml|b64|csv|ods|raw|tab|tsv|txt|xml|md|us)\\b",
       "name": "support.constant.execsql"
     },
     "secondary-operators": {

{execsql2-2.6.0 → execsql2-2.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "execsql2"
-version = "2.6.0"
+version = "2.8.0"
 description = "Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables."
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { file = "LICENSE.txt" }
@@ -56,7 +56,7 @@ firebird = ["firebird-driver"]
 oracle = ["oracledb"]
 odbc = ["pyodbc"]
 # Feature bundles
-formats = ["odfpy", "xlrd", "openpyxl", "Jinja2", "polars", "tables"]
+formats = ["odfpy", "xlrd", "openpyxl", "Jinja2", "polars", "tables", "PyYAML"]
 auth = ["keyring"]
 # Convenience groups
 all-db = [
@@ -158,7 +158,7 @@ skip-magic-trailing-comma = false
 line-ending = "auto"
 
 [tool.bumpversion]
-current_version = "2.6.0"
+current_version = "2.8.0"
 commit = true
 commit_args = "--no-verify"
 tag = true

{execsql2-2.6.0 → execsql2-2.8.0}/src/execsql/cli/__init__.py

@@ -254,6 +254,11 @@ def main(
         "--dump-keywords",
         help="Dump all metacommand keywords as JSON and exit.",
     ),
+    profile: bool = typer.Option(
+        False,
+        "--profile",
+        help="Record per-statement execution times and print a timing summary after the script completes.",
+    ),
     version: bool | None = typer.Option(
         None,
         "--version",
@@ -415,6 +420,7 @@ def main(
         dsn=dsn,
         output_dir=output_dir,
         progress=progress,
+        profile=profile,
     )
 
 

{execsql2-2.6.0 → execsql2-2.8.0}/src/execsql/cli/run.py

@@ -20,11 +20,11 @@ from execsql.cli.dsn import _parse_connection_string
 from execsql.cli.help import _console, _err_console
 from execsql.config import ConfigData, StatObj
 from execsql.exceptions import ConfigError, ErrInfo
-from execsql.script import SubVarSet, current_script_line, read_sqlfile, read_sqlstring, runscripts
+from execsql.script import SubVarSet, current_script_line, read_sqlfile, read_sqlstring, runscripts, substitute_vars
 from execsql.utils.fileio import FileWriter, Logger, filewriter_end
 from execsql.utils.gui import gui_connect, gui_console_isrunning, gui_console_off, gui_console_on, gui_console_wait_user
 
-__all__ = ["_connect_initial_db", "_print_dry_run", "_run"]
+__all__ = ["_connect_initial_db", "_print_dry_run", "_print_profile", "_run"]
 
 
 # ---------------------------------------------------------------------------
@@ -33,7 +33,16 @@ __all__ = ["_connect_initial_db", "_print_dry_run", "_run"]
 
 
 def _print_dry_run(cmdlist: object) -> None:
-    """Print the parsed command list for --dry-run mode."""
+    """Print the parsed command list for --dry-run mode.
+
+    Substitution variables (``$VAR``, ``&ENV``, ``@COUNTER``) that are already
+    populated — from environment variables, ``--assign-arg`` values, or config —
+    are expanded in the displayed text.  System variables that are set at
+    execution time (e.g. ``$CURRENT_TIME``, ``$DB_NAME``, ``$TIMER``) will
+    appear unexpanded because ``set_system_vars()`` has not yet been called.
+    Local ``~``-prefixed script-scope variables are also not expanded (no script
+    execution context exists in dry-run mode).
+    """
     if cmdlist is None or not cmdlist.cmdlist:
         _console.print("[yellow]No commands found in script.[/yellow]")
         return
@@ -43,7 +52,71 @@ def _print_dry_run(cmdlist: object) -> None:
     for i, cmd in enumerate(cmdlist.cmdlist, 1):
         ctype = "SQL    " if cmd.command_type == "sql" else "METACMD"
         source_info = f"[dim]{cmd.source}:{cmd.line_no}[/dim]"
-        _console.print(f"  [dim]{i:>4}[/dim]  [bold green]{ctype}[/bold green]  {source_info}  {cmd.commandline()}")
+        raw = cmd.commandline()
+        try:
+            expanded = substitute_vars(raw)
+        except Exception:
+            # Cycle detection or other expansion errors — fall back to raw text.
+            expanded = raw
+        _console.print(f"  [dim]{i:>4}[/dim]  [bold green]{ctype}[/bold green]  {source_info}  {expanded}")
+
+
+# ---------------------------------------------------------------------------
+# Profile report helper
+# ---------------------------------------------------------------------------
+
+
+def _print_profile(profile_data: list[tuple]) -> None:
+    """Print a per-statement timing summary to stdout.
+
+    Args:
+        profile_data: List of ``(source, line_no, command_type, elapsed_secs,
+            command_text_preview)`` tuples collected during execution.
+    """
+    if not profile_data:
+        _console.print("[dim]Profile: no statements recorded.[/dim]")
+        return
+
+    total_secs = sum(row[3] for row in profile_data)
+    n = len(profile_data)
+
+    # Sort descending by elapsed time; show top 20 (or all if <= 20).
+    sorted_data = sorted(profile_data, key=lambda r: r[3], reverse=True)
+    display = sorted_data[:20]
+
+    _console.print()
+    _console.print(f"[bold cyan]Profile:[/bold cyan] {n} statement{'s' if n != 1 else ''} in {total_secs:.3f}s")
+    _console.print()
+
+    header = f"  {'Time (s)':<10}  {'Pct':<7}  {'Source:Line':<20}  {'Type':<7}  Command"
+    sep = f"  {'-' * 10}  {'-' * 7}  {'-' * 20}  {'-' * 7}  {'-' * 40}"
+    _console.print(f"[dim]{header}[/dim]")
+    _console.print(f"[dim]{sep}[/dim]")
+
+    for source, line_no, command_type, elapsed, preview in display:
+        pct = (elapsed / total_secs * 100) if total_secs > 0 else 0.0
+        source_col = f"{source}:{line_no}"
+        if len(source_col) > 20:
+            source_col = "..." + source_col[-17:]
+        ctype_label = "SQL    " if command_type == "sql" else "METACMD"
+        preview_short = preview[:50].replace("\n", " ").strip()
+        if len(preview) > 50:
+            preview_short += "..."
+        _console.print(
+            f"  [yellow]{elapsed:<10.3f}[/yellow]  "
+            f"[dim]{pct:<6.1f}%[/dim]  "
+            f"[cyan]{source_col:<20}[/cyan]  "
+            f"[green]{ctype_label:<7}[/green]  "
+            f"{preview_short}",
+        )
+
+    if len(sorted_data) > 20:
+        omitted = len(sorted_data) - 20
+        _console.print(
+            f"[dim]  ... {omitted} more statement{'s' if omitted != 1 else ''} not shown (top 20 by time)[/dim]",
+        )
+
+    _console.print()
 
 
 # ---------------------------------------------------------------------------
@@ -76,6 +149,7 @@ def _run(
     dsn: str | None = None,
     output_dir: str | None = None,
     progress: bool = False,
+    profile: bool = False,
 ) -> None:
     """Initialise state, connect to the database, load the script, and run it.
 
@@ -378,7 +452,10 @@ def _run(
     atexit.register(_state.dbs.closeall)
     _state.dbs.do_rollback = True
 
-    _execute_script_direct(conf)
+    if profile:
+        _state.profile_data = []
+
+    _execute_script_direct(conf, profile=profile)
 
 
 # ---------------------------------------------------------------------------
@@ -437,8 +514,15 @@ def _execute_script_textual_console(conf: ConfigData) -> None:
     _state.exec_log.log_exit_end()
 
 
-def _execute_script_direct(conf: ConfigData) -> None:
-    """Run runscripts() in the current (main) thread — used when Textual is not active."""
+def _execute_script_direct(conf: ConfigData, *, profile: bool = False) -> None:
+    """Run runscripts() in the current (main) thread — used when Textual is not active.
+
+    Args:
+        conf: The active configuration object.
+        profile: When ``True``, print a per-statement timing summary after the
+            script completes.  Timing data must already have been activated on
+            ``_state.profile_data`` before this function is called.
+    """
     import execsql.state as _state
     import execsql.utils.gui as _gui
 
@@ -463,6 +547,8 @@ def _execute_script_direct(conf: ConfigData) -> None:
             if gui_console_isrunning():
                 gui_console_off()
         _state.exec_log.log_status_info(f"{_state.cmds_run} commands run")
+        if profile and _state.profile_data is not None:
+            _print_profile(_state.profile_data)
         sys.exit(exc.code)
     except ConfigError:
         raise
@@ -489,6 +575,8 @@ def _execute_script_direct(conf: ConfigData) -> None:
         if gui_console_isrunning():
             gui_console_off()
     _state.exec_log.log_status_info(f"{_state.cmds_run} commands run")
+    if profile and _state.profile_data is not None:
+        _print_profile(_state.profile_data)
     _state.exec_log.log_exit_end()
 
 

{execsql2-2.6.0 → execsql2-2.8.0}/src/execsql/exporters/__init__.py

@@ -9,9 +9,9 @@ handlers can access them via ``_state.write_query_to_csv`` etc. without
 importing directly from here.
 
 Sub-modules: ``base``, ``delimited``, ``json``, ``xml``, ``html``,
-``latex``, ``ods``, ``xls``, ``zip``, ``raw``, ``pretty``, ``values``,
-``templates``, ``feather``, ``parquet``, ``duckdb``, ``sqlite``,
-``protocol``.
+``latex``, ``markdown``, ``ods``, ``xls``, ``zip``, ``raw``, ``pretty``,
+``values``, ``templates``, ``feather``, ``parquet``, ``duckdb``,
+``sqlite``, ``protocol``.
 """
 
 from execsql.exporters.protocol import QueryExporter, RowsetExporter

{execsql2-2.6.0 → execsql2-2.8.0}/src/execsql/exporters/delimited.py

@@ -27,7 +27,7 @@ from execsql.exceptions import ErrInfo
 from execsql.models import DataTable
 from execsql.utils.errors import exception_desc
 from execsql.utils.fileio import filewriter_close
-from execsql.utils.strings import clean_words, fold_words
+from execsql.utils.strings import clean_words, dedup_words, fold_words
 
 __all__ = ["LineDelimiter", "CsvFile", "CsvWriter", "DelimitedWriter", "write_delimited_file"]
 
@@ -677,7 +677,7 @@ class CsvFile(EncodedFile):
         if conf.fold_col_hdrs != "no":
             colnames = fold_words(colnames, conf.fold_col_hdrs)
         if conf.dedup_col_hdrs:
-            colnames = _state.dedup_words(colnames)
+            colnames = dedup_words(colnames)
         return colnames
 
     def column_headers(self) -> list[str]: