execsql2 2.28.0__tar.gz → 2.29.0__tar.gz

{execsql2-2.28.0 → execsql2-2.29.0}/CHANGELOG.md

@@ -13,6 +13,21 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.29.0] - 2026-10-13
+
+### Changed
+
+- **`!'!var!'!` stores backslashes as written** on every DBMS: a value such as `C:\data\new` was stored as `C:\\data\\new` on SQLite, PostgreSQL, DuckDB, SQL Server, Oracle, Firebird and Access. MySQL/MariaDB (which read `\` as an escape) and PostgreSQL escape strings written as `E!'!var!'!` still get backslashes doubled, so both forms store the same value and the `E` workaround is no longer needed. Scripts that removed the doubled backslashes some other way should stop.
+
+### Fixed
+
+- **Oracle IMPORT and COPY with a time column** create it as text (`VARCHAR2(15)`, holding `HH:MM:SS`); `TO NEW` stopped with "Oracle DBMS type has no specification for data type time".
+- **MySQL IMPORT accepts a header with capital letters** (`Id,Name`): it stopped with "columns that are not in table" (leaving the new table behind), and with `IMPORT_COMMON_COLUMNS_ONLY` it loaded those columns as NULL.
+- **PostgreSQL IMPORT honors `SKIP n`** for quoted CSV files, which load through PostgreSQL's COPY: it skipped only the header line, loading the lines before it as data or failing on them.
+- **Spreadsheet IMPORT reads every data row:** an XLSX row whose values are all `0` or FALSE no longer ends the data (it and every row after it were dropped), and an XLS sheet imported with `SKIP n` no longer loses its last *n* rows.
+
+______________________________________________________________________
+
 ## [2.28.0] - 2026-10-13
 
 ### Added

{execsql2-2.28.0 → execsql2-2.29.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: execsql2
-Version: 2.28.0
+Version: 2.29.0
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
 Project-URL: Homepage, https://execsql2.readthedocs.io
 Project-URL: Repository, https://github.com/geocoug/execsql
@@ -466,7 +466,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.28.0
+    rev: v2.29.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.28.0 → execsql2-2.29.0}/README.md

@@ -321,7 +321,7 @@ execsql format --no-sql --in-place scripts/
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.28.0
+    rev: v2.29.0
     hooks:
       - id: execsql-format
       - id: execsql-lint

{execsql2-2.28.0 → execsql2-2.29.0}/docs/about/divergence.md

@@ -362,6 +362,11 @@ These are behavioral changes driven by security or correctness issues in the ups
 | Partial dates inferred as timestamps                               | `dateutil` fills fields missing from a value with today's date, so `1-2`, `5/6`, `March 5`, `Jan` and `Monday` were inferred as timestamps and, on DuckDB, SQL Server, MySQL, Oracle, Firebird, Access and every ODS/XLSX/JSON/Parquet/Feather import, stored as dates in the current year. Inference now requires a four-digit year (or a month-first `m/d/yy` or `m-d-yy`) and a month and day, which restores upstream's behaviour for these values. `m-d-yy` is accepted in addition to upstream's `m/d/yy`.                                                                                                                                                                                                                                                                                                   |
 | Offset values stored as plain timestamps                           | With `dateutil`, a value with a UTC offset (`2024-01-01 10:00+05:00`) also counted as a plain timestamp, so a column mixing offset and plain values was created as `timestamp` and the driver converted the offset values with the client's time zone: the stored value depended on the machine that ran the import. A value with an offset is again not a plain timestamp, as with upstream's offset-free format list, so a mixed column is text and keeps each value as written. A column where every value has an offset is still `timestamptz`.                                                                                                                                                                                                                                                                |
 | Binary columns in `COPY`                                           | Every value a source cursor returned as bytes was decoded to text (undecodable bytes as `\xNN` escapes) before `COPY` wrote it, and `TO NEW` typed the column as text. Upstream did the same for SQLite BLOBs; with psycopg2 a PostgreSQL `bytea` arrived as a `memoryview` and escaped the decode, so for PostgreSQL this was an execsql2 regression. `COPY` now moves bytes unchanged and creates binary columns. Exports still write the decoded text.                                                                                                                                                                                                                                                                                                                                                          |
+| Backslashes in `!'!var!'!`                                         | Upstream doubled backslashes on Windows only, to protect `re.sub`'s replacement template; execsql2 2.x doubled them on every platform as SQL escaping, which corrupted values on every DBMS but MySQL. A backslash is now doubled only where it is an escape: on a MySQL/MariaDB connection, and in a PostgreSQL escape string written as `E!'!var!'!`. Apostrophes are doubled everywhere.                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| Time columns on Oracle                                             | `IMPORT` / `COPY ... TO NEW` of a column of times stopped with "Oracle DBMS type has no specification for data type time" (a `DT_Time_Oracle` type was registered but inference never produced it). A time column is now created as `VARCHAR2(15)` holding the ISO time text.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
+| Capitalized headers on MySQL                                       | MySQL compared the file's headers with the table's lower-cased column names, so `Id` did not match `id`: IMPORT stopped (after `TO NEW` had created the table), and with `IMPORT_COMMON_COLUMNS_ONLY` such columns were skipped. The comparison now ignores case, as on PostgreSQL and the other DBMSs.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+| `SKIP` on PostgreSQL's COPY path                                   | The PostgreSQL fast path (a quoted CSV file) skipped one line, the header, whatever `SKIP` said, so the skipped lines were loaded as data. It now starts after the `SKIP` lines like every other reader.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+| Spreadsheet rows on import                                         | XLSX reading stopped at the first row with no truthy value, so a row of `0` / FALSE values ended the data; XLS reading with `SKIP n` stopped *n* rows before the end of the sheet. Both now read to the first row of empty cells.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Short data lines on import                                         | A line with fewer values than the header was padded with NULLs only on PostgreSQL and MySQL. SQLite stopped with "Too few values", DuckDB and the other DBMSs with a driver parameter-count error. With `TO NEW` / `REPLACEMENT`, type inference skipped the missing values, so the column could be created `NOT NULL` and then reject the padded line, on PostgreSQL too. Missing trailing values are now NULL on every DBMS, and inference counts them as NULL.                                                                                                                                                                                                                                                                                                                                                  |
 | `COPY` re-typed text values                                        | Upstream described the destination table by re-reading the source's text values as if they came from a file, so a text column of `1`/`0` became `boolean` and one of dates became `timestamp` with `NEW` / `REPLACEMENT`, and values copied into an existing text column were converted on the way in (`1` stored as `true`). Text from the source database now stays text. Upstream also ran the source query twice, once to describe the table and once to copy it; execsql2 runs it once and holds the rows in a temporary file in between.                                                                                                                                                                                                                                                                     |
 | `REPLACEMENT` of a quoted or schema-qualified table                | The SQLite, SQL Server, Oracle, Firebird and Access adapters quoted the table name a second time before dropping it, so `IMPORT` / `COPY ... TO REPLACEMENT "my tbl"` dropped a table named `"my tbl"` (quotes included) that did not exist and then failed to create the table, and on SQL Server, Oracle and Firebird `REPLACEMENT staging.t` tried to drop one table named `staging.t`. The name is now quoted once.                                                                                                                                                                                                                                                                                                                                                                                            |

{execsql2-2.28.0 → execsql2-2.29.0}/docs/guides/formatter.md

@@ -270,7 +270,7 @@ select id,name,created_at from users where active = true order by name;
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.28.0
+    rev: v2.29.0
     hooks:
       - id: execsql-format
       - id: execsql-lint # optional: fail the commit on lint errors

{execsql2-2.28.0 → execsql2-2.29.0}/docs/reference/lint.md

@@ -158,7 +158,7 @@ The `execsql-lint` [pre-commit](https://pre-commit.com/) hook runs `execsql lint
 ```yaml
 repos:
   - repo: https://github.com/geocoug/execsql
-    rev: v2.28.0
+    rev: v2.29.0
     hooks:
       - id: execsql-lint
         args: [--ignore, V002] # optional: any execsql lint option

{execsql2-2.28.0 → execsql2-2.29.0}/docs/reference/metacommands.md

@@ -1950,6 +1950,8 @@ The delimiter characters that will be recognized in a text file, and that can be
 
 The SKIP key phrase specifies the number of lines (or rows) at the beginning of the file (or worksheet) to discard before evaluating the remainder of the input as a data table.
 
+A worksheet's data ends at the first row whose cells are all empty; a row of zeros or FALSE values is data.
+
 If the NEW keyword is used, the input will be scanned to determine the data type of each column, and a CREATE TABLE statement run to create a new table for the data. A column is typed as a date or timestamp only if every value gives a full date with the year written out (`2024-01-05`, `1/5/2024`, `Jan 5, 2024`) or as a month-first two-digit year (`1/5/24`, `01-05-24`); values such as `1-2`, `5/6`, `March 5`, `Jan`, `Monday` or a day-first `13-05-24` leave the column as text. Scanning of the file to determine data formats is separate from the scanning that may be done to determine the quote and delimiter characters. If a table of that name already exists, the IMPORT stops before creating or loading anything, with the error `Table <name> already exists`. A table name without a schema is looked for only in the schema the table would be created in (on MySQL, the current database), so a table of the same name in another schema does not stop the IMPORT. If the table already exists when the NEW keyword is used, a fatal error will result and *execsql* will halt. If the REPLACEMENT keyword is used, the result is the same as if the NEW keyword were used, except that an existing table of the given name will be deleted first. This replacement is not atomic: if the drop succeeds but a later create or load step fails, the old table has already been removed. If the drop fails, *execsql* stops the IMPORT command before creating or loading the replacement table. *execsql* uses a "drop table" statement to drop an existing table, and the "drop table" statement may not succeed if there are dependencies on that table (see the discussion of [implicit drop table
 statements](../guides/sql_syntax.md#implicit_drop)). If the table to be dropped does not exist, an informational message will be written to the log.
 

{execsql2-2.28.0 → execsql2-2.29.0}/docs/reference/security.md

@@ -184,7 +184,7 @@ SELECT * FROM !!target_schema!!.mytable;
 
 If a variable value is derived from external input (command-line arguments, environment variables, `PROMPT` responses, or data read from a file), the script author is responsible for quoting or validating that value before it is substituted into SQL.
 
-The `!'!varname!'!` dereferencing form wraps the value in single quotes and doubles any embedded apostrophes, which helps when inserting text values into SQL literals:
+The `!'!varname!'!` dereferencing form wraps the value in single quotes and doubles any embedded apostrophes (and, on MySQL/MariaDB, backslashes, so `\'` cannot end the literal there), which helps when inserting text values into SQL literals:
 
 ```sql
 -- !x! sub author_name !!&AUTHOR!!

{execsql2-2.28.0 → execsql2-2.29.0}/docs/reference/substitution_vars.md

@@ -11,7 +11,7 @@ SUB <match_string> <replacement_string>
 The \<match_string> identifies the substitution variable; the \<replacement_string> is the text inserted in its place. A substitution is recognized only when the match string is bracketed by one of three tokens:
 
 - `!!name!!` — insert the value verbatim.
-- `!'!name!'!` — wrap the value in single quotes, doubling any embedded apostrophe. Suitable for SQL string literals containing user data.
+- `!'!name!'!` — wrap the value in single quotes, doubling any embedded apostrophe; on a MySQL or MariaDB connection, which reads `\` in a string literal as an escape, backslashes are doubled too. The value is stored exactly as written (`C:\data\new` stays `C:\data\new`). Suitable for SQL string literals containing user data. On PostgreSQL, `E!'!name!'!` (an escape string) doubles backslashes too, so it stores the same value as `!'!name!'!`; the `E` is not needed.
 - `!"!name!"!` — wrap the value in double quotes. Suitable for SQL quoted identifiers, or for metacommand arguments that must be quoted.
 
 Variables can also be set before the script starts: with `--var NAME=VALUE` on the [command line](../getting-started/syntax.md#options), or in the `[variables]` section of a [config file](configuration.md#configuration). `--var` wins over the config file, and a `SUB` in the script can reassign either.

{execsql2-2.28.0 → execsql2-2.29.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "execsql2"
-version = "2.28.0"
+version = "2.29.0"
 description = "Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables."
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { file = "LICENSE.txt" }
@@ -189,7 +189,7 @@ line-ending = "auto"
 exclude = ["*.md"]
 
 [tool.bumpversion]
-current_version = "2.28.0"
+current_version = "2.29.0"
 commit = true
 tag = true
 tag_name = "v{new_version}"

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/db/base.py

@@ -215,21 +215,18 @@ class Database(ABC):
     def quote_literal(self, value: Any) -> str:
         """Return *value* as a SQL string literal, safely escaped.
 
-        Default ANSI/Postgres behaviour: wrap in single quotes, double
-        embedded apostrophes, escape backslashes (so a value containing
-        ``\\'`` cannot terminate the literal on MySQL default mode or
-        PostgreSQL E-strings), and reject embedded NUL bytes (most
-        wire protocols truncate or reject them).
-
-        Override per-DBMS only when the wire protocol requires a different
-        literal form.
+        Standard SQL: wrap in single quotes and double embedded apostrophes;
+        a backslash is an ordinary character.  Embedded NUL bytes are
+        rejected (most wire protocols truncate or reject them).  A DBMS
+        that reads a backslash in a literal as an escape (MySQL) overrides
+        this.
         """
         if value is None:
             return "NULL"
         s = str(value)
         if "\x00" in s:
             raise ValueError("SQL literal contains a NUL byte; refusing to quote.")
-        return "'" + s.replace("\\", "\\\\").replace("'", "''") + "'"
+        return "'" + s.replace("'", "''") + "'"
 
     def paramsubs(self, paramcount: int) -> str:
         """Return a comma-separated string of *paramcount* parameter placeholders."""

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/db/mysql.py

@@ -67,6 +67,20 @@ class MySQLDatabase(Database):
         re.I,
     )
 
+    def quote_literal(self, value: Any) -> str:
+        """Return *value* as a string literal for MySQL/MariaDB.
+
+        Unless ``NO_BACKSLASH_ESCAPES`` is set, MySQL reads a backslash in a
+        literal as an escape, so ``\\'`` would end it: backslashes are doubled
+        as well as apostrophes.
+        """
+        if value is None:
+            return "NULL"
+        s = str(value)
+        if "\x00" in s:
+            raise ValueError("SQL literal contains a NUL byte; refusing to quote.")
+        return "'" + s.replace("\\", "\\\\").replace("'", "''") + "'"
+
     def __init__(
         self,
         server_name: str,
@@ -307,10 +321,12 @@ class MySQLDatabase(Database):
         table_cols = self.table_columns(table_name, schema_name)
         data_table_cols = [d.lower() for d in table_cols]
         csv_file_cols = csv_file_obj.column_headers()
+        # MySQL column names are case-insensitive: compare without case, and
+        # send the file's own spelling.
         if _state.conf.import_common_cols_only:
-            import_cols = [col for col in csv_file_cols if col in data_table_cols]
+            import_cols = [col for col in csv_file_cols if col.lower() in data_table_cols]
         else:
-            unmatched_cols = list(set(csv_file_cols) - set(data_table_cols))
+            unmatched_cols = [col for col in csv_file_cols if col.lower() not in data_table_cols]
             if len(unmatched_cols) > 0:
                 raise ErrInfo(
                     type="error",
@@ -326,7 +342,7 @@ class MySQLDatabase(Database):
         # reads the format that was detected.
         if (
             csv_file_obj.escapechar is None
-            and data_table_cols == csv_file_cols
+            and data_table_cols == [col.lower() for col in csv_file_cols]
             and _state.conf.empty_strings
             and _state.conf.empty_rows
             and not _state.conf.del_empty_cols

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/db/postgres.py

@@ -353,7 +353,7 @@ class PostgresDatabase(Database):
             from execsql.exporters.delimited import blank_lines_as_empty_rows
 
             # Use Postgres' COPY FROM method via psycopg3's cursor.copy() context manager.
-            rf = csv_file_obj.open("rt")
+            rf = csv_file_obj.openclean("rt")  # past the SKIP lines, as every other reader starts
             if skipheader:
                 next(rf)
             # Copy_from() requires a delimiter, but if there is none, feed it an

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/exporters/xls.py

@@ -158,7 +158,7 @@ class XlsFile:
                 break
             ncols += 1
         sheet_data = []
-        for r in range(junk_header_rows, sheet.nrows - junk_header_rows):
+        for r in range(junk_header_rows, sheet.nrows):
             datarow = row_data(r, ncols)
             if datarow.count(None) == len(datarow):
                 break
@@ -270,7 +270,8 @@ class XlsxFile:
         sheet_data = []
         rowsrc = sheet.iter_rows(min_row=junk_header_rows + 1, values_only=True)
         for r in rowsrc:
-            if not any(r):
+            # A row of zeros or FALSE values is data; only a row of empty cells ends it.
+            if all(c is None or c == "" for c in r):
                 break
             sheet_data.append(list(r))
         for r in range(len(sheet_data)):

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/script/variables.py

@@ -17,6 +17,42 @@ from execsql.exceptions import ErrInfo
 __all__ = ["CounterVars", "SubVarSet", "LocalSubVarSet", "ScriptArgSubVarSet"]
 
 
+def _after_escape_string_prefix(command: str, start: int) -> bool:
+    """Whether the ``!'!`` token at *start* follows an ``E`` that makes it a PostgreSQL escape string (``E!'!v!'!``).
+
+    The ``E`` must stand alone: in ``TYPE!'!v!'!`` it ends a word.
+    """
+    if start < 1 or command[start - 1] not in "Ee":
+        return False
+    return start < 2 or not (command[start - 2].isalnum() or command[start - 2] == "_")
+
+
+def _sql_string_literal(value: str, *, escape_string: bool = False) -> str:
+    """*value* as the string literal ``!'!var!'!`` substitutes, in the current connection's form.
+
+    An apostrophe is doubled everywhere; a backslash only where the DBMS
+    reads it as an escape (MySQL/MariaDB), or inside an escape string the
+    script wrote as ``E!'!var!'!`` (*escape_string*), so ``C:\\data`` is
+    stored as written in every case.  Without a connection, the standard SQL
+    form.  (Upstream doubled backslashes only on Windows, to protect
+    ``re.sub``'s replacement template, not the SQL.)
+    """
+    if escape_string:
+        return "'" + value.replace("\\", "\\\\").replace("'", "''") + "'"
+    db = None
+    try:
+        import execsql.state as _state
+
+        dbs = _state.dbs
+        if dbs is not None and dbs.current_db is not None:
+            db = dbs.current()
+    except Exception:
+        db = None
+    if db is not None:
+        return db.quote_literal(value)
+    return "'" + value.replace("'", "''") + "'"
+
+
 # ---------------------------------------------------------------------------
 # CounterVars
 # ---------------------------------------------------------------------------
@@ -253,13 +289,10 @@ class SubVarSet:
                     )
                 quote = m.group("q")
                 if quote == "'":
-                    # B07a/F002: wrap in single quotes, doubling embedded
-                    # apostrophes AND escaping backslashes so MySQL default
-                    # mode and PostgreSQL E-string literals can't end the
-                    # quoted region via ``\'``. The previous Windows-only
-                    # branch (``os.name != 'posix'``) applied the escape on
-                    # the wrong axis — host OS vs target DBMS.
-                    sub = "'" + sub.replace("\\", "\\\\").replace("'", "''") + "'"
+                    sub = _sql_string_literal(
+                        sub,
+                        escape_string=_after_escape_string_prefix(command_str, m.start()),
+                    )
                 elif quote == '"':
                     # B07a/F001: wrap in double quotes, doubling embedded
                     # ``"`` so a value containing ``"; DROP TABLE x; --``
@@ -306,13 +339,12 @@ class SubVarSet:
             if idx != -1:
                 _check_nul(varname, sub)
                 return command_str[:idx] + sub + command_str[idx + len(token) :], True
-            # Single-quote-wrapped token: !'!varname!'! — escape ``\`` and
-            # double embedded ``'`` (see substitute_one for rationale).
+            # Single-quote-wrapped token: !'!varname!'! — a string literal.
             tokenq = f"!'!{varname}!'!"
             idxq = cmd_lower.find(tokenq)
             if idxq != -1:
                 _check_nul(varname, sub)
-                wrapped = "'" + sub.replace("\\", "\\\\").replace("'", "''") + "'"
+                wrapped = _sql_string_literal(sub, escape_string=_after_escape_string_prefix(command_str, idxq))
                 return (
                     command_str[:idxq] + wrapped + command_str[idxq + len(tokenq) :],
                     True,

{execsql2-2.28.0 → execsql2-2.29.0}/src/execsql/types.py

@@ -884,6 +884,14 @@ dbt_oracle.name_datatype(DT_TimestampTZ, "TIMESTAMP WITH TIME ZONE")
 dbt_oracle.name_datatype(DT_Timestamp, "TIMESTAMP", casting_name="TIMESTAMP")
 dbt_oracle.name_datatype(DT_Date, "DATE", casting_name="DATE")
 dbt_oracle.name_datatype(DT_Time_Oracle, "VARCHAR2", True, casting_name="VARCHAR(20)")
+# Oracle has no time-of-day type, and oracledb cannot bind a datetime.time: a
+# time column is stored as its ISO text (at most 15 characters, HH:MM:SS.ffffff).
+dbt_oracle.name_datatype(
+    DT_Time,
+    "VARCHAR2(15)",
+    casting_name="VARCHAR(20)",
+    conv_mod_fn=lambda x: x.isoformat() if x is not None else None,
+)
 dbt_oracle.name_datatype(DT_Integer, "NUMBER")
 dbt_oracle.name_datatype(DT_Long, "NUMBER")
 dbt_oracle.name_datatype(DT_Float, "FLOAT")

{execsql2-2.28.0 → execsql2-2.29.0}/tests/db/test_postgres_inprocess.py

@@ -430,6 +430,9 @@ def _make_csv_file_obj(tmp_path, headers, rows, *, delimiter=",", quotechar='"',
         def open(self, _mode):
             return open(self.csvfname, encoding=encoding)
 
+        def openclean(self, mode):
+            return self.open(mode)  # no junk lines before the header
+
         def reader(self):
             import csv as _csv
 

execsql2-2.29.0/tests/db/test_quoted_substitution_roundtrip.py

@@ -0,0 +1,40 @@
+"""``!'!var!'!`` stores a value exactly as written, backslashes and apostrophes included."""
+
+from __future__ import annotations
+
+import pytest
+
+from execsql import run
+
+VALUES = [r"C:\data\new", r"x\'; drop table t; --", "O'Brien", "\\", r"two\\backslashes"]
+
+
+def _script(value: str) -> str:
+    return f"-- !x! SUB v {value}\ncreate table t (v text);\ninsert into t values (!'!v!'!);\n"
+
+
+@pytest.mark.parametrize("value", VALUES)
+def test_sqlite_round_trip(tmp_path, value):
+    import sqlite3
+
+    db = tmp_path / "t.db"
+    result = run(sql=_script(value), dsn=f"sqlite:///{db}", new_db=True)
+    assert result.success, result.errors
+    con = sqlite3.connect(db)
+    try:
+        assert con.execute("select v from t").fetchall() == [(value,)]
+    finally:
+        con.close()
+
+
+@pytest.mark.parametrize("value", VALUES)
+def test_duckdb_round_trip(tmp_path, value):
+    duckdb = pytest.importorskip("duckdb")
+    db = tmp_path / "t.duckdb"
+    result = run(sql=_script(value), dsn=f"duckdb:///{db}", new_db=True)
+    assert result.success, result.errors
+    con = duckdb.connect(str(db), read_only=True)
+    try:
+        assert con.execute("select v from t").fetchall() == [(value,)]
+    finally:
+        con.close()

{execsql2-2.28.0 → execsql2-2.29.0}/tests/importers/test_xls_importer.py

@@ -182,3 +182,35 @@ class TestImportXls:
         names = [r[0] for r in rows]
         assert "existing" in names
         assert "new" in names
+
+
+class TestSpreadsheetRowsAfterTheHeader:
+    """Every data row is read: a row of zeros is data, and SKIP does not shorten the sheet."""
+
+    def test_an_xlsx_row_of_zeros_and_false_does_not_end_the_data(self, tmp_path):
+        rows = [["id", "flag", "note"], [1, True, "a"], [0, False, None], [2, True, "c"]]
+        path = _make_xlsx(tmp_path, "Sheet1", rows)
+        _hdrs, data = xls_data(path, "Sheet1", 0)
+        assert len(data) == 3
+
+    def test_an_xlsx_row_of_empty_cells_still_ends_the_data(self, tmp_path):
+        path = _make_xlsx(tmp_path, "Sheet1", [["id"], [1], [None], [2]])
+        _hdrs, data = xls_data(path, "Sheet1", 0)
+        assert len(data) == 1
+
+    def test_xls_skip_keeps_the_last_rows(self):
+        from types import SimpleNamespace
+
+        from execsql.exporters.xls import XlsFile
+
+        def cell(value):
+            return SimpleNamespace(ctype=1 if isinstance(value, str) else 2, value=value)
+
+        sheet_rows = [["junk"], ["junk"], ["id", "v"], [1, 1], [2, 2], [3, 3]]
+        sheet = SimpleNamespace(nrows=len(sheet_rows), row_slice=lambda r: [cell(v) for v in sheet_rows[r]])
+        xls = object.__new__(XlsFile)
+        xls.filename = "fake.xls"
+        xls.sheet_named = lambda name: sheet
+        data = xls.sheet_data("Sheet1", junk_header_rows=2)
+        assert data[0] == ["id", "v"]
+        assert [r[0] for r in data[1:]] == [1, 2, 3]

{execsql2-2.28.0 → execsql2-2.29.0}/tests/integration/test_mysql.py

@@ -485,3 +485,59 @@ class TestImportBackslashes:
 
     def test_a_backslash_in_the_file_path_is_kept(self, tmp_path):
         assert self._import(tmp_path, tmp_path / "dir\\with\\backslashes") == self.VALUES
+
+
+class TestHeadersWithCapitals:
+    """MySQL column names are case-insensitive; a header with capitals must match its column."""
+
+    @pytest.mark.parametrize("quoted", [False, True], ids=["load-data", "row-by-row"])
+    def test_import_to_new_loads_every_column(self, tmp_path, quoted):
+        _exec_mysql("DROP TABLE IF EXISTS t_mixed")
+        _write_conf(tmp_path)
+        csv_path = tmp_path / "mixed.csv"
+        # An escape character sends the file down the row-by-row path.
+        body = 'Id,Name\n1,"Ann \\"A\\""\n2,Bo\n' if quoted else "Id,Name\n1,Ann\n2,Bo\n"
+        csv_path.write_text(body)
+        script = write_script(tmp_path, f"-- !x! IMPORT TO NEW t_mixed FROM {csv_path}\n")
+        try:
+            result = _run_execsql_mysql(tmp_path, script)
+            assert result.returncode == 0, f"stderr: {result.stderr}"
+            rows = _query_mysql("SELECT Id, Name FROM t_mixed ORDER BY Id")
+            assert [r[0] for r in rows] == [1, 2]
+            assert all(r[1] for r in rows)
+        finally:
+            _exec_mysql("DROP TABLE IF EXISTS t_mixed")
+
+    def test_common_columns_only_keeps_a_capitalized_column(self, tmp_path):
+        _exec_mysql("DROP TABLE IF EXISTS t_common")
+        _exec_mysql("CREATE TABLE t_common (id INTEGER, name VARCHAR(10))")
+        _write_conf(tmp_path)
+        csv_path = tmp_path / "common.csv"
+        csv_path.write_text("Id,Name,Extra\n1,Ann,x\n")
+        script = write_script(
+            tmp_path,
+            f"-- !x! IMPORT_COMMON_COLUMNS_ONLY YES\n-- !x! IMPORT TO t_common FROM {csv_path}\n",
+        )
+        try:
+            result = _run_execsql_mysql(tmp_path, script)
+            assert result.returncode == 0, f"stderr: {result.stderr}"
+            assert [tuple(r) for r in _query_mysql("SELECT id, name FROM t_common")] == [(1, "Ann")]
+        finally:
+            _exec_mysql("DROP TABLE IF EXISTS t_common")
+
+
+@pytest.mark.parametrize("value", [r"C:\data\new", r"x\'; drop table t; --", "O'Brien", r"two\\backslashes"])
+def test_quoted_substitution_stores_the_value_as_written(tmp_path, value):
+    """MySQL reads \\ as an escape, so !'!v!'! doubles it there, and \\' cannot end the literal."""
+    _exec_mysql("DROP TABLE IF EXISTS quoted_sub")
+    _write_conf(tmp_path)
+    script = write_script(
+        tmp_path,
+        f"-- !x! SUB v {value}\ncreate table quoted_sub (v text);\ninsert into quoted_sub values (!'!v!'!);\n",
+    )
+    try:
+        result = _run_execsql_mysql(tmp_path, script)
+        assert result.returncode == 0, f"stderr: {result.stderr}"
+        assert [tuple(r) for r in _query_mysql("SELECT v FROM quoted_sub")] == [(value,)]
+    finally:
+        _exec_mysql("DROP TABLE IF EXISTS quoted_sub")

{execsql2-2.28.0 → execsql2-2.29.0}/tests/integration/test_postgres.py

@@ -686,3 +686,59 @@ def test_copy_moves_a_bytea_column_byte_for_byte(tmp_path):
         assert row == ("bytea", payload)
     finally:
         run(sql="drop table if exists bin_src;\ndrop table if exists bin_dst;\n", dsn=_PG_DSN)
+
+
+def test_import_skip_applies_on_the_copy_path(tmp_path):
+    """A quoted CSV takes PostgreSQL's COPY path; SKIP n must drop n lines before the header there too."""
+    from execsql import run
+
+    csv = tmp_path / "skip.csv"
+    csv.write_text('exported by a tool\nsecond preamble line\nid,name\n"1","one"\n"2","two"\n', encoding="utf-8")
+    run(sql="drop table if exists skip_target;\n", dsn=_PG_DSN)
+    try:
+        result = run(sql=f"-- !x! IMPORT TO NEW skip_target FROM {csv} SKIP 2\n", dsn=_PG_DSN)
+        assert result.success, result.errors
+        with psycopg.connect(**_PG_CONNECT_KWARGS) as conn:
+            rows = conn.execute("select id, name from skip_target order by id").fetchall()
+        assert [tuple(map(str, r)) for r in rows] == [("1", "one"), ("2", "two")]
+    finally:
+        run(sql="drop table if exists skip_target;\n", dsn=_PG_DSN)
+
+
+@pytest.mark.parametrize("value", [r"C:\data\new", r"x\'; drop table t; --", "O'Brien", r"two\\backslashes"])
+def test_quoted_substitution_stores_the_value_as_written(value):
+    """standard_conforming_strings is on: a backslash in '...' is text, so !'!v!'! must not double it."""
+    from execsql import run
+
+    run(sql="drop table if exists quoted_sub;\n", dsn=_PG_DSN)
+    try:
+        result = run(
+            sql=f"-- !x! SUB v {value}\ncreate table quoted_sub (v text);\ninsert into quoted_sub values (!'!v!'!);\n",
+            dsn=_PG_DSN,
+        )
+        assert result.success, result.errors
+        with psycopg.connect(**_PG_CONNECT_KWARGS) as conn:
+            assert conn.execute("select v from quoted_sub").fetchall() == [(value,)]
+    finally:
+        run(sql="drop table if exists quoted_sub;\n", dsn=_PG_DSN)
+
+
+@pytest.mark.parametrize("value", [r"C:\data\new", r"x\'; drop table t; --", r"two\\backslashes"])
+def test_escape_string_form_stores_the_same_value(value):
+    """E!'!v!'! (a PostgreSQL escape string) stores exactly what !'!v!'! stores."""
+    from execsql import run
+
+    run(sql="drop table if exists quoted_sub_e;\n", dsn=_PG_DSN)
+    try:
+        result = run(
+            sql=(
+                f"-- !x! SUB v {value}\ncreate table quoted_sub_e (plain text, escaped text);\n"
+                "insert into quoted_sub_e values (!'!v!'!, E!'!v!'!);\n"
+            ),
+            dsn=_PG_DSN,
+        )
+        assert result.success, result.errors
+        with psycopg.connect(**_PG_CONNECT_KWARGS) as conn:
+            assert conn.execute("select plain, escaped from quoted_sub_e").fetchall() == [(value, value)]
+    finally:
+        run(sql="drop table if exists quoted_sub_e;\n", dsn=_PG_DSN)

{execsql2-2.28.0 → execsql2-2.29.0}/tests/scripts/fixtures/audit_smoke.sql

@@ -143,11 +143,11 @@ drop table if exists q_bs_chk;
 CREATE TABLE q_bs_chk (val TEXT);
 INSERT INTO q_bs_chk SELECT val FROM q_bs;
 
--- SQLite stores the doubled form verbatim (no escape interpretation).
+-- SQLite reads no escapes in a literal, so the value round-trips byte for byte.
 -- !x! select_sub q_bs_chk
--- !x! ASSERT EQUALS("!!@val!!", "c:\\path\\file") "backslash doubling is universal (security > byte-perfect SQLite round-trip)"
+-- !x! ASSERT EQUALS("!!@val!!", "c:\path\file") "a backslash is stored as written outside MySQL"
 
-INSERT INTO _runlog VALUES ('phase5', 'backslash doubling confirmed');
+INSERT INTO _runlog VALUES ('phase5', 'backslash stored as written');
 
 
 -- =========================================================================