execsql2 2.12.1__tar.gz → 2.12.3__tar.gz

execsql2-2.12.3/.claude/agents/liaison.md

@@ -0,0 +1,107 @@
+---
+name: The Liaison
+description: Integration specialist for pg-upsert and execsql2. Tracks the pg-upsert codebase (../pg-upsert), designs the optional dependency integration, and owns the UPSERT metacommand. Reads both codebases — writes only in execsql2.
+model: sonnet
+color: yellow
+---
+
+You are a senior Python engineer who specializes in integrating the **pg-upsert** library with the **execsql2** project. You are the bridge between two codebases maintained by the same author:
+
+- **execsql2** — `/Users/cgrant/GitHub/geocoug/execsql/` (this repo)
+- **pg-upsert** — `/Users/cgrant/GitHub/geocoug/pg-upsert/` (sibling repo)
+
+Your job is to keep the integration plan current, design the `UPSERT` metacommand, and ensure changes in either codebase don't break the integration path.
+
+## Expertise
+
+You have deep, working knowledge of:
+
+**pg-upsert internals:** The `PgUpsert` class (QA checks: null, PK, FK, check constraint validation), `PostgresDB` connection wrapper, the staging-to-base upsert workflow, topological table ordering, the `ups_control` temporary table, and the `psycopg2.sql` parameterized SQL generation patterns.
+
+**execsql2 extension points:** The metacommand dispatch system (`MetaCommandList`, `build_dispatch_table()`, `x_*` handlers), `DatabasePool` connection management, substitution variables (`SubVarSet`), transaction/autocommit model, the `PostgresDatabase` adapter, and the existing upsert SQL templates in `templates/pg_upsert.sql`.
+
+**Integration patterns:** Optional dependency detection (`importlib.util.find_spec`), connection sharing between libraries, adapter patterns for bridging different transaction models, and graceful degradation when an optional package is not installed.
+
+______________________________________________________________________
+
+## First Actions (always, before doing any work)
+
+1. **Read `.claude/project_context.md`** — load the execsql2 module layout and architectural overview.
+2. **Read your briefing** if one exists at `.claude/comms/briefings/liaison-*.md` — follow the DBA's specific instructions.
+3. **Read the integration plan** at `../pg-upsert/.claude/plans/refactor-and-execsql-integration.md` — this is the canonical plan. Understand the current phase and what's been decided.
+4. **Check pg-upsert's current state** — read `../pg-upsert/src/pg_upsert/upsert.py` and `../pg-upsert/src/pg_upsert/postgres.py` to understand the latest API surface. pg-upsert is under active refactoring; the API may have changed since the plan was written.
+5. **Check execsql2's metacommand system** — read `src/execsql/metacommands/dispatch.py` to understand registration patterns.
+
+______________________________________________________________________
+
+## Key Design Decisions (established)
+
+These have been agreed upon. Do not revisit unless the human asks:
+
+1. **pg-upsert is an optional dependency** — execsql2 must work without it installed. The `UPSERT` metacommand raises a clear error if pg-upsert is missing.
+2. **Connection sharing** — the metacommand passes execsql's existing `psycopg2` connection to `PgUpsert(conn=...)`. No second connection.
+3. **Results map to substitution variables** — upsert stats (rows updated, inserted, QA errors) are exposed as `!!$UPSERT_*!!` substitution variables.
+4. **execsql's transaction model governs** — the metacommand respects execsql's `AUTOCOMMIT ON/OFF` state, not pg-upsert's `do_commit` flag.
+5. **No Tkinter dependency** — the interactive GUI is not used when called from execsql. QA results go to execsql's logging/debug REPL instead.
+
+______________________________________________________________________
+
+## pg-upsert API Surface (reference)
+
+The integration depends on these pg-upsert interfaces. If any change, the integration plan must be updated.
+
+```python
+# Core class — this is what the metacommand will call
+PgUpsert(
+    conn=psycopg2_connection,  # Shared from execsql
+    tables=["t1", "t2"],
+    staging_schema="staging",
+    base_schema="public",
+    do_commit=False,           # execsql controls commits
+    interactive=False,         # No GUI from execsql
+    upsert_method="upsert",   # "upsert" | "update" | "insert"
+    exclude_cols=["col1"],
+    exclude_null_check_cols=["col2"],
+)
+
+# Methods the metacommand will call
+.qa_all()       # Run all QA checks, returns self
+.upsert_all()   # Run upsert on all tables, returns self
+.commit()       # Commit/rollback based on do_commit + qa_passed
+.run()          # qa_all() → upsert_all() → commit()
+
+# State the metacommand will read
+.qa_passed      # bool — did all QA checks pass?
+.control_table  # str — name of temp table with per-table results
+```
+
+______________________________________________________________________
+
+## What to Produce
+
+Depending on the task, you may produce:
+
+1. **Integration status reports** — what's changed in pg-upsert, what that means for execsql integration
+2. **Metacommand design** — regex patterns, handler signatures, SQL syntax for `UPSERT` metacommand
+3. **Implementation code** — `src/execsql/metacommands/upsert.py` and dispatch registration
+4. **Compatibility notes** — API changes in pg-upsert that require adaptation
+5. **Test specifications** — what the QA agent should test for the integration
+
+______________________________________________________________________
+
+## Syndicate Protocol
+
+When working as part of the SQL Syndicate:
+
+1. Read your briefing from `.claude/comms/briefings/liaison-*.md`
+2. Do your research across both codebases
+3. Write your report to `.claude/comms/reports/liaison-{YYYY-MM-DD}.md`
+4. Write integration artifacts to `.claude/plans/` (for design) or `.claude/patches/` (for implementation)
+
+## Constraints
+
+- **Writes only in execsql2.** You may read `../pg-upsert/` freely but never modify files there.
+- **pg-upsert is a moving target.** Always re-read the current pg-upsert source before making claims about its API — don't rely on cached knowledge.
+- **Optional means optional.** Never add pg-upsert to execsql2's required dependencies. Use `extras_require` / optional dependency groups.
+- **Preserve existing upsert templates.** The SQL templates in `templates/pg_upsert.sql` must continue to work. The metacommand is an alternative, not a replacement.
+- Follow all execsql2 project constraints from CLAUDE.md (ruff, Python 3.10+, coverage floor, changelog, docs, divergence tracking).

{execsql2-2.12.1 → execsql2-2.12.3}/CHANGELOG.md

@@ -13,6 +13,29 @@ ______________________________________________________________________
 
 ______________________________________________________________________
 
+## [2.12.3] - 2026-04-02
+
+### Changed
+
+- Performance: split `set_system_vars()` into static (once per script + on CONNECT/CHDIR) and dynamic (per statement) — eliminates ~14 redundant `add_substitution` calls and 2 `Path.resolve()` filesystem syscalls per statement.
+- Performance: `$RANDOM` and `$UUID` are now lazy — computed only when actually referenced in a statement, not generated unconditionally for every statement.
+- Performance: `LineDelimiter.delimited()` caches `quote_all_text` at construction time instead of reading `_state.conf` via module proxy on every row during export.
+- Performance: CSV/TSV import uses Python's `csv` module as a fast path for standard delimited formats (comma, tab, semicolon, pipe) with doubled-quote escaping. Falls back to the character-at-a-time parser for non-standard formats (space-delimiter collapsing, escape characters).
+
+______________________________________________________________________
+
+## [2.12.2] - 2026-04-02
+
+### Added
+
+- Documentation: keyring setup guide for headless Linux servers (encrypted and plaintext file backends) in the Security reference page, with a cross-reference from the Installation page.
+
+### Changed
+
+- `ASSERT` failures now report `**** Assertion failed.` instead of `**** Error in metacommand.` to distinguish intentional script-level checks from actual metacommand errors.
+
+______________________________________________________________________
+
 ## [2.12.1] - 2026-04-02
 
 ### Changed

{execsql2-2.12.1 → execsql2-2.12.3}/CLAUDE.md

@@ -22,6 +22,7 @@ A multi-agent system where specialized agents collaborate to improve, extend, de
 - `.claude/docs-drafts/` — Scribe's documentation drafts
 - `.claude/releases/` — Herald's release notes and changelog entries
 - `.claude/state/` — Shared state (current phase, active task, agent status)
+- `../pg-upsert/.claude/plans/` — pg-upsert integration plan (read by The Liaison)
 
 ## Communication Protocol
 

{execsql2-2.12.1 → execsql2-2.12.3}/PKG-INFO

@@ -1,7 +1,8 @@
 Metadata-Version: 2.4
 Name: execsql2
-Version: 2.12.1
+Version: 2.12.3
 Summary: Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables.
+Project-URL: Homepage, https://execsql2.readthedocs.io
 Project-URL: Repository, https://github.com/geocoug/execsql
 Project-URL: Issues, https://github.com/geocoug/execsql/issues
 Author-email: Dreas Nielsen <cortice@tutanota.com>

{execsql2-2.12.1 → execsql2-2.12.3}/docs/about/divergence.md

@@ -163,6 +163,12 @@ All 33 mutable runtime globals in `state.py` have been consolidated into a `Runt
 
 - **Cycle detection** — `substitute_vars()` raises an error after 100 iterations to prevent infinite loops when variables reference each other cyclically. Upstream had no protection.
 - **O(1) substitution** — Variable substitution uses a single combined regex and dict lookup instead of O(V) per-variable regex passes. Behavior is identical; performance is improved.
+- **Lazy `$RANDOM`/`$UUID`** — These system variables are now computed on first access rather than generated unconditionally for every statement. Behavior is identical when referenced; scripts that never reference them skip the computation entirely.
+- **Static/dynamic system var split** — System substitution variables are split into static (set once per script and refreshed on CONNECT/CHDIR) and dynamic (refreshed per statement). Eliminates redundant `Path.resolve()` syscalls and database pool lookups per statement.
+
+### CSV Import
+
+- **Fast-path CSV reader** — Standard delimited imports (comma, tab, semicolon, pipe with doubled-quote escaping) now use Python's `csv` module. Non-standard formats (space-delimiter collapsing, escape characters) fall back to the original character-at-a-time parser.
 
 ### Database Adapters
 
@@ -174,6 +180,7 @@ All 33 mutable runtime globals in `state.py` have been consolidated into a `Runt
 
 - **Exception hierarchy** — All custom exceptions inherit from `ExecSqlError`, enabling `except ExecSqlError` to catch any execsql-originated error.
 - **Exception chaining** — All `raise` statements inside `except` blocks preserve the original traceback via `from`.
+- **ASSERT error type** — `ASSERT` failures now use a dedicated `"assert"` error type that produces `**** Assertion failed.` instead of `**** Error in metacommand.`. This distinguishes intentional script-level checks from actual metacommand errors. Upstream did not have `ASSERT`.
 
 ______________________________________________________________________
 

{execsql2-2.12.1 → execsql2-2.12.3}/docs/getting-started/installation.md

@@ -30,3 +30,7 @@ pip install "execsql2[all]"         # Everything
 ```
 
 In addition to the *execsql* program itself, additional Python libraries may need to be installed to use *execsql* with specific types of DBMSs and spreadsheets. The additional libraries that may be needed are listed in the [Requirements](requirements.md#requirements) section.
+
+!!! tip "Keyring on headless Linux"
+
+    If you install `execsql2[auth]` on a headless Linux server (no desktop environment), the keyring backend needs manual configuration. See [Keyring Platform Setup](../reference/security.md#keyring_setup) for instructions.

{execsql2-2.12.1 → execsql2-2.12.3}/docs/reference/metacommands.md

@@ -56,7 +56,9 @@ ASSERT <condition> "<failure message>"
 ASSERT <condition> '<failure message>'
 ```
 
-Evaluates `<condition>` using the same expression engine as [IF](#if_cmd). If the condition is `True`, execution continues silently (and the result is written to the log). If the condition is `False`, an error is raised with the provided failure message. If no message is supplied, the default message is `Assertion failed: <condition>`.
+Evaluates `<condition>` using the same expression engine as [IF](#if_cmd). If the condition is `True`, execution continues silently (and the result is written to the log). If the condition is `False`, an assertion error is raised with the provided failure message. If no message is supplied, the default message is `Assertion failed: <condition>`.
+
+A failed assertion produces `**** Assertion failed.` (not `**** Error in metacommand.`) to make it clear that the script's own check caught a problem, not that execsql encountered an internal error.
 
 When [HALT_ON_METACOMMAND_ERROR](#config) is `ON` (the default), a failed assertion halts the script. When it is `OFF`, execution continues after the failure is logged.
 

{execsql2-2.12.1 → execsql2-2.12.3}/docs/reference/security.md

@@ -29,7 +29,7 @@ When execsql needs a database password and none is stored or configured, it prom
 
 ### OS credential store (keyring)
 
-When the optional `keyring` package is installed (`pip install execsql2[auth]`), execsql checks the OS credential store before prompting. After a successful interactive prompt, the password is automatically stored for future use. Supported stores are macOS Keychain, Windows Credential Manager, and Linux SecretService. Keyring service names follow the pattern:
+When the optional `keyring` package is installed (`pip install execsql2[auth]`), execsql checks the OS credential store before prompting. After a successful interactive prompt, the password is automatically stored for future use. Keyring service names follow the pattern:
 
 ```text
 execsql/<db_type>/<server>/<database>
@@ -37,6 +37,64 @@ execsql/<db_type>/<server>/<database>
 
 To disable keyring integration, set `use_keyring = No` in the `[connect]` section of `execsql.conf`.
 
+#### Platform setup { #keyring_setup }
+
+**macOS** — Works out of the box. Keyring uses the macOS Keychain automatically.
+
+**Windows** — Works out of the box. Keyring uses the Windows Credential Manager automatically.
+
+**Linux (desktop with GNOME/KDE)** — Works out of the box if a SecretService provider (GNOME Keyring or KWallet) is running.
+
+**Linux (headless / remote / server)** — No secret service is typically available, so keyring silently fails to store passwords. You need to configure a file-based backend. There are two options:
+
+##### Option A: Encrypted file backend (recommended) { #keyring_encrypted }
+
+Passwords are stored encrypted on disk. Requires a master password the first time keyring is used per session.
+
+```bash
+pip install keyrings.alt pycryptodome
+mkdir -p ~/.config/python_keyring
+cat > ~/.config/python_keyring/keyringrc.cfg << 'EOF'
+[backend]
+default-keyring=keyrings.alt.file.EncryptedKeyring
+EOF
+```
+
+The encrypted keyring file is stored at `~/.local/share/python_keyring/crypted_pass.cfg`. You will be prompted for a master password once per session (e.g., when you first run execsql after logging in).
+
+##### Option B: Plaintext file backend (no prompts) { #keyring_plaintext }
+
+Passwords are stored in plain text on disk. No master password is needed — execsql will never prompt for a password after the first successful entry.
+
+```bash
+pip install keyrings.alt
+mkdir -p ~/.config/python_keyring
+cat > ~/.config/python_keyring/keyringrc.cfg << 'EOF'
+[backend]
+default-keyring=keyrings.alt.file.PlaintextKeyring
+EOF
+```
+
+Passwords are stored at `~/.local/share/python_keyring/keyring_pass.cfg`. Restrict access to this file:
+
+```bash
+chmod 600 ~/.local/share/python_keyring/keyring_pass.cfg
+```
+
+!!! warning "Plaintext storage"
+
+    The plaintext backend stores passwords without encryption. Only use this on machines where the filesystem is already secured (encrypted disk, single-user access, restricted permissions). Prefer the encrypted backend when possible.
+
+##### Verifying keyring setup { #keyring_verify }
+
+After configuring a backend, verify it works:
+
+```bash
+python -c "import keyring; keyring.set_password('test', 'user', 'pw'); print(keyring.get_password('test', 'user'))"
+```
+
+This should print `pw` without errors. If it does, the next time execsql prompts for a database password, it will store it and skip the prompt on future runs.
+
 ### `enc_password` in execsql.conf
 
 !!! warning "Obfuscation only — not encryption"

{execsql2-2.12.1 → execsql2-2.12.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "execsql2"
-version = "2.12.1"
+version = "2.12.3"
 description = "Runs a SQL script against a PostgreSQL, SQLite, MariaDB/MySQL, DuckDB, Firebird, MS-Access, MS-SQL-Server, or Oracle database, or an ODBC DSN. Provides metacommands to import and export data, copy data between databases, conditionally execute SQL and metacommands, and dynamically alter SQL and metacommands with substitution variables."
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { file = "LICENSE.txt" }
@@ -91,6 +91,7 @@ execsql = "execsql.cli:_legacy_main"
 execsql-format = "execsql.format:main"
 
 [project.urls]
+Homepage = "https://execsql2.readthedocs.io"
 Repository = "https://github.com/geocoug/execsql"
 Issues = "https://github.com/geocoug/execsql/issues"
 
@@ -158,7 +159,7 @@ skip-magic-trailing-comma = false
 line-ending = "auto"
 
 [tool.bumpversion]
-current_version = "2.12.1"
+current_version = "2.12.3"
 commit = true
 commit_args = "--no-verify"
 tag = true

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/db/base.py

@@ -686,6 +686,13 @@ class DatabasePool:
             )
             self.pool[db_alias].close()
         self.pool[db_alias] = db_obj
+        # Refresh static system vars so $DB_NAME, $DB_USER, etc. reflect the new connection.
+        try:
+            from execsql.script.engine import set_static_system_vars
+
+            set_static_system_vars()
+        except Exception:
+            pass  # Engine not yet initialized (early startup).
 
     def aliases(self) -> list[str]:
         """Return a list of all currently registered database aliases."""

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/exceptions.py

@@ -132,6 +132,8 @@ class ErrInfo(ExecSqlError):
 
         if self.type == "db":
             self.error_message = "**** Error in SQL statement."
+        elif self.type == "assert":
+            self.error_message = "**** Assertion failed."
         elif self.type == "cmd":
             self.error_message = "**** Error in metacommand."
         elif self.type == "log":

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/exporters/delimited.py

@@ -40,6 +40,7 @@ class LineDelimiter:
         self.delimiter = delim
         self.joinchar = delim if delim else ""
         self.quotechar = quote
+        self.quote_all_text = _state.conf.quote_all_text if _state.conf else False
         if quote:
             if escchar:
                 self.quotedquote = escchar + quote
@@ -50,13 +51,12 @@ class LineDelimiter:
 
     def delimited(self, datarow: Any, add_newline: bool = True) -> str:
         """Format a sequence of values as a single delimited text line."""
-        conf = _state.conf
         if self.quotechar:
             d_row = []
             for e in datarow:
                 if isinstance(e, str):
                     if (
-                        conf.quote_all_text
+                        self.quote_all_text
                         or (self.quotechar in e)
                         or (self.delimiter is not None and self.delimiter in e)
                         or ("\n" in e)
@@ -609,10 +609,52 @@ class CsvFile(EncodedFile):
             raise ErrInfo("error", other_msg=", ".join(self.parse_errors))
         return elements
 
+    def _can_use_fast_csv_reader(self) -> bool:
+        """Return True if the detected format is compatible with Python's csv module."""
+        # The csv module handles comma/tab delimiters with doubled-quote escaping.
+        # It cannot handle: space-delimiter collapsing, escape chars, or no delimiter.
+        if self.delimiter is None or self.delimiter == " ":
+            return False
+        return self.escapechar is None
+
     def reader(self) -> Any:
         """Yield parsed rows from the file as lists of field values."""
         conf = _state.conf
         self.evaluate_line_format()
+        if self._can_use_fast_csv_reader():
+            yield from self._fast_reader(conf)
+        else:
+            yield from self._slow_reader(conf)
+
+    def _fast_reader(self, conf: Any) -> Any:
+        """Read using Python's csv module (fast path for standard delimited formats)."""
+        import csv
+
+        f = self.openclean("rt")
+        try:
+            csv_reader = csv.reader(
+                f,
+                delimiter=self.delimiter,
+                quotechar=self.quotechar,
+                doublequote=True,
+                strict=False,
+            )
+            for elements in csv_reader:
+                if len(elements) == 0:
+                    break
+                # Normalize empty strings to None for parity with the slow reader.
+                elements = [e if e != "" else None for e in elements]
+                if conf.del_empty_cols and len(self.blank_cols) > 0:
+                    blanks = copy.copy(self.blank_cols)
+                    while len(blanks) > 0:
+                        b = blanks.pop()
+                        del elements[b]
+                yield elements
+        finally:
+            f.close()
+
+    def _slow_reader(self, conf: Any) -> Any:
+        """Read using the character-at-a-time state machine (fallback for non-standard formats)."""
         f = self.openclean("rt")
         line_no = 0
         try:

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/metacommands/control.py

@@ -66,7 +66,7 @@ def x_assert(**kwargs: Any) -> None:
         if _state.exec_log is not None:
             _state.exec_log.log_user_msg(f"ASSERT passed: {condition}")
     else:
-        raise ErrInfo(type="cmd", other_msg=message)
+        raise ErrInfo(type="assert", other_msg=message)
 
 
 def x_if(**kwargs: Any) -> None:

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/metacommands/io_fileops.py

@@ -241,6 +241,10 @@ def x_cd(**kwargs: Any) -> None:
     os.chdir(new_dir)
     script, lno = current_script_line()
     _state.exec_log.log_status_info(f"Current directory changed to {new_dir} at line {lno} of {script}")
+    if _state.subvars is not None:
+        from execsql.script.engine import set_static_system_vars
+
+        set_static_system_vars()
     return None
 
 

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/script/__init__.py

@@ -63,6 +63,8 @@ from execsql.script.engine import (
     read_sqlfile,
     read_sqlstring,
     runscripts,
+    set_dynamic_system_vars,
+    set_static_system_vars,
     set_system_vars,
     substitute_vars,
 )
@@ -86,6 +88,8 @@ __all__ = [
     "CommandListUntilLoop",
     "ScriptFile",
     "ScriptExecSpec",
+    "set_dynamic_system_vars",
+    "set_static_system_vars",
     "set_system_vars",
     "substitute_vars",
     "runscripts",

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/script/engine.py

@@ -704,9 +704,45 @@ class ScriptExecSpec:
 # ---------------------------------------------------------------------------
 
 
-def set_system_vars() -> None:
-    """Refresh all built-in system substitution variables (``$CURRENT_TIME``, ``$DB_NAME``, etc.)."""
-    # (Re)define the system substitution variables that are not script-specific.
+def set_static_system_vars() -> None:
+    """Set system substitution variables that only change on CONNECT or CHDIR.
+
+    Called once before the execution loop.  These values are expensive to compute
+    (filesystem syscalls, database pool lookups) but rarely change — only on
+    ``CONNECT``, ``USE``, or ``CHDIR`` metacommands.  The ``runscripts()`` loop
+    calls this once up front; metacommand handlers that change the connection or
+    working directory should call it again afterward.
+    """
+    import random
+
+    cwd = str(Path(".").resolve())
+    _state.subvars.add_substitution("$CURRENT_DIR", cwd)
+    _state.subvars.add_substitution("$CURRENT_PATH", cwd + os.sep)
+    _state.subvars.add_substitution("$CURRENT_ALIAS", _state.dbs.current_alias())
+    db = _state.dbs.current()
+    _state.subvars.add_substitution("$DB_USER", db.user if db.user else "")
+    _state.subvars.add_substitution(
+        "$DB_SERVER",
+        db.server_name if db.server_name else "",
+    )
+    _state.subvars.add_substitution("$DB_NAME", db.db_name)
+    _state.subvars.add_substitution("$DB_NEED_PWD", "TRUE" if db.need_passwd else "FALSE")
+    _state.subvars.add_substitution("$VERSION1", str(_state.primary_vno))
+    _state.subvars.add_substitution("$VERSION2", str(_state.secondary_vno))
+    _state.subvars.add_substitution("$VERSION3", str(_state.tertiary_vno))
+    # Register lazy providers for $RANDOM and $UUID — computed only when referenced.
+    _state.subvars.register_lazy("$random", lambda: str(random.random()))
+    _state.subvars.register_lazy("$uuid", lambda: str(uuid.uuid4()))
+
+
+def set_dynamic_system_vars() -> None:
+    """Refresh system substitution variables that change every statement.
+
+    Called once per statement in the execution loop.  Includes cheap boolean-to-string
+    conversions for halt states and autocommit (which can change on any CONFIG or
+    AUTOCOMMIT metacommand) plus ``$TIMER`` and lazy-variable cache reset.
+    """
+    # Halt/config state vars — cheap to set, can change on any CONFIG metacommand.
     _state.subvars.add_substitution("$CANCEL_HALT_STATE", "ON" if _state.status.cancel_halt else "OFF")
     _state.subvars.add_substitution("$ERROR_HALT_STATE", "ON" if _state.status.halt_on_err else "OFF")
     _state.subvars.add_substitution(
@@ -718,27 +754,22 @@ def set_system_vars() -> None:
         "ON" if _state.conf.gui_wait_on_error_halt else "OFF",
     )
     _state.subvars.add_substitution("$CONSOLE_WAIT_WHEN_DONE_STATE", "ON" if _state.conf.gui_wait_on_exit else "OFF")
-    # $CURRENT_TIME is set per-statement in run_and_increment() for accuracy.
-    _state.subvars.add_substitution("$CURRENT_DIR", str(Path(".").resolve()))
-    _state.subvars.add_substitution("$CURRENT_PATH", str(Path(".").resolve()) + os.sep)
-    _state.subvars.add_substitution("$CURRENT_ALIAS", _state.dbs.current_alias())
     db = _state.dbs.current()
     _state.subvars.add_substitution("$AUTOCOMMIT_STATE", "ON" if db.autocommit else "OFF")
+    # $CURRENT_TIME is set per-statement in run_and_increment() for accuracy.
     _state.subvars.add_substitution("$TIMER", str(datetime.timedelta(seconds=_state.timer.elapsed())))
-    _state.subvars.add_substitution("$DB_USER", db.user if db.user else "")
-    _state.subvars.add_substitution(
-        "$DB_SERVER",
-        db.server_name if db.server_name else "",
-    )
-    _state.subvars.add_substitution("$DB_NAME", db.db_name)
-    _state.subvars.add_substitution("$DB_NEED_PWD", "TRUE" if db.need_passwd else "FALSE")
-    import random
+    _state.subvars.clear_lazy_cache()
 
-    _state.subvars.add_substitution("$RANDOM", str(random.random()))
-    _state.subvars.add_substitution("$UUID", str(uuid.uuid4()))
-    _state.subvars.add_substitution("$VERSION1", str(_state.primary_vno))
-    _state.subvars.add_substitution("$VERSION2", str(_state.secondary_vno))
-    _state.subvars.add_substitution("$VERSION3", str(_state.tertiary_vno))
+
+def set_system_vars() -> None:
+    """Refresh all built-in system substitution variables.
+
+    Convenience wrapper that calls both :func:`set_static_system_vars` and
+    :func:`set_dynamic_system_vars`.  Retained for backward compatibility with
+    tests and any external callers.
+    """
+    set_static_system_vars()
+    set_dynamic_system_vars()
 
 
 _MAX_SUBSTITUTION_DEPTH = 100
@@ -779,11 +810,12 @@ def substitute_vars(command_str: str, localvars: SubVarSet | None = None) -> str
 
 def runscripts() -> None:
     """Drive execution until the command-list stack is empty."""
-    # Repeatedly run the next statement from the script at the top of the
-    # command list stack until there are no more statements.
+    # Set static vars once before the loop; they are refreshed by metacommand
+    # handlers (CONNECT, CONFIG, AUTOCOMMIT, CHDIR) when state changes.
+    set_static_system_vars()
     while len(_state.commandliststack) > 0:
         current_cmds = _state.commandliststack[-1]
-        set_system_vars()
+        set_dynamic_system_vars()
         try:
             current_cmds.run_next()
         except StopIteration:

{execsql2-2.12.1 → execsql2-2.12.3}/src/execsql/script/variables.py

@@ -89,6 +89,7 @@ class SubVarSet:
     # compatibility with external code.
     def __init__(self) -> None:
         self._subs_dict: dict[str, Any] = {}
+        self._lazy_providers: dict[str, Any] = {}
         self.prefix_list: list[str] = ["$", "&", "@"]
         # Don't construct/compile on init because deepcopy() can't handle compiled regexes.
         self.var_rx = None
@@ -120,6 +121,30 @@ class SubVarSet:
         if not self.var_name_ok(varname.lower()):
             raise ErrInfo("error", other_msg=f"Invalid variable name ({varname}) in this context.")
 
+    def register_lazy(self, varname: str, provider: Any) -> None:
+        """Register a lazy variable whose value is computed on first access per cycle.
+
+        The *provider* callable is invoked only when the variable is actually
+        referenced (via :meth:`substitute`, :meth:`varvalue`, etc.).  The result
+        is cached in ``_subs_dict`` until :meth:`clear_lazy_cache` is called.
+        """
+        self.check_var_name(varname)
+        self._lazy_providers[varname.lower()] = provider
+
+    def clear_lazy_cache(self) -> None:
+        """Remove materialized lazy values so they regenerate on next access."""
+        for key in self._lazy_providers:
+            self._subs_dict.pop(key, None)
+
+    def _materialize_lazy(self, varname: str) -> str | None:
+        """If *varname* has a lazy provider, invoke it, cache the result, and return it."""
+        provider = self._lazy_providers.get(varname)
+        if provider is not None:
+            value = str(provider())
+            self._subs_dict[varname] = value
+            return value
+        return None
+
     def remove_substitution(self, template_str: str) -> None:
         """Remove the variable named *template_str* from the substitution pool."""
         self.check_var_name(template_str)
@@ -143,7 +168,11 @@ class SubVarSet:
     def varvalue(self, varname: str) -> str | None:
         """Return the value of *varname*, or ``None`` if it is not defined."""
         self.check_var_name(varname)
-        return self._subs_dict.get(varname.lower())
+        key = varname.lower()
+        val = self._subs_dict.get(key)
+        if val is None and key in self._lazy_providers:
+            return self._materialize_lazy(key)
+        return val
 
     def increment_by(self, varname: str, numeric_increment: Any) -> None:
         self.check_var_name(varname)
@@ -165,13 +194,15 @@ class SubVarSet:
     def sub_exists(self, template_str: str) -> bool:
         """Return True if the variable named *template_str* is defined."""
         self.check_var_name(template_str)
-        return template_str.lower() in self._subs_dict
+        key = template_str.lower()
+        return key in self._subs_dict or key in self._lazy_providers
 
     def merge(self, other_subvars: SubVarSet | None) -> SubVarSet:
         """Return a new SubVarSet with this object's variables merged with other_subvars."""
         if other_subvars is not None:
             newsubs = SubVarSet()
             newsubs._subs_dict = {**self._subs_dict, **other_subvars._subs_dict}
+            newsubs._lazy_providers = {**self._lazy_providers, **other_subvars._lazy_providers}
             newsubs.prefix_list = list(set(self.prefix_list + other_subvars.prefix_list))
             newsubs.compile_var_rx()
             return newsubs
@@ -201,6 +232,8 @@ class SubVarSet:
         m = self._TOKEN_RX.search(command_str)
         while m:
             varname = m.group("varname").lower()
+            if varname not in self._subs_dict and varname in self._lazy_providers:
+                self._materialize_lazy(varname)
             if varname in self._subs_dict:
                 sub = self._subs_dict[varname]
                 if sub is None: