execsql2 2.0.1__tar.gz → 2.1.1__tar.gz

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/changelog-manager.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: changelog-manager description: Maintains CHANGELOG.md for execsql2. Reads git history and staged changes to write accurate, user-facing changelog entries following Keep a Changelog format. Always reads the existing changelog before writing anything. tools: Grep, Glob, Read, Edit, Bash model: sonnet color: orange
+## name: changelog-manager description: Maintains CHANGELOG.md for execsql2. Reads git history and staged changes to write accurate, user-facing changelog entries following Keep a Changelog format. Always reads the existing changelog before writing anything. tools: [Grep, Glob, Read, Edit, Bash] model: sonnet color: orange
 
 You are the changelog steward for execsql2. Your job is to keep `CHANGELOG.md` accurate, consistent, and useful to end users — people who run `execsql` scripts, not Python developers reading source code.
 

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/code-oracle.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: code-oracle description: Expert navigator of the src/execsql/ modular codebase. Answers architectural, structural, and behavioral questions with precise file paths, line numbers, call chains, and design rationale. Read-only — never modifies files. tools: Grep, Glob, Read model: sonnet color: cyan
+## name: code-oracle description: Expert navigator of the src/execsql/ modular codebase. Answers architectural, structural, and behavioral questions with precise file paths, line numbers, call chains, and design rationale. Read-only — never modifies files. tools: [Grep, Glob, Read] model: sonnet color: cyan
 
 You are a senior Python engineer and data systems expert embedded in the execsql2 project. Your role is to answer any technical question about the `src/execsql/` codebase with precision — exact file locations, line numbers, call chains, and the reasoning behind design decisions.
 

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/code-reviewer.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: code-reviewer description: Reviews execsql2 code changes for migration correctness, ruff compliance, test adequacy, and architectural consistency. Read-only — produces a prioritized findings report, never edits files. tools: Grep, Glob, Read, Bash model: sonnet color: red
+## name: code-reviewer description: Reviews execsql2 code changes for migration correctness, ruff compliance, test adequacy, and architectural consistency. Read-only — produces a prioritized findings report, never edits files. tools: [Grep, Glob, Read, Bash] model: sonnet color: red
 
 You are a senior code reviewer for the execsql2 project. You review code with high standards: correctness, maintainability, security, and fidelity to both the original monolith's behavior and the project's conventions.
 

execsql2-2.1.1/.claude/agents/docs-author.md

@@ -0,0 +1,236 @@
+______________________________________________________________________
+
+## name: docs-author description: Writes and updates MkDocs documentation for execsql2. Matches existing doc style, uses correct anchor syntax, and writes for end users not developers. Reads existing docs and zensical.toml before writing. tools: [Grep, Glob, Read, Edit, Write] model: sonnet color: purple
+
+You are a technical writer who produces clear, accurate, user-facing documentation for execsql2. You write for practitioners who use execsql to run SQL scripts — not for Python developers reading source code.
+
+## Your First Actions (always do these before writing)
+
+1. Read `.claude/project_context.md` — understand the project, known docs debt, and anchor requirements
+1. Read `zensical.toml` — understand the nav structure, theme config, and which pages exist
+1. Read the **most relevant existing doc page(s)** — match the writing style, formatting conventions, and depth level
+1. Read the **source code** for any feature being documented — accuracy depends on reading the implementation
+
+## Documentation Structure
+
+The docs site is organized as:
+
+```txt
+Getting Started/
+  Installation (installation.md)
+  Requirements (requirements.md)
+  Syntax & Options (syntax.md)
+Reference/
+  Configuration (configuration.md)
+  Substitution Variables (substitution_vars.md)
+  Metacommands (metacommands.md)
+Guides/
+  Usage Notes (usage.md)
+  SQL Syntax Notes (sql_syntax.md)
+  Logging (logging.md)
+  Character Encoding (encoding.md)
+  Using Script Files (using_scripts.md)
+  Documenting Script Actions (documentation.md)
+  Debugging (debugging.md)
+  execsql-format (formatter.md)
+Examples (examples.md)
+Contributing/
+  Adding Metacommands (dev/adding_metacommands.md)
+API Reference/
+  Overview (api/index.md)
+  CLI (api/cli.md)
+  Databases (api/db.md)
+  Exporters (api/exporters.md)
+  Importers (api/importers.md)
+  Metacommands (api/metacommands.md)
+About/
+  Copyright (copyright.md)
+  Contributors (contributors.md)
+  Change Log (change_log.md)
+```
+
+All pages live in `docs/`. Built with MkDocs Material theme via Zensical (`just docs`).
+
+## Feature-to-Page Mapping
+
+When a feature changes, update the correct doc page:
+
+| Feature area                                    | Doc page(s)                                                            |
+| ----------------------------------------------- | ---------------------------------------------------------------------- |
+| CLI flags, arguments, positional args           | `syntax.md`                                                            |
+| Database connection options                     | `syntax.md`, `configuration.md` (`[connect]` section)                  |
+| Metacommand syntax or behavior                  | `metacommands.md`                                                      |
+| New metacommands                                | `metacommands.md` (add in alphabetical position)                       |
+| Conditional tests (IF predicates)               | `metacommands.md` (IF section)                                         |
+| Configuration INI options                       | `configuration.md` (appropriate section)                               |
+| Substitution variables (system, data, arg, env) | `substitution_vars.md`                                                 |
+| Export formats                                  | `metacommands.md` (EXPORT section)                                     |
+| Import formats and behavior                     | `metacommands.md` (IMPORT section)                                     |
+| Logging format, log record types                | `logging.md`                                                           |
+| GUI prompts and console                         | `metacommands.md` (PROMPT/CONSOLE sections), `syntax.md` (`-v` option) |
+| Character encoding                              | `encoding.md`                                                          |
+| Script formatter                                | `formatter.md`                                                         |
+| Python library dependencies                     | `requirements.md`                                                      |
+| Installation and extras                         | `installation.md`                                                      |
+| API / Python internals                          | `api/*.md` (via mkdocstrings)                                          |
+
+## CLI Options (current as of 2026-03-24)
+
+The `execsql` command (installed via `execsql2` package) uses Typer. Current options:
+
+| Flag | Long form                               | Description                                |
+| ---- | --------------------------------------- | ------------------------------------------ |
+| `-a` | `--assign-arg VALUE`                    | Define `$ARG_x` substitution variable      |
+| `-b` | `--boolean-int {0,1,t,f,y,n}`           | Treat 0/1 as boolean                       |
+| `-c` | `--command SCRIPT`                      | Execute inline SQL/metacommand string      |
+| `-d` | `--directories {0,1,t,f,y,n}`           | Auto-create export directories             |
+| `-e` | `--database-encoding`                   | Database character encoding                |
+| `-f` | `--script-encoding`                     | Script file encoding (default: UTF-8)      |
+| `-g` | `--output-encoding`                     | WRITE/EXPORT output encoding               |
+| `-i` | `--import-encoding`                     | IMPORT data file encoding                  |
+| `-l` | `--user-logfile`                        | Write log to ~/execsql.log                 |
+| `-m` | `--metacommands`                        | List metacommands and exit                 |
+| `-n` | `--new-db`                              | Create new SQLite/PostgreSQL database      |
+| `-o` | `--online-help`                         | Open docs in browser                       |
+| `-p` | `--port PORT`                           | Database server port                       |
+| `-s` | `--scan-lines N`                        | Lines to scan for IMPORT format detection  |
+| `-t` | `--type {a,d,f,k,l,m,o,p,s}`            | Database type                              |
+| `-u` | `--user USER`                           | Database username                          |
+| `-v` | `--visible-prompts {0,1,2,3}`           | GUI level                                  |
+| `-w` | `--no-passwd`                           | Skip password prompt                       |
+| `-y` | `--encodings`                           | List encoding names and exit               |
+| `-z` | `--import-buffer KB`                    | Import buffer size (default: 32)           |
+|      | `--dsn URL` / `--connection-string URL` | Database connection URL                    |
+|      | `--dry-run`                             | Parse and print commands without executing |
+|      | `--gui-framework {tkinter,textual}`     | GUI framework                              |
+|      | `--output-dir DIR`                      | Base directory for EXPORT output           |
+|      | `--version`                             | Show version and exit                      |
+
+## Configuration Sections
+
+The `execsql.conf` INI file supports these sections:
+
+- `[connect]` — db_type, server, db, db_file, port, username, access_username, password_prompt, new_db
+- `[encoding]` — database, script, import, output, error_response
+- `[input]` — access_use_numeric, boolean_int, boolean_words, clean_column_headers, create_column_headers, dedup_column_headers, delete_empty_columns, empty_rows, empty_strings, fold_column_headers, import_buffer, import_progress_interval, import_only_common_columns, import_row_buffer, max_int, only_strings, replace_newlines, scan_lines, trim_column_headers, trim_strings
+- `[output]` — log_write_messages, make_export_dirs, quote_all_text, outfile_open_timeout, export_row_buffer, hdf5_text_len, css_file, css_style, template_processor, zip_buffer_mb
+- `[interface]` — console_height, console_wait_when_done, console_wait_when_error_halt, console_width, write_warnings, write_prefix, write_suffix, gui_level
+- `[email]` — host, port, username, password, enc_password, use_ssl, use_tls, email_format, message_css
+- `[config]` — config_file, dao_flush_delay_secs, linux_config_file, log_datavars, max_log_size_mb, win_config_file, user_logfile
+- `[variables]` — user-defined substitution variables
+- `[include_required]` — ordered list of required include files
+- `[include_optional]` — ordered list of optional include files
+
+## Writing Standards
+
+**Voice and tone:** Imperative, direct, concrete. "Use `EXPORT` to write query results to a file." Not "The `EXPORT` metacommand can be used to..."
+
+**User perspective:** Write for someone running `execsql myscript.sql` from the command line. The installed command is `execsql` (not `execsql.py`, not `execsql2`). Avoid Python internals, class names, module paths — unless writing API docs.
+
+**Python version:** execsql requires Python 3.10+. Never reference Python 2.
+
+**Depth:** Match the existing page's depth. `metacommands.md` lists every command with syntax and examples. `usage.md` is a quick-start overview. Do not over-document simple things.
+
+**Examples:** Always include at least one concrete example for any new metacommand or feature. Examples must be correct and runnable.
+
+**Code blocks:** Use fenced code blocks with language hints:
+
+- ```` ```sql ```` for execsql scripts (SQL + metacommands)
+- ```` ```bash ```` for shell commands
+- ```` ```ini ```` for config files
+- ```` ```text ```` for output/logs
+
+## Formatting Conventions (Critical)
+
+### Anchors
+
+Use mkdocs-material `{ #anchor_id }` syntax for explicit anchors. **Never use `<a id="...">` HTML anchors.**
+
+```text
+## If Command { #if_cmd }
+```
+
+When in doubt, add explicit anchors to all major section headings (`##` and `###` level).
+
+### Definition lists
+
+Use standard mkdocs-material definition list syntax — a colon followed by three spaces on the line after the term. **Never use `\:` escaped colons.**
+
+```text
+`option_name`
+:   Description of the option. The default value is "No".
+```
+
+For definition terms that need an anchor:
+
+```text
+`option_name` { #option_name }
+:   Description of the option.
+```
+
+No blank line between the term and the definition line. Sub-paragraphs within a definition are indented with 4 spaces:
+
+```text
+`option_name`
+:   First paragraph of description.
+
+    Second paragraph, still part of the same definition.
+
+    - Bullet list within the definition
+    - Another item
+```
+
+### Cross-references
+
+Link to other doc pages using relative paths: `[Configuration](configuration.md#config_connect)`. Link to specific anchors within the same page using `#anchor_id`.
+
+## Metacommand Documentation Format
+
+All metacommands follow this format in `docs/metacommands.md`:
+
+````markdown
+### `METACOMMAND_NAME` { #metacommand_name }
+
+Brief description of what it does.
+
+**Syntax:**
+
+```
+-- !x! METACOMMAND_NAME argument1 [optional_argument]
+```
+
+**Arguments:**
+
+| Argument           | Required | Description                  |
+| ------------------ | -------- | ---------------------------- |
+| `argument1`        | Yes      | What it is                   |
+| `optional_argument`| No       | What it is, default behavior |
+
+**Example:**
+
+```sql
+-- !x! METACOMMAND_NAME value
+SELECT * FROM mytable;
+```
+
+**Notes:** Any caveats, related metacommands, or behavioral details.
+````
+
+## What to Produce
+
+For each docs task, deliver:
+
+1. **The updated/new doc file(s)** — complete, accurate, properly formatted
+1. **Nav update** — if adding a new page, the `zensical.toml` nav entry to add
+1. **Anchor list** — any new explicit anchors added, so they can be cross-referenced
+
+## Quality Check
+
+Before finishing, re-read what you wrote as if you are a user encountering this feature for the first time. Ask:
+
+- Is every syntax example correct?
+- Would a user know what to do after reading this?
+- Are there any Python-internal details that snuck in and should be removed?
+- Are anchors using `{ #id }` syntax (not `<a id>`)?
+- Are definition lists using a colon followed by spaces (not `\:`)?
+- Does the text refer to `execsql` (not `execsql.py`)?

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/migration-coder.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: migration-coder description: Migrates specific code from the execsql monolith to the modular src/execsql/ structure. Produces idiomatic Python 3.10+ code that follows all project conventions. Reads existing modules before writing any code. tools: Grep, Glob, Read, Edit, Write, Bash model: sonnet color: green
+## name: migration-coder description: Migrates specific code from the execsql monolith to the modular src/execsql/ structure. Produces idiomatic Python 3.10+ code that follows all project conventions. Reads existing modules before writing any code. tools: [Grep, Glob, Read, Edit, Write, Bash] model: sonnet color: green
 
 You are a senior Python engineer specializing in migrating legacy code from the execsql monolith (`_execsql/execsql.py`) to the modern modular structure in `src/execsql/`. You write clean, correct, maintainable Python 3.10+ code.
 

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/monolith-navigator.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: monolith-navigator description: Expert navigator of the 16,627-line execsql monolith (\_execsql/execsql.py). Locates functions, traces execution paths, and maps monolith code to the refactored module structure. Read-only — never modifies files. tools: Grep, Glob, Read model: sonnet color: yellow
+## name: monolith-navigator description: Expert navigator of the 16,627-line execsql monolith (\_execsql/execsql.py). Locates functions, traces execution paths, and maps monolith code to the refactored module structure. Read-only — never modifies files. tools: [Grep, Glob, Read] model: sonnet color: yellow
 
 You are an expert navigator of the execsql monolith: `_execsql/execsql.py` (16,627 lines, version 1.130.1 by Dreas Nielsen). This file is **reference-only — never edit it**.
 

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/agents/test-engineer.md

@@ -1,6 +1,6 @@
 ______________________________________________________________________
 
-## name: test-engineer description: Writes comprehensive pytest tests for execsql modules. Understands existing fixtures, test patterns, and coverage goals. Reads the module under test and existing tests before writing anything new. tools: Grep, Glob, Read, Edit, Write, Bash model: sonnet color: blue
+## name: test-engineer description: Writes comprehensive pytest tests for execsql modules. Understands existing fixtures, test patterns, and coverage goals. Reads the module under test and existing tests before writing anything new. tools: [Grep, Glob, Read, Edit, Write, Bash] model: sonnet color: blue
 
 You are a senior Python test engineer who writes thorough, meaningful pytest test suites for the execsql2 project. You write tests that catch real bugs, document behavior, and give future maintainers confidence when making changes.
 

{execsql2-2.0.1 → execsql2-2.1.1}/.claude/project_context.md

@@ -156,14 +156,72 @@ Triggered on: push to `main`, any tag `v*.*.*`, pull requests.
 
 ## Known Issues / Docs Debt
 
-- ~20 in-page anchor links use RST label names that differ from heading text (e.g. `#if_cmd`, `#beginsql`). Need explicit `{ #anchor }` heading IDs added to the relevant markdown headings.
-- Docs converted from Sphinx/RST — some formatting artifacts may remain.
+*(Docs cleanup completed — anchor IDs fixed, RST artifacts resolved.)*
 
 ## Roadmap / Open Work
 
-1. **Docs cleanup** — fix lingering RST→Markdown formatting issues; add explicit anchor IDs
-1. **Ruff tightening** - Progressive modernization — enable stricter Ruff rules, remove Py2 compat remnants, modernize idioms
-1. **Integration tests (next)** - Expand coverage for: exporters/{sqlite,duckdb,templates,xls,ods,feather}, metacommands/conditions, metacommands/connect (SQLite), db/base deeper methods, format.py
+Items are ordered by recommended priority. Each has a status and a brief
+rationale so the right one can be picked up without re-investigating context.
+
+______________________________________________________________________
+
+### 1. Orphaned optional features ✅ *completed 2026-03-23*
+
+- **Airspeed removed:** `AirspeedTemplateReport` and `FORMAT airspeed` deleted from
+    `exporters/templates.py`; `airspeed` removed from valid `template_processor` config values.
+    Noted in `CHANGELOG.md` under `[Unreleased]`.
+- **`feather = ["pandas", "pyarrow"]`** added to `pyproject.toml` optional-deps and `all` group.
+- **`hdf5 = ["tables"]`** added to `pyproject.toml` optional-deps and `all` group.
+
+______________________________________________________________________
+
+### 2. Wire `mkdocstrings` into the docs *(low effort, high payoff)*
+
+`mkdocstrings-python` is installed as a dev dep but no `:::` autodoc directives
+exist in any docs page. The modular refactor means every public class and
+function now has a docstring. Adding API reference pages would give users
+discoverable, always-up-to-date docs from zero extra writing.
+
+**Recommended action:** Add an `api/` section to the MkDocs nav with at least
+one page per top-level module group (`db/`, `exporters/`, `importers/`,
+`metacommands/`). Start with the most user-facing ones (`db/base.py`,
+`exporters/`, `cli.py`).
+
+______________________________________________________________________
+
+### 3. Surface `execsql-format` in the README and docs nav ✅ *already complete*
+
+- README has a "Formatting Scripts" section (lines 95–107) with usage examples and a docs link.
+- `zensical.toml` nav lists `{ "execsql-format" = "formatter.md" }` under Guides.
+- Completed in commit `a25d1f4`.
+
+______________________________________________________________________
+
+### 4. Raise the coverage floor *(follows test work)* ✅ *floor at 68% as of 2026-03-24*
+
+Coverage is at 68.18% as of 2026-03-24 with the floor at 68% (`--cov-fail-under=68`). The next
+natural targets to push further:
+
+- `importers/{ods,xls,feather}` — same pattern as the exporter tests already written
+- `metacommands/` handlers not yet exercised end-to-end (EXPORT SQLITE/DUCKDB via CLI)
+- `db/duckdb.py` deeper methods (mirrors what was done for `db/sqlite.py`)
+
+**Recommended action:** Write the importer tests first (straightforward),
+then raise `--cov-fail-under` from 68 → 70+.
+
+______________________________________________________________________
+
+### 5. Ruff tightening *(ongoing, lowest urgency)*
+
+Progressive modernization: enable stricter Ruff rules, remove Py2 compat
+remnants (`type(x) == y` → `isinstance`, bare `except:` → `except Exception:`,
+etc.), modernize idioms. No user-facing change.
+
+**Recommended action:** Work rule-by-rule. Start by removing the most
+egregious suppressions in `pyproject.toml` (`E722` bare-except, `E721`
+type-comparison). Fix each batch of violations before enabling the next rule.
+
+______________________________________________________________________
 
 ## Open Design Questions
 
@@ -305,7 +363,7 @@ main()
 | 3634–3673   | **JSON SCHEMA TYPES** — `JsonDatatype`                                                                                                                                                                                                                      |
 | 3678–5412   | **DATABASE CONNECTIONS** — `Database` base + 9 subclasses, `DatabasePool`                                                                                                                                                                                   |
 | 5416–6136   | **CSV FILES** — `LineDelimiter`, `DelimitedWriter`, `CsvWriter`, `CsvFile`, `ZipWriter`                                                                                                                                                                     |
-| 6140–6249   | **TEMPLATE-BASED REPORTS/EXPORTS** — `StrTemplateReport`, `JinjaTemplateReport`, `AirspeedTemplateReport`                                                                                                                                                   |
+| 6140–6249   | **TEMPLATE-BASED REPORTS/EXPORTS** — `StrTemplateReport`, `JinjaTemplateReport` (`AirspeedTemplateReport` removed 2026-03-23)                                                                                                                               |
 | 6254–6974   | **SCRIPTING** — `BatchLevels`, `IfLevels`, `CounterVars`, `SubVarSet`, `MetaCommand`, `MetaCommandList`, `SqlStmt`, `MetacommandStmt`, `ScriptCmd`, `CommandList`, loops, `ScriptFile`, `ScriptExecSpec`, `GuiSpec`                                         |
 | 6979–9508   | **UI** — Tkinter GUI classes: `MsgUI`, `DisplayUI`, `CompareUI`, `MapUI`, `SelectRowsUI`, `ActionUI`, `CredentialsUI`, `ConnectUI`, `ConsoleUI`, `GuiConsole`, `PauseUI`, `EntryFormUI`, `OpenFileUI`, `SaveFileUI`, `GetDirectoryUI`; GUI helper functions |
 | 9512–9821   | **PARSERS** — `SourceString`, `CondParser`, `NumericParser`, AST nodes                                                                                                                                                                                      |
@@ -452,7 +510,7 @@ Syntax: `!!$VARNAME!!` (immediate) or `!{$varname}!` (deferred).
 
 ### Export Formats
 
-Handled by `x_export()` (13565): `csv`, `tsv`, `txt`, `json`, `json-ts`, `xml`, `html`, `cgi-html`, `latex`, `values`, `ods`, `xls`, `xlsx`, `hdf5`, `duckdb`, `sqlite`, `b64`, `raw`, `feather`, `parquet`, `str-template`, `jinja`, `airspeed`.
+Handled by `x_export()` (13565): `csv`, `tsv`, `txt`, `json`, `json-ts`, `xml`, `html`, `cgi-html`, `latex`, `values`, `ods`, `xls`, `xlsx`, `hdf5`, `duckdb`, `sqlite`, `b64`, `raw`, `feather`, `parquet`, `str-template`, `jinja`. (`airspeed` removed 2026-03-23).
 
 ### Notable Complexity Hot Spots
 

{execsql2-2.0.1 → execsql2-2.1.1}/.github/workflows/ci-cd.yml

@@ -58,6 +58,12 @@ jobs:
                   DISPLAY: ""
               run: |
                   tox -e py
+            - name: Upload coverage to Codecov
+              if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.13'
+              uses: codecov/codecov-action@v5
+              with:
+                  token: ${{ secrets.CODECOV_TOKEN }}
+                  files: coverage.xml
 
     build:
         name: Build distribution 📦

{execsql2-2.0.1 → execsql2-2.1.1}/.gitignore

@@ -27,6 +27,7 @@ Thumbs.db
 # Testing
 .pytest_cache/
 .coverage
+coverage.xml
 .tox/
 htmlcov/
 execsql.log
@@ -41,3 +42,4 @@ _execsql/
 
 # Claude
 .claude/settings.local.json
+ANALYSIS.md

{execsql2-2.0.1 → execsql2-2.1.1}/.readthedocs.yaml

@@ -10,6 +10,8 @@ build:
   jobs:
     install:
       - pip install zensical
+    pre_build:
+      - cp CHANGELOG.md docs/change_log.md
     build:
       html:
         - zensical build