dbctl 0.7.3__tar.gz → 0.7.7__tar.gz

{dbctl-0.7.3 → dbctl-0.7.7}/CHANGELOG.md

@@ -5,6 +5,131 @@ All notable changes to this project will be documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.7.7] — 2026-08-10
+
+### Fixed
+
+- **`ruff format` failed in CI** on the v0.7.6 commit — three files
+  (`dbctl/cli.py`, `dbctl/runtime.py`, `tests/test_execute.py`) had
+  manually-typed signatures that fit on one line per `ruff format`'s
+  rules but were wrapped in the source. Reformatted to the canonical
+  layout (`def opened_engine(ctx: click.Context, canonical: str, conn:
+  Connection) -> Iterator[OpenedStub]:` on one line; same for the
+  `execute_cmd` callback signature and the inline-URL error `print`
+  call). Re-running `uv run ruff format --check dbctl tests` is now
+  clean and `make format` is a no-op on top of this commit.
+- **Typo in `docs/execute.md`** — `# dduckdb inline URL` comment →
+  `# duckdb inline URL`.
+- **README project layout** now lists `execute.md` under `docs/` so the
+  new reference page is discoverable from the layout block.
+
+No behaviour change; re-running `dbctl execute --help` produces the
+same output as v0.7.6.
+
+## [0.7.6] — 2026-08-10
+
+### Added
+
+- **`dbctl execute`** — run ad-hoc SQL without declaring an operation in
+  `operations.yaml`. The new top-level verb takes a connection name *or* a
+  full SQLAlchemy URL, the SQL string as a single positional argument, and
+  the short option set the existing operation dispatch uses:
+
+  ```bash
+  dbctl execute -c pg -o json "SELECT * FROM users"
+  dbctl execute -c pg --show-sql --apply "DELETE FROM users WHERE name='bob'"
+  dbctl execute -c "postgresql+psycopg://u:p@host:5432/db" -o csv "SELECT 1"
+  dbctl execute -c pg -o yaml "SELECT * FROM users LIMIT 10"
+  ```
+
+  Behaviour:
+
+  - **`-c` / `--connection`** accepts a registered connection name /
+    alias (resolved via `connections.yaml`) **or** a full SQLAlchemy URL
+    (detected by the presence of `://`). Inline URLs build a transient
+    `direct` Connection so the SSM/SSH password-source plumbing, native-lib
+    install hints, and `connect_args` (connect-timeout, Windows SSO) all
+    apply uniformly.
+  - **`-o` / `--output`** chooses `table` (default) / `json` / `csv` /
+    `yaml` for SELECT-shaped results, exactly like declared `fetch` ops.
+  - The SQL is auto-classified by its first verb: `SELECT` / `WITH` /
+    `SHOW` / `EXPLAIN` / `DESCRIBE` / `PRAGMA` / `VALUES` / `TABLE` runs
+    as a query (rendered via `--output`); anything else runs as DML
+    (INSERT / UPDATE / DELETE / CREATE / DROP / ALTER / ...) inside
+    `engine.begin()`.
+  - DML respects the connection's `safety.confirm` (dry-run-by-default
+    unless `--apply`), `safety.read_only` (DML blocked with exit 6), and
+    prompts before commit unless `--yes` / `-y` is passed — exactly the
+    same gate as a declared `mode: execute` operation. Inline URLs
+    default to `safety.confirm: true`.
+  - DDL that returns `rowcount = -1` (CREATE TABLE, DROP, ...) is rendered
+    as `OK in <ms>ms` instead of the misleading `OK -1 row(s) affected`.
+    Real INSERT/UPDATE/DELETE row counts are still reported.
+  - Every run is appended to `~/.dbctl/history.jsonl` as
+    `operation="execute"` with the SQL (truncated to 500 chars) in the
+    `params.sql` field, so it shows up under `dbctl history list`. The
+    audit `connection` field is the canonical connection name, or the
+    literal `<inline>` token for inline URL runs.
+  - `--show-sql` prints the resolved SQL before executing; `--` separates
+    options from positional SQL that begins with a dash (Click would
+    otherwise treat the leading dash as an unknown option).
+  - The new verb is registered as a top-level static command alongside
+    `connections` / `operations` / `doctor` / `init` / `history` /
+    `tunnel` / `ui`, so shell completion picks it up.
+
+  This closes the long-standing "no ad-hoc query command" gap without
+  weakening the declarative operation registry: the declared
+  `operations.yaml` model remains the recommended way to make "what can
+  be run against this DB" discoverable from a versioned file, while
+  `execute` covers the exploratory / break-glass path the TUI already
+  serves — both share the same tunnel / safety / audit plumbing so every
+  run is still logged and policy-checked.
+
+## [0.7.4] — 2026-08-10
+
+### Added
+
+- **`dbctl ui`** — an interactive [Textual](https://textual.textualize.io/)
+  TUI on top of the same connections/operations/tunnels/audit-log stack as
+  the CLI (`textual` is a base dependency, no separate install step). See
+  `docs/tui.md` for the full reference.
+  - **Connection tree** doubling as a lazily-loaded schema browser: `m`
+    toggles a simple `connection → table → column` view and a fuller
+    `connection → schema → Tables/Views → table/view → Columns/Indexes`
+    view; `g` toggles a flat list and a grouping that treats `-` as a path
+    separator (`in-gateway-ifp-dev` nests under `in-gateway` → `ifp`), with
+    single-child folders compressed and support for a name that's both a
+    group and a real connection at once (`ifp` / `ifp-gateway`). `c`/`d`/`t`
+    connect/disconnect/test-tunnel; `a`/`e` open `connections.yaml` in
+    `$EDITOR`.
+  - **Tabbed workspace**: a SQL editor (syntax-highlighted) or an
+    operation-launcher (a form built from a declared operation's
+    parameters) per tab, each with an independently resizable results
+    table and a status line (row count / rows-affected + duration) below
+    it. `Ctrl+N` new tab, `Ctrl+O` searchable operation launcher (type to
+    filter), `Ctrl+W` close, `Ctrl+R` run. Activating a table/view in the
+    tree opens a SQL tab pre-filled with a dialect-correct preview query
+    (`LIMIT` / `TOP` / `FETCH FIRST … ROWS ONLY`, with per-dialect
+    identifier quoting for mixed-case table/schema names).
+  - Connect/disconnect/test-tunnel and every SQL/operation run happen on a
+    background thread with a loading indicator, so the UI stays responsive
+    instead of freezing while a tunnel spins up or a query runs.
+  - Every run still respects `safety.read_only` / `safety.confirm` /
+    `allowed_operations` and is appended to `~/.dbctl/history.jsonl`,
+    exactly like a CLI-driven run.
+
+### Fixed
+
+- **`uv.lock` could resolve against a contributor's local package-index
+  override instead of public PyPI** — a machine-level `uv` config pointing
+  at an internal package mirror silently became the index every regenerated
+  `uv.lock` entry was resolved and hashed against, which both leaked an
+  internal hostname into the lock file and broke `uv sync --frozen` for
+  anyone (CI included) without access to that mirror. Pinned
+  `[[tool.uv.index]]` in `pyproject.toml` with `default = true` so this
+  project always locks against `pypi.org`/`files.pythonhosted.org`
+  regardless of what index a contributor's environment defaults to.
+
 ## [0.7.3] — 2026-08-04
 
 ### Added

{dbctl-0.7.3 → dbctl-0.7.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dbctl
-Version: 0.7.3
+Version: 0.7.7
 Summary: Generic CLI to monitor, control, and administer multiple databases via SSM, SSH, or direct connection.
 Author: dbctl contributors
 License: MIT
@@ -25,8 +25,10 @@ Requires-Dist: pyodbc>=5
 Requires-Dist: pyyaml>=6
 Requires-Dist: rich>=13
 Requires-Dist: sqlalchemy>=2
+Requires-Dist: textual>=0.60
 Provides-Extra: dev
 Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
 Requires-Dist: pytest-cov>=5; extra == 'dev'
 Requires-Dist: pytest>=8; extra == 'dev'
 Requires-Dist: ruff>=0.5; extra == 'dev'
@@ -71,11 +73,13 @@ leaving the terminal.
 | DML is one `Ctrl-Enter` away                        | DML is **dry-run by default** until `--apply`; `--yes` skips the prompt |
 | No cross-DB diff                                     | `dbctl diff user-count prod-tenant1 int-tenant1`        |
 
-`dbctl` is **not** a schema browser or a query playground — it deliberately
-has no ad-hoc query command. Declaring operations in YAML keeps "what can be
-run against this DB" discoverable from a versioned file instead of buried in
-your shell history. (For ad-hoc exploration open the tunnel with
-`dbctl tunnel open <conn>` and point your favourite client at the local bind.)
+The declarative CLI above is still the primary interface: declaring
+operations in YAML keeps "what can be run against this DB" discoverable from
+a versioned file instead of buried in your shell history. For ad-hoc
+exploration, `dbctl ui` (see below) gives you a schema browser and a SQL
+editor without leaving `dbctl`'s tunnel/safety/audit model - or open the
+tunnel with `dbctl tunnel open <conn>` and point your favourite client at
+the local bind.
 
 ### How it reaches a database
 
@@ -118,9 +122,9 @@ the same params.
 
 ## Why "the operations are declarative"
 
-There is **no ad-hoc query command**. To run SQL through `dbctl` you must
-declare an operation in `operations.yaml` first — its name, its parameters
-(with types, descriptions, positional vs keyword), its SQL, and its mode
+The recommended path for permanent, repeatable work is to declare an
+operation in `operations.yaml` first — its name, its parameters (with
+types, descriptions, positional vs keyword), its SQL, and its mode
 (`execute` / `fetch` / `fetch_one` / `script` / `upsert` for single-DB;
 `diff` / `compare` / `copy` / `sync` / `validate` / `replay` for multi-DB).
 The CLI then synthesises a Click subcommand per operation, so:
@@ -131,7 +135,18 @@ The CLI then synthesises a Click subcommand per operation, so:
 - the audit log is queryable by operation name (`dbctl history list`).
 
 This keeps "what can be run against this DB" discoverable from a versioned
-config file, instead of buried in shell history.
+config file, instead of buried in shell history. For exploratory / break-glass
+SQL — the path the TUI serves — `dbctl execute` (added in v0.7.6) runs
+ad-hoc SQL through the same tunnel / safety / audit plumbing without
+requiring a declared operation:
+
+```bash
+dbctl execute -c pg -o json "SELECT * FROM users"
+dbctl execute -c "postgresql+psycopg://u:p@host:5432/db" -o csv "SELECT 1"
+dbctl execute -c pg --show-sql --apply "DELETE FROM users WHERE name='bob'"
+```
+
+See [`docs/execute.md`](docs/execute.md) for the full reference.
 
 ## Install (uv)
 
@@ -184,6 +199,12 @@ dbctl pg add-user stephen 12 --show-sql        # dry-run (prints SQL)
 dbctl pg add-user stephen 12 --apply --yes     # commit (no prompt)
 dbctl pg increase-credits alice 10 --apply -y    # +10% on alice's credits
 
+# ad-hoc SQL without declaring an operation (since v0.7.6):
+dbctl execute -c pg -o json "SELECT * FROM users"
+dbctl execute -c "postgresql+psycopg://u:p@host:5432/db" -o csv "SELECT 1"
+dbctl execute -c pg --show-sql --apply "DELETE FROM users WHERE name='bob'"
+dbctl execute -c pg -o yaml -- "SELECT * FROM users WHERE name='-alice'"  # SQL starting with a dash needs `--`
+
 # multi-DB modes (operation-first, preferred since v0.6):
 dbctl user-count pg my                        # multi-DB diff
 dbctl compare-credits pg my Daily
@@ -352,6 +373,41 @@ reference.
 Secret-typed **operation** parameters (`type: secret`) are redacted in the
 audit log regardless of which DB password source the connection uses.
 
+## Interactive UI (`dbctl ui`)
+
+A [Textual](https://textual.textualize.io/) TUI for interactive work, on top
+of the same connections/operations/tunnels/audit-log stack as the CLI - not
+a separate code path with its own rules. `textual` is a base dependency, so
+no separate install step.
+
+```bash
+dbctl ui
+```
+
+- **Left pane**: a connection tree that doubles as a lazily-loaded schema
+  browser (`m` toggles a simple `connection → table → column` view and a
+  fuller `connection → schema → Tables/Views → table/view →
+  Columns/Indexes` view). Expanding a connection connects it if needed;
+  activating (Enter) a table/view opens a SQL tab pre-filled with a
+  dialect-correct preview query using the real table name. `g` toggles a
+  grouped view that treats `-` as a path separator (`in-gateway-ifp-dev`
+  nests under `in-gateway` → `ifp`) - handy once a fleet has more than a
+  handful of connections.
+- **Right pane**: a tabbed workspace of SQL editor / operation-launcher
+  tabs (`Ctrl+N` new tab, `Ctrl+O` searchable operation launcher, `Ctrl+W`
+  close, `Ctrl+R` run), each with an independently resizable results table
+  below it and a status line (row count / rows affected + duration) after
+  each run.
+- Connecting, disconnecting, testing a tunnel, and every SQL/operation run
+  happen on a background thread with a loading indicator, so the UI stays
+  responsive instead of freezing while a tunnel spins up or a query runs.
+- Every run still respects `safety.read_only` / `safety.confirm` /
+  `allowed_operations` and is appended to `~/.dbctl/history.jsonl`, exactly
+  like a CLI-driven run.
+
+See [`docs/tui.md`](docs/tui.md) for the full keybinding reference and
+per-dialect SQL details (row-limit clause, identifier quoting).
+
 ## Shell completion
 
 ```bash
@@ -375,6 +431,8 @@ dbctl/
 ├── docs/
 │   ├── connections.md      # connections.yaml reference
 │   ├── operations.md       # operations.yaml reference
+│   ├── execute.md          # dbctl execute (ad-hoc SQL) reference
+│   ├── tui.md              # dbctl ui reference (keybindings, dialect SQL)
 │   └── DESIGN.md           # architecture and design decisions
 └── dbctl/
     ├── cli.py              # dynamic groups + per-op Click commands
@@ -388,7 +446,21 @@ dbctl/
     ├── reports.py          # rich tables / json / csv / yaml + diff + copy/sync/validate rendering
     ├── audit.py            # history.jsonl
     ├── runtime.py          # opened_conn() ctx-mgr (tunnel+engine+healthcheck)
-    └── init.py             # dbctl init wizard
+    ├── init.py             # dbctl init wizard
+    └── ui/                 # Textual TUI (`dbctl ui`)
+        ├── app.py          # DbctlApp: connection tree + tabbed workspace
+        ├── session.py      # per-connection tunnel+engine kept open across tab-runs
+        ├── connection_tree.py  # lazy schema browser (simple/normal + flat/grouped views)
+        ├── grouping.py     # "-" as a path separator -> compressed connection-name trie
+        ├── schema.py       # sqlalchemy.inspect() wrapper for the tree
+        ├── editor_tab.py   # SQL editor tab
+        ├── operation_tab.py    # operation-launcher tab
+        ├── sql_templates.py    # dialect-aware default SELECT + identifier quoting
+        ├── screens.py      # confirm / new-tab / operation-launcher modal screens
+        ├── splitter.py     # draggable pane-resize bars
+        ├── tabs.py         # shared tab base: resize, toolbar, status bar, loading indicator
+        ├── results.py      # DataTable rendering helpers
+        └── registry.py     # tolerant connections/operations loading
 ```
 
 ## Development
@@ -397,7 +469,7 @@ dbctl/
 uv sync --extra dev
 make help            # list all Makefile targets
 make check           # lint + unit tests (the pre-commit gate)
-make test            # unit tests (~80 tests, in-memory SQLite, no docker)
+make test            # unit tests (~270 tests, sqlite-backed, no docker)
 make typecheck       # mypy strict (pre-existing debt; non-blocking)
 make smoke           # docker compose up + dbctl doctor against the fleet
 make build           # wheel + sdist via uv

{dbctl-0.7.3 → dbctl-0.7.7}/README.md

@@ -37,11 +37,13 @@ leaving the terminal.
 | DML is one `Ctrl-Enter` away                        | DML is **dry-run by default** until `--apply`; `--yes` skips the prompt |
 | No cross-DB diff                                     | `dbctl diff user-count prod-tenant1 int-tenant1`        |
 
-`dbctl` is **not** a schema browser or a query playground — it deliberately
-has no ad-hoc query command. Declaring operations in YAML keeps "what can be
-run against this DB" discoverable from a versioned file instead of buried in
-your shell history. (For ad-hoc exploration open the tunnel with
-`dbctl tunnel open <conn>` and point your favourite client at the local bind.)
+The declarative CLI above is still the primary interface: declaring
+operations in YAML keeps "what can be run against this DB" discoverable from
+a versioned file instead of buried in your shell history. For ad-hoc
+exploration, `dbctl ui` (see below) gives you a schema browser and a SQL
+editor without leaving `dbctl`'s tunnel/safety/audit model - or open the
+tunnel with `dbctl tunnel open <conn>` and point your favourite client at
+the local bind.
 
 ### How it reaches a database
 
@@ -84,9 +86,9 @@ the same params.
 
 ## Why "the operations are declarative"
 
-There is **no ad-hoc query command**. To run SQL through `dbctl` you must
-declare an operation in `operations.yaml` first — its name, its parameters
-(with types, descriptions, positional vs keyword), its SQL, and its mode
+The recommended path for permanent, repeatable work is to declare an
+operation in `operations.yaml` first — its name, its parameters (with
+types, descriptions, positional vs keyword), its SQL, and its mode
 (`execute` / `fetch` / `fetch_one` / `script` / `upsert` for single-DB;
 `diff` / `compare` / `copy` / `sync` / `validate` / `replay` for multi-DB).
 The CLI then synthesises a Click subcommand per operation, so:
@@ -97,7 +99,18 @@ The CLI then synthesises a Click subcommand per operation, so:
 - the audit log is queryable by operation name (`dbctl history list`).
 
 This keeps "what can be run against this DB" discoverable from a versioned
-config file, instead of buried in shell history.
+config file, instead of buried in shell history. For exploratory / break-glass
+SQL — the path the TUI serves — `dbctl execute` (added in v0.7.6) runs
+ad-hoc SQL through the same tunnel / safety / audit plumbing without
+requiring a declared operation:
+
+```bash
+dbctl execute -c pg -o json "SELECT * FROM users"
+dbctl execute -c "postgresql+psycopg://u:p@host:5432/db" -o csv "SELECT 1"
+dbctl execute -c pg --show-sql --apply "DELETE FROM users WHERE name='bob'"
+```
+
+See [`docs/execute.md`](docs/execute.md) for the full reference.
 
 ## Install (uv)
 
@@ -150,6 +163,12 @@ dbctl pg add-user stephen 12 --show-sql        # dry-run (prints SQL)
 dbctl pg add-user stephen 12 --apply --yes     # commit (no prompt)
 dbctl pg increase-credits alice 10 --apply -y    # +10% on alice's credits
 
+# ad-hoc SQL without declaring an operation (since v0.7.6):
+dbctl execute -c pg -o json "SELECT * FROM users"
+dbctl execute -c "postgresql+psycopg://u:p@host:5432/db" -o csv "SELECT 1"
+dbctl execute -c pg --show-sql --apply "DELETE FROM users WHERE name='bob'"
+dbctl execute -c pg -o yaml -- "SELECT * FROM users WHERE name='-alice'"  # SQL starting with a dash needs `--`
+
 # multi-DB modes (operation-first, preferred since v0.6):
 dbctl user-count pg my                        # multi-DB diff
 dbctl compare-credits pg my Daily
@@ -318,6 +337,41 @@ reference.
 Secret-typed **operation** parameters (`type: secret`) are redacted in the
 audit log regardless of which DB password source the connection uses.
 
+## Interactive UI (`dbctl ui`)
+
+A [Textual](https://textual.textualize.io/) TUI for interactive work, on top
+of the same connections/operations/tunnels/audit-log stack as the CLI - not
+a separate code path with its own rules. `textual` is a base dependency, so
+no separate install step.
+
+```bash
+dbctl ui
+```
+
+- **Left pane**: a connection tree that doubles as a lazily-loaded schema
+  browser (`m` toggles a simple `connection → table → column` view and a
+  fuller `connection → schema → Tables/Views → table/view →
+  Columns/Indexes` view). Expanding a connection connects it if needed;
+  activating (Enter) a table/view opens a SQL tab pre-filled with a
+  dialect-correct preview query using the real table name. `g` toggles a
+  grouped view that treats `-` as a path separator (`in-gateway-ifp-dev`
+  nests under `in-gateway` → `ifp`) - handy once a fleet has more than a
+  handful of connections.
+- **Right pane**: a tabbed workspace of SQL editor / operation-launcher
+  tabs (`Ctrl+N` new tab, `Ctrl+O` searchable operation launcher, `Ctrl+W`
+  close, `Ctrl+R` run), each with an independently resizable results table
+  below it and a status line (row count / rows affected + duration) after
+  each run.
+- Connecting, disconnecting, testing a tunnel, and every SQL/operation run
+  happen on a background thread with a loading indicator, so the UI stays
+  responsive instead of freezing while a tunnel spins up or a query runs.
+- Every run still respects `safety.read_only` / `safety.confirm` /
+  `allowed_operations` and is appended to `~/.dbctl/history.jsonl`, exactly
+  like a CLI-driven run.
+
+See [`docs/tui.md`](docs/tui.md) for the full keybinding reference and
+per-dialect SQL details (row-limit clause, identifier quoting).
+
 ## Shell completion
 
 ```bash
@@ -341,6 +395,8 @@ dbctl/
 ├── docs/
 │   ├── connections.md      # connections.yaml reference
 │   ├── operations.md       # operations.yaml reference
+│   ├── execute.md          # dbctl execute (ad-hoc SQL) reference
+│   ├── tui.md              # dbctl ui reference (keybindings, dialect SQL)
 │   └── DESIGN.md           # architecture and design decisions
 └── dbctl/
     ├── cli.py              # dynamic groups + per-op Click commands
@@ -354,7 +410,21 @@ dbctl/
     ├── reports.py          # rich tables / json / csv / yaml + diff + copy/sync/validate rendering
     ├── audit.py            # history.jsonl
     ├── runtime.py          # opened_conn() ctx-mgr (tunnel+engine+healthcheck)
-    └── init.py             # dbctl init wizard
+    ├── init.py             # dbctl init wizard
+    └── ui/                 # Textual TUI (`dbctl ui`)
+        ├── app.py          # DbctlApp: connection tree + tabbed workspace
+        ├── session.py      # per-connection tunnel+engine kept open across tab-runs
+        ├── connection_tree.py  # lazy schema browser (simple/normal + flat/grouped views)
+        ├── grouping.py     # "-" as a path separator -> compressed connection-name trie
+        ├── schema.py       # sqlalchemy.inspect() wrapper for the tree
+        ├── editor_tab.py   # SQL editor tab
+        ├── operation_tab.py    # operation-launcher tab
+        ├── sql_templates.py    # dialect-aware default SELECT + identifier quoting
+        ├── screens.py      # confirm / new-tab / operation-launcher modal screens
+        ├── splitter.py     # draggable pane-resize bars
+        ├── tabs.py         # shared tab base: resize, toolbar, status bar, loading indicator
+        ├── results.py      # DataTable rendering helpers
+        └── registry.py     # tolerant connections/operations loading
 ```
 
 ## Development
@@ -363,7 +433,7 @@ dbctl/
 uv sync --extra dev
 make help            # list all Makefile targets
 make check           # lint + unit tests (the pre-commit gate)
-make test            # unit tests (~80 tests, in-memory SQLite, no docker)
+make test            # unit tests (~270 tests, sqlite-backed, no docker)
 make typecheck       # mypy strict (pre-existing debt; non-blocking)
 make smoke           # docker compose up + dbctl doctor against the fleet
 make build           # wheel + sdist via uv