dbctl 0.5.2__tar.gz → 0.5.3__tar.gz

{dbctl-0.5.2 → dbctl-0.5.3}/.dbctl/operations.yaml

@@ -21,6 +21,21 @@ operations:
             type         = EXCLUDED.type,
             updated_at   = NOW()
 
+  increase-quota:
+    description: "Increase a user's daily and yearly quota by a percentage"
+    scope: single
+    mode: execute
+    confirm: true
+    parameters:
+      - { name: name, type: string, required: true, position: 1, description: "User to bump" }
+      - { name: pct,  type: float,  required: true, position: 2, description: "Percentage increase (e.g. 10 = +10%)" }
+    sql: |
+      UPDATE users
+         SET quota_daily  = (quota_daily  * (1 + $pct / 100.0))::integer,
+             quota_yearly = (quota_yearly * (1 + $pct / 100.0))::integer,
+             updated_at   = NOW()
+       WHERE name = $name
+
   list-users:
     description: "List users (top N)"
     scope: single

{dbctl-0.5.2 → dbctl-0.5.3}/CHANGELOG.md

@@ -5,6 +5,46 @@ All notable changes to this project will be documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.5.3] — 2026-08-01
+
+### Fixed
+
+- **`$name::type` casts broke psycopg** (the `::type` suffix collides with
+  the bind-param parser). `to_bindparams` now rewrites the Postgres cast
+  idiom to the SQL-standard `CAST(:name AS type)` form, so the same
+  operation YAML works cross-dialect. Parenthesised precision
+  (`$x::numeric(10, 2)`) and schema-qualified types (`$x::pg.text`) are
+  preserved. The bundled `report-logs` operation (which uses
+  `$since::timestamp`) now runs instead of failing with
+  `syntax error at or near ":"`.
+- **`dbctl connections show <alias>`** reported "unknown connection" for a
+  registered alias (e.g. `postgres`) because `connections_show` did a direct
+  dict lookup instead of going through the alias-aware `resolve()`. Now
+  resolves aliases to the canonical connection and dumps it under the
+  canonical name.
+- **`dbctl diff <op> <conn> <unknown>`** crashed with a raw
+  `UnknownConnectionError` traceback. The multi-op callback now catches the
+  `KeyError` and emits a one-line message on stderr with exit code 2.
+- **Negative positional arguments** (e.g.
+  `dbctl pg increase-quota alice -10`) failed with the opaque Click error
+  `No such option '-1'`. Dynamic commands now append a hint pointing at the
+  `--` separator and showing the user's actual token (`-10`).
+
+### Added
+
+- **`increase-quota`** bundled operation — bumps a user's daily and yearly
+  quota by a percentage (positive or negative via `--`), with postgres
+  `::integer` rounding.
+- **README + docs** rewritten to reflect the tool's actual thesis — a
+  one-command replacement for the DBeaver multi-click flow when
+  administering a database explosion (multiple environments × tenants)
+  through SSO + SSH tunnels. Adds a "Why not DBeaver?" comparison, documents
+  the `password` / `password_env` / `prompt` credential sources (the docs
+  previously claimed there was no plaintext-password field, which was
+  wrong), the loader-resilience guarantee (one bad connection no longer
+  nukes the registry), the `$name::type` CAST syntax, and the
+  negative-positional-arg `--` hint.
+
 ## [0.5.2] — 2026-08-01
 
 ### Fixed

{dbctl-0.5.2 → dbctl-0.5.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dbctl
-Version: 0.5.2
+Version: 0.5.3
 Summary: Generic CLI to monitor, control, and administer multiple databases via SSM, SSH, or direct connection.
 Author: dbctl contributors
 License: MIT
@@ -34,13 +34,51 @@ Description-Content-Type: text/markdown
 # dbctl
 
 `dbctl` is a small Python CLI for **monitoring, controlling, and administering
-multiple databases** through a single declarative config. It supports three
-ways of reaching a database:
+multiple databases** through a single declarative config. It is built for
+operators who manage a **database explosion** — the same application deployed
+across several environments (dev / test / int / prod) and several tenants
+(Germany / USA / India / …) — where administering a row means opening a GUI
+client, navigating a connection tree, opening a tunnel, and clicking through
+to the right schema. `dbctl` replaces that multi-click flow with one command:
+
+```text
+dbctl prod-de increase-quota alice 10 --apply     # bump a user's quota, in prod, Germany
+dbctl prod-us increase-quota alice 10 --apply     # same op, different tenant
+dbctl int-in  doctor                              # healthcheck integration India
+```
+
+Every connection's tunnel (`ssm` / `ssh` / `k8s` / `direct`), credentials,
+safety policy, and introspection queries live in one versioned
+`~/.dbctl/connections.yaml`; every common SQL action lives in one
+`~/.dbctl/operations.yaml`. `dbctl` synthesises one Click subcommand per
+connection × operation, opens the tunnel, runs the SQL with bind parameters,
+writes an audit-log entry, and tears the tunnel down — without you ever
+leaving the terminal.
+
+### Why not DBeaver / a GUI client?
+
+| GUI client (DBeaver, etc.)                          | `dbctl`                                       |
+|-----------------------------------------------------|-----------------------------------------------|
+| Click through a connection tree every time          | `dbctl <conn> <op>` — one command             |
+| Re-establish SSO / SSH per session, often by hand  | Tunnels auto-opened from YAML, SSO re-used    |
+| Ad-hoc SQL with no audit trail                      | Every run appended to `~/.dbctl/history.jsonl` (secrets redacted) |
+| "What can I run here?" is whatever you remember     | Operations are declared + versioned; `--help` lists them |
+| DML is one `Ctrl-Enter` away                        | DML is **dry-run by default** until `--apply`; `--yes` skips the prompt |
+| No cross-DB diff                                     | `dbctl diff user-count prod-de int-de`        |
+
+`dbctl` is **not** a schema browser or a query playground — it deliberately
+has no ad-hoc query command. Declaring operations in YAML keeps "what can be
+run against this DB" discoverable from a versioned file instead of buried in
+your shell history. (For ad-hoc exploration open the tunnel with
+`dbctl tunnel open <conn>` and point your favourite client at the local bind.)
+
+### How it reaches a database
 
 | type     | how                                                            | external deps |
 |----------|----------------------------------------------------------------|---------------|
 | `ssm`    | AWS SSM port-forward through an EC2 bastion to a private host | `aws` CLI on PATH, active SSO session |
 | `ssh`    | Classic `ssh -N -L` port-forward through a bastion             | `ssh` CLI on PATH, an SSH key file |
+| `k8s`    | `kubectl port-forward` to a Service / Pod in a cluster         | `kubectl` CLI on PATH, a valid kubeconfig |
 | `direct` | No tunnel — connect to the upstream host:port directly        | none |
 
 Each connection declares its SQLAlchemy URL scheme (`postgresql+psycopg`,
@@ -121,12 +159,8 @@ multi-connection `diff` commands.
 ```bash
 docker compose up -d
 
-# passwords for the sample config (set them in your shell):
-export DBCTL_PG_PASSWORD=pwd_postgres
-export DBCTL_MY_PASSWORD=pwd_mysql
-export DBCTL_MS_PASSWORD='PwdSqlServer2026!'
-
-# install the sample config
+# install the sample config (uses plaintext passwords out of the box for dev;
+# swap to `password_env: VARNAME` or `prompt: true` for real environments)
 mkdir -p ~/.dbctl
 cp .dbctl/connections.yaml ~/.dbctl/connections.yaml
 cp .dbctl/operations.yaml  ~/.dbctl/operations.yaml
@@ -139,6 +173,7 @@ dbctl pg info row_counts
 dbctl pg list-users 10
 dbctl pg add-user stephen 12 --show-sql        # dry-run (prints SQL)
 dbctl pg add-user stephen 12 --apply --yes     # commit (no prompt)
+dbctl pg increase-quota alice 10 --apply -y    # +10% on alice's quotas
 
 dbctl diff user-count pg my                    # multi-DB diff
 dbctl diff compare-quotas pg my Daily
@@ -167,6 +202,21 @@ dbctl tunnel open pg                           # hold tunnel for ad-hoc psql
 writes it back to `~/.dbctl/connections.yaml` — it will test the tunnel +
 healthcheck before saving (for `direct` connections).
 
+### Loader resilience
+
+Each connection is validated **independently**, not as one big
+`ConnectionsFile`. A mis-configured connection (e.g. a reference template
+with all password sources commented out, or a typo'd driver) is skipped with
+a one-line warning on stderr — the rest of the registry still loads so the
+dashboard, `--help`, and the good connections keep working:
+
+```
+connections.yaml: 1 invalid connection skipped:
+  pg-ssm: set 'password', 'password_env' or 'prompt: true' for each connection
+```
+
+`dbctl operations validate` checks `operations.yaml` the same way.
+
 ## Adding an operation
 
 Operations are global — every connection sees them. Declare one:
@@ -197,9 +247,21 @@ dbctl pg reset-quota alice --apply --yes     # commit
 
 `$name` and `$floor` are rewritten to SQLAlchemy bind-params (`:name`,
 `:floor`) — they are always parameterised, never string-interpolated, so SQL
-injection through a parameter value is not possible. (For *dynamic
-identifiers* such as table names, write one operation per table for now;
-`${var}` identifier interpolation is on the v2 roadmap.)
+injection through a parameter value is not possible. The Postgres cast idiom
+`$name::type` is rewritten to the portable `CAST(:name AS type)` form so the
+same operation YAML works cross-dialect:
+
+```yaml
+sql: |
+  SELECT level, COUNT(*) AS events
+    FROM logs
+   WHERE created_at >= $since::timestamp     # → CAST(:since AS timestamp)
+     AND created_at <  $until::timestamp
+   GROUP BY level
+```
+
+(For *dynamic identifiers* such as table names, write one operation per table
+for now; `${var}` identifier interpolation is on the v2 roadmap.)
 
 See [`docs/operations.md`](docs/operations.md) for the full parameter and
 mode reference.
@@ -250,10 +312,16 @@ Each connection has a `safety` block:
 `dbctl` **confirms before opening the transaction** — saying `N` at the
 prompt leaves the database untouched.
 
-Only `password_env: VARNAME` and `prompt: true` are accepted as DB password
-sources — no plaintext in YAML, no Secrets Manager code (you said your SSO
-session already wraps the secrets you need). Secret-typed operation parameters
-(`type: secret`) are redacted in the audit log.
+DB password sources are mutually exclusive — pick **one** per connection:
+
+| source          | use |
+|-----------------|------|
+| `password: "…"`     | plaintext (local dev only). Don't commit real secrets. |
+| `password_env: VAR` | read from the named environment variable. Recommended for any shared/CI host. |
+| `prompt: true`      | prompt for the DB password interactively each run. Good for break-glass access. |
+
+Secret-typed **operation** parameters (`type: secret`) are redacted in the
+audit log regardless of which DB password source the connection uses.
 
 ## Shell completion
 
@@ -298,11 +366,13 @@ dbctl/
 ```bash
 uv sync --extra dev
 uv run ruff check dbctl tests   # lint
-uv run pytest tests/             # 15 unit tests
-uv run mypy dbctl                 # type-check
+uv run pytest tests/             # ~60 unit tests against in-memory SQLite
+uv run mypy dbctl                # type-check (pre-existing debt; non-blocking in CI)
 ```
 
 The unit tests use an in-memory SQLite database so they run without docker.
+For a live end-to-end smoke test, `docker compose up -d` and follow the
+[quick start](#quick-start-with-the-bundled-docker-compose-fleet) above.
 
 ## Roadmap
 

{dbctl-0.5.2 → dbctl-0.5.3}/README.md

@@ -1,13 +1,51 @@
 # dbctl
 
 `dbctl` is a small Python CLI for **monitoring, controlling, and administering
-multiple databases** through a single declarative config. It supports three
-ways of reaching a database:
+multiple databases** through a single declarative config. It is built for
+operators who manage a **database explosion** — the same application deployed
+across several environments (dev / test / int / prod) and several tenants
+(Germany / USA / India / …) — where administering a row means opening a GUI
+client, navigating a connection tree, opening a tunnel, and clicking through
+to the right schema. `dbctl` replaces that multi-click flow with one command:
+
+```text
+dbctl prod-de increase-quota alice 10 --apply     # bump a user's quota, in prod, Germany
+dbctl prod-us increase-quota alice 10 --apply     # same op, different tenant
+dbctl int-in  doctor                              # healthcheck integration India
+```
+
+Every connection's tunnel (`ssm` / `ssh` / `k8s` / `direct`), credentials,
+safety policy, and introspection queries live in one versioned
+`~/.dbctl/connections.yaml`; every common SQL action lives in one
+`~/.dbctl/operations.yaml`. `dbctl` synthesises one Click subcommand per
+connection × operation, opens the tunnel, runs the SQL with bind parameters,
+writes an audit-log entry, and tears the tunnel down — without you ever
+leaving the terminal.
+
+### Why not DBeaver / a GUI client?
+
+| GUI client (DBeaver, etc.)                          | `dbctl`                                       |
+|-----------------------------------------------------|-----------------------------------------------|
+| Click through a connection tree every time          | `dbctl <conn> <op>` — one command             |
+| Re-establish SSO / SSH per session, often by hand  | Tunnels auto-opened from YAML, SSO re-used    |
+| Ad-hoc SQL with no audit trail                      | Every run appended to `~/.dbctl/history.jsonl` (secrets redacted) |
+| "What can I run here?" is whatever you remember     | Operations are declared + versioned; `--help` lists them |
+| DML is one `Ctrl-Enter` away                        | DML is **dry-run by default** until `--apply`; `--yes` skips the prompt |
+| No cross-DB diff                                     | `dbctl diff user-count prod-de int-de`        |
+
+`dbctl` is **not** a schema browser or a query playground — it deliberately
+has no ad-hoc query command. Declaring operations in YAML keeps "what can be
+run against this DB" discoverable from a versioned file instead of buried in
+your shell history. (For ad-hoc exploration open the tunnel with
+`dbctl tunnel open <conn>` and point your favourite client at the local bind.)
+
+### How it reaches a database
 
 | type     | how                                                            | external deps |
 |----------|----------------------------------------------------------------|---------------|
 | `ssm`    | AWS SSM port-forward through an EC2 bastion to a private host | `aws` CLI on PATH, active SSO session |
 | `ssh`    | Classic `ssh -N -L` port-forward through a bastion             | `ssh` CLI on PATH, an SSH key file |
+| `k8s`    | `kubectl port-forward` to a Service / Pod in a cluster         | `kubectl` CLI on PATH, a valid kubeconfig |
 | `direct` | No tunnel — connect to the upstream host:port directly        | none |
 
 Each connection declares its SQLAlchemy URL scheme (`postgresql+psycopg`,
@@ -88,12 +126,8 @@ multi-connection `diff` commands.
 ```bash
 docker compose up -d
 
-# passwords for the sample config (set them in your shell):
-export DBCTL_PG_PASSWORD=pwd_postgres
-export DBCTL_MY_PASSWORD=pwd_mysql
-export DBCTL_MS_PASSWORD='PwdSqlServer2026!'
-
-# install the sample config
+# install the sample config (uses plaintext passwords out of the box for dev;
+# swap to `password_env: VARNAME` or `prompt: true` for real environments)
 mkdir -p ~/.dbctl
 cp .dbctl/connections.yaml ~/.dbctl/connections.yaml
 cp .dbctl/operations.yaml  ~/.dbctl/operations.yaml
@@ -106,6 +140,7 @@ dbctl pg info row_counts
 dbctl pg list-users 10
 dbctl pg add-user stephen 12 --show-sql        # dry-run (prints SQL)
 dbctl pg add-user stephen 12 --apply --yes     # commit (no prompt)
+dbctl pg increase-quota alice 10 --apply -y    # +10% on alice's quotas
 
 dbctl diff user-count pg my                    # multi-DB diff
 dbctl diff compare-quotas pg my Daily
@@ -134,6 +169,21 @@ dbctl tunnel open pg                           # hold tunnel for ad-hoc psql
 writes it back to `~/.dbctl/connections.yaml` — it will test the tunnel +
 healthcheck before saving (for `direct` connections).
 
+### Loader resilience
+
+Each connection is validated **independently**, not as one big
+`ConnectionsFile`. A mis-configured connection (e.g. a reference template
+with all password sources commented out, or a typo'd driver) is skipped with
+a one-line warning on stderr — the rest of the registry still loads so the
+dashboard, `--help`, and the good connections keep working:
+
+```
+connections.yaml: 1 invalid connection skipped:
+  pg-ssm: set 'password', 'password_env' or 'prompt: true' for each connection
+```
+
+`dbctl operations validate` checks `operations.yaml` the same way.
+
 ## Adding an operation
 
 Operations are global — every connection sees them. Declare one:
@@ -164,9 +214,21 @@ dbctl pg reset-quota alice --apply --yes     # commit
 
 `$name` and `$floor` are rewritten to SQLAlchemy bind-params (`:name`,
 `:floor`) — they are always parameterised, never string-interpolated, so SQL
-injection through a parameter value is not possible. (For *dynamic
-identifiers* such as table names, write one operation per table for now;
-`${var}` identifier interpolation is on the v2 roadmap.)
+injection through a parameter value is not possible. The Postgres cast idiom
+`$name::type` is rewritten to the portable `CAST(:name AS type)` form so the
+same operation YAML works cross-dialect:
+
+```yaml
+sql: |
+  SELECT level, COUNT(*) AS events
+    FROM logs
+   WHERE created_at >= $since::timestamp     # → CAST(:since AS timestamp)
+     AND created_at <  $until::timestamp
+   GROUP BY level
+```
+
+(For *dynamic identifiers* such as table names, write one operation per table
+for now; `${var}` identifier interpolation is on the v2 roadmap.)
 
 See [`docs/operations.md`](docs/operations.md) for the full parameter and
 mode reference.
@@ -217,10 +279,16 @@ Each connection has a `safety` block:
 `dbctl` **confirms before opening the transaction** — saying `N` at the
 prompt leaves the database untouched.
 
-Only `password_env: VARNAME` and `prompt: true` are accepted as DB password
-sources — no plaintext in YAML, no Secrets Manager code (you said your SSO
-session already wraps the secrets you need). Secret-typed operation parameters
-(`type: secret`) are redacted in the audit log.
+DB password sources are mutually exclusive — pick **one** per connection:
+
+| source          | use |
+|-----------------|------|
+| `password: "…"`     | plaintext (local dev only). Don't commit real secrets. |
+| `password_env: VAR` | read from the named environment variable. Recommended for any shared/CI host. |
+| `prompt: true`      | prompt for the DB password interactively each run. Good for break-glass access. |
+
+Secret-typed **operation** parameters (`type: secret`) are redacted in the
+audit log regardless of which DB password source the connection uses.
 
 ## Shell completion
 
@@ -265,11 +333,13 @@ dbctl/
 ```bash
 uv sync --extra dev
 uv run ruff check dbctl tests   # lint
-uv run pytest tests/             # 15 unit tests
-uv run mypy dbctl                 # type-check
+uv run pytest tests/             # ~60 unit tests against in-memory SQLite
+uv run mypy dbctl                # type-check (pre-existing debt; non-blocking in CI)
 ```
 
 The unit tests use an in-memory SQLite database so they run without docker.
+For a live end-to-end smoke test, `docker compose up -d` and follow the
+[quick start](#quick-start-with-the-bundled-docker-compose-fleet) above.
 
 ## Roadmap
 

{dbctl-0.5.2 → dbctl-0.5.3}/dbctl/cli.py

@@ -12,6 +12,7 @@ from __future__ import annotations
 
 import contextlib
 import os
+import re
 import sys
 import time
 from typing import Any
@@ -32,6 +33,48 @@ from dbctl.runtime import (
     registries,
 )
 
+# --------------------------------------------------------------------------- #
+# Negative-number hint for dynamic commands
+# --------------------------------------------------------------------------- #
+_NEG_OPT_RE = re.compile(r"No such option '(-\d[^']*)'")
+
+
+class _NegNumberHintCommand(click.Command):
+    """Click can't parse a negative number as a positional Argument value
+    (only Options support negative-number detection, and only with a Range
+    type and `min < 0`). When the user writes ``dbctl pg increase-quota
+    zelda -10`` Click reports ``No such option '-1'`` which is opaque.
+
+    This subclass intercepts that exact error and appends a hint pointing
+    the user at the ``--`` separator: ``dbctl <conn> <op> ... -- <args>``.
+    See https://github.com/pallets/click/issues/2169 for the upstream
+    limitation.
+    """
+
+    def parse_args(self, ctx, args):
+        # Click's parser mutates `args` in place while it consumes tokens,
+        # so by the time the except block runs the list is empty. Snapshot
+        # here while still pristine.
+        original_args = list(args)
+        try:
+            return super().parse_args(ctx, args)
+        except click.exceptions.UsageError as e:
+            m = _NEG_OPT_RE.match(e.message or "")
+            if m:
+                # Click truncates the bad token to the first option char
+                # (e.g. `-10` → `-1`), so scan the original argv for the
+                # offending token to show the user what *they* typed.
+                bad = next(
+                    (a for a in original_args if a.startswith("-") and len(a) > 1 and a[1].isdigit()),
+                    m.group(1),
+                )
+                e.message = (
+                    f"{e.message}\n  hint: a positional argument can't start "
+                    f"with '-'. Separate options from positionals with '--':\n"
+                    f"  {ctx.command_path} [opts] -- <positional args including {bad}>"
+                )
+            raise
+
 
 # --------------------------------------------------------------------------- #
 # param type mapping
@@ -241,7 +284,7 @@ def _make_single_op_command(conn_name: str, op_name: str, op: Operation) -> clic
     help_text = (op.description or "").strip()
     if sig:
         help_text = f"{help_text}\n\n  dbctl {conn_name} {op_name} {sig}"
-    return click.Command(
+    return _NegNumberHintCommand(
         name=op_name,
         params=click_params,
         callback=callback,
@@ -453,10 +496,16 @@ def _make_multi_op_command(verb: str, op_name: str, op: Operation) -> click.Comm
         try:
             for role in op.roles:
                 cname = role_conns[role]
-                canonical, conn = resolve(cname, conns_all)
+                try:
+                    canonical, conn = resolve(cname, conns_all)
+                except KeyError as e:
+                    err_console.print(f"[red]{e}[/red]")
+                    raise SystemExit(2)
                 oc = opened(canonical, conn).__enter__()
                 opened_list.append(oc)
                 results[role] = run_role(op, role, oc, bound)
+        except SystemExit:
+            raise
         except RuntimeError as e:
             err_console.print(f"[red]{e}[/red]")
             raise SystemExit(1)
@@ -497,7 +546,7 @@ def _make_multi_op_command(verb: str, op_name: str, op: Operation) -> click.Comm
     sig = " ".join(r.upper() for r in op.roles) + (
         " " + " ".join(p.name.upper() for p in pos_params) if pos_params else ""
     )
-    return click.Command(
+    return _NegNumberHintCommand(
         name=op_name,
         params=click_params,
         callback=callback,
@@ -613,12 +662,16 @@ def connections_list(ctx):
 @click.argument("name")
 @click.pass_context
 def connections_show(ctx, name):
+    from dbctl.connections import resolve
+
     conns, _ = registries(ctx)
-    if name not in conns:
-        err_console.print(f"[red]unknown connection {name!r}[/red]")
+    try:
+        canonical, conn = resolve(name, conns)
+    except KeyError as e:
+        err_console.print(f"[red]{e}[/red]")
         raise SystemExit(2)
     console.print(
-        yaml.safe_dump({name: conns[name].model_dump(mode="json")}, sort_keys=False, allow_unicode=True)
+        yaml.safe_dump({canonical: conn.model_dump(mode="json")}, sort_keys=False, allow_unicode=True)
     )
 
 

{dbctl-0.5.2 → dbctl-0.5.3}/dbctl/execute.py

@@ -14,18 +14,41 @@ from sqlalchemy.exc import SQLAlchemyError
 if TYPE_CHECKING:
     from dbctl.config import Operation
 
-_PLACEHOLDER = re.compile(r"\$([a-zA-Z_][a-zA-Z0-9_]*)")
+# A SQL type name after `$name::`. Allows schema-qualified names
+# (`pg.text`) and parenthesised precision/length (`varchar(255)`,
+# `numeric(10, 2)`, `timestamp(6)`), but NOT surrounding spaces or
+# trailing operators (so `$x::text || $y::int` rewrites cleanly).
+_TYPE = r"[a-zA-Z_][a-zA-Z0-9_]*(?:\.[a-zA-Z_][a-zA-Z0-9_]*)*(?:\([^)]*\))?"
+_PLACEHOLDER = re.compile(rf"\$([a-zA-Z_][a-zA-Z0-9_]*)(?:::({_TYPE}))?")
+
+
+def _replace(match: re.Match[str]) -> str:
+    name = match.group(1)
+    cast = match.group(2)
+    if cast:
+        # `$name::type` (Postgres idiom) — emit the SQL-standard
+        # `CAST(:name AS type)` form so the rewritten SQL works on
+        # every dialect (psycopg rejects `:name::type` because the
+        # `::type` suffix collides with its bind param syntax).
+        return f"CAST(:{name} AS {cast})"
+    return f":{name}"
 
 
 def to_bindparams(sql: str) -> str:
     """Rewrite ``$name`` → ``:name`` while leaving everything else (literals,
     operators, dollar-quoting edge cases) untouched.
 
+    Also handles the Postgres cast idiom ``$name::type`` by rewriting it to
+    the portable ``CAST(:name AS type)`` form. This is necessary because
+    SQLAlchemy/psycopg rejects ``:name::type`` (the trailing ``::type``
+    collides with the bind param name parser), and because ``CAST`` is
+    SQL-standard so the same operation YAML works cross-dialect.
+
     We deliberately do **not** touch ``$$ ... $$`` dollar-quoted strings: those
     are matched only when the char after ``$`` is a digit (which is an invalid
     bindparam name anyway), so they survive intact.
     """
-    return _PLACEHOLDER.sub(r":\1", sql)
+    return _PLACEHOLDER.sub(_replace, sql)
 
 
 @dataclass

{dbctl-0.5.2 → dbctl-0.5.3}/docs/connections.md

@@ -24,8 +24,9 @@ falling back to defaults.
 | `driver`        | string                                    | **yes**  | SQLAlchemy URL scheme. Supported conventions: `postgresql+psycopg`, `mysql+pymysql`, `mariadb+pymysql`, `mssql+pyodbc`. Any other SQLAlchemy scheme works as long as its driver is importable. |
 | `database`      | string                                    | **yes**  | database / catalog name passed to SQLAlchemy. |
 | `username`      | string                                    | **yes**  | DB user. |
-| `password_env`  | string                                    | see rule | name of the environment variable holding the DB password. Mutually exclusive with `prompt`. |
-| `prompt`        | bool                                      | see rule | prompt for the DB password interactively each run. Mutually exclusive with `password_env`. |
+| `password`      | string                                    | see rule | plaintext DB password (local dev only — don't commit real secrets to YAML). Mutually exclusive with `password_env` and `prompt`. |
+| `password_env`  | string                                    | see rule | name of the environment variable holding the DB password. Mutually exclusive with `password` and `prompt`. |
+| `prompt`        | bool                                      | see rule | prompt for the DB password interactively each run. Mutually exclusive with `password` and `password_env`. |
 | `ssm`           | [`SsmTunnel`](#ssm-block)                 | yes if `type: ssm` | tunnel params. |
 | `ssh`           | [`SshTunnel`](#ssh-block)                 | yes if `type: ssh` | tunnel params. |
 | `k8s`           | [`K8sTunnel`](#k8s-block)                 | yes if `type: k8s` | tunnel params. |
@@ -34,8 +35,30 @@ falling back to defaults.
 | `info`          | list of [`InfoQuery`](#info-query)        | no       | named introspection queries that `dbctl <conn> info <name>` can run. |
 | `safety`        | [`Safety`](#safety-block)                 | no       | gates DML. |
 
-**Credential rule.** Exactly one of `password_env` and `prompt` must be set;
-there is no plaintext-password field and no Secrets Manager integration.
+**Credential rule.** Exactly one of `password`, `password_env`, and `prompt`
+must be set; they are mutually exclusive. `password` (plaintext) is fine for
+local dev fleets like the bundled docker-compose sample; for anything shared
+or production-shaped prefer `password_env` (the value lives in your shell /
+secret manager, never in YAML) or `prompt: true` (interactive, leaves no
+trace). There is no Secrets Manager / Vault integration — the assumption is
+your existing SSO session and shell environment already wrap the secrets you
+need.
+
+### Loader resilience
+
+Each connection is validated **individually**, not as one all-or-nothing
+`ConnectionsFile`. A mis-configured connection (a reference template with no
+password source, a typo'd driver name, an unknown tunnel block, …) is skipped
+with a one-line warning on stderr, and the good connections still load so the
+dashboard and `--help` keep working:
+
+```
+connections.yaml: 1 invalid connection skipped:
+  pg-ssm: set 'password', 'password_env' or 'prompt: true' for each connection
+```
+
+The offending connection is simply absent from `dbctl connections list` until
+you fix it — the CLI never refuses to start because of one bad entry.
 
 ## `ssm` block
 

{dbctl-0.5.2 → dbctl-0.5.3}/docs/operations.md

@@ -78,12 +78,28 @@ sql: |
 
 - `$1`, `$2`, … are **not** rewritten (they're positional placeholders in
   some dialects) — the regex only matches `$<identifier>`.
+- The Postgres cast idiom `$name::type` is rewritten to the SQL-standard
+  `CAST(:name AS type)` form so the same operation YAML works cross-dialect
+  and doesn't collide with psycopg's bind-param parser. Parenthesised
+  precision / length is preserved, e.g. `$x::numeric(10, 2)` →
+  `CAST(:x AS numeric(10, 2))`, and schema-qualified types work too
+  (`$x::pg.text` → `CAST(:x AS pg.text)`).
 - PostgreSQL dollar-quoted strings (`$$ ... $$`) are not specially handled:
   avoid mixing dollar-quoting with `$name` parameters in the same operation.
   Use single-quoted string literals for any in-SQL text.
 - The same placeholder can appear twice in one statement
   (`VALUES ($quota, $quota * 365)`); the driver reuses the value.
 
+> **Negative positional arguments.** Click can't distinguish a negative
+> number from an unknown option when it appears as a positional `Argument`
+> value (e.g. `dbctl pg increase-quota alice -10` is parsed as the unknown
+> option `-1`). The error message includes a hint pointing at the `--`
+> separator, which you can use to unambiguously mark the start of positionals:
+>
+> ```bash
+> dbctl pg increase-quota --apply -y -- alice -10
+> ```
+
 ## <a name="mode"></a> `mode` (single-scope)
 
 | mode        | behaviour |
@@ -218,6 +234,21 @@ operations:
     sql: |
       SELECT name, quota_daily, is_active FROM users WHERE name ILIKE $prefix || '%'
 
+  increase-quota:
+    description: "Increase a user's daily and yearly quota by a percentage"
+    scope: single
+    mode: execute
+    confirm: true
+    parameters:
+      - { name: name, type: string, required: true, position: 1, description: "User to bump" }
+      - { name: pct,  type: float,  required: true, position: 2, description: "Percentage increase (e.g. 10 = +10%) — use `--` for negative values" }
+    sql: |
+      UPDATE users
+         SET quota_daily  = (quota_daily  * (1 + $pct / 100.0))::integer,
+             quota_yearly = (quota_yearly * (1 + $pct / 100.0))::integer,
+             updated_at   = NOW()
+       WHERE name = $name
+
   report-logs:
     description: "Summarise logs by level between two dates"
     scope: single

{dbctl-0.5.2 → dbctl-0.5.3}/docs/tutorial.md

@@ -43,7 +43,7 @@ Run `dbctl` once to confirm it's alive:
 
 ```bash
 ▶ uv run dbctl --version
-dbctl, version 0.1.0
+dbctl, version 0.5.3
 ```
 
 ---
@@ -85,19 +85,14 @@ You should see all three services `healthy`.
 ```
 
 The bundled config has three "live" connections — `pg`, `my`, `ms` — plus
-two reference-only templates — `pg-ssm`, `pg-ssh` — that we'll use in
-section 11.
-
-The sample config reads DB passwords from environment variables (there is
-no plaintext-password field in dbctl). Export them now:
-
-```bash
-▶ export DBCTL_PG_PASSWORD=pwd_postgres
-▶ export DBCTL_MY_PASSWORD=pwd_mysql
-▶ export DBCTL_MS_PASSWORD='PwdSqlServer2026!'
-```
-
-(These match the values in `docker-compose.yml`.)
+three reference-only templates — `pg-ssm`, `pg-k8s`, `pg-ssh` — that we'll
+use in section 11.
+
+The sample config ships with **plaintext passwords** out of the box so the
+docker fleet "just works" after `cp` — see the `password:` field in
+`~/.dbctl/connections.yaml`. For any real environment swap to
+`password_env: VARNAME` (read from your shell) or `prompt: true` (typed each
+run); the three sources are mutually exclusive.
 
 ---
 

{dbctl-0.5.2 → dbctl-0.5.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "dbctl"
-version = "0.5.2"
+version = "0.5.3"
 description = "Generic CLI to monitor, control, and administer multiple databases via SSM, SSH, or direct connection."
 readme = "README.md"
 requires-python = ">=3.11"

dbctl-0.5.3/tests/test_regressions.py

@@ -0,0 +1,210 @@
+"""Regression tests for bugs found while validating against the docker fleet."""
+
+from __future__ import annotations
+
+from contextlib import contextmanager
+from types import SimpleNamespace
+from unittest import mock
+
+from click.testing import CliRunner
+
+from dbctl.audit import append, read
+from dbctl.cli import _make_multi_op_command
+from dbctl.config import Operation
+
+
+# --------------------------------------------------------------------------- #
+# Multi-op CLI used to crash on every invocation: click lowercases argument
+# names when exposing them as kwargs, but the callback popped `r.upper()`.
+# Verifying the fix + guarding against a reintroduction.
+# --------------------------------------------------------------------------- #
+def test_multi_op_command_uses_lowercase_role_kwargs(tmp_path, monkeypatch):
+    op = Operation.model_validate(
+        {
+            "description": "test diff",
+            "scope": "multi",
+            "mode": "diff",
+            "roles": ["src", "trg"],
+            "queries": {
+                "src": "SELECT 'u' AS t, 1 AS n",
+                "trg": "SELECT 'u' AS t, 2 AS n",
+            },
+            "diff": {"key": ["t"], "show": ["n"]},
+        }
+    )
+
+    conns = {
+        "pg": SimpleNamespace(description="a", aliases=[]),
+        "my": SimpleNamespace(description="b", aliases=[]),
+    }
+
+    @contextmanager
+    def fake_opened(name, conn):
+        yield SimpleNamespace(name=name, engine=None, tunnel=mock.MagicMock())
+
+    captured: dict = {}
+
+    def fake_run_role(_op, role, opened_conn, _params):
+        captured.setdefault("role_conns", {})[role] = opened_conn.name
+        return [{"t": "u", "n": 1 if role == "src" else 2}]
+
+    monkeypatch.setattr("dbctl.cli.registries", lambda _ctx: (conns, {"test-diff": op}))
+    monkeypatch.setattr("dbctl.multi.opened", fake_opened)
+    monkeypatch.setattr("dbctl.multi.run_role", fake_run_role)
+    monkeypatch.setattr("dbctl.audit.history_path", lambda profile=None: tmp_path / "h.jsonl")
+
+    cmd = _make_multi_op_command("diff", "test-diff", op)
+    result = CliRunner().invoke(cmd, ["pg", "my"], obj={})
+    assert result.exit_code == 0, result.output
+    assert captured["role_conns"] == {"src": "pg", "trg": "my"}
+
+
+# --------------------------------------------------------------------------- #
+# audit.read() must skip corrupt lines *before* truncating to `limit`, not
+# after — otherwise a half-written trailing line evicts a valid older entry
+# from the window.
+# --------------------------------------------------------------------------- #
+def test_audit_read_skips_corrupt_lines_without_losing_history(tmp_path, monkeypatch):
+    from dbctl import audit
+
+    monkeypatch.setattr(audit, "history_path", lambda profile=None: tmp_path / "h.jsonl")
+
+    append(profile=None, connection="pg", operation="op-a", params={}, mode="execute", status="ok")
+    append(profile=None, connection="pg", operation="op-b", params={}, mode="execute", status="ok")
+
+    # append a corrupted tail (simulating a crash mid-write)
+    (tmp_path / "h.jsonl").open("a").write("{ broken json\n")
+
+    entries = read(None, limit=5)
+    assert len(entries) == 2
+    assert {e["operation"] for e in entries} == {"op-a", "op-b"}
+
+
+# --------------------------------------------------------------------------- #
+# Bug: `$name::type` cast broke psycopg (syntax error at ":").
+# `to_bindparams` now rewrites to `CAST(:name AS type)`.
+# --------------------------------------------------------------------------- #
+def test_to_bindparams_handles_cast_idiom():
+    from dbctl.execute import to_bindparams
+
+    assert to_bindparams("SELECT $x::timestamp") == "SELECT CAST(:x AS timestamp)"
+    assert (
+        to_bindparams("WHERE created_at >= $since::timestamp")
+        == "WHERE created_at >= CAST(:since AS timestamp)"
+    )
+
+
+# --------------------------------------------------------------------------- #
+# Bug: a negative number for a positional Argument (e.g.
+# `dbctl pg increase-quota zelda -10`) failed with the opaque
+# `No such option '-1'`. The CLI now sub-classes `click.Command` to
+# append a "-- separator" hint showing the user's actual token.
+# --------------------------------------------------------------------------- #
+def test_negative_positional_arg_emits_separator_hint():
+    from dbctl.cli import _make_single_op_command
+    from dbctl.config import Operation
+
+    op = Operation.model_validate(
+        {
+            "description": "bump pct",
+            "scope": "single",
+            "mode": "execute",
+            "confirm": False,  # don't prompt; we never reach execution
+            "parameters": [
+                {"name": "name", "type": "string", "required": True, "position": 1},
+                {"name": "pct", "type": "float", "required": True, "position": 2},
+            ],
+            "sql": "UPDATE users SET quota = quota * (1 + $pct / 100) WHERE name = $name",
+        }
+    )
+    cmd = _make_single_op_command("pg", "increase-quota", op)
+    result = CliRunner().invoke(cmd, ["zelda", "-10"], obj={}, color=False)
+    # Click returns exit code 2 on UsageError; we just want the hint.
+    assert "No such option '-1'" in result.output
+    assert "hint:" in result.output
+    # The user's actual token `-10` must appear (Click truncates to `-1`).
+    assert "-10" in result.output
+    assert "--" in result.output
+
+
+# --------------------------------------------------------------------------- #
+# Bug: `dbctl connections show <alias>` reported "unknown connection"
+# because `connections_show` did a direct dict lookup instead of going
+# through the alias-aware `resolve()`.
+# --------------------------------------------------------------------------- #
+def test_connections_show_resolves_alias(tmp_path, monkeypatch):
+    from dbctl.config import Connection, TunnelType
+
+    conns = {
+        "pg": Connection(
+            description="Postgres dev",
+            aliases=["postgres"],
+            type=TunnelType.direct,
+            driver="postgresql+psycopg",
+            database="app",
+            username="u",
+            password="p",
+            direct={"host": "127.0.0.1", "port": 5432},
+        )
+    }
+    monkeypatch.setattr("dbctl.cli.registries", lambda _ctx: (conns, {}))
+
+    from dbctl.cli import connections_cmd
+
+    # Canonical name resolves.
+    r = CliRunner().invoke(connections_cmd, ["show", "pg"], obj={}, color=False)
+    assert r.exit_code == 0, r.output
+    assert "pg:" in r.output
+    assert "Postgres dev" in r.output
+
+    # Alias resolves to the canonical connection (top key is "pg", not "postgres").
+    r = CliRunner().invoke(connections_cmd, ["show", "postgres"], obj={}, color=False)
+    assert r.exit_code == 0, r.output
+    assert "pg:" in r.output
+    assert "postgres:" not in r.output.split("pg:")[0]  # not the alias as the key
+
+    # Unknown connection still produces a clean error + exit code 2.
+    r = CliRunner().invoke(connections_cmd, ["show", "zzz"], obj={}, color=False)
+    assert r.exit_code == 2
+    assert "unknown connection 'zzz'" in r.output
+
+
+# --------------------------------------------------------------------------- #
+# Bug: `dbctl diff user-count pg zzz` crashed with a raw
+# `UnknownConnectionError` traceback. The multi-op callback now catches
+# `KeyError` and emits a one-line message on stderr instead.
+# --------------------------------------------------------------------------- #
+def test_multi_op_unknown_connection_emits_clean_error(tmp_path, monkeypatch):
+    op = Operation.model_validate(
+        {
+            "description": "test diff",
+            "scope": "multi",
+            "mode": "diff",
+            "roles": ["src", "trg"],
+            "queries": {"src": "SELECT 1", "trg": "SELECT 1"},
+        }
+    )
+    conns = {
+        "pg": SimpleNamespace(description="a", aliases=[]),
+        # note: no "zzz" connection
+    }
+
+    @contextmanager
+    def fake_opened(name, conn):
+        yield SimpleNamespace(name=name, engine=None, tunnel=mock.MagicMock())
+
+    def fake_run_role(_op, role, opened_conn, _params):
+        return [{"t": "u", "n": 1}]
+
+    monkeypatch.setattr("dbctl.cli.registries", lambda _ctx: (conns, {"test-diff": op}))
+    monkeypatch.setattr("dbctl.multi.opened", fake_opened)
+    monkeypatch.setattr("dbctl.multi.run_role", fake_run_role)
+    monkeypatch.setattr("dbctl.audit.history_path", lambda profile=None: tmp_path / "h.jsonl")
+
+    cmd = _make_multi_op_command("diff", "test-diff", op)
+    result = CliRunner().invoke(cmd, ["pg", "zzz"], obj={}, color=False)
+    assert result.exit_code == 2, result.output
+    assert "unknown connection" in result.output
+    # No traceback leaked to the user.
+    assert "Traceback" not in result.output
+    assert "UnknownConnectionError" not in result.output

{dbctl-0.5.2 → dbctl-0.5.3}/tests/test_smoke.py

@@ -96,6 +96,38 @@ def test_to_bindparams_leaves_dollar_digit_alone():
     assert to_bindparams("SELECT $1") == "SELECT $1"
 
 
+def test_to_bindparams_cast_simple():
+    # Postgres idiom `$name::type` rewrites to the SQL-standard
+    # `CAST(:name AS type)` so it works on every dialect and doesn't
+    # collide with psycopg's bindparam parser.
+    assert to_bindparams("SELECT $x::timestamp") == "SELECT CAST(:x AS timestamp)"
+
+
+def test_to_bindparams_cast_multiple_in_expression():
+    assert to_bindparams("$x::text || $y::int") == "CAST(:x AS text) || CAST(:y AS int)"
+
+
+def test_to_bindparams_cast_with_precision():
+    assert to_bindparams("SELECT $x::varchar(255)") == "SELECT CAST(:x AS varchar(255))"
+
+
+def test_to_bindparams_cast_with_numeric_precision():
+    assert to_bindparams("SELECT $x::numeric(10, 2)") == "SELECT CAST(:x AS numeric(10, 2))"
+
+
+def test_to_bindparams_cast_schema_qualified():
+    assert to_bindparams("SELECT $x::pg.text") == "SELECT CAST(:x AS pg.text)"
+
+
+def test_to_bindparams_cast_in_where_clause():
+    # The actual `report-logs` SQL shape that broke psycopg pre-fix.
+    sql = "WHERE created_at >= $since::timestamp AND created_at < $until::timestamp"
+    assert (
+        to_bindparams(sql)
+        == "WHERE created_at >= CAST(:since AS timestamp) AND created_at < CAST(:until AS timestamp)"
+    )
+
+
 # --------------------------------------------------------------------------- #
 # parameter binding
 # --------------------------------------------------------------------------- #

{dbctl-0.5.2 → dbctl-0.5.3}/uv.lock

@@ -168,7 +168,7 @@ toml = [
 
 [[package]]
 name = "dbctl"
-version = "0.5.2"
+version = "0.5.3"
 source = { editable = "." }
 dependencies = [
     { name = "click" },

dbctl-0.5.2/tests/test_regressions.py

@@ -1,80 +0,0 @@
-"""Regression tests for bugs found while validating against the docker fleet."""
-
-from __future__ import annotations
-
-from contextlib import contextmanager
-from types import SimpleNamespace
-from unittest import mock
-
-from click.testing import CliRunner
-
-from dbctl.audit import append, read
-from dbctl.cli import _make_multi_op_command
-from dbctl.config import Operation
-
-
-# --------------------------------------------------------------------------- #
-# Multi-op CLI used to crash on every invocation: click lowercases argument
-# names when exposing them as kwargs, but the callback popped `r.upper()`.
-# Verifying the fix + guarding against a reintroduction.
-# --------------------------------------------------------------------------- #
-def test_multi_op_command_uses_lowercase_role_kwargs(tmp_path, monkeypatch):
-    op = Operation.model_validate(
-        {
-            "description": "test diff",
-            "scope": "multi",
-            "mode": "diff",
-            "roles": ["src", "trg"],
-            "queries": {
-                "src": "SELECT 'u' AS t, 1 AS n",
-                "trg": "SELECT 'u' AS t, 2 AS n",
-            },
-            "diff": {"key": ["t"], "show": ["n"]},
-        }
-    )
-
-    conns = {
-        "pg": SimpleNamespace(description="a", aliases=[]),
-        "my": SimpleNamespace(description="b", aliases=[]),
-    }
-
-    @contextmanager
-    def fake_opened(name, conn):
-        yield SimpleNamespace(name=name, engine=None, tunnel=mock.MagicMock())
-
-    captured: dict = {}
-
-    def fake_run_role(_op, role, opened_conn, _params):
-        captured.setdefault("role_conns", {})[role] = opened_conn.name
-        return [{"t": "u", "n": 1 if role == "src" else 2}]
-
-    monkeypatch.setattr("dbctl.cli.registries", lambda _ctx: (conns, {"test-diff": op}))
-    monkeypatch.setattr("dbctl.multi.opened", fake_opened)
-    monkeypatch.setattr("dbctl.multi.run_role", fake_run_role)
-    monkeypatch.setattr("dbctl.audit.history_path", lambda profile=None: tmp_path / "h.jsonl")
-
-    cmd = _make_multi_op_command("diff", "test-diff", op)
-    result = CliRunner().invoke(cmd, ["pg", "my"], obj={})
-    assert result.exit_code == 0, result.output
-    assert captured["role_conns"] == {"src": "pg", "trg": "my"}
-
-
-# --------------------------------------------------------------------------- #
-# audit.read() must skip corrupt lines *before* truncating to `limit`, not
-# after — otherwise a half-written trailing line evicts a valid older entry
-# from the window.
-# --------------------------------------------------------------------------- #
-def test_audit_read_skips_corrupt_lines_without_losing_history(tmp_path, monkeypatch):
-    from dbctl import audit
-
-    monkeypatch.setattr(audit, "history_path", lambda profile=None: tmp_path / "h.jsonl")
-
-    append(profile=None, connection="pg", operation="op-a", params={}, mode="execute", status="ok")
-    append(profile=None, connection="pg", operation="op-b", params={}, mode="execute", status="ok")
-
-    # append a corrupted tail (simulating a crash mid-write)
-    (tmp_path / "h.jsonl").open("a").write("{ broken json\n")
-
-    entries = read(None, limit=5)
-    assert len(entries) == 2
-    assert {e["operation"] for e in entries} == {"op-a", "op-b"}