@carllee1983/dbcli 1.39.2 → 1.41.0

package/skills/dbcli/reference.md

@@ -105,6 +105,7 @@ dbcli schema --use prod             # Scan prod DB; saves to .dbcli/schemas/prod
 
 **Schema storage (v1.4+):** Schema is persisted as layered files under `.dbcli/schemas/`. With v2 multi-connection config each connection gets its own subdirectory (`.dbcli/schemas/<connection>/`). Run `dbcli schema --use <connection>` once per connection before querying it — otherwise `schema <table>` may return data from the wrong connection's cache.
 
+> **PostgreSQL:** Introspection uses the exact `public` catalog identity throughout. Full catalog/schema/table joins prevent a reused constraint name from contaminating another table; enum lookup includes its namespace; composite primary-key order comes from the exact table OID and index ordinality; and row estimates are scoped to the exact `public` relation. Row-count SQL qualifies and quotes both `"public"` and the exact table identifier, escaping embedded quotes so mixed-case or punctuation-bearing names remain distinct and safe.
 > **Redis:** `schema <key>` is required (no full scan). The output exposes `type`, `ttl`, `size`, and a small `sample` (e.g. first 5 hash keys). `--reset` / `--refresh` are rejected — Redis caches no schema.
 > **Elasticsearch:** `schema [index]` flattens the `_mapping` properties (nested `a.b.c`) and emits each `.fields` multi-field as a separate column (e.g. `text` + `text.keyword`). Full scan iterates all non-system indices and stores per-connection caches alongside SQL engines.
 > **MongoDB:** schema is sampled via `$sample` (default 100, max 1000). `--sample-method natural` switches to `find().limit()`; `random` (default) falls back to natural order on driver error. Output columns surface nested dot-paths with `presence` (0..1) and `redacted: true` flags for blacklist-matched paths. The persisted cache records `sampleMethod` and `sampleSize`; `dbcli doctor` reports them via a `sampled: method=…, size=…` line.
@@ -192,9 +193,134 @@ dbcli explain --bulk @analytics/*                     # glob over saved queries
 | `nested-loop-large` | yellow | PG `Nested Loop` with planner rows > 10,000 |
 
 > Notes:
-> - `--analyze` executes the statement — do not use against destructive SQL.
+> - `--analyze` executes the statement, so dbcli accepts it only for SQL that is
+>   structurally proven to be a read-only, function-free `SELECT` (including
+>   SELECT-only CTEs). Explicit function and table-function calls are unproven
+>   because user-defined and built-in functions may have side effects. DML, DDL,
+>   data-modifying CTEs, session assignments, function-bearing SQL, and
+>   unrecognized SQL are rejected before the adapter is invoked; use plain
+>   `dbcli explain` for those statements.
 > - Auto-`LIMIT` is **not** applied to EXPLAIN statements (since v1.23 P1).
 
+### lint
+
+Static, report-only SQL anti-pattern analysis for PostgreSQL, MySQL, and
+MariaDB. `lint` never opens a database connection, never runs the SQL, and
+never applies a rewrite. Schema-aware findings use only the layered schema
+cache under `.dbcli/schemas/`.
+
+```text
+dbcli lint [queries...]
+dbcli lint --bulk <input>
+dbcli --use <conn> lint [queries...]
+```
+
+An input may be inline SQL, a saved query such as `@analytics/live-summary`, a
+SQL file such as `@queries.sql`, or a saved-query/filesystem glob such as
+`@analytics/*` or `@queries/**/*.sql`. `--bulk` accepts a comma-separated mix
+of those `@file`, `@glob`, and `@saved-query` inputs; quote a filesystem glob
+in a shell so the `@` reference reaches dbcli unchanged.
+
+```bash
+dbcli lint "SELECT * FROM users WHERE email LIKE '%@example.com'" --format json
+dbcli lint --bulk '@queries/**/*.sql' --format markdown
+dbcli --use staging lint @analytics/live-summary --min-severity warn
+```
+
+| Option | Default | Meaning |
+|---|---|---|
+| `--format <text\|json\|markdown>` | `text` | Render one report per resolved input. |
+| `--min-severity <info\|warn\|error>` | `info` | Omit findings below the selected severity. |
+| `--no-schema` | off | Skip schema-only checks without reading schema-cache paths; static `NOT IN` NULL checks still run. |
+| `--bulk <input>` | none | Resolve a comma-separated list of `@file`, `@glob`, or `@saved-query` inputs. |
+| `--recovery` | off | On command failure, emit and save a linked `RecoveryEnvelope`. |
+| global `--use <conn>` | configured default | Select a v2 named connection and its isolated cache; place it before `lint`: `dbcli --use <conn> lint …`. |
+
+**Rules:**
+
+| Rule | Severity | What it reports |
+|---|---|---|
+| `select-star` | warn | A top-level `SELECT *`; when one table and its cached columns are unambiguous, the finding may include a column-list rewrite draft. |
+| `unanchored-like` | warn | A `LIKE` / `ILIKE` pattern beginning with `%`, which a conventional B-tree index cannot anchor. |
+| `missing-limit-offset` | info | Deep pagination with `OFFSET >= 1000`; prefer keyset pagination. |
+| `non-sargable-where` | warn | A function or arithmetic expression applied to the column side of a predicate. |
+| `or-to-union` | info | A top-level `OR` across different columns that can complicate index selection; any UNION alternative must preserve identity and multiplicity. |
+| `subquery-to-join` | info | `IN (SELECT …)` where an equivalent `EXISTS`, or a JOIN with proven uniqueness/deduplication, may plan better. |
+| `distinct-groupby-abuse` | warn | Redundant `DISTINCT` when simple projected columns exactly cover the `GROUP BY` columns. |
+| `implicit-cast` | warn | A schema-verified column/literal type mismatch that can disable index use; safe, unambiguous numeric drafts may be included. |
+| `not-in-nullable` | warn | A right-hand `NOT IN` value that can be NULL: explicit `NULL`, outer-join null extension, a nullable subquery projection, or a known nullable CASE/cast/aggregate expression. A nullable left-hand column is not this rule. |
+
+`implicit-cast` and the schema-enriched portion of `not-in-nullable` read the
+selected cache through the schema loader abstraction. Static `not-in-nullable`
+checks still run without it. All schema caches live beneath `.dbcli/schemas/`. A v2
+configuration always uses `.dbcli/schemas/<resolved-connection>/`, including
+the configured default. The root `.dbcli/schemas/` directory is only the
+v1/legacy unnamed cache. Global `dbcli --use <conn> lint …` selects another
+named v2 slot. The command never refreshes the cache and never falls back to
+schema embedded in config.
+
+Skipped rules are returned with machine-readable `blocked:` reasons:
+
+- Invalid SQL blocks all nine rules with `blocked: parse failed` and includes
+  `parseError`.
+- `--no-schema` blocks `implicit-cast` and the schema-dependent portion of
+  `not-in-nullable` with `blocked: --no-schema`; static RHS hazards still run.
+- A missing layered cache records
+  `blocked: schema cache unavailable (run dbcli schema)` for those unavailable
+  schema checks while retaining static RHS findings.
+
+Every finding includes its rule, severity, source span, message, and
+`schemaVerified` state. Some findings also carry a confidence-labelled rewrite
+draft and a shell-safe verification command. It uses
+`dbcli explain --analyze` only when the statement is structurally proven read-only;
+function-bearing and session-assignment statements are unproven, so lint
+falls back to plain `dbcli explain`. These are suggestions only: `lint` neither
+executes the verification command nor changes the query.
+
+When schema identifiers collide after case folding, schema-aware findings and
+rewrites are withheld. The SQL parser does not preserve reliable quote
+provenance, so an exact-looking mixed-case AST identifier cannot disambiguate
+that collision. CTE, derived, schema-qualified, and database-qualified
+relations also never borrow facts from the unqualified cache.
+
+For `not-in-nullable`, remove or filter right-hand NULL values. In a subquery,
+filter the projected value with `IS NOT NULL`; `NOT EXISTS` may be a better
+semantic form when appropriate. dbcli does not automatically rewrite this case
+unless correlation, type classification, qualified-column resolution, and the
+rewrite target are all unambiguous. A direct or `AND`-conjoined `IS NOT NULL`
+filter on the exact projected expression suppresses the finding; aggregates
+apply the same proof in `HAVING`. Filters under `OR` or ambiguous expression
+matches do not. The rule recursively checks projection, JOIN `ON`, `WHERE`, and
+`HAVING` expressions, using each nested SELECT/CTE/derived statement's own
+scope. Qualified outer-join null extension remains detectable without a cache,
+but a join's synthetic NULL row is not applied inside that join's own `ON`;
+declared nullability and completed earlier joins still apply there.
+
+Trimmed JSON example:
+
+```json
+[
+  {
+    "sql": "SELECT * FROM users",
+    "dialect": "postgresql",
+    "findings": [
+      {
+        "rule": "select-star",
+        "severity": "warn",
+        "message": "SELECT * fetches every column; list the columns you need.",
+        "span": { "start": 0, "end": 8 },
+        "schemaVerified": false
+      }
+    ],
+    "skippedRules": [],
+    "relatedCommands": [
+      "dbcli guide missing-index-for \"SELECT * FROM users\"",
+      "dbcli explain --analyze \"SELECT * FROM users\""
+    ]
+  }
+]
+```
+
 ### plan
 
 Static SQL risk analyzer. Classifies a statement into the same permission tiers
@@ -583,6 +709,147 @@ dbcli diff --against before.json --format json
 **Options:** `--snapshot <path>`, `--against <path>`, `--format <json|table>`
 **Permission:** query-only+
 
+#### `diff --against-orm`
+
+Compare an ORM definition with the local SQL schema cache. This mode reads
+`config.schema`; it does not open a database connection, refresh the cache, or
+execute a proposal. An empty cache fails with
+`Schema cache is empty. Run 'dbcli schema' first.` Snapshot mode remains a
+separate `--snapshot` / `--against` workflow.
+
+```bash
+# Prisma and normalized JSON accept exactly one file
+dbcli diff --against-orm prisma/schema.prisma --format json
+dbcli diff --against-orm schema.normalized.json --orm-format json --format table
+
+# DDL accepts repeatable or comma-separated paths and real filesystem globs
+dbcli diff --against-orm "migrations/*.sql" --format markdown
+dbcli diff --against-orm migrations/base.sql,migrations/accounts.sql \
+  --against-orm migrations/orders.sql --orm-format ddl --format json
+
+# Ignore patterns are comma-separated and match qualified table identity
+dbcli diff --against-orm prisma/schema.prisma --ignore 'public.audit_*,public.Legacy'
+```
+
+| Option | Behavior |
+| :--- | :--- |
+| `--against-orm <paths>` | Repeatable or comma-separated input. DDL inputs support real filesystem globs; matches are deduplicated and put in deterministic path order, then parsed as one shared ordered context so an index in a later file can attach to a table declared in an earlier file. Prisma and normalized JSON accept exactly one file, and globs are rejected for those formats. |
+| `--orm-format prisma\|ddl\|json` | Override extension/content detection. Without it, dbcli detects Prisma, DDL, or normalized JSON from the path and content. |
+| `--ignore <globs>` | Comma-separated, case-sensitive table globs. Patterns match the qualified display identity (for example `public.Users`). `_prisma_migrations` is always unmanaged. |
+| `--format json\|table\|markdown` | Select machine JSON, human table, or Markdown output. Markdown is available only in ORM drift mode. |
+| `--recovery` | On an I/O, configuration, empty-cache, invalid-format, or unsupported-engine failure, emit and save a structured recovery envelope. Invalid Prisma/DDL constructs normally become `unparsed` entries instead of throwing. |
+
+The command supports PostgreSQL, MySQL, and MariaDB configurations. Only
+error-level **scored drift** determines the report's drift exit code: one or more
+scored errors exits `1`; warnings, infos, `unmanaged`, or `unparsed` entries alone
+exit `0`. Command/configuration failures independently exit code `1`. The four
+drift categories and tolerance rules are:
+
+| Category | Severity and comparison rule |
+| :--- | :--- |
+| `missing_in_db` | `error` — a table, column, or index exists in the ORM definition but not in the cached DB schema. |
+| `missing_in_orm` | `warn` — a table, column, or index exists in the cached DB schema but not in the ORM definition. |
+| `mismatch` | `error` when the type family or nullability differs; `info` for same-family type spelling, default, or primary-key differences. |
+| `unmanaged` | `info`, excluded from error/warn scoring — the table matched the built-in or user `--ignore` patterns. |
+
+Type-family tolerance deliberately treats engine spellings such as `text` and
+`varchar(191)` as the same family: the spelling difference is still visible as
+`info`, while an integer/text family difference is an `error`. Indexes compare
+by structural index signatures — ordered, case-folded column names plus
+uniqueness — rather than by engine-specific index names. Duplicate signatures
+are emitted once. Drift entries sort deterministically by table, object, category,
+and detail using Unicode code-point order, never locale-dependent collation.
+
+**Schema and table identity.** Storage preserves exact, case-sensitive schema
+and table names from the database catalog. Exact, case-sensitive `(schema, table)`
+tuples are the comparison key, so PostgreSQL `users` and `"Users"` can coexist.
+DDL resolution rules: unquoted SQL identifiers fold to lowercase; quoted identifiers match exactly.
+For example, unquoted `Users` resolves to `users`, and quoted
+`"Users"` resolves only to `Users`. Quote state comes from the parsed identifier representation;
+dbcli never infers it from display text, catalog spelling, or a
+Prisma mapping. Qualified components resolve independently, and unqualified ORM
+identities use the cached DB default schema when one is known. Qualified display
+names and `--ignore` matching remain case-sensitive. Duplicate exact or
+duplicate resolved table identities fail closed instead of overwriting one
+another.
+
+**Prisma subset.** The parser supports `model` blocks; scalar `String`, `Int`,
+`BigInt`, `Float`, `Decimal`, `Boolean`, `DateTime`, `Json`, and `Bytes` fields;
+`?`; relation-side `[]`; `@id`, `@unique`, `@default(...)`, `@map("...")`,
+`@@map("...")`, `@@index([...])`, `@@unique([...])`; relations with
+`fields` / `references`; and the validated native mappings `@db.Text`,
+`@db.VarChar(n)`, `@db.Uuid`, `@db.Timestamptz([precision])`, `@db.Date`,
+`@db.SmallInt`, and `@db.JsonB`. Views, composite types, enums used as scalar
+columns, multi-schema datasource configuration, malformed declarations, unknown
+attributes, and unsupported native mappings are never guessed.
+
+Prisma and DDL constructs outside the supported subset are retained in
+`unparsed` with a `blocked:` reason. These entries are separate from scored drift:
+inspect and resolve them before treating an otherwise clean summary as complete.
+Multi-file DDL is consumed as one deterministic shared ordered statement context,
+so later `CREATE INDEX` statements can reference tables declared in earlier
+files. PostgreSQL `PARTITION BY` and MySQL/MariaDB table engine, charset, and
+other `CREATE TABLE` table options are unsupported: the construct produces a
+`blocked:` `unparsed` entry and does not emit a managed ORM table.
+The normalized JSON escape hatch is Zod-validated and uses an array of tables
+with explicit exact `identity` objects; optional parsed identifiers must include
+their `quoted` flags, and every normalized JSON `unparsed.reason` must start with
+`blocked:`.
+
+```json
+{
+  "ormSource": "prisma",
+  "entries": [
+    {
+      "category": "missing_in_db",
+      "severity": "error",
+      "table": "public.users",
+      "object": "email",
+      "detail": "column 'email' (text) is defined in prisma but absent in the database",
+      "proposedCommands": [
+        "# escalate: schema-qualified table 'public.users' is not losslessly representable by dbcli migrate — run: dbcli skill tasks plan migration-review"
+      ]
+    }
+  ],
+  "unparsed": [],
+  "summary": { "errors": 1, "warns": 0, "infos": 0, "unmanaged": 0 }
+}
+```
+
+Missing unqualified columns and indexes may receive shell-safe, dry-run-by-default
+`dbcli migrate add-column` or `add-index` proposal strings. Simple arguments stay
+unquoted; unsafe shell characters are POSIX single-quoted. Table creation,
+removal, mismatch, and DB-only drift escalate to `migration-review`. A
+schema-qualified target, or index columns that the current `migrate --columns`
+CLI cannot represent losslessly, also escalates instead of emitting a corrupt
+command. Any table, column, or type positional beginning with `-` also escalates
+so Commander cannot reinterpret it as an option. A leading-dash option value is
+rendered with option-safe attached syntax, for example `--default=-1` or
+`--columns=--config,email`. Proposals are text only and never add `--execute`.
+
+For a guided, cache-refreshing review, use the built-in `orm-drift-review` pack:
+
+```bash
+dbcli skill tasks plan orm-drift-review \
+  --param orm_path=prisma/schema.prisma \
+  --format json
+```
+
+The plan is `blacklist list` → `schema --format json` →
+`diff --against-orm ... --format json`. Run any proposed `migrate` command in its
+default dry-run mode, capture the emitted DDL, confirm its exact target, and pass
+both values to the separate migration review:
+
+```sh
+dbcli skill tasks plan migration-review \
+  --param "table=${exact_table}" \
+  --param "ddl=${captured_ddl}"
+```
+
+Both parameters are required. Keep each expansion as one quoted shell argument;
+never use `eval`, and consider `--execute` only after the plan and captured DDL
+have been reviewed.
+
 ### snapshot
 
 Capture a **result fingerprint** of a query (not schema): `rowCount` plus per-column
@@ -1713,8 +1980,12 @@ a read-only (`plan-only`) pack taking a required `table` parameter that walks
 in recent audit activity. Additional read-only packs ship for common agent
 workflows: `audit-permissions` (permission/blacklist audit), `safe-backfill`
 (plan a write with blacklist+schema+risk checks), `schema-drift-review` (cached
-vs live schema diff), and `connection-health` (reachability/config/capacity
-triage). Run `dbcli skill tasks list` for the full set.
+vs live schema diff), `orm-drift-review` (ORM definition vs cached DB schema),
+and `connection-health` (reachability/config/capacity
+triage). **MongoDB packs:** `mongo-safe-backfill` (dry-run–previewed backfill)
+and `mongo-schema-drift-review` (sampled dot-path drift, with a `sample_size` knob
+to damp sampling noise); filter them with `dbcli skill tasks list --engine mongodb`.
+Run `dbcli skill tasks list` for the full set.
 
 ```bash
 dbcli skill tasks plan analyze-table-perf --param table=betting_logs --format json