@carllee1983/dbcli 1.41.0 → 1.43.0

package/skills/dbcli/reference.md

@@ -112,7 +112,7 @@ dbcli schema --use prod             # Scan prod DB; saves to .dbcli/schemas/prod
 
 ### query
 
-Execute SQL query (MySQL/PostgreSQL/MariaDB) or JSON filter/pipeline (MongoDB).
+Execute a SQL statement, MongoDB filter/pipeline, allow-listed Redis command, or Elasticsearch DSL/Lucene query.
 
 ```bash
 # SQL databases
@@ -143,13 +143,84 @@ dbcli query "SELECT day, dau FROM dau_daily" --ui                # open in brows
 dbcli query "SELECT * FROM orders" --format html > orders.html   # pipe to stdout
 ```
 
-**Options:** `--format <table|json|csv|html>`, `--ui` (open the dashboard in the system browser; implies `--format html`), `--limit <number>`, `--no-limit`, `--collection <name>` (MongoDB / Elasticsearch), `--index <name>` (Elasticsearch alias for `--collection`), `--recovery`
+**Options:** `--format <table|json|csv|html>`, `--ui` (open the dashboard in the system browser; implies `--format html`), `--limit <number>`, `--no-limit`, `--collection <name>` (MongoDB / Elasticsearch), `--index <name>` (Elasticsearch alias for `--collection`), `--fields <list>`, `--truncate <number>` / `--no-truncate`, `-f, --query-file <path>`, `--use <name[,name]>`, `--recovery`
 **Permission:** query-only+ (Redis: per-command; Elasticsearch: per HTTP method/path)
 
+#### Field projection (`--fields`)
+
+```bash
+dbcli query "SELECT * FROM bet_log" --fields sn,currency,bet
+dbcli query "SELECT * FROM bet_log" --fields=-raw_response,-created_at   # exclusion
+dbcli query '{"station_code":"cmg9998"}' --collection raw_bet_log --fields sn,bet
+```
+
+Include and exclude forms cannot be mixed. Dotted paths (`user.email`) are supported.
+On MongoDB the selection becomes a driver-level `projection` (find) or a trailing
+`$project` stage (aggregate), so the omitted fields never leave the server; `_id` is
+excluded unless listed explicitly. On SQL the rows are projected after fetch — write
+an explicit column list in the `SELECT` when you also want to cut transfer cost.
+Blacklisted columns stay blacklisted: naming one in `--fields` yields no value and the
+result still carries the blacklist `securityNotification`. A requested field that does
+not exist in the result comes back as `null`.
+
+#### Cell truncation (`--truncate`)
+
+```bash
+dbcli query "SELECT sn, raw_response FROM bet_log"                 # table: 120-char default
+dbcli query "SELECT sn, raw_response FROM bet_log" --truncate 40
+dbcli query "SELECT sn, raw_response FROM bet_log" --no-truncate
+```
+
+Table output truncates each serialized cell at 120 Unicode code points by default and
+appends `…(+N chars)`; counting by code point keeps multi-byte characters and emoji
+intact. `--truncate <n>` sets the width, `--no-truncate` disables it. Explicit truncation
+flags are rejected with JSON, CSV, HTML, and `--ui` output.
+
+#### Query from a file or stdin (`-f`)
+
+```bash
+dbcli query -f report.sql
+dbcli query --collection raw_bet_log -f - <<'EOF'
+[{"$match": {"sn": {"$regex": "^SN0000"}}}, {"$group": {"_id": "$currency", "n": {"$sum": 1}}}]
+EOF
+```
+
+Avoids shell quoting entirely — the usual reason a Mongo pipeline containing `$regex`
+or nested date objects fails. Supplying both `--query-file` and positional query text
+is an error rather than a silent choice, and an empty file or empty stdin is refused.
+`-f -` requires piped input: on an interactive terminal dbcli refuses immediately
+instead of waiting silently for input that is never coming.
+
+#### One-shot connection selection and read-only fan-out
+
+Selection precedence is explicit `--use`, then `DBCLI_CONNECTION`, then the saved default.
+`query`, `schema`, `list`, `export`, and `check` accept command-level `--use`; for other
+commands use root-level `dbcli --use <name> <command>`. One-shot selectors never update the
+saved default and require a v2 config. A legacy v1 single-connection config rejects them
+instead of silently running its only connection.
+
+An explicit comma-separated `--use primary,staging` fans one query out to several named
+connections. `DBCLI_CONNECTION` always names one literal connection and never enables
+fan-out. SQL permits `SELECT`, `SHOW`, `DESCRIBE`, and `EXPLAIN`; MongoDB permits filters and
+read-only pipelines without top-level `$out` / `$merge`; Elasticsearch permits searches.
+Redis, writes, `--recovery`, `--ui`, CSV, and HTML are rejected before execution. Each
+connection keeps its own blacklist, limit metadata, audit entry, and error. Aggregate exit
+codes are `0` when all succeed, `2` for mixed outcomes, and `1` when all fail or preflight
+rejects the request.
+
+#### Truncation is stated, not implied
+
+When the query-only auto-limit trims the result, the table footer reads
+`Rows: 1000 (truncated; limit 1000)`, `--format json` carries
+`metadata.truncated` and `metadata.limit_applied`, and CSV appends a `#` comment line.
+dbcli fetches one row past the cap to decide this, so a result of exactly 1000 rows is
+reported as `truncated: false` — never infer truncation from a round row count.
+
 > **MongoDB notes:**
 > - SQL syntax is rejected — use JSON object (filter) or JSON array (pipeline)
 > - `--collection <name>` is required
-> - Auto-limit does not apply; use `$limit` in your pipeline if needed
+> - Query-only auto-limit applies to filters and to pipelines without their own
+>   `$limit`; the applied cap and truncation are reported in the result metadata
 
 > **Redis notes:**
 > - The first token must be an allow-listed command (`GET`/`SET`/`HGET`/`HSET`/`DEL`/...). Unknown commands are refused.
@@ -661,9 +732,13 @@ dbcli export orders --no-limit --format jsonl             # scroll the whole ind
 **Options:** `--format <json|jsonl|csv|html>` (required), `--output <path>`, `--force`, `--recovery`, `--collection <name>` (MongoDB collection) / `--index <name>` (Elasticsearch index; alias for `--collection`), `--limit <number>` (overrides auto-limit), `--no-limit` (Elasticsearch full-index scroll)
 **Permission:** query-only+ — SQL, MongoDB, and **(v1.22)** Elasticsearch.
 
+If the query-only auto-limit would omit rows, export fails closed with exit code `1` and
+writes no partial file. Re-run with `--no-limit` to export everything, or `--limit N` to
+accept a bounded export explicitly. This applies to SQL, MongoDB, and Elasticsearch.
+
 The `html` format emits the same self-contained dashboard as `query --ui` (see [Interactive HTML dashboard](#interactive-html-dashboard)). Because `export` runs raw SQL (no snippet metadata), the HTML report is always rendered as a sortable / filterable table — no KPIs or charts. Use `dbcli q @<name> --format html` (or `--ui`) for the charted view.
 
-> **Elasticsearch export (v1.22):** pass a search DSL with `--index <index>` to export the hits, or pass an index name as the query to scroll the whole index via `match_all`. Default cap is 1000 rows; `--no-limit` streams the full index via scroll in batches. Index-level blacklist is checked before export and an audit record is written.
+> **Elasticsearch export (v1.22):** pass a search DSL with `--index <index>` to export the hits, or pass an index name as the query to scroll the whole index via `match_all`. Default cap is 1000 rows; reaching it fails closed unless the caller explicitly uses `--limit N`, while `--no-limit` streams the full index via scroll in batches. Index-level blacklist is checked before export and an audit record is written.
 
 ### blacklist
 
@@ -722,6 +797,10 @@ separate `--snapshot` / `--against` workflow.
 dbcli diff --against-orm prisma/schema.prisma --format json
 dbcli diff --against-orm schema.normalized.json --orm-format json --format table
 
+# Drizzle requires a PostgreSQL drizzle-kit v7 snapshot (generate it first)
+drizzle-kit generate
+dbcli diff --against-orm drizzle/meta/0001_snapshot.json --orm-format drizzle --format table
+
 # DDL accepts repeatable or comma-separated paths and real filesystem globs
 dbcli diff --against-orm "migrations/*.sql" --format markdown
 dbcli diff --against-orm migrations/base.sql,migrations/accounts.sql \
@@ -731,11 +810,60 @@ dbcli diff --against-orm migrations/base.sql,migrations/accounts.sql \
 dbcli diff --against-orm prisma/schema.prisma --ignore 'public.audit_*,public.Legacy'
 ```
 
+##### TypeORM
+
+TypeORM entities are not parsed directly. `schema:log` prints the SQL that
+`schema:sync` would execute without applying it; `-d` is the required data-source
+path. Generate that DDL, then select the `typeorm` alias so the report is tagged
+`ormSource: typeorm`:
+
+```bash
+bunx typeorm schema:log -d <path/to/datasource> > schema.sql
+dbcli diff --against-orm schema.sql --orm-format typeorm --format table
+```
+
+With `--orm-format typeorm`, `typeorm_metadata` and `migrations` are
+default-ignored and appear as `unmanaged` rather than scored drift. Passing a
+TypeORM `.ts`, `.js`, `.mjs`, or `.cjs` source file is rejected with the
+`schema:log` command to run. See the
+[TypeORM CLI documentation](https://typeorm.io/docs/using-cli) and
+[`SchemaLogCommand`](https://github.com/typeorm/typeorm/blob/master/src/commands/SchemaLogCommand.ts).
+
+##### Sequelize
+
+Sequelize CLI does not provide a universal `db:migrate --dry-run`. Point the
+project's existing Sequelize configuration at an empty scratch database, apply
+the migrations there, and dump definitions without row data:
+
+```bash
+# Configure Sequelize for an empty scratch database first
+bunx sequelize-cli db:migrate
+
+# PostgreSQL scratch database
+pg_dump --schema-only <scratch-database> > schema.sql
+
+# MySQL scratch database
+mysqldump --no-data <database> > schema.sql
+
+dbcli diff --against-orm schema.sql --orm-format sequelize --format json
+```
+
+With `--orm-format sequelize`, `SequelizeMeta` is default-ignored and appears as
+`unmanaged` rather than scored drift. Passing a Sequelize `.ts`, `.js`, `.mjs`,
+or `.cjs` model file is rejected with the scratch-database and schema-only dump
+recipe. See the
+[Sequelize CLI migration command](https://github.com/sequelize/cli/blob/main/src/commands/migrate.js),
+[PostgreSQL `pg_dump`](https://www.postgresql.org/docs/current/app-pgdump.html),
+and [MySQL `mysqldump`](https://dev.mysql.com/doc/refman/8.4/en/mysqldump-definition-data-dumps.html)
+references.
+
 | Option | Behavior |
 | :--- | :--- |
-| `--against-orm <paths>` | Repeatable or comma-separated input. DDL inputs support real filesystem globs; matches are deduplicated and put in deterministic path order, then parsed as one shared ordered context so an index in a later file can attach to a table declared in an earlier file. Prisma and normalized JSON accept exactly one file, and globs are rejected for those formats. |
-| `--orm-format prisma\|ddl\|json` | Override extension/content detection. Without it, dbcli detects Prisma, DDL, or normalized JSON from the path and content. |
-| `--ignore <globs>` | Comma-separated, case-sensitive table globs. Patterns match the qualified display identity (for example `public.Users`). `_prisma_migrations` is always unmanaged. |
+| `--against-orm <paths>` | Repeatable or comma-separated input. DDL-family inputs (raw DDL, TypeORM, and Sequelize) support real filesystem globs; matches are deduplicated and put in deterministic path order, then parsed as one shared ordered context so an index in a later file can attach to a table declared in an earlier file. Prisma, normalized JSON, and Drizzle accept exactly one file, and globs are rejected for those formats. |
+| Drizzle input | Run `drizzle-kit generate`, then pass the PostgreSQL drizzle-kit v7 snapshot at `drizzle/meta/<NNNN>_snapshot.json`. TypeScript ORM schema sources (`.ts` or `.TS`) are rejected with that snapshot-generation hint; dbcli does not parse them directly. |
+| TypeORM / Sequelize input | Generate DDL with the ORM/database tooling, then pass the SQL file with the matching `typeorm` or `sequelize` alias. Entity/model source files are rejected rather than parsed. |
+| `--orm-format prisma\|ddl\|json\|drizzle\|typeorm\|sequelize` | Override extension/content detection. The `typeorm` and `sequelize` aliases use the DDL adapter while preserving the source tag and ORM-specific default ignores. Without an override, dbcli detects Prisma, raw DDL, normalized JSON, or a Drizzle snapshot from the path and content. |
+| `--ignore <globs>` | Comma-separated, case-sensitive table globs. Patterns match the qualified display identity (for example `public.Users`). `_prisma_migrations` is always unmanaged; the TypeORM alias additionally ignores `typeorm_metadata` and `migrations`, and the Sequelize alias additionally ignores `SequelizeMeta`. |
 | `--format json\|table\|markdown` | Select machine JSON, human table, or Markdown output. Markdown is available only in ORM drift mode. |
 | `--recovery` | On an I/O, configuration, empty-cache, invalid-format, or unsupported-engine failure, emit and save a structured recovery envelope. Invalid Prisma/DDL constructs normally become `unparsed` entries instead of throwing. |
 
@@ -783,9 +911,11 @@ another.
 columns, multi-schema datasource configuration, malformed declarations, unknown
 attributes, and unsupported native mappings are never guessed.
 
-Prisma and DDL constructs outside the supported subset are retained in
+Prisma, DDL, and Drizzle constructs outside the supported subset are retained in
 `unparsed` with a `blocked:` reason. These entries are separate from scored drift:
 inspect and resolve them before treating an otherwise clean summary as complete.
+Drizzle enums and other unsupported snapshot constructs therefore appear as blocked
+`unparsed` entries rather than managed tables or columns.
 Multi-file DDL is consumed as one deterministic shared ordered statement context,
 so later `CREATE INDEX` statements can reference tables declared in earlier
 files. PostgreSQL `PARTITION BY` and MySQL/MariaDB table engine, charset, and