@carllee1983/dbcli 1.6.0 → 1.8.0

package/CHANGELOG.md

@@ -5,6 +5,36 @@ All notable changes to dbcli are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.8.0] - 2026-05-06
+
+### Added
+
+- **Redis 與 Elasticsearch 支援**：`init`、`list`、`schema`、`query`、`status`、`use`、`doctor`、`upgrade`、`completion` 在兩個系統皆真實可用。
+  - Redis：`list` 透過 SCAN 取 keys；`schema <key>` 顯示 type/TTL/size/sample；`query` 執行白名單 Redis 指令並走原本權限與黑名單檢查。
+  - Elasticsearch：`list` 顯示 indices 與文件數；`schema [index]` 攤平 mapping、揭露 `.fields` multi-fields；`query` 接受 DSL JSON 或 Lucene 字串。
+- 文件：`assets/SKILL.md` 與 `assets/reference.md` 同步加入 Redis / Elasticsearch 章節。
+
+### Fixed
+
+- **`insert` / `update` / `delete` / `export` / `diff` 對 Redis / Elasticsearch 的早期錯誤訊息**：先前會落入 SQL DataExecutor 出現「Column ... not found in table」之類誤導訊息，現在直接回傳明確的「不支援」JSON，並指引正確替代路徑（Redis 改用 `query`、Elasticsearch 改用外部工具或 `query --index`）。
+- **TypeScript 嚴格度**：`bun run typecheck` 從 43 個錯誤降為 0。
+  - `ConnectionConfig` union 加入 `ElasticsearchConnectionConfig`。
+  - `ResolvedConnection.connection.system`、`ReplContext.system` 涵蓋 `'elasticsearch'`。
+  - `ExecutionResult` 補上 optional `rowCount` / `columnNames`。
+  - `getDefaultsForSystem` 涵蓋 redis (6379) / elasticsearch (9200) 預設值。
+
+## [1.7.0] - 2026-05-04
+
+### Added
+
+- `dbcli q @<name>` 執行已保存的參數化 SELECT 片段
+- `dbcli queries list/show/new/edit/check` 管理片段
+- 兩層片段儲存：`.dbcli-shared/queries/`（共享）+ `.dbcli/queries/`（個人覆蓋）
+- 完整安全 invariants：拒絕非 SELECT/WITH、多語句、`${...}` / `{{...}}` 模板語法
+- 子查詢式 size guard 包裹 (`SELECT * FROM (...) AS _dbcli_guard LIMIT 1000`)
+- 內建 YAML 子集 frontmatter parser（無新增 npm 依賴）
+- `queries list/show --format json` 為未來 MCP server 預留契約
+
 ## [1.6.0] - 2026-04-23
 
 ### Added

package/README.md

@@ -114,6 +114,8 @@ dbcli query '{"status":"active"}' --collection users --use atlas
 
 For MongoDB, `list` and `query` operate on the database configured for the connection, and `query` requires `--collection <name>`.
 
+For a command-by-command support matrix across PostgreSQL, MySQL, MariaDB, and MongoDB, see [docs/feature-matrix.md](./docs/feature-matrix.md).
+
 ---
 
 ## Multi-connection Support (v2)
@@ -769,6 +771,43 @@ dbcli migrate drop-enum status --execute --force
 
 ---
 
+## Query Risk Planning
+
+Use `plan` to inspect SQL safety before execution. It reads local dbcli config, permissions, blacklist rules, and cached schema metadata only; it does not connect to the database.
+
+```bash
+dbcli plan "UPDATE users SET status='inactive'" --format json
+```
+
+Decisions are:
+
+- `ALLOW` — no obvious risk was detected.
+- `WARN` — inspect warnings before executing.
+- `BLOCK` — unsafe, unsupported, or violates configured safety constraints.
+
+Text output is concise for humans:
+
+```text
+Decision: BLOCK
+Operation: UPDATE
+Target tables: users
+
+Risk factors:
+- UPDATE statement has no WHERE clause.
+
+Recommendations:
+- Add a WHERE clause.
+- Use --dry-run on the actual write command.
+```
+
+JSON output includes `suggestedCommands` for agents:
+
+```bash
+dbcli plan "SELECT id FROM users WHERE id = 1 LIMIT 1" --format json
+```
+
+---
+
 ## Global Options
 
 All commands support these global options:
@@ -957,6 +996,22 @@ A Query-only agent cannot write to any table, and also cannot read blacklisted t
 
 ---
 
+## Saved queries
+
+Save parameterised SELECT snippets and re-run them by name:
+
+```bash
+dbcli queries list
+dbcli queries show @dau
+dbcli q @dau --param days=30 --format json
+```
+
+Snippets live in `.dbcli-shared/queries/` (committed) or `.dbcli/queries/`
+(gitignored, personal override). Each `.sql` file declares its frontmatter
+in a `-- ---` block. Read `assets/reference.md` for the full schema.
+
+---
+
 ## AI Integration Guide
 
 dbcli ships AI-consumable skill files (`assets/SKILL.md` and `assets/reference.md`) and can copy them into your favorite AI tool directories.
@@ -1258,12 +1313,15 @@ chmod +x dist/cli.mjs
 
 ```bash
 bun test                  # full test suite (Bun test runner)
+bun run typecheck         # TypeScript compile-time validation
 bun run test:unit         # unit + core tests only
 bun run test:integration  # integration tests
 bun run test:docker       # integration tests with docker-compose.test.yml (MySQL + PostgreSQL)
 bun run build             # bundle CLI to dist/ (used before publish)
 ```
 
+CI treats `bun run typecheck` and `bun test` as the required pass/fail validation gate on every push and pull request. Lint, build, smoke checks, and benchmarks run in addition to that gate.
+
 Live database integration tests use `.dbcli/config.json` by default. If your live
 config lives elsewhere, set `LIVE_DB_CONFIG_PATH=/path/to/.dbcli` before running:
 

package/README.zh-TW.md

@@ -779,6 +779,43 @@ dbcli migrate drop-enum status --execute --force
 
 ---
 
+## 查詢風險規劃
+
+使用 `plan` 在執行前檢查 SQL 安全性。它只讀取本機 dbcli 設定、權限、黑名單規則與已快取的 schema metadata；不會連線到資料庫。
+
+```bash
+dbcli plan "UPDATE users SET status='inactive'" --format json
+```
+
+決策結果：
+
+- `ALLOW` — 未偵測到明顯風險。
+- `WARN` — 執行前應檢查警告。
+- `BLOCK` — 不安全、不支援，或違反安全設定。
+
+文字輸出保持精簡，適合人工閱讀：
+
+```text
+Decision: BLOCK
+Operation: UPDATE
+Target tables: users
+
+Risk factors:
+- UPDATE statement has no WHERE clause.
+
+Recommendations:
+- Add a WHERE clause.
+- Use --dry-run on the actual write command.
+```
+
+JSON 輸出會包含給 AI agent 使用的 `suggestedCommands`：
+
+```bash
+dbcli plan "SELECT id FROM users WHERE id = 1 LIMIT 1" --format json
+```
+
+---
+
 ## 全域選項
 
 所有指令皆支援下列全域選項：
@@ -963,6 +1000,23 @@ Query-only 代理無法寫入任何表，也無法讀取黑名單表或欄位
 
 ---
 
+## 已保存查詢片段（Saved queries）
+
+將參數化的 SELECT 片段保存於版控，並依名稱重複執行：
+
+```bash
+dbcli queries list
+dbcli queries show @dau
+dbcli q @dau --param days=30 --format json
+```
+
+片段位於兩層儲存：`.dbcli-shared/queries/`（commit 進版控、團隊共享）與
+`.dbcli/queries/`（gitignore、個人覆蓋）。每個 `.sql` 檔以 `-- ---` 區塊
+宣告 frontmatter（name、description、engine、params、tags），完整 schema 見
+`assets/reference.md`。
+
+---
+
 ## AI 整合指南
 
 dbcli 內建供 AI 使用的 skill 文件（`assets/SKILL.md` 與 `assets/reference.md`），並可複製到常見 AI 開發工具的目錄。

package/assets/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: dbcli
-description: Database CLI for AI agents with permission-based access control. Use to query, inspect schemas, insert/update/delete, export results, and blacklist sensitive columns/tables. Supports MySQL, PostgreSQL, MariaDB, and MongoDB with multiple named connections per project and custom env files. Trigger when working with databases, running SQL or MongoDB JSON queries, exploring table/collection structures, switching database environments, or protecting sensitive data from AI access. For exhaustive flags and examples, read the sibling `reference.md`.
+description: Database CLI for AI agents with permission-based access control. Use to query, inspect schemas, insert/update/delete, export results, and blacklist sensitive columns/tables. Supports MySQL, PostgreSQL, MariaDB, MongoDB, Redis, and Elasticsearch with multiple named connections per project and custom env files. Trigger when working with databases, running SQL / MongoDB JSON / Redis commands / Elasticsearch DSL, exploring table/collection/key/index structures, switching database environments, or protecting sensitive data from AI access. For exhaustive flags and examples, read the sibling `reference.md`.
 ---
 
 # dbcli
@@ -33,21 +33,21 @@ dbcli query "SELECT * FROM users"   # Execute SQL (auto LIMIT 1000)
 |---------|-----------------|---------|
 | `init` | n/a | Create `.dbcli` (v1 single or v2 multi via `--conn-name` / `--env-file`). **Usually run by the human** — do NOT re-run to strip `{"$env"}` references; that format is intentional. |
 | `use` | n/a | Show/switch default named connection (v2 only). |
-| `list` | query-only+ | Tables (SQL) or collections (MongoDB). |
-| `schema` | query-only+ | Per-table or full scan into `.dbcli/schemas/`; use `--use` for the correct connection cache. |
-| `query` | query-only+ | SQL, or Mongo JSON filter / pipeline with `--collection`. |
-| `insert` / `update` | read-write+ | JSON `--data` / `--set`; `--where` required on `update`; `--dry-run` first. |
-| `delete` | data-admin+ | `--where` required; `--dry-run` first. |
-| `export` | query-only+ | Query → CSV/JSON file or stdout. |
+| `list` | query-only+ | Tables (SQL), collections (MongoDB), keys (Redis), or indices (Elasticsearch). |
+| `schema` | query-only+ | SQL: per-table or full scan into `.dbcli/schemas/` (use `--use` for the right cache). MongoDB: sampled. ES: flattened mapping. Redis: per-key only (type/TTL/size). |
+| `query` | query-only+ | SQL, Mongo JSON (`--collection`), Redis command, or ES DSL/Lucene (`--collection`/`--index`). |
+| `insert` / `update` | read-write+ | SQL or MongoDB only. JSON `--data` / `--set`; `--where` required on `update`; `--dry-run` first. Redis writes go through `query`. |
+| `delete` | data-admin+ | SQL or MongoDB only. `--where` required; `--dry-run` first. |
+| `export` | query-only+ | SQL or MongoDB only. Query → CSV/JSON(L) file or stdout. |
 | `blacklist` | n/a | `list` / `table` / `column` subcommands redact sensitive data from query results. |
-| `check` | query-only+ | Table health: nulls, duplicates, orphans, rowCount, size. |
-| `diff` | query-only+ | Save/compare schema snapshots. |
+| `check` | query-only+ | SQL only (best on MySQL/MariaDB). |
+| `diff` | query-only+ | SQL only. Save/compare schema snapshots. |
 | `status` | query-only+ | Safe JSON/text summary (no credentials). |
-| `doctor` | n/a | Environment, config, connection, SRV diagnostics (Mongo), schema cache age. |
+| `doctor` | n/a | Environment, config, connection, SRV diagnostics (Mongo), schema cache age; ES has a dedicated path. |
 | `completion` | n/a | bash / zsh / fish scripts. |
 | `upgrade` | n/a | Self-update from npm; 24h-cached version hints on every command. |
-| `shell` | (same as query+) | Interactive REPL. |
-| `migrate` | admin | **DDL; dry-run by default** — needs `--execute`; DROP also needs `--force`. |
+| `shell` | (same as query+) | Interactive REPL. SQL engines + MongoDB shell only; Redis/ES not currently exposed in REPL. |
+| `migrate` | admin | SQL only. **DDL; dry-run by default** — needs `--execute`; DROP also needs `--force`. Mongo/Redis exit with error. |
 
 `--use <name>` on any subcommand targets a v2 connection without changing the default.
 
@@ -69,11 +69,75 @@ dbcli query "SELECT * FROM users"   # Execute SQL (auto LIMIT 1000)
 ## MongoDB
 
 - JSON filter object (`find`) or JSON array (`aggregate`); SQL is rejected. `--collection <name>` is required on `query`.
-- **Supported:** `init`, `list`, `query`, `status`, `use`, `shell`, `doctor`, `upgrade`, `completion`.
-- **Not supported:** `schema`, `insert`, `update`, `delete`, `export`, `diff`, `migrate`, `check`.
-- No auto-limit on MongoDB queries — use `$limit` in the pipeline if needed.
+- **Supported:** `init`, `list`, `schema` (sampled), `query`, `insert`, `update`, `delete`, `export`, `status`, `use`, `shell`, `doctor`, `upgrade`, `completion`.
+- **Not supported:** `q` (saved queries), `diff`, `migrate`, `check`.
+- Schema is **sampled** (default 50 docs, `--sample-size`); types are JS `typeof` strings — no PK/FK/index info.
+- `--limit` applies on `find`/aggregate; query-only mode caps at 1000 unless `--no-limit`.
 - See reference.md MongoDB section for full syntax and examples.
 
+## Redis
+
+- Command-style execution; `query` runs a whitelisted Redis command (e.g. `GET`, `HSET`, `DEL`).
+- **Supported:** `init`, `list` (keys via SCAN), `schema <key>` (type / TTL / size / sample), `query`, `status`, `use`, `doctor`, `upgrade`, `completion`.
+- **Not supported:** `schema` full scan / `--refresh` / `--reset`, `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `q`.
+  Use `query "DEL <key>"` etc. for writes — they go through the same permission gate.
+- Permission tiers map to commands: read commands → `query-only`; mutators (`SET`, `HSET`, `EXPIRE`, …) → `read-write`; `DEL` / `HDEL` / `UNLINK` → `data-admin`; `KEYS`, `FLUSHDB`, `CONFIG`, `INFO`, … → `admin`. Unlisted commands are denied.
+- `database` field is the logical DB index (default `0`); `list` returns ≤ 100 000 keys via SCAN.
+- See reference.md Redis section.
+
+## Elasticsearch
+
+- DSL (JSON body) or Lucene query string; `--collection <index>` (or `--index <index>`) is required on `query`.
+- **Supported:** `init`, `list` (indices with doc count), `schema [index]` (flattened mapping), `query`, `status`, `use`, `doctor`, `upgrade`, `completion`.
+- **Not supported:** `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `q`.
+  Writes are not exposed via dedicated subcommands yet — use external tooling for indexing.
+- Auth: API key (`apiKey`), Basic (`user`/`password`), or Cloud ID (`cloudId`); supports HTTPS, custom CA (`caPath`), and multi-node arrays (`nodes`).
+- Query-only mode caps at 1000 hits; `--no-limit` is bounded at 10 000 (use saved searches with `search_after` beyond that).
+- Schema flattens nested fields (`a.b.c`) and surfaces `.fields` multi-fields (e.g. `text.keyword`).
+- See reference.md Elasticsearch section.
+
+## Saved queries
+
+Run reusable parameterised SELECT snippets stored in your repo.
+
+| Step | Command |
+|------|---------|
+| 1. Discover | `dbcli queries list` |
+| 2. Inspect | `dbcli queries show @<name>` |
+| 3. Run     | `dbcli q @<name> --param k=v` |
+
+Snippets resolve from three layers, **local > shared > builtin** (local wins):
+- `builtin` — bundled with dbcli (e.g. `@diag/*`); read-only at runtime
+- `.dbcli-shared/queries/` — committed, team-shared
+- `.dbcli/queries/` — gitignored, personal override
+
+Manage local snippets with `queries new | edit | delete | rename | copy | import | export`
+(see reference.md). Use `copy` / `import` to fork a builtin or shared snippet into the
+local layer for editing.
+
+Each `.sql` file may declare YAML frontmatter inside `-- ---` blocks
+(name, description, engine, params, tags). See `dbcli queries show @<name> --format json`
+for the machine-readable contract.
+
+### Built-in diagnostic snippets
+
+dbcli ships ready-made diagnostic queries. Run with `dbcli q @diag/<topic>`:
+
+| key                     | purpose                                  |
+|-------------------------|------------------------------------------|
+| `@diag/connections`     | active sessions                          |
+| `@diag/long-running`    | queries above `min_seconds` (default 30) |
+| `@diag/table-sizes`     | table data/index size with row counts    |
+| `@diag/index-usage`     | indexes by scan count                    |
+| `@diag/missing-indexes` | tables dominated by sequential scans     |
+| `@diag/locks`           | lock-wait chains                         |
+| `@diag/db-size`         | database size summary                    |
+| `@diag/cache-hit`       | buffer cache hit ratios                  |
+
+Engine variants are picked automatically based on the active connection.
+Override any of them by placing a same-named file under `.dbcli-shared/queries/`
+or `.dbcli/queries/`.
+
 ## Common workflows
 
 - **Debug odd state:** `schema` → `check` → `query` with tight `WHERE` → follow FKs from schema JSON. Evidence over theory.