@carllee1983/dbcli 1.9.0 → 1.10.0

package/CHANGELOG.md

@@ -5,6 +5,48 @@ All notable changes to dbcli are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.10.0] - 2026-05-08
+
+### Added
+
+- **Saved Queries 擴展至 Elasticsearch 與 Redis**：`dbcli q @<name>` 與 `queries` 子命令現在能依 frontmatter `engine` 自動切換到對應引擎，並走各引擎專屬的安全管線。
+  - **Engine strategy 重構**：runner 透過 `EngineStrategy` 介面分派到 SQL / Elasticsearch / Redis 三個獨立 strategy；既有 SQL 行為以 strategy 形式保留，無行為變更。
+  - **Elasticsearch strategy**：
+    - Frontmatter 接受 `engine: elasticsearch` 與 `index` 欄位；body 必須是合法 JSON，含 `script` 欄位的 query 直接拒絕。
+    - JSON-aware 參數注入：`:name` 僅在 JSON 字串脈絡裡替換，避免破壞語法。
+    - Size guard：自動補 `size` 上限；`aggs` 模式下放行但加註警告，分頁 (`from + size`) 過大時提示。
+  - **Redis strategy**：
+    - 命令白名單（read-only 為主）+ body validation；直接拒絕 unsupported 或寫入命令。
+    - Raw 參數注入：`:name` 直接代入字面量並打印 foot-gun 警告，提醒使用者 saved query 內不可放使用者輸入。
+    - Size guard：對 range / SCAN 命令的 `COUNT` / `LIMIT` 加上保險上限。
+  - **`q` 命令分派**：根據 prepared execution 的 engine family 呼叫對應 adapter，`--dry-run` 依 engine 用對應格式輸出（SQL 維持 SQL、ES 印 JSON body、Redis 印 argv）。
+  - **內建診斷 snippet**：
+    - `assets/snippets/diag/es-cluster-health.elasticsearch.sql` — ES 叢集健康度摘要。
+    - `assets/snippets/diag/redis-key-stats.redis.sql` — Redis key 數量 / type 分佈快照。
+- **整合測試**：新增 ES / Redis end-to-end saved query 測試（依本機是否有 Docker 而 skip，與既有 PG / MySQL 測試一致）。
+
+### Changed
+
+- **Redis 驅動**：改用 Bun 內建 `RedisClient`，移除外部 `ioredis` 依賴。
+- **Elasticsearch adapter**：refactor 並收斂錯誤訊息與 ExecutionResult 形狀，與 SQL / Mongo / Redis 對齊。
+- **文件**：`assets/SKILL.md` 與 `assets/reference.md` 補上 ES / Redis snippet 工作流；`docs/feature-matrix.md` 更新 saved-queries 欄位。
+
+### Fixed
+
+- **`dbcli export`（Redis 分支）**：`result.rowCount` 在 Redis 上可能 undefined 時導致 `tsc --noEmit` 報 TS2322；改為 `result.rowCount ?? result.rows.length ?? 0`，release gate 中的 typecheck 回到 0 錯誤。
+
+## [1.9.1] - 2026-05-07
+
+### Changed
+
+- **Skill 連線設定指引**：`assets/SKILL.md` 加入「Connection setup」章節，補齊 AI agent 協助使用者建立資料庫連線時所需的決策樹與各 engine essentials。
+  - 決策樹：v1 vs v2、credentials 來源（`.env` / env-refs / 明文）、權限 tier、`status` + `doctor` 驗證。
+  - Per-engine essentials：PostgreSQL / MySQL / MariaDB / MongoDB（含 `mongodb+srv://`）/ Redis（`--name` 為 logical DB index）/ Elasticsearch（basic / Cloud ID / API key）。
+  - v2 multi-connection 範例（`--conn-name`、`--env-file`、`use --list`、`--rename`、`--remove`）與 per-connection schema cache 注意事項。
+  - env-refs（`{ "$env": "..." }`）說明，以及「不要用 `--force` 把 env-refs 蓋成明文」的 guard。
+  - 常見陷阱：SRV DNS、URL 中特殊字元編碼、Redis `--name` 限制、Elasticsearch TLS 設定需手動編輯 `.dbcli`。
+  - 同步擴充 frontmatter `description`，加入 `init` / `.dbcli` / auth modes 觸發詞，提升 skill 觸發精準度。
+
 ## [1.9.0] - 2026-05-06
 
 ### Added

package/assets/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: dbcli
-description: Database CLI for AI agents with permission-based access control. Use to query, inspect schemas, insert/update/delete, export results, and blacklist sensitive columns/tables. Supports MySQL, PostgreSQL, MariaDB, MongoDB, Redis, and Elasticsearch with multiple named connections per project and custom env files. Trigger when working with databases, running SQL / MongoDB JSON / Redis commands / Elasticsearch DSL, exploring table/collection/key/index structures, switching database environments, or protecting sensitive data from AI access. For exhaustive flags and examples, read the sibling `reference.md`.
+description: Database CLI for AI agents with permission-based access control. Use to set up new connections, query, inspect schemas, insert/update/delete, export results, and blacklist sensitive columns/tables. Supports MySQL, PostgreSQL, MariaDB, MongoDB, Redis, and Elasticsearch with multiple named connections per project and custom env files. Trigger when configuring a database connection (`.dbcli` / `.env`), choosing between v1 single and v2 multi-connection layouts, picking auth modes (URI, env refs, Cloud ID, API key), running SQL / MongoDB JSON / Redis commands / Elasticsearch DSL, exploring table/collection/key/index structures, switching database environments, or protecting sensitive data from AI access. For exhaustive flags and examples, read the sibling `reference.md`.
 ---
 
 # dbcli
@@ -46,6 +46,107 @@ dbcli schema                        # Scan all tables → .dbcli/schemas/
 dbcli query "SELECT * FROM users"   # Execute SQL (auto LIMIT 1000)
 ```
 
+If `.dbcli` does not yet exist, route through **Connection setup** below before
+touching `schema` / `query`.
+
+## Connection setup (helping the user wire up a database)
+
+When the user asks "how do I connect to X?", "set up dbcli for our staging DB",
+or `doctor` / `status` reports a missing or invalid config, follow this flow.
+
+> **Default to guiding, not running.** `init` writes credentials to disk. Only
+> execute it for the user with explicit permission and confirmed values.
+> If a `.dbcli` already contains `{"$env": "..."}` references, **do not** rerun
+> `init` to "fill them in" — the env-ref form is intentional for CI/multi-env.
+
+### Decision tree (ask before writing)
+
+1. **One DB or many environments?** One → v1 (single connection). Multiple
+   environments / tenants / replicas → v2 (`--conn-name <name>`, optionally
+   `--env-file <path>` per connection).
+2. **Where do credentials live?**
+   - Already in a `.env` (`DATABASE_URL` or `DB_HOST` / `DB_PORT` / `DB_USER` /
+     `DB_PASSWORD` / `DB_NAME` | `DB_DATABASE`) → `init` parses it automatically.
+   - Need to keep secrets out of `.dbcli` (CI/CD, multi-env) → `--use-env-refs`
+     plus `--env-host` / `--env-port` / `--env-user` / `--env-password` / `--env-database`.
+   - Plain values are acceptable → pass `--host` / `--port` / `--user` /
+     `--password` / `--name` (and `--system`).
+3. **What permission tier?** Default to the **lowest** that satisfies the task:
+   `query-only` → `read-write` → `data-admin` → `admin`. Set with `--permission`.
+4. **Verify, never assume.** After init: `dbcli status` (system + permission +
+   blacklist summary, no creds) and `dbcli doctor --format json` (env, config
+   shape, connectivity, schema-cache age, Mongo SRV path).
+
+### Per-engine essentials
+
+```bash
+# PostgreSQL / MySQL / MariaDB (v1, plain values)
+dbcli init --system postgresql --host localhost --port 5432 \
+  --user app --password '<secret>' --name appdb --permission query-only
+
+# Reuse an existing .env (DATABASE_URL=postgresql://user:pw@host:5432/db)
+dbcli init                                                # parses .env in cwd
+
+# MongoDB — full URI (Atlas / replica sets / authSource)
+dbcli init --system mongodb \
+  --uri "mongodb+srv://user:pw@cluster.example.mongodb.net/mydb?authSource=admin"
+# MongoDB — discrete params (no auth = omit --user/--password)
+dbcli init --system mongodb --host localhost --port 27017 --name mydb
+
+# Redis — `--name` is the LOGICAL DB INDEX ("0".."15"), not a database name
+dbcli init --system redis --host localhost --port 6379 --password '<secret>' --name 0
+
+# Elasticsearch — basic auth, Cloud ID, or API key
+dbcli init --system elasticsearch --host localhost --port 9200 \
+  --user elastic --password '<secret>'
+dbcli init --system elasticsearch \
+  --cloud-id "myCluster:dXMtZWFzdC0xLmF3..." --api-key "<base64>"
+# Multi-node / custom CA / self-signed: edit `.dbcli` directly to add
+# `nodes: [...]`, `protocol: https`, `caPath`, `rejectUnauthorized: false`.
+```
+
+### Multi-connection (v2)
+
+```bash
+dbcli init --conn-name staging --env-file .env.staging --permission query-only
+dbcli init --conn-name prod    --env-file .env.production --use-env-refs --skip-test
+dbcli use --list                          # show all, * marks default
+dbcli use prod                            # switch default
+dbcli query --use staging "SELECT 1"      # one-shot override
+dbcli init --rename staging:stg           # rename
+dbcli init --remove stg                   # remove
+```
+
+Per-connection schema cache lives at `.dbcli/schemas/<connection>/`. Run
+`dbcli schema --use <name>` once per connection before `schema <table>` —
+otherwise the cache may serve another connection's columns.
+
+### env-refs (keep secrets out of `.dbcli`)
+
+```bash
+dbcli init --use-env-refs \
+  --env-host DB_HOST --env-port DB_PORT \
+  --env-user DB_USER --env-password DB_PASSWORD --env-database DB_NAME
+```
+
+Stored as `{ "$env": "DB_HOST" }` etc. and resolved at runtime. Pair with
+`--env-file <path>` (v2) when each connection has its own env file.
+
+### Common gotchas
+
+- **MongoDB `mongodb+srv://`** — `dbcli doctor` reports whether SRV resolves
+  natively or via the DoH fallback; useful when the runtime restricts DNS.
+- **MySQL/Postgres password with `@` `:` `/`** — when using `DATABASE_URL`,
+  percent-encode (`@` → `%40`); discrete `--password` flags do not need encoding.
+- **Redis `--name`** — accepts only the logical DB index string; non-numeric
+  values are rejected.
+- **Elasticsearch TLS** — `caPath` and `rejectUnauthorized` are not exposed as
+  flags; edit `.dbcli` after `init` to add them.
+- **Re-running `init`** — refuses to overwrite without `--force`; never use
+  `--force` to "fix" a config full of `{ "$env": "..." }` refs.
+
+Full flags and edge cases: see [reference.md](reference.md) `init` section.
+
 ## Command overview
 
 | Command | Min permission | Summary |
@@ -136,6 +237,18 @@ Each `.sql` file may declare YAML frontmatter inside `-- ---` blocks
 (name, description, engine, params, tags). See `dbcli queries show @<name> --format json`
 for the machine-readable contract.
 
+### Engine-specific bodies
+
+Each snippet's body format is determined by the `engine` frontmatter field:
+
+| Engine            | Body format            | Notes |
+|-------------------|------------------------|-------|
+| postgres / mysql  | Single SELECT or WITH  | `:name` → driver bind (`$1` / `?`) |
+| elasticsearch     | JSON DSL               | `:name` → JSON-aware substitution; `index:` field required |
+| redis             | Single Redis command   | `:name` → raw text; only read commands allowed |
+
+Mixed-family `engine` arrays (e.g. `[postgres, elasticsearch]`) are rejected at parse time.
+
 ### Built-in diagnostic snippets
 
 dbcli ships ready-made diagnostic queries. Run with `dbcli q @diag/<topic>`:
@@ -150,6 +263,8 @@ dbcli ships ready-made diagnostic queries. Run with `dbcli q @diag/<topic>`:
 | `@diag/locks`           | lock-wait chains                         |
 | `@diag/db-size`         | database size summary                    |
 | `@diag/cache-hit`       | buffer cache hit ratios                  |
+| `@diag/es-cluster-health` | document counts per index (ES connections) |
+| `@diag/redis-key-stats`   | sample SCAN over keyspace (Redis connections) |
 
 Engine variants are picked automatically based on the active connection.
 Override any of them by placing a same-named file under `.dbcli-shared/queries/`

package/assets/reference.md

@@ -230,6 +230,62 @@ Param placeholders use `:name`. They are rewritten to `$1, $2, …` (Postgres) o
 - Files exceeding 64 KiB are rejected.
 - `--no-limit` is honoured only at the outermost level; nested subqueries are still wrapped by the size guard.
 
+##### Elasticsearch snippets
+
+Body is a JSON DSL `_search` request body. Frontmatter requires an `index:` field (may contain `:param`).
+
+Example:
+
+    -- ---
+    -- name: events-by-day
+    -- engine: elasticsearch
+    -- index: 'events-:date'
+    -- params:
+    --   date:    { type: date,   required: true }
+    --   user_id: { type: int,    required: true }
+    -- ---
+    {
+      "query": {
+        "bool": {
+          "filter": [
+            { "term": { "user_id": :user_id } }
+          ]
+        }
+      },
+      "size": 100
+    }
+
+Substitution rules (type-aware JSON injection):
+
+- `int` / `float` / `bool` outside string literals → bare value (`42`, `1.5`, `true`)
+- `string` / `date` / `datetime` outside string literals → JSON-quoted (`"Alice"`, `"2026-05-08"`)
+- Any param inside a JSON string literal → escaped inner form (`"prefix-:name"` works)
+
+`script` and `script_fields` are rejected anywhere in the body.
+
+Size guard: if `size` is missing, `1000` is injected (or `0` when `aggs` is present); explicit `size > 1000` is overridden with a warning unless `--no-limit`.
+
+##### Redis snippets
+
+Body is a single Redis command on one line. Only read-only commands are allowed:
+`GET MGET HGET HGETALL HMGET HKEYS HVALS HLEN HEXISTS LRANGE LLEN LINDEX SMEMBERS SISMEMBER SCARD ZRANGE ZRANGEBYSCORE ZRANGEBYLEX ZSCORE ZCARD ZCOUNT ZRANK TYPE EXISTS TTL PTTL STRLEN OBJECT SCAN HSCAN SSCAN ZSCAN`.
+
+`KEYS`, `EVAL`, `FLUSHDB`, `FLUSHALL`, `CONFIG`, `DEBUG`, `SHUTDOWN`, `SCRIPT` and any write command are rejected.
+
+Example:
+
+    -- ---
+    -- name: cache-user
+    -- engine: redis
+    -- params:
+    --   id: { type: int, required: true }
+    -- ---
+    HGETALL user::id
+
+Substitution rules: pure raw text — `:name` becomes the value's `String()` form. **Quoting is the snippet author's responsibility**: wrap `:name` in double quotes if the value may contain whitespace. The parser warns when a `string`-typed `:name` is adjacent to non-whitespace and unquoted.
+
+Size guard: `LRANGE` / `ZRANGE` stop overridden when `< 0` or `> 1000`; `SCAN` / `HSCAN` / `SSCAN` / `ZSCAN` get `COUNT 1000` injected if absent. `--no-limit` disables.
+
 ### queries
 
 Manage saved snippets — discover, inspect, scaffold, and edit local copies. Mutating
@@ -582,7 +638,9 @@ Redis connections speak Redis commands rather than SQL. The adapter uses the `io
 
 **Supported commands:** `init`, `use`, `list`, `schema`, `query`, `status`, `doctor`, `upgrade`, `completion`
 
-**Not supported (exit with error or unsupported error):** `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `q` (saved queries), `shell`. For writes, run the equivalent Redis command via `query` — the same permission gate applies.
+**Saved queries:** `q` is supported for read-only Redis commands (see "Redis snippets" below).
+
+**Not supported (exit with error or unsupported error):** `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `shell`. For writes, run the equivalent Redis command via `query` — the same permission gate applies.
 
 ### Connection and configuration
 
@@ -648,7 +706,9 @@ Elasticsearch connections speak the REST API. The adapter is fetch-based (no SDK
 
 **Supported commands:** `init`, `use`, `list`, `schema`, `query`, `status`, `doctor`, `upgrade`, `completion`
 
-**Not supported (use external tooling):** `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `q`, `shell`. The permission classifier already understands `_doc` / `_update` / `_bulk` so future write surfaces can be wired in without changing tiers.
+**Saved queries:** `q` is supported for ES JSON DSL bodies (see "Elasticsearch snippets" below).
+
+**Not supported (use external tooling):** `insert`, `update`, `delete`, `export`, `check`, `diff`, `migrate`, `shell`. The permission classifier already understands `_doc` / `_update` / `_bulk` so future write surfaces can be wired in without changing tiers.
 
 ### Connection and configuration
 

package/assets/snippets/diag/es-cluster-health.elasticsearch.sql

@@ -0,0 +1,15 @@
+-- ---
+-- name: es-cluster-health
+-- description: Document counts per index across the cluster
+-- engine: elasticsearch
+-- index: '*'
+-- tags: [diag, elasticsearch]
+-- ---
+{
+  "size": 0,
+  "aggs": {
+    "by_index": {
+      "terms": { "field": "_index", "size": 50 }
+    }
+  }
+}

package/assets/snippets/diag/redis-key-stats.redis.sql

@@ -0,0 +1,11 @@
+-- ---
+-- name: redis-key-stats
+-- description: Sample keyspace via SCAN with explicit COUNT bound
+-- engine: redis
+-- params:
+--   match:
+--     type: string
+--     default: '*'
+-- tags: [diag, redis]
+-- ---
+SCAN 0 MATCH :match COUNT 100