@carllee1983/dbcli 0.5.0-beta → 1.1.0

package/CHANGELOG.md

@@ -5,6 +5,138 @@ All notable changes to dbcli are documented here.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.1.0] - 2026-03-30
+
+### Changed
+
+- **Adapter `execute()` 回傳型別重構**: 從 `T[]` 改為 `ExecutionResult<T>`，包含 `rows`、`affectedRows`、`lastInsertId` 欄位，DML 操作（INSERT/UPDATE/DELETE）現在回傳正確的 affected rows 計數
+- **Export 覆寫確認**: `export --output` 寫入已存在檔案時會提示確認，可用 `--force` 跳過
+- **`ExecutionResult<T>` 介面**: 新增統一的查詢結果型別定義於 `src/adapters/types.ts`
+
+---
+
+## [1.0.0] - 2026-03-28
+
+### Stable Release
+
+dbcli v1.0.0 is the first stable release. All three milestones are complete:
+- **M1 (v0.6.0):** Smart REPL — interactive shell with SQL + dbcli commands
+- **M2 (v0.8.0):** Schema DDL — CREATE/DROP/ALTER TABLE, INDEX, CONSTRAINT, ENUM
+- **M3 (v1.0.0):** Stabilization — documentation, permission matrix, known limitations update
+
+### Added
+
+- **`dbcli migrate` command group** (12 subcommands): Full DDL operations with cross-database support
+  - `migrate create <table>` — CREATE TABLE with `--column` spec format (`"id:serial:pk"`)
+  - `migrate drop <table>` — DROP TABLE with double confirmation (`--execute --force`)
+  - `migrate add-column` / `drop-column` / `alter-column` — Column management
+  - `migrate add-index` / `drop-index` — Index management (MySQL `--table` option for DROP)
+  - `migrate add-constraint` / `drop-constraint` — FK, UNIQUE, CHECK constraints
+  - `migrate add-enum` / `alter-enum` / `drop-enum` — PostgreSQL native ENUM support
+- **DDLGenerator interface** with PostgreSQL and MySQL/MariaDB dialect implementations
+  - PostgreSQL: SERIAL, native ENUM types, ALTER COLUMN TYPE, double-quote identifiers
+  - MySQL: AUTO_INCREMENT, inline ENUM, MODIFY COLUMN, backtick identifiers
+- **DDLExecutor**: Unified execution pipeline — admin permission check → blacklist protection → SQL generation → dry-run/execute → schema cache auto-refresh
+- **Default dry-run for DDL**: All `migrate` commands preview SQL without `--execute`. Destructive operations also require `--force`
+- **142 new tests**: column-parser (17), PG DDL (35), MySQL DDL (25), factory (5), DDL executor (22), schema cache DDL (6), CLI migrate (26), live-db migrate lifecycle (6)
+
+### Fixed
+
+- **Schema comment encoding**: Fixed double-encoded UTF-8 comments from MySQL/MariaDB `information_schema` (e.g., `帳號` → `帳號`)
+- **MySQL connection charset**: Added `charset: utf8mb4` and `SET NAMES utf8mb4`
+- **DDL multi-line SQL execution**: Fixed statement splitting to use `;\n` instead of `\n`
+- **MySQL DROP INDEX**: Added `--table` option (MariaDB requires `ON <table>`)
+
+### Changed
+
+- **Permission model**: 4 levels — query-only, read-write, data-admin, admin (DDL requires admin)
+- **Known Limitations**: Removed "Read-only schema" and "CLI-only" (both resolved). Added "No migration version tracking" as post-v1.0 item
+- **Test infrastructure**: `docker-compose.test.yml` for MySQL 8 + PostgreSQL 16 integration testing
+- **Package scripts**: Added `test:unit`, `test:integration`, `test:docker`
+- **SKILL.md**: Updated with full `migrate` command reference and AI agent guidelines
+
+### Test Results (v1.0.0)
+
+- Unit/Core: 1082 pass, 0 fail
+- Live DB (MariaDB 10.11): 61 pass
+- Docker Adapter (MySQL 8 + PG 16): 18 pass
+
+---
+
+## [0.6.1-beta] - 2026-03-28
+
+### Encoding Fix & Test Infrastructure
+
+### Fixed
+
+- **Schema comment encoding**: Fixed double-encoded UTF-8 comments from MySQL/MariaDB `information_schema`. Comments stored through latin1 (cp1252) connections now correctly display CJK characters (e.g., `帳號` → `帳號`)
+- **MySQL connection charset**: Added `charset: utf8mb4` and `SET NAMES utf8mb4` to MySQL adapter connections
+
+### Added
+
+- **`fixDoubleEncodedUtf8()` utility** (`src/utils/encoding.ts`): Detects and reverses cp1252-to-UTF-8 double encoding with full cp1252 reverse mapping table. Applied to schema comments in both MySQL and PostgreSQL adapters
+- **`docker-compose.test.yml`**: MySQL 8.4 (port 3307) + PostgreSQL 16 (port 5433) for integration testing, with health checks and tmpfs for fast ephemeral storage
+- **Environment-driven adapter tests**: `mysql.test.ts` and `postgresql.test.ts` now read connection from `MYSQL_*` / `PG_*` env vars, falling back to docker-compose defaults. Auto-skip when DB is unreachable
+- **`live-db.test.ts`**: 55 comprehensive CLI-level integration tests covering all commands against live MariaDB — list, schema, query, blacklist CRUD, insert/update/delete lifecycle, export, check, diff, status, doctor, shell, format validation, SQL injection protection
+- **New test scripts**: `test:unit`, `test:integration`, `test:docker` in package.json
+
+### Test Results
+
+- Unit/Core: 940 pass
+- Live DB (MariaDB 10.11): 55 pass
+- Adapter (Docker MySQL 8 + PG 16): 18 pass
+
+---
+
+## [0.6.0-beta] - 2026-03-28
+
+### Interactive Shell — Smart REPL
+
+### Added
+
+- **`dbcli shell` command:** Interactive database shell with SQL execution and dbcli command dispatch
+- **SQL-only mode:** `--sql` flag restricts to SQL statements only
+- **Auto-completion (Tab):** Context-aware completion for SQL keywords, table names, column names, and dbcli commands
+- **Multi-line SQL:** Accumulates input until `;` is found, with `...>` continuation prompt
+- **SQL syntax highlighting:** Real-time colorization of keywords, strings, and numbers in verbose mode
+- **Meta commands:** `.help`, `.quit`/`.exit`, `.clear`, `.format`, `.history`, `.timing`
+- **Persistent history:** Stored in `~/.dbcli_history` (max 1000 entries), with up/down navigation and Ctrl+R search
+- **Permission & blacklist integration:** Full enforcement within REPL session — SQL goes through PermissionGuard, query results go through blacklist filtering
+- **Auto-reconnect:** Attempts to reconnect once on connection errors, then displays error without crashing the session
+- **Error resilience:** SQL/permission/connection errors never crash the session
+- **i18n support:** All shell messages available in English and Traditional Chinese
+- **102 new tests:** input-classifier (25), multiline-buffer (10), meta-commands (15), completer (17), history-manager (8), command-dispatcher (12), repl-engine (12), shell-command (3)
+
+---
+
+## [0.5.2-beta] - 2026-03-27
+
+### Fixed
+
+- **`init --use-env-refs` permission bug**: Interactive env-ref mode now correctly offers all 4 permission levels (was missing `data-admin`)
+- **`init` i18n completeness**: All 10 hardcoded English messages replaced with i18n keys (supports en/zh-TW)
+- **`init` duplicate code**: Extracted shared `.dbcli exists` overwrite check into `checkOverwrite()` helper
+- **`--use-env-refs` help text**: Improved option description to clarify CI/CD and multi-env use case
+- **Documentation**: Added `--use-env-refs` to README (en/zh-TW), CHANGELOG, and SKILL.md with AI agent guidance
+
+---
+
+## [0.5.1-beta] - 2026-03-27
+
+### Added
+
+- **Database version check**: Warns on stderr when connected database version is below minimum supported (PostgreSQL 12+, MySQL 8.0+, MariaDB 10.5+). Non-blocking — connection proceeds normally.
+- **`dbcli doctor` DB version check**: New "Database version" item in Connection & Data group.
+- **`dbcli init --use-env-refs`**: Store environment variable references (`{"$env": "DB_HOST"}`) in config instead of actual values. Supports interactive and non-interactive modes with `--env-host`, `--env-port`, `--env-user`, `--env-password`, `--env-database` options. Suitable for CI/CD and multi-environment deployments.
+
+### Fixed
+
+- **`init` permission bug**: Interactive env-ref mode now correctly offers all 4 permission levels (was missing `data-admin`)
+- **`init` i18n**: All hardcoded English messages in init command replaced with i18n keys (10 messages)
+- **`init` duplicate code**: Extracted shared `.dbcli exists` overwrite check into `checkOverwrite()` helper
+
+---
+
 ## [0.5.0-beta] - 2026-03-27
 
 ### UX & Developer Experience — Colors, Logging, Diagnostics, and Tooling
@@ -236,12 +368,11 @@ dbcli v0.1.0-beta is a complete, production-ready CLI tool enabling AI agents an
 
 ---
 
-## Known Limitations (V1)
+## Known Limitations
 
-- **Single database per project:** Multi-connection support deferred to a future version
-- **No audit logging:** WHO/WHAT/WHEN tracking deferred to a future version
-- **Read-only schema:** No schema modification commands (ALTER TABLE, etc.) in V1
-- **CLI-only:** No visual schema designer, REPL, or interactive shell in V1
+- **Single database per project:** Each directory uses one `.dbcli` config. For multi-database setups, use separate directories or `--config` flag. This is by design, not a technical limitation.
+- **No audit logging:** WHO/WHAT/WHEN tracking deferred to post-v1.0
+- **No migration version tracking:** `migrate` commands execute DDL directly without version history or rollback. The `migrate` namespace is reserved for future migration tracking support.
 
 ---
 

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Carl Lee
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -96,9 +96,27 @@ Initialize a new dbcli project with database connection configuration.
 
 **Usage:**
 ```bash
-dbcli init
+dbcli init [OPTIONS]
 ```
 
+**Options:**
+- `--system <type>` — Database system: `postgresql`, `mysql`, `mariadb`
+- `--host <host>` — Database host
+- `--port <port>` — Database port
+- `--user <user>` — Database user
+- `--password <pass>` — Database password
+- `--name <db>` — Database name
+- `--permission <level>` — Permission level: `query-only`, `read-write`, `data-admin`, `admin`
+- `--use-env-refs` — Store environment variable references instead of actual values in config
+- `--env-host <var>` — Env var name for host (with `--use-env-refs`)
+- `--env-port <var>` — Env var name for port (with `--use-env-refs`)
+- `--env-user <var>` — Env var name for user (with `--use-env-refs`)
+- `--env-password <var>` — Env var name for password (with `--use-env-refs`)
+- `--env-database <var>` — Env var name for database (with `--use-env-refs`)
+- `--skip-test` — Skip connection test
+- `--no-interactive` — Non-interactive mode (requires all options)
+- `--force` — Overwrite existing config without confirmation
+
 **Behavior:**
 - Reads `.env` file if present (auto-fills DATABASE_URL, DB_* variables)
 - Prompts for missing values (host, port, user, password, database name, permission level)
@@ -116,8 +134,20 @@ dbcli init
 
 # Specify permission level
 echo "PERMISSION_LEVEL=admin" >> .env && dbcli init
+
+# Store env var references instead of values (interactive)
+dbcli init --use-env-refs
+
+# Store env var references (non-interactive)
+dbcli init --use-env-refs --system mysql \
+  --env-host DB_HOST --env-port DB_PORT \
+  --env-user DB_USER --env-password DB_PASSWORD \
+  --env-database DB_DATABASE \
+  --no-interactive
 ```
 
+> **`--use-env-refs`:** When enabled, the config stores environment variable names (e.g., `{"$env": "DB_HOST"}`) instead of actual values. This avoids writing sensitive credentials into the config file, making it suitable for multi-environment deployments and CI/CD pipelines. At connection time, dbcli automatically reads the actual values from the referenced environment variables.
+
 ---
 
 #### `dbcli list`
@@ -489,6 +519,72 @@ dbcli upgrade --check           # Only check, do not upgrade
 **Options:** `--check` — check only, don't install
 **Background check:** dbcli silently checks npm registry once per 24 hours. If a newer version is found, a hint is shown after command output.
 
+#### `dbcli shell`
+
+Interactive database shell with SQL execution, auto-completion, and syntax highlighting.
+
+**Usage:**
+```bash
+dbcli shell          # Interactive mode (SQL + dbcli commands)
+dbcli shell --sql    # SQL-only mode
+```
+
+**Inside the shell:**
+- Type SQL statements ending with `;` to execute queries
+- Type dbcli commands without the `dbcli` prefix (e.g., `schema users`, `list`)
+- Press Tab for context-aware auto-completion (SQL keywords, table/column names)
+- Use `.help` for meta commands (`.quit`, `.clear`, `.format`, `.history`, `.timing`)
+- Multi-line SQL: input accumulates until `;` is found
+- History persists across sessions in `~/.dbcli_history`
+
+**Permission:** Inherits from config. SQL and commands are fully permission/blacklist enforced.
+
+#### `dbcli migrate`
+
+Schema DDL operations. **All commands default to dry-run** — use `--execute` to actually run the SQL.
+
+**Usage:**
+```bash
+# Create table
+dbcli migrate create posts \
+  --column "id:serial:pk" \
+  --column "title:varchar(200):not-null" \
+  --column "body:text" \
+  --column "created_at:timestamp:default=now()"
+
+# Execute (actually run the SQL)
+dbcli migrate create posts --column "id:serial:pk" --execute
+
+# Drop table (destructive — requires --execute --force)
+dbcli migrate drop posts --execute --force
+
+# Column operations
+dbcli migrate add-column users bio text --nullable --execute
+dbcli migrate alter-column users name --type "varchar(200)" --execute
+dbcli migrate alter-column users email --rename user_email --execute
+dbcli migrate drop-column users temp_field --execute --force
+
+# Index operations
+dbcli migrate add-index users --columns email --unique --execute
+dbcli migrate drop-index idx_users_email --table users --execute --force
+
+# Constraint operations
+dbcli migrate add-constraint orders --fk user_id --references users.id --on-delete cascade --execute
+dbcli migrate add-constraint users --unique email --execute
+dbcli migrate add-constraint users --check "age >= 0" --execute
+dbcli migrate drop-constraint orders fk_orders_user_id --execute --force
+
+# Enum (PostgreSQL only)
+dbcli migrate add-enum status active inactive suspended --execute
+dbcli migrate alter-enum status --add-value archived --execute
+dbcli migrate drop-enum status --execute --force
+```
+
+**Column spec format:** `name:type[:modifier...]` — Modifiers: `pk`, `not-null`, `unique`, `auto-increment`, `default=<value>`, `references=<table>.<column>`
+
+**Options (all subcommands):** `--execute` (run SQL), `--force` (skip confirmation for DROP), `--config <path>`
+**Permission:** admin only
+
 ---
 
 ## Global Options
@@ -512,9 +608,10 @@ dbcli implements a coarse-grained permission system with three levels. Permissio
 
 | Level | Allowed Commands | Blocked Commands | Use Case |
 |-------|------------------|------------------|----------|
-| **Query-only** | `init`, `list`, `schema`, `query`, `export` (limited to 1000 rows) | `insert`, `update`, `delete` | Read-only AI agents, data analysts, reporting |
-| **Read-Write** | + `insert`, `update` | `delete` | Application developers, content managers |
-| **Admin** | All commands | — | Database administrators, schema modifications |
+| **Query-only** | `init`, `list`, `schema`, `query`, `export` (limited to 1000 rows) | `insert`, `update`, `delete`, `migrate` | Read-only AI agents, data analysts, reporting |
+| **Read-Write** | + `insert`, `update` | `delete`, `migrate` | Application developers, content managers |
+| **Data-Admin** | + `delete` | `migrate` | Full DML access, no DDL |
+| **Admin** | All commands including `migrate` (DDL) | — | Database administrators, schema modifications |
 
 ### Configuration
 
@@ -553,11 +650,12 @@ dbcli delete users --where "id=1"  # ERROR: Admin only
 
 #### Admin Mode (Database Administrator)
 ```bash
-# Allowed: Everything
+# Allowed: Everything including DDL
 dbcli query "SELECT * FROM users"
 dbcli insert users --data '{"name": "Eve"}'
 dbcli update users --where "id=1" --set '{"status": "active"}'
-dbcli delete users --where "id=1" --force  # Only Admin can delete
+dbcli delete users --where "id=1" --force  # Data-Admin+ can delete
+dbcli migrate create posts --column "id:serial:pk" --execute  # Admin only
 ```
 
 ### Best Practices

package/README.zh-TW.md

@@ -238,8 +238,19 @@ dbcli init
 
 dbcli init --host db.example.com --port 5432 --user admin --password secret --name prod_db --system postgresql
 # 非交互式初始化
+
+dbcli init --use-env-refs
+# 交互式：提示輸入環境變數名稱，config 中儲存 {"$env": "DB_HOST"} 而非實際值
+
+dbcli init --use-env-refs --system mysql \
+  --env-host DB_HOST --env-port DB_PORT \
+  --env-user DB_USER --env-password DB_PASSWORD \
+  --env-database DB_DATABASE --no-interactive
+# 非交互式環境變數參照模式，適合 CI/CD
 ```
 
+> **`--use-env-refs` 說明：** 使用此選項時，config 中儲存的是環境變數名稱（如 `{"$env": "DB_HOST"}`）而非實際值。這樣可以避免將敏感資訊寫入 config 檔案，適合多環境部署或 CI/CD 場景。連線時 dbcli 會自動從環境變數讀取實際值。
+
 ### 列出表
 
 ```bash

package/assets/SKILL.md

@@ -30,6 +30,8 @@ dbcli init --no-interactive --force # Non-interactive, skip overwrite confirmati
 
 **Key options:** `--system <postgresql|mysql|mariadb>`, `--permission <query-only|read-write|data-admin|admin>`, `--use-env-refs`, `--skip-test`, `--no-interactive`, `--force`
 
+> **AI agent note on `--use-env-refs`:** If an existing `.dbcli` config contains `{"$env": "DB_HOST"}` style references, the connection values are read from environment variables at runtime. Do NOT re-run `init` to replace these references with actual values — the env-ref format is intentional for CI/CD and multi-environment setups.
+
 ### list
 
 List all tables.
@@ -114,10 +116,11 @@ Export query results to file or stdout.
 
 ```bash
 dbcli export "SELECT * FROM users" --format csv --output users.csv
+dbcli export "SELECT * FROM users" --format csv --output users.csv --force  # Skip overwrite confirmation
 dbcli export "SELECT * FROM users" --format json | jq '.[]'
 ```
 
-**Options:** `--format <json|csv>` (required), `--output <path>`
+**Options:** `--format <json|csv>` (required), `--output <path>`, `--force`
 **Permission:** query-only+
 
 ### blacklist
@@ -220,6 +223,75 @@ dbcli upgrade --check           # Only check, do not upgrade
 
 **Background check:** Every command silently checks the npm registry for a newer version (at most once per 24 hours, cached in `.dbcli/version-check.json`). If a newer version is found, a one-line hint is printed to stderr after the command completes. Pass `-q` / `--quiet` to suppress the hint.
 
+### `dbcli shell`
+
+Start an interactive database shell.
+
+```bash
+dbcli shell          # Interactive mode with SQL + dbcli commands
+dbcli shell --sql    # SQL-only mode
+```
+
+Inside the shell:
+- Type SQL statements ending with `;` to execute
+- Type dbcli commands without the `dbcli` prefix (e.g., `schema users`)
+- Use Tab for auto-completion (SQL keywords, table names, column names)
+- Type `.help` for meta commands (.quit, .clear, .format, .history, .timing)
+- Multi-line SQL: keeps accumulating until `;` is found
+- History persists across sessions (~/.dbcli_history)
+
+### migrate
+
+Schema DDL operations. **All commands default to dry-run** — use `--execute` to actually run the SQL. Destructive operations (DROP) also require `--force`.
+
+```bash
+# Create table
+dbcli migrate create posts \
+  --column "id:serial:pk" \
+  --column "title:varchar(200):not-null" \
+  --column "body:text" \
+  --column "created_at:timestamp:default=now()"
+
+# Drop table (dry-run by default)
+dbcli migrate drop posts
+dbcli migrate drop posts --execute --force   # Actually drop
+
+# Add/drop/alter column
+dbcli migrate add-column users bio text --nullable
+dbcli migrate drop-column users temp_field --execute --force
+dbcli migrate alter-column users name --type "varchar(200)"
+dbcli migrate alter-column users email --rename user_email
+dbcli migrate alter-column users status --set-default "'active'"
+dbcli migrate alter-column users bio --drop-default
+dbcli migrate alter-column users bio --set-nullable
+dbcli migrate alter-column users email --drop-nullable
+
+# Index management
+dbcli migrate add-index users --columns email --unique
+dbcli migrate add-index users --columns "last_name,first_name" --name idx_fullname
+dbcli migrate drop-index idx_fullname --execute --force
+
+# Constraint management
+dbcli migrate add-constraint orders --fk user_id --references users.id --on-delete cascade
+dbcli migrate add-constraint users --unique email
+dbcli migrate add-constraint users --check "age >= 0"
+dbcli migrate drop-constraint orders fk_orders_user_id --execute --force
+
+# Enum (PostgreSQL only — MySQL uses inline ENUM in column type)
+dbcli migrate add-enum status active inactive suspended
+dbcli migrate alter-enum status --add-value archived
+dbcli migrate drop-enum status --execute --force
+```
+
+**Column spec format:** `name:type[:modifier[:modifier...]]`
+- Modifiers: `pk`, `not-null`, `unique`, `auto-increment`, `default=<value>`, `references=<table>.<column>`
+- Serial types: `serial`, `bigserial`, `smallserial` (auto-expand per DB dialect)
+
+**Options (all subcommands):** `--execute`, `--force`, `--config <path>`
+**Permission:** admin
+
+**AI agent note:** Always use dry-run first (no `--execute`) to preview generated SQL. Only add `--execute` after confirming the SQL is correct. For DROP operations, both `--execute` and `--force` are required.
+
 ## Permission Levels
 
 | Level | Allowed Operations |
@@ -227,7 +299,7 @@ dbcli upgrade --check           # Only check, do not upgrade
 | query-only | SELECT, list, schema, export |
 | read-write | query-only + INSERT, UPDATE |
 | data-admin | read-write + DELETE (full DML, no DDL) |
-| admin | data-admin + DROP, ALTER, CREATE, TRUNCATE |
+| admin | data-admin + DDL (migrate create/drop/alter, DROP, ALTER, CREATE, TRUNCATE) |
 
 Set via `dbcli init --permission <level>` or in `.dbcli` config.