@clickzetta/cz-cli-darwin-arm64 0.3.18 → 0.3.19

package/bin/skills/clickzetta-monitoring/SKILL.md

@@ -0,0 +1,199 @@
+---
+name: clickzetta-monitoring
+description: |
+  监控和分析 ClickZetta Lakehouse 作业运行状态、性能和资源使用情况，
+  以及通过 INFORMATION_SCHEMA 查询元数据（表、列、Schema、工作空间等）。
+  覆盖 SHOW JOBS 实时查看作业、information_schema.job_history 历史分析、
+  慢查询识别、集群负载分析、缓存命中率统计、失败作业排查、
+  information_schema.tables/columns/schemas 元数据查询等完整监控与治理工作流。
+  当用户说"查看作业"、"作业历史"、"SHOW JOBS"、"慢查询"、"查询性能"、
+  "集群负载"、"作业失败"、"查询失败"、"监控"、"job history"、
+  "information_schema"、"缓存命中率"、"查询耗时"、"作业状态"、
+  "元数据查询"、"查看所有表"、"表大小"、"列信息"、"资产盘点"时触发。
+  Keywords: monitoring, job status, performance, resource usage, SHOW JOBS, slow query
+---
+
+# ClickZetta 作业监控与分析
+
+阅读 [references/show-jobs.md](references/show-jobs.md) 了解 SHOW JOBS 语法。
+阅读 [references/job-history-analysis.md](references/job-history-analysis.md) 了解历史分析查询。
+
+---
+
+## 实时查看作业（SHOW JOBS）
+
+```sql
+-- 查看所有作业（最近7天）
+SHOW JOBS;
+
+-- 查看指定集群的作业
+SHOW JOBS IN VCLUSTER default_ap;
+
+-- 查看执行时间超过2分钟的慢查询
+SHOW JOBS WHERE execution_time > INTERVAL 2 MINUTE;
+
+-- 查看失败的作业
+SHOW JOBS WHERE status = 'FAILED';
+
+-- 限制返回数量
+SHOW JOBS IN VCLUSTER default_ap LIMIT 50;
+```
+
+---
+
+## 历史作业分析（information_schema）
+
+### 集群负载分析
+
+```sql
+SELECT
+    virtual_cluster,
+    COUNT(*) AS job_count,
+    AVG(execution_time) AS avg_seconds,
+    ROUND(SUM(CASE WHEN status = 'SUCCEED' THEN 1 ELSE 0 END) * 100.0 / COUNT(*), 2) AS success_rate
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+GROUP BY virtual_cluster
+ORDER BY job_count DESC;
+```
+
+### 慢查询 TOP 20
+
+```sql
+SELECT job_id, virtual_cluster, execution_time, status, start_time
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+ORDER BY execution_time DESC
+LIMIT 20;
+```
+
+### 失败作业统计
+
+```sql
+SELECT
+    virtual_cluster,
+    COUNT(*) AS failed_count,
+    DATE(start_time) AS date
+FROM sys.information_schema.job_history
+WHERE status = 'FAILED'
+  AND start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+GROUP BY virtual_cluster, DATE(start_time)
+ORDER BY date DESC;
+```
+
+### 高峰期识别
+
+```sql
+SELECT
+    HOUR(start_time) AS hour_of_day,
+    COUNT(*) AS job_count,
+    AVG(execution_time) AS avg_execution_time
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+GROUP BY HOUR(start_time)
+ORDER BY hour_of_day;
+```
+
+---
+
+## query_tag 标记与过滤
+
+给作业打标，便于按来源过滤：
+
+```sql
+-- 在 SQL 中设置 query_tag
+SET query_tag = 'etl_daily';
+SELECT * FROM orders;
+
+-- 按 query_tag 过滤作业历史
+SELECT job_id, execution_time, status
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+  AND query_tag = 'etl_daily';
+```
+
+JDBC URL 中设置：
+```
+jdbc:clickzetta://instance.region.api.clickzetta.com/workspace?query_tag=my_app
+```
+
+---
+
+## 常见问题排查
+
+| 现象 | 排查方向 |
+|---|---|
+| 作业长时间"等待执行" | 集群资源不足，考虑扩容 VCluster |
+| 作业长时间"集群启动中" | VCluster 冷启动慢，联系技术支持 |
+| 大量失败作业 | 查看 job_id 详情，检查 SQL 语法或权限 |
+| 平均执行时间突然变长 | 检查数据量变化、索引状态、缓存命中率 |
+
+---
+
+## INFORMATION_SCHEMA 元数据查询
+
+除了 `job_history`，INFORMATION_SCHEMA 还提供丰富的元数据视图，用于资产盘点和治理。
+
+### 空间级视图（当前工作空间）
+
+```sql
+-- 查看当前空间下所有 Schema
+SELECT * FROM information_schema.schemas;
+
+-- 查看所有表及其大小、行数
+SELECT table_schema, table_name, table_type, row_count, bytes
+FROM information_schema.tables
+ORDER BY bytes DESC;
+
+-- 查看所有列的详细信息（字段名、类型、是否可空、注释）
+SELECT table_schema, table_name, column_name, data_type, is_nullable, comment
+FROM information_schema.columns
+WHERE table_schema = 'public';
+
+-- 查看排序列推荐
+SELECT * FROM information_schema.sortkey_candidates;
+```
+
+### 实例级视图（需要 instance_admin 权限，使用 sys 库）
+
+```sql
+-- 查看实例下所有工作空间
+SELECT * FROM sys.information_schema.workspaces;
+
+-- 查看实例下所有 Schema（跨工作空间）
+SELECT * FROM sys.information_schema.schemas;
+
+-- 查看实例用量（费用分析）
+SELECT * FROM sys.information_schema.instance_usage
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY;
+```
+
+### 常用元数据分析场景
+
+```sql
+-- 找出最大的 10 张表
+SELECT table_schema, table_name, row_count, bytes
+FROM information_schema.tables
+WHERE table_type = 'TABLE'
+ORDER BY bytes DESC
+LIMIT 10;
+
+-- 找出没有注释的表
+SELECT table_schema, table_name
+FROM information_schema.tables
+WHERE comment IS NULL OR comment = '';
+
+-- 找出没有注释的字段
+SELECT table_schema, table_name, column_name
+FROM information_schema.columns
+WHERE (comment IS NULL OR comment = '')
+  AND table_schema NOT IN ('information_schema');
+
+-- 统计各 Schema 下的表数量和总存储
+SELECT table_schema,
+       COUNT(*) AS table_count,
+       SUM(bytes) AS total_storage
+FROM information_schema.tables
+GROUP BY table_schema
+ORDER BY total_storage DESC;
+```

package/bin/skills/clickzetta-monitoring/eval_cases.jsonl

@@ -0,0 +1,5 @@
+{"case_id":"001","type":"should_call","user_input":"SHOW JOBS 的语法是什么？怎么过滤特定状态的作业？","expected_skill":"clickzetta-monitoring","expected_output_contains":["SHOW JOBS"]}
+{"case_id":"002","type":"should_call","user_input":"怎么查询失败的作业？用什么 SQL？","expected_skill":"clickzetta-monitoring","expected_output_contains":["FAILED"]}
+{"case_id":"003","type":"should_call","user_input":"过去 7 天各集群的作业执行情况怎么查？","expected_skill":"clickzetta-monitoring","expected_output_contains":["job_history","virtual_cluster"]}
+{"case_id":"004","type":"should_call","user_input":"怎么查看慢查询？执行超过 2 分钟的作业有哪些？","expected_skill":"clickzetta-monitoring","expected_output_contains":["execution_time"]}
+{"case_id":"005","type":"should_call","user_input":"怎么通过 information_schema 查看所有表的大小和行数？","expected_skill":"clickzetta-monitoring","expected_output_contains":["information_schema","tables"]}

package/bin/skills/clickzetta-monitoring/references/job-history-analysis.md

@@ -0,0 +1,97 @@
+# information_schema 作业历史分析参考
+
+> 来源：https://www.yunqi.tech/documents/job_history_analysis_with_information_schema
+
+## 数据源
+
+表名：`sys.information_schema.job_history`
+
+### 关键字段
+
+| 字段 | 类型 | 说明 |
+|---|---|---|
+| workspace_name | String | 工作空间名称 |
+| virtual_cluster | String | 计算集群名称 |
+| job_id | String | 作业唯一标识 |
+| execution_time | Float | 执行时长（秒） |
+| start_time | Timestamp | 开始时间 |
+| status | String | 状态（SUCCEED/FAILED/CANCELLED/...） |
+| input_tables | String | 输入表（JSON 格式） |
+| input_bytes | String | 读取字节数 |
+| cache_hit | String | 缓存命中字节数 |
+
+---
+
+## 常用分析查询
+
+### 1. 集群负载分析（近 30 天）
+
+```sql
+SELECT
+    virtual_cluster,
+    COUNT(*) AS job_count,
+    SUM(execution_time) AS total_execution_time,
+    AVG(execution_time) AS avg_execution_time,
+    ROUND(SUM(CASE WHEN status = 'SUCCEED' THEN 1 ELSE 0 END) * 100.0 / COUNT(*), 2) AS success_rate
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 30 DAY
+GROUP BY virtual_cluster
+ORDER BY total_execution_time DESC;
+```
+
+### 2. 慢查询分析（执行时间 TOP 20）
+
+```sql
+SELECT
+    job_id,
+    virtual_cluster,
+    execution_time,
+    status,
+    start_time
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+ORDER BY execution_time DESC
+LIMIT 20;
+```
+
+### 3. 失败作业分析
+
+```sql
+SELECT
+    virtual_cluster,
+    COUNT(*) AS failed_count,
+    DATE(start_time) AS date
+FROM sys.information_schema.job_history
+WHERE status = 'FAILED'
+  AND start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+GROUP BY virtual_cluster, DATE(start_time)
+ORDER BY date DESC, failed_count DESC;
+```
+
+### 4. 缓存命中率分析
+
+```sql
+SELECT
+    virtual_cluster,
+    SUM(CAST(input_bytes AS BIGINT)) AS total_input_bytes,
+    SUM(CAST(cache_hit AS BIGINT)) AS total_cache_hit,
+    ROUND(SUM(CAST(cache_hit AS BIGINT)) * 100.0 /
+          NULLIF(SUM(CAST(input_bytes AS BIGINT)), 0), 2) AS cache_hit_rate
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+  AND input_bytes IS NOT NULL
+GROUP BY virtual_cluster;
+```
+
+### 5. 按小时统计作业量（识别高峰期）
+
+```sql
+SELECT
+    HOUR(start_time) AS hour_of_day,
+    COUNT(*) AS job_count,
+    AVG(execution_time) AS avg_execution_time
+FROM sys.information_schema.job_history
+WHERE start_time >= CURRENT_DATE() - INTERVAL 7 DAY
+GROUP BY HOUR(start_time)
+ORDER BY hour_of_day;
+```

package/bin/skills/clickzetta-monitoring/references/show-jobs.md

@@ -0,0 +1,48 @@
+# SHOW JOBS 参考
+
+> 来源：https://www.yunqi.tech/documents/show-jobs
+
+## 语法
+
+```sql
+SHOW JOBS [IN VCLUSTER vc_name] [LIKE 'pattern'] [WHERE <expr>] [LIMIT num];
+```
+
+## 参数说明
+
+- `IN VCLUSTER vc_name`：（可选）指定计算集群，筛选该集群下的作业
+- `WHERE <expr>`：（可选）按字段过滤，支持 SHOW JOBS 显示的所有字段
+- `LIMIT num`：（可选）限制返回数量，范围 1-10000
+- `LIKE 'pattern'`：（可选）按 job_id 模式匹配（支持 `%` 和 `_`）
+
+默认显示最近 7 天内的作业，最多 10000 条。
+
+## 示例
+
+```sql
+-- 查看所有作业（最近7天）
+SHOW JOBS;
+
+-- 查看指定集群的作业
+SHOW JOBS IN VCLUSTER default_ap;
+
+-- 查看执行时间超过2分钟的作业
+SHOW JOBS IN VCLUSTER default_ap WHERE execution_time > INTERVAL 2 MINUTE;
+
+-- 限制返回100条
+SHOW JOBS LIMIT 100;
+
+-- 按 job_id 模糊匹配
+SHOW JOBS LIKE '2024%';
+```
+
+## 作业状态说明
+
+| 状态 | 含义 |
+|---|---|
+| 初始化 | SQL 编译优化阶段 |
+| 集群启动中 | 等待 VCluster 启动 |
+| 等待执行 | 排队等待资源 |
+| 正在执行 | 正在处理数据 |
+| 执行成功 | 运行结束 |
+| 执行失败 | 运行失败 |

package/bin/skills/clickzetta-query-optimizer/eval_cases.jsonl

@@ -0,0 +1,5 @@
+{"case_id":"001","type":"should_call","user_input":"为什么这个 JOIN 查询这么慢？SELECT t1.user_id, t2.tenant_name FROM public.dim_studio_user_dmin_f t1 JOIN public.dim_studio_tenant_dmin_f t2 ON t1.tenant_id = t2.tenant_id","expected_skill":"clickzetta-query-optimizer","expected_output_contains":["JOIN","优化"]}
+{"case_id":"002","type":"should_call","user_input":"public.dwd_studio_lakehouse_jobs_dd_i 表适合设置什么 Sort Key？","expected_skill":"clickzetta-query-optimizer","expected_output_contains":["sort","key"]}
+{"case_id":"003","type":"should_call","user_input":"怎么开启结果缓存？Result Cache 的使用限制是什么？","expected_skill":"clickzetta-query-optimizer","expected_output_contains":["cache","cz.sql.enable"]}
+{"case_id":"004","type":"should_call","user_input":"表有很多小文件影响查询性能，怎么合并优化？","expected_skill":"clickzetta-query-optimizer","expected_output_contains":["OPTIMIZE","小文件"]}
+{"case_id":"005","type":"should_call","user_input":"怎么用 EXPLAIN 分析执行计划？Map Join 什么时候用？","expected_skill":"clickzetta-query-optimizer","expected_output_contains":["EXPLAIN","Map Join"]}

package/bin/skills/cz-cli/SKILL.md

@@ -1,5 +1,5 @@
 ---
-name: cz-cli-v2
+name: cz-cli
 description: Route ALL ClickZetta Lakehouse operations to cz-cli: SQL, Studio tasks, tables, pipelines, profiles. Use when user mentions ClickZetta, Lakehouse, cz-cli, or needs profile/connection configuration.
 ---
 

package/bin/skills/cz-cli-inner/SKILL.md

@@ -70,6 +70,14 @@ cz-cli job result <job-id>                    Fetch job result set
 
 cz-cli status                                 Check connection status
 cz-cli profile list                           List connection profiles
+
+cz-cli datasource list [--type <type>] [--name <filter>]
+                                              List external data sources (type: mysql/kafka/redis/postgresql/...)
+cz-cli datasource catalogs <name_or_id>      List catalogs (databases/topics/buckets) in a data source
+cz-cli datasource objects <name_or_id> <catalog>
+                                              List objects (tables/topics/collections) in a catalog
+cz-cli datasource describe <name_or_id> <catalog> <object>
+                                              Show object metadata (columns, types)
 ```
 
 ## Output Formats

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@clickzetta/cz-cli-darwin-arm64",
-  "version": "0.3.18",
+  "version": "0.3.19",
   "description": "cz-cli binary for macOS ARM64 (Apple Silicon)",
   "os": ["darwin"],
   "cpu": ["arm64"],