@clickzetta/cz-cli-darwin-x64 0.3.87-dev.20260528223948 → 0.3.88

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. package/bin/cz-cli +0 -0
  2. package/bin/skills/clickzetta-dynamic-table/SKILL.md +169 -169
  3. package/bin/skills/clickzetta-dynamic-table/best-practices/dimension-table-join-guide.md +126 -126
  4. package/bin/skills/clickzetta-dynamic-table/best-practices/medallion-and-stream-patterns.md +25 -25
  5. package/bin/skills/clickzetta-dynamic-table/best-practices/non-partitioned-merge-into-warning.md +48 -48
  6. package/bin/skills/clickzetta-dynamic-table/best-practices/performance-optimization.md +51 -51
  7. package/bin/skills/clickzetta-dynamic-table/best-practices/scheduling-guide.md +59 -59
  8. package/bin/skills/clickzetta-dynamic-table/dt-creator/SKILL.md +8 -7
  9. package/bin/skills/clickzetta-dynamic-table/dt-creator/references/dt-declaration-strategy.md +99 -99
  10. package/bin/skills/clickzetta-dynamic-table/dt-creator/references/incremental-config-reference.md +188 -188
  11. package/bin/skills/clickzetta-dynamic-table/dt-creator/references/refresh-history-guide.md +117 -117
  12. package/bin/skills/clickzetta-dynamic-table/dt-creator/references/sql-limitations.md +29 -29
  13. package/bin/skills/clickzetta-dynamic-table/dynamic-table-alter/SKILL.md +80 -79
  14. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/SKILL.md +15 -15
  15. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-column-validation-rules.md +61 -61
  16. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-conversion-rules.md +100 -100
  17. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-placeholder-rules.md +64 -64
  18. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-refresh-rules.md +32 -32
  19. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-self-reference-rules.md +21 -21
  20. package/bin/skills/clickzetta-dynamic-table/sql-to-dt/references/sql2dt-workflow.md +71 -71
  21. package/bin/skills/clickzetta-sql-pipeline-manager/SKILL.md +203 -202
  22. package/bin/skills/clickzetta-sql-pipeline-manager/references/dynamic-table.md +62 -62
  23. package/bin/skills/clickzetta-sql-pipeline-manager/references/materialized-view.md +34 -34
  24. package/bin/skills/clickzetta-sql-pipeline-manager/references/pipe.md +61 -61
  25. package/bin/skills/clickzetta-sql-pipeline-manager/references/table-stream.md +41 -41
  26. package/bin/skills/clickzetta-table-stream-pipeline/SKILL.md +103 -101
  27. package/package.json +1 -1
@@ -1,34 +1,34 @@
1
- # Dynamic Table 自引用表转换规则
1
+ # Dynamic Table Self-referencing Table Conversion Rules
2
2
 
3
- 你是一个 SQL 转换专家。当 INSERT OVERWRITE 的目标表同时出现在查询的 FROM/JOIN 中时,这是一个自引用(self-reference)场景,需要特殊处理。
3
+ You are a SQL conversion expert. When the target table of INSERT OVERWRITE also appears in the FROM/JOIN of the query, this is a self-reference scenario that requires special handling.
4
4
 
5
- ## 自引用检测
5
+ ## Self-reference Detection
6
6
 
7
- ### 判断条件
7
+ ### Detection Criteria
8
8
 
9
- 1. INSERT OVERWRITE 语句中提取目标表名(含 schema)
10
- 2. SELECT 查询的 FROM JOIN 子句中搜索该表名
11
- 3. 排除 PARTITION 子句中的表名引用(不算自引用)
12
- 4. 如果在 FROM/JOIN 中找到目标表名 判定为自引用
9
+ 1. Extract the target table name (including schema) from the INSERT OVERWRITE statement
10
+ 2. Search for that table name in the FROM and JOIN clauses of the SELECT query
11
+ 3. Exclude table name references in the PARTITION clause (these do not count as self-references)
12
+ 4. If the target table name is found in FROM/JOIN → classify as self-reference
13
13
 
14
- ### 示例
14
+ ### Example
15
15
 
16
16
  ```sql
17
- -- 目标表: kscdm.daily_sales
17
+ -- Target table: kscdm.daily_sales
18
18
  INSERT OVERWRITE TABLE kscdm.daily_sales PARTITION(ds='${ds}')
19
19
  SELECT current.id, current.amount
20
20
  FROM source_sales current
21
- LEFT JOIN kscdm.daily_sales prev ON current.id = prev.id -- ← 自引用
21
+ LEFT JOIN kscdm.daily_sales prev ON current.id = prev.id -- ← self-reference
22
22
  WHERE prev.ds = '${ds - 1}';
23
23
  ```
24
24
 
25
- ## 转换规则
25
+ ## Conversion Rules
26
26
 
27
- 自引用表的转换与普通表基本相同,但有以下区别:
27
+ Self-referencing table conversion is essentially the same as regular tables, with the following differences:
28
28
 
29
- ### 1. 显式 Schema 声明
29
+ ### 1. Explicit Schema Declaration
30
30
 
31
- 自引用表必须在 CREATE DYNAMIC TABLE 中显式声明完整的列定义(含类型),因为 SQL 引擎需要这些信息来推理自依赖列的类型:
31
+ Self-referencing tables must explicitly declare complete column definitions (including types) in CREATE DYNAMIC TABLE, because the SQL engine needs this information to infer the types of self-dependent columns:
32
32
 
33
33
  ```sql
34
34
  CREATE OR REPLACE DYNAMIC TABLE kscdm.daily_sales (
@@ -45,23 +45,23 @@ LEFT JOIN kscdm.daily_sales prev ON current.id = prev.id
45
45
  WHERE prev.ds = DATE_FORMAT(sub_days(SESSION_CONFIGS()['dt.args.ds'], 1), 'yyyy-MM-dd')::STRING;
46
46
  ```
47
47
 
48
- ### 2. 查询中保留自引用
48
+ ### 2. Retain Self-reference in Query
49
49
 
50
- 转换后的 AS 子句中,自引用表名保持不变,不做任何替换。SQL 引擎会自动处理自引用的版本管理。
50
+ In the converted AS clause, the self-referencing table name remains unchanged without any substitution. The SQL engine automatically handles version management for self-references.
51
51
 
52
- ## 常见自引用场景
52
+ ## Common Self-reference Scenarios
53
53
 
54
- ### 日环比计算
54
+ ### Day-over-day Comparison
55
55
 
56
56
  ```sql
57
- -- 输入
57
+ -- Input
58
58
  INSERT OVERWRITE TABLE metrics PARTITION(ds='${ds}')
59
59
  SELECT t.id, t.value,
60
60
  t.value - prev.value AS daily_change
61
61
  FROM source t
62
62
  LEFT JOIN metrics prev ON t.id = prev.id AND prev.ds = '${ds - 1}';
63
63
 
64
- -- 输出
64
+ -- Output
65
65
  CREATE OR REPLACE DYNAMIC TABLE metrics (
66
66
  id BIGINT, value DECIMAL(10,2), daily_change DECIMAL(10,2), ds STRING
67
67
  )
@@ -1,109 +1,109 @@
1
- # SQL → Dynamic Table 完整转换工作流
1
+ # SQL → Dynamic Table Complete Conversion Workflow
2
2
 
3
- 当用户给你一组 CREATE TABLE DDL INSERT OVERWRITE SQL,要求转换为 Dynamic Table 时,按以下步骤顺序执行。
3
+ When the user gives you a set of CREATE TABLE DDL and INSERT OVERWRITE SQL and asks to convert them to a Dynamic Table, execute the following steps in order.
4
4
 
5
- 每一步的详细规则在对应的 skill 文件中,你需要同时引用它们。
5
+ The detailed rules for each step are in the corresponding skill files, which you need to reference simultaneously.
6
6
 
7
- ## 工作流步骤
7
+ ## Workflow Steps
8
8
 
9
- ### Step 1: 预处理输入
9
+ ### Step 1: Pre-process Input
10
10
 
11
- INSERT OVERWRITE 文件中移除:
12
- - 所有 `ALTER TABLE` 语句
13
- - `ANALYZE TABLE` 语句
14
- - SQL 注释(`--` `/* */`)
11
+ Remove from the INSERT OVERWRITE file:
12
+ - All `ALTER TABLE` statements
13
+ - `ANALYZE TABLE` statements
14
+ - SQL comments (`--` and `/* */`)
15
15
 
16
- 保留:CREATE TABLEINSERT OVERWRITEWITHSETCREATE TEMPORARY FUNCTION
16
+ Retain: CREATE TABLE, INSERT OVERWRITE, WITH, SET, CREATE TEMPORARY FUNCTION.
17
17
 
18
- ### Step 2: 占位符替换
18
+ ### Step 2: Placeholder Replacement
19
19
 
20
- #[[file:sql2dt-placeholder-rules.md]] 中的规则:
21
- 1. 统一占位符格式(`{{ }}` → `${ }`)
22
- 2. 替换所有占位符为 `SESSION_CONFIGS()` 调用
23
- 3. 处理 nodash 变量、日期运算、macros 函数
24
- 4. 根据引号上下文决定处理方式(去引号 / CONCAT / 直接替换)
20
+ Follow the rules in #[[file:sql2dt-placeholder-rules.md]]:
21
+ 1. Normalize placeholder format (`{{ }}` → `${ }`)
22
+ 2. Replace all placeholders with `SESSION_CONFIGS()` calls
23
+ 3. Handle nodash variables, date arithmetic, macros functions
24
+ 4. Decide handling based on quote context (remove quotes / CONCAT / direct replacement)
25
25
 
26
- ### Step 3: 自引用检测
26
+ ### Step 3: Self-reference Detection
27
27
 
28
- #[[file:sql2dt-self-reference-rules.md]] 中的规则:
29
- 1. 检查 INSERT OVERWRITE 目标表是否出现在 FROM/JOIN
30
- 2. 如果是自引用表,标记并在后续步骤中添加注释、使用显式 schema
28
+ Follow the rules in #[[file:sql2dt-self-reference-rules.md]]:
29
+ 1. Check whether the INSERT OVERWRITE target table appears in FROM/JOIN
30
+ 2. If it is a self-referencing table, mark it and add comments and use explicit schema in subsequent steps
31
31
 
32
- ### Step 4: 核心转换
32
+ ### Step 4: Core Conversion
33
33
 
34
- #[[file:sql2dt-conversion-rules.md]] 中的规则:
35
- 1. 解析 CREATE TABLE DDL(提取列、分区、属性等)
36
- 2. 解析 INSERT OVERWRITE(提取查询、分区类型)
37
- 3. 组装 `CREATE OR REPLACE DYNAMIC TABLE ... AS SELECT ...`
38
- 4. 注入静态分区值到 SELECT(智能引号处理)
39
- 5. 合并表属性模板(默认 `data_lifecycle=15`)
40
- 6. 处理 UNION ALL(每个分支独立注入)
41
- 7. 日期函数后处理:将所有 `DATE_SUB/DATE_ADD` 统一转为 `sub_days`
34
+ Follow the rules in #[[file:sql2dt-conversion-rules.md]]:
35
+ 1. Parse CREATE TABLE DDL (extract columns, partitions, properties, etc.)
36
+ 2. Parse INSERT OVERWRITE (extract query, partition type)
37
+ 3. Assemble `CREATE OR REPLACE DYNAMIC TABLE ... AS SELECT ...`
38
+ 4. Inject static partition values into SELECT (smart quote handling)
39
+ 5. Merge table property template (default `data_lifecycle=15`)
40
+ 6. Handle UNION ALL (inject into each branch independently)
41
+ 7. Date function post-processing: convert all `DATE_SUB/DATE_ADD` to `sub_days`
42
42
 
43
- ### Step 5: 列校验
43
+ ### Step 5: Column Validation
44
44
 
45
- #[[file:sql2dt-column-validation-rules.md]] 中的规则:
46
- 1. 计算 schema 列数和 SELECT 列数
47
- 2. 验证两者相等
48
- 3. 检查重复别名和缺失分区列
49
- 4. UNION ALL 分支列数一致性检查
45
+ Follow the rules in #[[file:sql2dt-column-validation-rules.md]]:
46
+ 1. Count schema columns and SELECT columns
47
+ 2. Verify they are equal
48
+ 3. Check for duplicate aliases and missing partition columns
49
+ 4. UNION ALL branch column count consistency check
50
50
 
51
- ### Step 6: 生成配套文件
51
+ ### Step 6: Generate Companion Files
52
52
 
53
- #[[file:sql2dt-refresh-rules.md]] 中的规则:
54
- 1. DDL 中提取所有 SESSION_CONFIGS 变量
55
- 2. 生成当前周期 refresh 语句
56
- 3. 生成上一周期 prev_refresh 语句
57
- 4. 生成回填 backfill 语句
53
+ Follow the rules in #[[file:sql2dt-refresh-rules.md]]:
54
+ 1. Extract all SESSION_CONFIGS variables from the DDL
55
+ 2. Generate current-cycle refresh statement
56
+ 3. Generate previous-cycle prev_refresh statement
57
+ 4. Generate backfill statement
58
58
 
59
- ### Step 7: 转换后改进建议
59
+ ### Step 7: Post-conversion Improvement Suggestions
60
60
 
61
- DDL 生成完成后,对转换结果做以下检查,并主动向用户提出改进建议:
61
+ After DDL generation is complete, check the conversion result and proactively offer improvement suggestions to the user:
62
62
 
63
- **检查 1:非分区表 + 持续写入风险**
63
+ **Check 1: Non-partitioned table + continuous write risk**
64
64
 
65
- #[[file:../best-practices/non-partitioned-merge-into-warning.md]] 中的判断逻辑:
66
- - 生成的 DT 是非分区表(无 `PARTITIONED BY` 也无 `SESSION_CONFIGS()`)
67
- - SQL 中包含 `ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ... DESC) WHERE rn = 1` 去重模式
65
+ Follow the judgment logic in #[[file:../best-practices/non-partitioned-merge-into-warning.md]]:
66
+ - The generated DT is a non-partitioned table (no `PARTITIONED BY` and no `SESSION_CONFIGS()`)
67
+ - And the SQL contains the `ROW_NUMBER() OVER (PARTITION BY ... ORDER BY ... DESC) WHERE rn = 1` deduplication pattern
68
68
 
69
- 满足条件时,使用该文档中的告警话术模板向用户发出风险提示,并建议改用 MERGE INTO + Table Stream 方案。
69
+ When conditions are met, use the alert message template from that document to warn the user of the risk, and suggest switching to the MERGE INTO + Table Stream approach.
70
70
 
71
- **检查 2SQL 性能优化机会**
71
+ **Check 2: SQL performance optimization opportunities**
72
72
 
73
- #[[file:../best-practices/performance-optimization.md]] 中的规则,扫描生成的 DT SQL
74
- - 存在 `LEFT/RIGHT/FULL OUTER JOIN` → 提示如果业务允许,改用 INNER JOIN 可提升增量效率
75
- - 存在无 `PARTITION BY` 的窗口函数 提示添加 PARTITION BY,否则每次增量都全量重算
76
- - `GROUP BY` 使用了复杂表达式(如 `DATE_TRUNC`、`SUBSTR`)→ 提示考虑在上游预计算或拆分为多级 DT
73
+ Follow the rules in #[[file:../best-practices/performance-optimization.md]], scan the generated DT SQL:
74
+ - Contains `LEFT/RIGHT/FULL OUTER JOIN` → suggest switching to INNER JOIN if business allows, to improve incremental efficiency
75
+ - Contains window functions without `PARTITION BY` → suggest adding PARTITION BY; otherwise every incremental refresh will do a full recomputation
76
+ - `GROUP BY` uses complex expressions (e.g., `DATE_TRUNC`, `SUBSTR`) suggest pre-computing upstream or splitting into multi-level DTs
77
77
 
78
- **检查 3:JOIN 中是否有维度表**
78
+ **Check 3: Whether there are dimension tables in JOINs**
79
79
 
80
- #[[file:../best-practices/dimension-table-join-guide.md]] 中的推荐场景:
81
- - SQL 中存在 JOIN → 询问用户右侧表是否为低频变更的维度表(码表、字典表、配置表等)
82
- - 如果是建议在 TBLPROPERTIES 中添加 `mv_const_tables` 配置,并说明其行为和数据一致性权衡
80
+ Follow the recommended scenarios in #[[file:../best-practices/dimension-table-join-guide.md]]:
81
+ - SQL contains JOIN → ask the user whether the right-side table is a low-frequency-change dimension table (lookup table, dictionary table, config table, etc.)
82
+ - If yes suggest adding `mv_const_tables` configuration in TBLPROPERTIES, and explain its behavior and data consistency tradeoffs
83
83
 
84
- ## 输出清单
84
+ ## Output Checklist
85
85
 
86
- 对每个表,最终输出:
86
+ For each table, the final output is:
87
87
 
88
- | 文件 | 内容 | 条件 |
88
+ | File | Content | Condition |
89
89
  |------|------|------|
90
- | `表名.sql` | Dynamic Table DDL | 始终生成 |
91
- | `表名_refresh.sql` | 当前周期 REFRESH 语句 | 始终生成 |
92
- | `表名_prev_refresh.sql` | 上一周期 REFRESH 语句 | 仅有分区变量时 |
93
- | `表名_backfill.sql` | 回填语句 | 仅有分区变量时 |
90
+ | `table_name.sql` | Dynamic Table DDL | Always generated |
91
+ | `table_name_refresh.sql` | Current-cycle REFRESH statement | Always generated |
92
+ | `table_name_prev_refresh.sql` | Previous-cycle REFRESH statement | Only when partition variables exist |
93
+ | `table_name_backfill.sql` | Backfill statement | Only when partition variables exist |
94
94
 
95
- ## 快速判断路径
95
+ ## Quick Decision Path
96
96
 
97
97
  ```
98
- 输入 DDL + INSERT OVERWRITE
98
+ Input DDL + INSERT OVERWRITE
99
99
 
100
- ├─ 有占位符? → Step 2 占位符替换
100
+ ├─ Has placeholders? → Step 2 placeholder replacement
101
101
 
102
- ├─ 自引用? → Step 3 特殊处理
102
+ ├─ Self-reference? → Step 3 special handling
103
103
 
104
- ├─ 有静态分区? → Step 4 注入分区值到 SELECT
104
+ ├─ Has static partitions? → Step 4 inject partition values into SELECT
105
105
 
106
- ├─ UNION ALL → Step 4 每个分支独立注入
106
+ ├─ Has UNION ALL? → Step 4 inject into each branch independently
107
107
 
108
- └─ 生成 DDL → Step 5 校验 → Step 6 生成配套文件 → Step 7 改进建议
108
+ └─ Generate DDL → Step 5 validate → Step 6 generate companion files → Step 7 improvement suggestions
109
109
  ```