@clickzetta/cz-cli-darwin-arm64 0.3.94 → 0.3.96

package/bin/skills/clickzetta-sql-pipeline-manager/references/dynamic-table.md

@@ -1,14 +1,14 @@
 # Dynamic Table SQL Reference
 
-> **⚠️ ClickZetta-specific syntax**
-> - Refresh schedule syntax: `REFRESH INTERVAL 5 MINUTE vcluster default` (not `TARGET_LAG`)
-> - Modifying the schedule interval or compute cluster requires `CREATE OR REPLACE`; `ALTER` does not support this
+> **⚠️ ClickZetta-specific Syntax**
+> - Refresh scheduling: `REFRESH INTERVAL 5 MINUTE vcluster default` (not `TARGET_LAG`)
+> - Modifying refresh interval or compute cluster requires `CREATE OR REPLACE`, `ALTER` does not support this
 > - `ALTER DYNAMIC TABLE` only supports: SUSPEND / RESUME / SET COMMENT / RENAME COLUMN / CHANGE COLUMN COMMENT / SET/UNSET PROPERTIES
-> - Drop with `DROP DYNAMIC TABLE` (not `DROP TABLE`)
-> - Restore with `UNDROP TABLE` (not `UNDROP DYNAMIC TABLE`)
-> - Describe with `DESC TABLE name` (does not support `DESC DYNAMIC TABLE name EXTENDED`)
+> - Drop using `DROP DYNAMIC TABLE` (not `DROP TABLE`)
+> - Restore using `UNDROP TABLE` (not `UNDROP DYNAMIC TABLE`)
+> - DESC using `DESC TABLE name` (does not support `DESC DYNAMIC TABLE name EXTENDED`)
 
-Dynamic Tables are the core incremental computation objects in ClickZetta Lakehouse. Defined by a SQL query, they refresh automatically and incrementally without manual scheduling.
+Dynamic Tables are core incremental computation objects in ClickZetta Lakehouse. They are defined by SQL queries and refresh incrementally without manual scheduling.
 
 ## CREATE DYNAMIC TABLE
 
@@ -25,11 +25,11 @@ AS
   <query>;
 ```
 
-**Key parameters:**
-- `REFRESH INTERVAL <n> MINUTE`: refresh interval, minimum 1 minute
-- `vcluster`: name of the compute cluster to run refresh jobs (name directly, no equals sign or quotes)
-- `OR REPLACE`: replaces an existing Dynamic Table with the same name (required when modifying SQL logic or scheduling config)
-- Recommended: use a GP-type cluster (e.g., `default`); AP-type clusters do not support small file compaction
+**Key Parameters:**
+- `REFRESH INTERVAL <n> MINUTE`: Refresh interval, minimum 1 minute
+- `vcluster`: Name of compute cluster for running refresh tasks (directly followed by name, no equals sign or quotes)
+- `OR REPLACE`: Replace if dynamic table with same name exists (must use this to modify SQL logic or scheduling configuration)
+- Recommended to use GP-type cluster (e.g. `default`), AP-type clusters do not support small file compaction
 
 **Examples:**
 ```sql
@@ -81,62 +81,62 @@ ALTER DYNAMIC TABLE <name> SET PROPERTIES ('key' = 'value');
 ALTER DYNAMIC TABLE <name> UNSET PROPERTIES ('key');
 ```
 
-> Note: To modify the refresh interval, compute cluster, or SQL query logic, use `CREATE OR REPLACE DYNAMIC TABLE`. ALTER does not support these operations.
+> Note: To modify refresh interval, compute cluster, or SQL query logic, must use `CREATE OR REPLACE DYNAMIC TABLE`, ALTER does not support these operations.
 
 ## REFRESH DYNAMIC TABLE (Manual Trigger)
 
 ```sql
--- Manually trigger a single refresh
+-- Manually trigger one refresh
 REFRESH DYNAMIC TABLE <name>;
 ```
 
 ## DROP DYNAMIC TABLE
 
 ```sql
--- ⚠️ Must use DROP DYNAMIC TABLE, not DROP TABLE
+-- ⚠️ Must use DROP DYNAMIC TABLE, cannot use DROP TABLE
 DROP DYNAMIC TABLE [ IF EXISTS ] <name>;
 
--- Restore a dropped Dynamic Table (⚠️ use UNDROP TABLE, not UNDROP DYNAMIC TABLE)
+-- Restore deleted dynamic table (⚠️ use UNDROP TABLE, not UNDROP DYNAMIC TABLE)
 UNDROP TABLE <name>;
 ```
 
 ## SHOW / DESC
 
 ```sql
--- List all Dynamic Tables in the current schema
+-- List all dynamic tables in current schema
 SHOW TABLES WHERE is_dynamic = true;
 
--- List Dynamic Tables in a specific schema
+-- List dynamic tables in specified schema
 SHOW TABLES IN <schema_name> WHERE is_dynamic = true;
 
--- View Dynamic Table structure
+-- View dynamic table structure
 DESC TABLE <name>;
 
--- View full CREATE statement
+-- View complete CREATE TABLE statement
 SHOW CREATE TABLE <name>;
 
--- View refresh history (status, duration, trigger type, incremental row count)
+-- View refresh history (status, duration, trigger method, incremental rows)
 SHOW DYNAMIC TABLE REFRESH HISTORY WHERE name = '<dt_name>' LIMIT 20;
 ```
 
-> ⚠️ **DESC note**: Use `DESC TABLE name` for Dynamic Tables. `DESC DYNAMIC TABLE name EXTENDED` is not supported (EXTENDED will cause an error).
+> ⚠️ **DESC Note**: For dynamic tables use `DESC TABLE name`, does not support `DESC DYNAMIC TABLE name EXTENDED` (EXTENDED will error).
 
-## Notes
+## Important Notes
 
-- To modify SQL logic, refresh interval, or compute cluster → use `CREATE OR REPLACE`; `ALTER` is not supported for these
+- To modify SQL logic, refresh interval, compute cluster → use `CREATE OR REPLACE`, cannot use `ALTER`
 - Minimum refresh interval is 1 minute
-- Drop with `DROP DYNAMIC TABLE` (not `DROP TABLE`)
-- Restore with `UNDROP TABLE` (not `UNDROP DYNAMIC TABLE`)
-- Refresh failures do not affect queryability (returns data from the last successful version)
-- A `CREATE OR REPLACE` that is not a simple add/drop column will trigger a full refresh
-- Recommended: use a GP-type cluster (e.g., `default`); AP-type clusters do not support small file compaction
+- Drop using `DROP DYNAMIC TABLE` (not `DROP TABLE`)
+- Restore using `UNDROP TABLE` (not `UNDROP DYNAMIC TABLE`)
+- Refresh failures do not affect table queryability (returns data from last successful version)
+- Non-simple add/drop column `CREATE OR REPLACE` triggers a full refresh
+- Recommended to use GP-type cluster (e.g. `default`), AP-type clusters do not support small file compaction
 
-## Parameterized Dynamic Table (SESSION_CONFIGS)
+## Parameterized Dynamic Tables (SESSION_CONFIGS)
 
-Use the `SESSION_CONFIGS()` function to define parameterized queries, passing partition values at refresh time to control the refresh scope:
+Use `SESSION_CONFIGS()` function to define parameterized queries, pass partition values during refresh to control refresh scope:
 
 ```sql
--- Create a parameterized Dynamic Table
+-- Create parameterized dynamic table
 CREATE OR REPLACE DYNAMIC TABLE dwd.orders_partitioned
   REFRESH INTERVAL 30 MINUTE vcluster default
 AS
@@ -144,19 +144,19 @@ SELECT order_id, user_id, amount, dt
 FROM ods.orders
 WHERE dt = SESSION_CONFIGS('target_date', CAST(CURRENT_DATE() AS STRING));
 
--- Manually trigger refresh with parameters
+-- Manually trigger refresh with parameter
 REFRESH DYNAMIC TABLE dwd.orders_partitioned
   WITH PROPERTIES ('target_date' = '2024-06-15');
 ```
 
-Use case: migrating traditional daily full ETL jobs to incremental jobs, replacing scheduling variables with SESSION_CONFIGS.
+Use case: Transforming traditional daily full ETL into incremental tasks, use SESSION_CONFIGS to replace scheduling variables.
 
 ## Dynamic Table DML Operations
 
-Dynamic Tables do not support DML by default. You must enable the parameter first (must be set before each DML operation):
+Dynamic tables do not support DML by default, must enable parameter first (need SET before each DML):
 
 ```sql
--- ⚠️ Must execute SET in the same session/batch before the DML
+-- ⚠️ Must execute SET in same session/batch, then execute DML
 SET cz.sql.dt.allow.dml = true;
 INSERT INTO <name> VALUES (...);
 
@@ -165,11 +165,11 @@ SET cz.sql.dt.allow.dml = true;
 DELETE FROM <name> WHERE ...;
 ```
 
-> ⚠️ **DML notes**:
-> - `SET cz.sql.dt.allow.dml = true` must be in the same execution batch as the DML statement
-> - After a DML operation, the next automatic refresh will trigger a **full refresh** (not incremental), which may take longer
-> - UPDATE may fail due to internal hidden columns (`MV__KEY`); use DELETE + INSERT instead
-> - Use DML only for special cases such as data correction
+> ⚠️ **DML Notes**:
+> - `SET cz.sql.dt.allow.dml = true` must be in same execution batch with DML statement
+> - After executing DML, next auto-refresh will trigger **full refresh** (not incremental), may take longer
+> - UPDATE may error due to internal hidden column (`MV__KEY`), recommend using DELETE + INSERT instead
+> - Only use DML in special scenarios like data correction
 
 ## Reference Documentation
 
@@ -178,8 +178,8 @@ DELETE FROM <name> WHERE ...;
 - [DROP DYNAMIC TABLE](https://www.yunqi.tech/documents/drop-dynamic-table)
 - [SHOW DYNAMIC TABLES](https://www.yunqi.tech/documents/show-dynamic-table)
 - [SHOW DYNAMIC TABLE REFRESH HISTORY](https://www.yunqi.tech/documents/refresh-history)
-- [Dynamic Table Overview](https://www.yunqi.tech/documents/dynamic_table_summary)
+- [Dynamic Table Introduction](https://www.yunqi.tech/documents/dynamic_table_summary)
 - [View Dynamic Table Refresh Mode](https://www.yunqi.tech/documents/dynamic-table-incre)
-- [Migrating Traditional Offline Jobs to Incremental](https://www.yunqi.tech/documents/transformt-dt)
-- [Parameterized Dynamic Table](https://www.yunqi.tech/documents/dynamicTable-parmaters)
-- [Dynamic Table DML Support](https://www.yunqi.tech/documents/dynamicTable-dml)
+- [Traditional Offline Task to Incremental Practice](https://www.yunqi.tech/documents/transformt-dt)
+- [Dynamic Table Parameterized Definition Support](https://www.yunqi.tech/documents/dynamicTable-parmaters)
+- [Dynamic Table DML Statement Support](https://www.yunqi.tech/documents/dynamicTable-dml)

package/bin/skills/clickzetta-sql-pipeline-manager/references/materialized-view.md

@@ -1,11 +1,11 @@
 # Materialized View SQL Reference
 
-> **⚠️ ClickZetta-specific syntax**
-> - Scheduled refresh: `REFRESH INTERVAL 10 MINUTE vcluster default` (same syntax as Dynamic Table)
+> **⚠️ ClickZetta-specific Syntax**
+> - Scheduled refresh: `REFRESH INTERVAL 10 MINUTE vcluster default` (same syntax as dynamic tables)
 > - Manual refresh: `REFRESH MATERIALIZED VIEW <name>;`
-> - Modify comments with `ALTER TABLE`, not `ALTER MATERIALIZED VIEW`
+> - Modify comment using `ALTER TABLE`, not `ALTER MATERIALIZED VIEW`
 
-Materialized Views pre-compute and physically store query results, making them ideal for fixed-dimension aggregation acceleration. Unlike Dynamic Tables, Materialized Views support manual or scheduled refresh but do not support incremental refresh.
+Materialized views pre-compute query results and store them physically, suitable for fixed-dimension aggregation acceleration scenarios. Difference from dynamic tables: materialized views support manual or scheduled refresh, do not support incremental refresh.
 
 ## CREATE MATERIALIZED VIEW
 
@@ -19,15 +19,15 @@ AS
   <query>;
 ```
 
-**Key parameters:**
-- `REFRESH INTERVAL 10 MINUTE vcluster default`: scheduled automatic refresh (same syntax as Dynamic Table)
-- Omitting the REFRESH clause: only manual refresh via `REFRESH MATERIALIZED VIEW <name>;`
-- `BUILD DEFERRED`: deferred build — does not compute results immediately at creation time
-- `DISABLE QUERY REWRITE`: disables query rewrite (MV will not automatically accelerate queries)
+**Key Parameters:**
+- `REFRESH INTERVAL 10 MINUTE vcluster default`: Scheduled auto-refresh (same syntax as dynamic tables)
+- Without REFRESH clause: can only manually trigger `REFRESH MATERIALIZED VIEW <name>;`
+- `BUILD DEFERRED`: Deferred build, does not compute results immediately upon creation
+- `DISABLE QUERY REWRITE`: Disable query rewrite (does not automatically use MV to accelerate queries)
 
 **Examples:**
 ```sql
--- Materialized View with scheduled auto-refresh (every 10 minutes)
+-- Scheduled auto-refresh materialized view (every 10 minutes)
 CREATE MATERIALIZED VIEW mv_dept_stats
 REFRESH INTERVAL 10 MINUTE vcluster default
 AS
@@ -64,25 +64,25 @@ REFRESH MATERIALIZED VIEW mv_dept_stats;
 ## ALTER MATERIALIZED VIEW
 
 ```sql
--- Suspend automatic refresh
+-- Suspend auto-refresh
 ALTER MATERIALIZED VIEW <name> SUSPEND;
 
--- Resume automatic refresh
+-- Resume auto-refresh
 ALTER MATERIALIZED VIEW <name> RESUME;
 
 -- Modify comment
 ALTER TABLE <mv_name> SET COMMENT '<comment>';
 
--- Modify column comment (Materialized Views use ALTER TABLE syntax)
+-- Modify column comment (materialized view uses ALTER TABLE syntax)
 ALTER TABLE <mv_name> CHANGE COLUMN <col_name> COMMENT '<comment>';
 ```
 
-> Note: Use `ALTER TABLE` (not `ALTER MATERIALIZED VIEW`) to modify comments on a Materialized View.
+> Note: Modifying comments for materialized views uses `ALTER TABLE`, not `ALTER MATERIALIZED VIEW`.
 
 ## REFRESH MATERIALIZED VIEW
 
 ```sql
--- Manually trigger a full refresh
+-- Manually trigger full refresh
 REFRESH MATERIALIZED VIEW <name>;
 ```
 
@@ -95,28 +95,28 @@ DROP MATERIALIZED VIEW [ IF EXISTS ] <name>;
 ## SHOW / DESC
 
 ```sql
--- List all Materialized Views in the current schema
+-- List all materialized views in current schema
 SHOW TABLES WHERE is_materialized_view = true;
 
 -- Filter by name
 SHOW TABLES LIKE 'mv_%' WHERE is_materialized_view = true;
 
--- View Materialized View structure
+-- View materialized view structure
 DESC MATERIALIZED VIEW <name>;
 DESCRIBE MATERIALIZED VIEW <name> EXTENDED;
 
--- View full CREATE statement
+-- View complete CREATE TABLE statement
 SHOW CREATE TABLE <name>;
 ```
 
-## Dynamic Table vs Materialized View — Selection Guide
+## Dynamic Table vs Materialized View Selection Guide
 
-| Scenario | Recommended |
+| Scenario | Recommendation |
 |---|---|
-| Need second/minute-level automatic incremental refresh | Dynamic Table |
+| Need second/minute-level auto incremental refresh | Dynamic Table |
 | Fixed aggregation, manual or low-frequency refresh | Materialized View |
-| Need CDC change detection | Dynamic Table + Table Stream |
-| Accelerate BI queries, real-time data not required | Materialized View |
+| Need CDC change awareness | Dynamic Table + Table Stream |
+| Accelerate BI queries, data does not need to be real-time | Materialized View |
 
 ## Reference Documentation
 
@@ -125,5 +125,5 @@ SHOW CREATE TABLE <name>;
 - [REFRESH MATERIALIZED VIEW](https://www.yunqi.tech/documents/REFRESH)
 - [DROP MATERIALIZED VIEW](https://www.yunqi.tech/documents/DROPMATERIALIZEDVIEW)
 - [SHOW MATERIALIZED VIEWS](https://www.yunqi.tech/documents/show-materialized-view)
-- [Materialized View Concepts and Use Cases](https://www.yunqi.tech/documents/MATERIALIZEDVIEW)
-- [Materialized View DDL Summary](https://www.yunqi.tech/documents/materialized_ddl)
+- [Materialized View Concepts and Scenarios](https://www.yunqi.tech/documents/MATERIALIZEDVIEW)
+- [Materialized View DDL Summary](https://www.yunqi.tech/documents/materialized_ddl)

package/bin/skills/clickzetta-sql-pipeline-manager/references/pipe.md

@@ -1,14 +1,14 @@
 # Pipe SQL Reference
 
-> **⚠️ ClickZetta-specific syntax**
-> - The Kafka read function is `read_kafka(...)`, using **positional parameters** (not named parameters with `=>`)
+> **⚠️ ClickZetta-specific Syntax**
+> - Kafka read function is `read_kafka(...)`, using **positional parameters** (not named parameters `=>`)
 > - JSON field extraction uses `parse_json(value::string)['field']::TYPE` syntax
-> - A Pipe starts automatically after creation; no manual RESUME is needed
-> - For OSS Pipes, `PURGE=true` follows immediately after `USING <format>` (e.g., `USING CSV PURGE=true`)
+> - Pipe auto-starts after creation, no need for manual RESUME
+> - OSS Pipe's `PURGE=true` follows immediately after `USING <format>` (e.g. `USING CSV PURGE=true`)
 
-Pipe is the continuous data ingestion object in ClickZetta Lakehouse. Defined by SQL, it automatically and continuously imports data from Kafka or object storage (OSS/S3/COS) into a target table without external scheduling.
+Pipe is ClickZetta Lakehouse's continuous data ingestion object, defined by SQL to automatically and continuously import data from Kafka or object storage (OSS/S3/COS) to target tables without external scheduling.
 
-## CREATE PIPE — Ingest from Kafka
+## CREATE PIPE — Import from Kafka
 
 ```sql
 CREATE [ OR REPLACE ] PIPE <pipe_name>
@@ -21,13 +21,13 @@ AS
 COPY INTO <target_table> FROM (
   SELECT <expr> [, ...]
   FROM read_kafka(
-    '<bootstrap_servers>',   -- required: Kafka cluster address
-    '<topic>',               -- required: topic name
-    '',                      -- reserved (leave empty string)
-    '<group_id>',            -- required: persistent consumer group ID
-    '', '', '', '',          -- positional params left empty, managed by Pipe automatically
-    'raw',                   -- key format (only 'raw' supported currently)
-    'raw',                   -- value format (only 'raw' supported currently)
+    '<bootstrap_servers>',   -- Required: Kafka cluster address
+    '<topic>',               -- Required: Topic name
+    '',                      -- Reserved (fill with empty string)
+    '<group_id>',            -- Required: Persistent consumer group ID
+    '', '', '', '',          -- Positional parameters left empty, managed by Pipe
+    'raw',                   -- key format (currently only supports raw)
+    'raw',                   -- value format (currently only supports raw)
     0,                       -- max_errors
     MAP(<kafka_config>)      -- Kafka configuration parameters
   )
@@ -36,7 +36,7 @@ COPY INTO <target_table> FROM (
 
 **Examples:**
 ```sql
--- Continuously ingest JSON data from Kafka
+-- Continuously import JSON data from Kafka
 CREATE OR REPLACE PIPE kafka_orders_pipe
   VIRTUAL_CLUSTER = 'default'
   BATCH_INTERVAL_IN_SECONDS = '60'
@@ -83,9 +83,9 @@ COPY INTO ods.secure_events FROM (
 );
 ```
 
-## Verify Kafka Connection (Before Creating a Pipe)
+## Verify Kafka Connection (Before Creating Pipe)
 
-When using `read_kafka` standalone to explore data, you can set `kafka.auto.offset.reset` in the MAP:
+When independently using `read_kafka` to explore data, you can set `kafka.auto.offset.reset` in MAP:
 
 ```sql
 -- Verify connection and data format
@@ -102,11 +102,11 @@ FROM read_kafka(
 LIMIT 10;
 ```
 
-> ⚠️ **Standalone exploration vs inside a Pipe**:
-> - Standalone exploration: you can set `kafka.auto.offset.reset` to `earliest` in the MAP to read historical data
-> - Inside a Pipe: positional parameters must be left empty; the consumer offset is controlled by the Pipe's `RESET_KAFKA_GROUP_OFFSETS` parameter
+> ⚠️ **Independent Exploration vs In Pipe Difference**:
+> - Independent exploration: can set `kafka.auto.offset.reset` to `earliest` in MAP to read historical data
+> - In Pipe: positional parameters must be left empty, consumption offset controlled by Pipe's `RESET_KAFKA_GROUP_OFFSETS` parameter
 
-## CREATE PIPE — Ingest from Object Storage
+## CREATE PIPE — Import from Object Storage
 
 ```sql
 CREATE [ OR REPLACE ] PIPE [ IF NOT EXISTS ] <pipe_name>
@@ -120,13 +120,13 @@ FROM VOLUME <volume_name>
 USING <csv | parquet | orc | json> [OPTIONS ('<key>' = '<value>', ...)] PURGE=true;
 ```
 
-**Key parameters:**
-- `VIRTUAL_CLUSTER`: specifies the virtual cluster name (required for OSS Pipes)
-- `INGEST_MODE = 'LIST_PURGE'`: general mode, periodically scans the file list; `PURGE=true` must be set
-- `INGEST_MODE = 'EVENT_NOTIFICATION'`: event notification mode, low latency (Alibaba Cloud OSS + AWS S3 only); `PURGE=true` is not required
-- `COMMENT 'text'`: no equals sign (`COMMENT = 'text'` will cause an error)
-- `PURGE=true`: placed at the end, after OPTIONS: `USING CSV OPTIONS (...) PURGE=true`
-- COPY statements inside a PIPE do not support `files`, `regexp`, or `subdirectory` parameters
+**Key Parameters:**
+- `VIRTUAL_CLUSTER`: Specify virtual cluster name (required for OSS Pipe)
+- `INGEST_MODE = 'LIST_PURGE'`: General mode, periodically scan file list, must set `PURGE=true`
+- `INGEST_MODE = 'EVENT_NOTIFICATION'`: Event notification mode, low latency (only Alibaba Cloud OSS + AWS S3), does not need `PURGE=true`
+- `COMMENT 'text'`: Without equals sign (`COMMENT = 'text'` will error)
+- `PURGE=true`: Place at end, OPTIONS before it: `USING CSV OPTIONS (...) PURGE=true`
+- COPY statement in PIPE does not support `files`, `regexp`, `subdirectory` parameters
 
 **Examples:**
 ```sql
@@ -149,7 +149,7 @@ COPY INTO ods.csv_data
 FROM VOLUME my_csv_volume
 USING CSV OPTIONS ('header' = 'true', 'sep' = ',') PURGE=true;
 
--- EVENT_NOTIFICATION mode (PURGE not required)
+-- EVENT_NOTIFICATION mode (does not need PURGE)
 CREATE PIPE oss_event_pipe
   VIRTUAL_CLUSTER = 'default'
   INGEST_MODE = 'EVENT_NOTIFICATION'
@@ -160,10 +160,10 @@ FROM VOLUME my_oss_event_volume
 USING PARQUET;
 ```
 
-## Start / Stop a Pipe
+## Start/Stop Pipe
 
 ```sql
--- Pause Pipe
+-- Suspend Pipe
 ALTER PIPE <pipe_name> SET PIPE_EXECUTION_PAUSED = true;
 
 -- Resume Pipe
@@ -173,20 +173,20 @@ ALTER PIPE <pipe_name> SET PIPE_EXECUTION_PAUSED = false;
 ## Modify Pipe Properties
 
 ```sql
--- Only one property can be modified at a time
+-- Can only modify one property at a time
 ALTER PIPE <pipe_name> SET VIRTUAL_CLUSTER = 'new_vc';
 ALTER PIPE <pipe_name> SET COPY_JOB_HINT = '{"cz.sql.split.kafka.strategy":"size","cz.mapper.kafka.message.size":"200000"}';
 ```
 
-> ⚠️ **Supported ALTER PIPE properties**:
+> ⚠️ **ALTER PIPE Supported Properties**:
 > - ✅ `PIPE_EXECUTION_PAUSED`
 > - ✅ `VIRTUAL_CLUSTER`
 > - ✅ `COPY_JOB_HINT`
-> - ❌ `BATCH_INTERVAL_IN_SECONDS` (not supported; must drop and recreate)
-> - ❌ `BATCH_SIZE_PER_KAFKA_PARTITION` (not supported; must drop and recreate)
+> - ❌ `BATCH_INTERVAL_IN_SECONDS` (not supported to modify, needs drop and recreate)
+> - ❌ `BATCH_SIZE_PER_KAFKA_PARTITION` (not supported to modify, needs drop and recreate)
 >
-> Modifying the COPY/INSERT statement logic is not supported; drop the Pipe and recreate it.
-> Modifying `COPY_JOB_HINT` overwrites all existing hints; all parameters must be set at once.
+> Does not support modifying COPY/INSERT statement logic, needs to drop Pipe then recreate.
+> `COPY_JOB_HINT` modification will overwrite all existing hints, need to set all parameters at once.
 
 ## DROP PIPE
 
@@ -197,7 +197,7 @@ DROP PIPE [ IF EXISTS ] <pipe_name>;
 ## SHOW PIPE
 
 ```sql
--- List all Pipes in the current schema
+-- List all Pipes in current schema
 SHOW PIPES;
 
 -- View Pipe details (status, latency, definition)
@@ -205,18 +205,18 @@ DESC PIPE <pipe_name>;
 DESC PIPE EXTENDED <pipe_name>;
 ```
 
-## Notes
+## Important Notes
 
-- A Pipe starts automatically after creation; no manual RESUME is needed
-- Kafka Pipes use a consumer group to manage offsets; keeping the same group_id when recreating a Pipe allows resuming from the last offset
-- Object storage Pipes detect new files via file list scanning or event notifications; `load_history` deduplication records are retained for 7 days
-- Pipes do not support modifying the AS clause; drop and recreate (not `CREATE OR REPLACE`)
-- Kafka Pipes only support PLAINTEXT and SASL_PLAINTEXT security protocols; SSL is not supported
+- Pipe auto-starts after creation, no need for manual RESUME
+- Kafka Pipe uses consumer group to manage offset, recreating Pipe with same group_id can continue from last offset
+- Object storage Pipe detects new files through file list or event notification, `load_history` deduplication records retained for 7 days
+- Pipe does not support modifying AS clause, needs to drop then recreate (not `CREATE OR REPLACE`)
+- Kafka Pipe only supports PLAINTEXT and SASL_PLAINTEXT security protocols, does not support SSL
 
 ## Reference Documentation
 
-- [Pipe Overview](https://www.yunqi.tech/documents/pipe-summary)
-- [Continuous Ingestion with read_kafka](https://www.yunqi.tech/documents/pipe-kafka)
-- [Continuous Ingestion with Kafka External Table Stream](https://www.yunqi.tech/documents/pipe-kafka-table-stream)
-- [Best Practices: Efficient Kafka Ingestion with Pipe](https://www.yunqi.tech/documents/pipe-kafka-bestpractice-1)
-- [Continuous Ingestion from Object Storage with Pipe](https://www.yunqi.tech/documents/pipe-storage-object)
+- [Pipe Introduction](https://www.yunqi.tech/documents/pipe-summary)
+- [Continuous Import with read_kafka Function](https://www.yunqi.tech/documents/pipe-kafka)
+- [Continuous Import with Kafka External Table Stream](https://www.yunqi.tech/documents/pipe-kafka-table-stream)
+- [Best Practice: Efficiently Ingesting Kafka Data with Pipe](https://www.yunqi.tech/documents/pipe-kafka-bestpractice-1)
+- [Continuous Import of Object Storage Data with Pipe](https://www.yunqi.tech/documents/pipe-storage-object)

package/bin/skills/clickzetta-sql-pipeline-manager/references/scenarios.md

@@ -0,0 +1,141 @@
+# SQL Pipeline Scenarios
+
+Worked examples for common pipeline patterns. Read this file when the user asks to build a specific type of pipeline.
+
+## Scenario A: Kafka → Dynamic Table (real-time ETL)
+
+```sql
+-- Step 1: Create Pipe to continuously ingest Kafka data to ODS layer
+-- ⚠️ ClickZetta does not support CREATE OR REPLACE PIPE — use CREATE PIPE or DROP then CREATE
+CREATE PIPE kafka_orders_pipe
+  VIRTUAL_CLUSTER = 'default'
+  BATCH_INTERVAL_IN_SECONDS = '60'
+AS
+COPY INTO ods.orders FROM (
+  SELECT
+    j['order_id']::STRING,
+    j['user_id']::STRING,
+    j['amount']::DECIMAL(10,2),
+    j['status']::STRING,
+    j['created_at']::TIMESTAMP
+  FROM (
+    SELECT parse_json(value::string) AS j
+    FROM read_kafka(
+      'kafka.example.com:9092',  -- bootstrap_servers
+      'orders',                   -- topic
+      '',                         -- reserved
+      'lakehouse_ingest',         -- group_id
+      '', '', '', '',
+      'raw', 'raw', 0,
+      MAP('kafka.security.protocol', 'PLAINTEXT')
+    )
+  )
+);
+
+-- Step 2: Dynamic Table for DWD layer cleansing (incremental refresh every minute)
+CREATE OR REPLACE DYNAMIC TABLE dwd.orders_clean
+  REFRESH INTERVAL 1 MINUTE vcluster default
+AS
+SELECT
+  order_id, user_id, amount,
+  UPPER(status) AS status,
+  created_at,
+  DATE(created_at) AS dt
+FROM ods.orders
+WHERE amount > 0;
+
+-- Step 3: Dynamic Table for DWS layer aggregation (refresh every 5 minutes)
+CREATE OR REPLACE DYNAMIC TABLE dws.order_hourly
+  REFRESH INTERVAL 5 MINUTE vcluster default
+AS
+SELECT
+  DATE_TRUNC('hour', created_at) AS hour,
+  status,
+  COUNT(*) AS order_cnt,
+  SUM(amount) AS total_amount
+FROM dwd.orders_clean
+GROUP BY 1, 2;
+```
+
+## Scenario B: Table Stream + Dynamic Table (CDC UPSERT)
+
+```sql
+-- Step 1: Create Stream on source table to capture changes
+CREATE TABLE STREAM ods.orders_stream
+  ON TABLE ods.orders
+  WITH PROPERTIES ('TABLE_STREAM_MODE' = 'STANDARD');
+
+-- Step 2: Dynamic Table consumes Stream, filters latest state
+CREATE OR REPLACE DYNAMIC TABLE dwd.orders_latest
+  REFRESH INTERVAL 2 MINUTE vcluster default
+AS
+SELECT order_id, user_id, amount, status, created_at
+FROM ods.orders_stream
+WHERE __change_type IN ('INSERT', 'UPDATE_AFTER');
+```
+
+## Scenario C: Materialized View for BI query acceleration
+
+```sql
+-- ⚠️ ClickZetta does not support CREATE OR REPLACE MATERIALIZED VIEW
+-- Method 1: DROP then CREATE (recommended)
+DROP MATERIALIZED VIEW IF EXISTS dws.mv_daily_revenue;
+CREATE MATERIALIZED VIEW dws.mv_daily_revenue
+  COMMENT 'Daily revenue summary for BI tools'
+  REFRESH INTERVAL 60 MINUTE vcluster default
+AS
+SELECT
+  DATE(created_at) AS day,
+  region,
+  SUM(amount) AS revenue,
+  COUNT(DISTINCT user_id) AS uv
+FROM dwd.orders_clean
+GROUP BY 1, 2;
+
+-- Manual refresh
+REFRESH MATERIALIZED VIEW dws.mv_daily_revenue;
+
+-- Drop (⚠️ must use DROP MATERIALIZED VIEW, not DROP TABLE)
+DROP MATERIALIZED VIEW dws.mv_daily_revenue;
+```
+
+## Scenario D: Operations
+
+```sql
+-- Suspend / resume dynamic table
+ALTER DYNAMIC TABLE dwd.orders_clean SUSPEND;
+ALTER DYNAMIC TABLE dwd.orders_clean RESUME;
+
+-- View refresh history to diagnose failures
+SHOW DYNAMIC TABLE REFRESH HISTORY dwd.orders_clean LIMIT 10;
+
+-- Suspend / resume Pipe
+ALTER PIPE kafka_orders_pipe SET PIPE_EXECUTION_PAUSED = true;
+ALTER PIPE kafka_orders_pipe SET PIPE_EXECUTION_PAUSED = false;
+```
+
+## Scenario E: Parameterized Dynamic Table (partition-based refresh)
+
+```sql
+-- Create parameterized dynamic table using SESSION_CONFIGS
+CREATE OR REPLACE DYNAMIC TABLE dwd.orders_partitioned
+  REFRESH INTERVAL 30 MINUTE vcluster default
+AS
+SELECT order_id, user_id, amount, status, created_at, DATE(created_at) AS dt
+FROM ods.orders
+WHERE dt = SESSION_CONFIGS('target_date', CAST(CURRENT_DATE() AS STRING));
+
+-- Manually trigger refresh with parameter
+REFRESH DYNAMIC TABLE dwd.orders_partitioned
+  WITH PROPERTIES ('target_date' = '2024-06-15');
+```
+
+Use case: migrating traditional daily/hourly full ETL jobs to incremental — replace scheduling variables (e.g. `${bizdate}`) with SESSION_CONFIGS for parameterized partition refresh.
+
+## Scenario F: Dynamic Table DML (manual data correction)
+
+Dynamic Tables **do not support DML** (INSERT/UPDATE/DELETE). To correct data:
+
+**Option 1: Rebuild (recommended)** — correct data in the source table and wait for the next auto-refresh.
+
+**Option 2: Use a regular table** — for scenarios requiring frequent manual corrections, use a regular table + scheduled Studio task instead of a dynamic table.