@clickzetta/cz-cli-darwin-x64 0.3.94 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/windowframe.md

@@ -0,0 +1,143 @@
+# Window Frame
+
+Window Frame is a subset defined when using window functions to limit the calculation range of the window function, only calculating data within the window frame. Through the window frame, you can more precisely control the calculation method and results of the window function. Below are detailed explanations and usage examples of the window frame.
+
+## Window Frame Syntax
+
+When using window functions, you can define the window frame through the `OVER` clause, with the following syntax:
+
+```SQL
+<window_function> OVER (
+    [PARTITION BY <partition_expression>]
+    [ORDER BY <order_expression>]
+    [<frame_clause>]
+)
+```
+
+The syntax includes:
+
+* `<window_function>`: Specifies the window function name (e.g., `ROW_NUMBER(), RANK(), SUM()`)
+* `PARTITION BY`: Defines data partitioning criteria, creating distinct groups called partitions
+* `ORDER BY`: Determines the sort order within partitions, crucial for functions like `ROW_NUMBER()` and `RANK()`
+* frame\_clause: Specifies window frame boundaries, available in either ROWS or RANGE format
+
+### ROWS frame
+
+ROWS frame is a row-based window frame used to specify the number or position of rows included in the window frame. The syntax is as follows:
+
+```SQL
+ROWS BETWEEN <start_boundary> AND <end_boundary>
+```
+
+`<start_boundary>` and `<end_boundary>` are used to define the start and end boundaries of the window frame, and can be one of the following values:
+
+* `UNBOUNDED PRECEDING`: Starts from the first row of the partition.
+* `UNBOUNDED FOLLOWING`: Ends at the last row of the partition.
+* `CURRENT ROW`: The current row.
+* `<offset> PRECEDING`: The `<offset>`th row before the current row, where `<offset>` is a non-negative integer.
+* `<offset> FOLLOWING`: The `<offset>`th row after the current row, where `<offset>` is a non-negative integer.
+
+### RANGE frame
+
+`RANGE frame` is a value-based window frame used to specify the range of values for the rows included in the window frame. The syntax is as follows:
+
+```SQL
+RANGE BETWEEN <start_boundary> AND <end_boundary>
+```
+
+`<start_boundary>` and `<end_boundary>` are used to define the start and end boundaries of the window frame, and can be one of the following values:
+
+* `UNBOUNDED PRECEDING`: Starts from the minimum value in the partition.
+* `UNBOUNDED FOLLOWING`: Ends at the maximum value in the partition.
+* `CURRENT ROW`: The value of the current row.
+* `<value> PRECEDING`: The value of the current row minus `<value>`, where `<value>` is a non-negative number.
+* `<value> FOLLOWING`: The value of the current row plus `<value>`, where `<value>` is a non-negative number.
+
+## Example Usage of Window Frame
+
+Below are some examples of SQL queries using window frames and their output results. Suppose we have a table named sales that contains data on monthly sales and profit margins, as shown below:
+
+|       |       |        |
+| ----- | ----- | ------ |
+| month | sales | profit |
+| 1     | 100   | 0.1    |
+| 2     | 120   | 0.15   |
+| 3     | 80    | 0.05   |
+| 4     | 150   | 0.2    |
+| 5     | 90    | 0.1    |
+| 6     | 110   | 0.12   |
+
+### ROWS frame
+
+* Query: Use the `SUM()` function and `ROWS frame` to calculate the total sales for each month along with the sales of the previous two months and the next two months.
+
+```SQL
+SELECT month, sales, SUM(sales) OVER (ORDER BY month ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING) AS sum_sales
+FROM sales;
+
++-------+-------+-----------+
+| month | sales | sum_sales |
++-------+-------+-----------+
+| 1     | 100   | 300       |
+| 2     | 120   | 450       |
+| 3     | 80    | 540       |
+| 4     | 150   | 550       |
+| 5     | 90    | 430       |
+| 6     | 110   | 350       |
++-------+-------+-----------+
+```
+
+* Query: Use the `AVG()` function and `ROWS frame` to calculate the average profit margin for each month with the profit margin of the previous month and the next month.
+
+```SQL
+SELECT month, profit, AVG(profit) OVER (ORDER BY month ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) AS avg_profit
+FROM sales;
++-------+--------+---------------------+
+| month | profit |     avg_profit      |
++-------+--------+---------------------+
+| 1     | 0.1    | 0.125               |
+| 2     | 0.15   | 0.09999999999999999 |
+| 3     | 0.05   | 0.13333333333333333 |
+| 4     | 0.2    | 0.11666666666666665 |
+| 5     | 0.1    | 0.14                |
+| 6     | 0.12   | 0.11                |
++-------+--------+---------------------+
+```
+
+### RANGE frame
+
+* Query: Use the `COUNT()` function and `RANGE frame` to calculate the distribution of each month's sales in the annual sales, that is, how many months' sales differ from the current month's sales by no more than 10.
+
+```SQL
+SELECT month, sales, COUNT(*) OVER (ORDER BY sales RANGE BETWEEN 10 PRECEDING AND 10 FOLLOWING) AS count_sales
+FROM sales;
++-------+-------+-------------+
+| month | sales | count_sales |
++-------+-------+-------------+
+| 3     | 80    | 2           |
+| 5     | 90    | 3           |
+| 1     | 100   | 3           |
+| 6     | 110   | 3           |
+| 2     | 120   | 2           |
+| 4     | 150   | 1           |
++-------+-------+-------------+
+```
+
+* Query: Use the `MAX()` function and `RANGE frame` to calculate the maximum profit margin for each month compared to the previous month and the following month.
+
+```SQL
+SELECT month, profit, MAX(profit) OVER (ORDER BY month RANGE BETWEEN 1 PRECEDING AND 1 FOLLOWING) AS max_profit
+FROM sales;
++-------+--------+------------+
+| month | profit | max_profit |
++-------+--------+------------+
+| 1     | 0.1    | 0.15       |
+| 2     | 0.15   | 0.15       |
+| 3     | 0.05   | 0.2        |
+| 4     | 0.2    | 0.2        |
+| 5     | 0.1    | 0.2        |
+| 6     | 0.12   | 0.12       |
++-------+--------+------------+
+```
+
+^

package/bin/skills/lakehouse-doc-en/references/working_with_Vclusters.md

@@ -0,0 +1,175 @@
+# Using Lakehouse Compute Cluster
+
+## 1. Overview: Why do we need a "Compute Cluster"?
+
+In Singdata Lakehouse, whether executing offline/real-time data integration, performing complex ETL tasks, or conducting real-time/offline interactive queries, it is necessary to use the compute cluster (Virtual Cluster, abbreviated as Vcluster) provided by Lakehouse. The compute cluster supports:
+
+* **On-demand start and stop**: Automatically stops when there are no queries or jobs, avoiding cost wastage due to idle resources; automatically starts when new jobs or queries arrive, ensuring timely processing.
+* **Flexible specification selection**: Quickly "vertically scale" or "horizontally expand" based on job load and concurrency.
+* **Multiple usage methods**: Cluster creation, start, stop, resizing, and permission management can be done through the web interface or SQL DDL commands.
+
+This document will combine the features and operation methods of **Lakehouse Compute Cluster (Virtual Cluster, abbreviated as Vcluster**) with practical application scenarios to provide some application examples and best practices.
+
+## 2. Core Concepts and Main Functions
+
+### 1. Cluster Name
+
+Each Singdata compute cluster has a unique name that cannot be duplicated within its workspace. Once a compute cluster is created, its name cannot be changed.
+
+### 2. Specifications
+
+In Lakehouse, specifications are measured in "CRU (Compute Resource Unit)", such as (1CRU, 2CRU, 4CRU... up to 256CRU). Depending on the needs, you can choose **General Purpose/Analytics Purpose/Integration Purpose** clusters and configure the corresponding cluster size.
+
+**Note**: Larger cluster specifications mean higher load capacity and query performance, but also higher costs. You can adjust and optimize cluster specifications during use based on business load and cost expectations.
+
+### 3. Cluster Types
+
+Lakehouse compute clusters are divided into General Purpose (GP), Analytics Purpose (AP), and Integration Purpose (Integration).
+
+* General Purpose (GP): Suitable for offline jobs or comprehensive scenarios, where job resources in the same compute cluster are shared and fairly scheduled. The specification size can automatically scale up and down (vertical scaling).
+* Analytics Purpose (AP): Suitable for ad-hoc and high-concurrency online job scenarios, where jobs in the same compute cluster exclusively occupy resources and are queued for execution based on job order. The number of replicas can automatically scale up and down based on concurrent load (horizontal scaling).
+* Integration Purpose (Integration): Suitable for real-time/offline data integration and other synchronization tasks, where multiple integration tasks can share one instance, and the specification size can automatically scale up and down based on load (vertical scaling).
+
+### 4. Auto Start & Auto Stop
+
+**Auto Start**: When a new job or query is submitted, if the cluster is in a "suspended" state, it will automatically start to execute the task.
+
+**Auto Stop**: After a period of no job/query tasks, the cluster will automatically stop and release compute resources, no longer incurring charges.
+
+### 5. Multi-instance Scaling
+
+The analytics compute cluster in Lakehouse can set "minimum instances" and "maximum instances". When concurrent requests are high, instances are automatically added, and when concurrent requests decrease, instances are automatically reclaimed, meeting query business needs while saving resource costs.
+
+### 6. Billing Method
+
+Billing is based on the actual running time of the compute cluster, accurate to the second, with a minimum billing cycle of 1 minute. By reasonably setting compute cluster specifications and strategies such as auto stop and auto start, compute costs can be effectively saved.
+
+## 3. Common Usage Scenarios and Examples
+
+### Scenario 1: Offline ETL Job Load
+
+**Requirement**: Perform data cleaning/transformation every day or every hour, with large data volumes, taking tens of minutes to several hours, but do not want to incur compute cluster costs during idle periods.
+**Solution**:
+
+1. Create a General Purpose Compute Cluster
+
+In the workspace where ETL jobs need to be executed, create a General Purpose compute cluster. Configure the cluster size based on data volume and job completion expectations. You can refer to the table in [Compute Cluster](virtual-cluster.md) for estimation and configuration:
+
+| Business Scenario         | Load Type                         | Execution Frequency | Job Concurrency | Data Volume | VCluster Type   | Job Latency SLA        | VCluster Specification |
+| ------------------------- | --------------------------------- | ------------------- | --------------- | ----------- | --------------- | ---------------------- | ---------------------- |
+| ETL Scheduling Jobs       | Near real-time offline processing | Hourly              | 1               | 1   TB      | General Purpose | 15 Min                 | 4                      |
+|                           | T+1 Offline Processing            | Days                | 1               | 10 TB       | General Purpose | 4 Hours                | 8                      |
+| Tableau/FineBI            | Ad-Hoc Analytics                  | Ad-Hoc              | 8               | 1   TB      | General Purpose | <1 Min&#xA;TP90 <5s    | 16                     |
+| Data Application Products | Applications                      | On demand           | 8               | 100 GB      | Analytical      | <3 seconds             | 4                      |
+|                           |                                   | On demand           | 96              | 100 MB      | Analytical      | <3 seconds             | 4                      |
+| Singdata Web-UI         | Ad-Hoc Analytics                  | Ad-Hoc              | 8               | 3   TB      | General Purpose | < 1 Min&#xA;TP90 <15 s | 16                     |
+
+Assume the data to be processed is 10TB, and it is expected to be completed within 4 hours. Therefore, a general-purpose computing cluster with a specification of 8 CRU needs to be created.
+
+2. Set the automatic stop time to 1 minute to ensure resources are released promptly after the job ends.
+
+3. Enable the "Auto Start" configuration so that the computing cluster can be automatically started whenever the ETL scheduling task is initiated.
+
+:-: ![](.topwrite/assets/image_1739274962311.png =514)
+
+4. In the scheduling configuration of the ETL task (ETL\_task\_demo), configure the computing cluster for task execution to the ETL cluster set in the previous step. For more scheduling configuration operations, refer to the relevant content in the [Task Development and Scheduling](taskdevelop.md) document.
+
+:-: ![](.topwrite/assets/image_1739276187369.png =791)
+
+***
+
+**Effect**:
+
+1. When the ETL task (ETL\_task\_demo) starts executing, the general-purpose computing cluster ETL will automatically start.
+
+2. After the ETL task (ETL\_task\_demo) finishes executing, the general-purpose computing cluster ETL will automatically shut down after being idle for 1 minute.
+
+^
+
+### Scenario 2: Online Reports + Real-time Queries
+
+**Requirement**: During working hours, multiple business personnel/analysts simultaneously start business queries through the APP or BI, requiring low latency for query results and short waiting queues. To improve resource utilization efficiency, it is hoped that the resource consumption of the computing cluster can dynamically scale with the usage scale of the business personnel, and the cluster can automatically shut down when there is no business, saving costs.
+**Solution**:
+
+1. Create an analytical computing cluster and set:
+
+Instance specification to **4 CRU**;
+
+Minimum instance value to **1**, maximum value to **10**;
+
+Maximum concurrency per instance to **8**;
+
+Enable **Auto Start**;
+
+Automatic stop time to **10 minutes**.
+
+This way, complex queries with an expected data volume of 100GiB can return query results within 3 seconds. When the computing cluster is running, it can support 8 concurrent queries at the minimum scale and up to 80 concurrent queries at the maximum scale. When the concurrency exceeds 80, new concurrent requests will start queuing. After being idle for more than 30 minutes, the computing cluster will automatically shut down, stopping the computation cost. Extending the shutdown time to 30 minutes instead of 1 minute can minimize the impact of frequent restarts on the query experience and retain the cache for faster response to subsequent queries, balancing cost and query performance. To proactively preload hot tables into the cluster cache, see [Compute Cluster Cache](vc_cache.md).
+
+:-: ![](.topwrite/assets/image_1739277930228.png =580)
+
+^
+
+2. When executing query jobs in the APP or BI linked to Lakehouse, configure the use of the ANALYTICS\_BI computing cluster to execute query tasks. For example, the connection string for JDBC is as follows:
+
+```SQL
+jdbc:clickzetta://<your_instance_name>.api.singdata.com/<your_workspace>?username=<your_username>&password=<your_password>&virtualCluster=ANALYTICS_BI&schema=public
+```
+
+The virtualCluster=ANALYTICS\_BI parameter specifies that the compute cluster for executing query tasks is ANALYTICS\_BI.
+
+**Effect**:
+
+High real-time query processing efficiency, automatically scaling new instances when user query volume surges to avoid queuing. Ensures good performance under high concurrency. Scales down when idle to save costs.
+
+^
+
+### Scenario 3: Temporary Analysis/Data Exploration
+
+**Requirement**: Data scientists perform exploratory analysis on large-scale data, including repeated multi-table joins and advanced aggregations. Need to quickly return query results for complex queries.
+**Solution**:
+
+1\. Create a **large-scale analytical computing** cluster, such as 16CRU, to enhance computing parallelism and shorten query time. Minimum instance count is 1, maximum instance count is 2, to control costs.
+
+:-: ![](.topwrite/assets/image_1739326621174.png =378)
+
+2\. During the analysis phase, **disable auto-stop** or set the auto-stop time to more than 30 minutes to retain cache, significantly improving speed during repeated queries. To proactively preload frequently used tables into the cluster cache, see [Compute Cluster Cache](vc_cache.md).
+
+:-: ![](.topwrite/assets/image_1739326785003.png =692)
+
+^
+
+:-: ![](.topwrite/assets/image_1739327118802.png =384)
+
+3\. Manually stop or reset the auto-pause duration to 1 minute after the analysis is complete to save costs.
+
+:-: ![](.topwrite/assets/image_1739327178912.png =383)
+
+4\. If performance is found to be insufficient during use, you can "manually adjust" the computing specifications at any time to achieve better query performance.
+
+:-: ![](.topwrite/assets/image_1739327260338.png =391)
+
+**Effect**:
+
+* Large-scale analytical computing clusters process large volumes of complex SQL faster, shortening the analysis cycle.
+* No need to always occupy large-scale resources, you can manually adjust cluster specifications and auto-start/stop times at any time without affecting ongoing analysis tasks.
+
+^
+
+### Scenario 4: Multi-task Isolation
+
+:-: **Requirement**: Within the same department, there are periodic ELT tasks, Ad-Hoc analysis, and online report requests. Resources should not compete with each other, and resources should not be wasted.
+**Solution**:
+![](.topwrite/assets/image_1739329227325.png =709)
+
+1. Create multiple clusters:
+   * **ETL dedicated cluster**: Use a general-purpose computing cluster, configure specifications that meet job SLA expectations based on business needs, and set a short auto-pause time to promptly shut down the cluster after job completion to reduce costs. Multiple ETL dedicated clusters can be created to assign ETL tasks with different SLA requirements to different clusters, avoiding resource competition within the same cluster that could affect important task SLAs.
+   * **Online report cluster**: Use an analytical computing cluster, enable multi-instance mode, allowing the computing cluster to automatically scale up or down with the number of concurrent users, and appropriately extend the auto-stop time to utilize cache and improve response speed, providing stable online services.
+   * **Ad-Hoc analysis cluster**: Use an analytical computing cluster, configure larger specifications to provide better query performance for complex queries. Manually adjust cluster specifications during use based on actual query needs. Set a longer auto-stop time to retain cache and further enhance query performance.
+2. Configure dedicated clusters for different scheduling tasks, analytical queries, and online reports to isolate the load of different tasks.
+
+**Effect**:
+
+* Workloads do not interfere with each other, maximizing resource utilization.
+* SLAs for different scenarios are guaranteed, avoiding situations where "one large job slows down all other queries" or "online queries crowd out offline tasks."
+
+^

package/bin/skills/lakehouse-doc-en/references/working_with_cache.md

@@ -0,0 +1,96 @@
+# Understanding and Using Result Cache
+
+## Introduction
+
+Singdata Lakehouse uses caching technology to accelerate query performance and efficiency. The platform provides three types of Cache to improve query performance.
+
+1. Query Result Cache
+2. Metadata Cache
+3. Virtual Cluster Local Disk Cache
+
+![](.topwrite/assets/image_1713780513737.png)
+
+Among them:
+
+* Metadata Cache and Query Result Cache services belong to the service layer and can be shared within the workspace.
+* Virtual Cluster Local Disk Cache is stored on the local nodes of the cluster and can only be used when using the specified virtual cluster.
+
+This article will introduce the working principles of the caches and how to use them.
+
+## Result Cache
+
+When executing queries in Singdata Lakehouse, query results that meet certain conditions will be automatically retained for a period of time and will be cleared after the period ends. These temporarily stored query results are called Query Result Cache.
+
+Result Cache needs to meet the following conditions to be reused:
+
+* The underlying data of the tables used in the query does not change. If the data in any table used in the query changes, the result cache cannot be used;
+* The query does not include queries on views. If the query object contains views, the query result cache is not supported;
+* The newly initiated SQL query statement can exactly match the previously executed query in syntax;
+* The query does not include non-deterministic functions (e.g., CURRENT\_TIMESTAMP()), user-defined functions (UDF);
+* The previous Result Cache has not expired and been deleted.
+
+## Result Cache Expiration Period
+
+After the result cache is successfully created, the default retention period is 24 hours.
+
+If subsequent queries using the query result cache are run within 24 hours, the result cache expiration time will be extended by an additional 24 hours. Otherwise, after 24 hours, the query result cache will be cleared.
+
+## Enabling and Disabling Result Cache
+
+Use the parameter `cz.sql.enable.shortcut.result.cache` at the SESSION level to enable or disable it, as shown below.
+```Python
+--Enable result cache
+set cz.sql.enable.shortcut.result.cache=true;
+
+--Disable result cache
+set cz.sql.enable.shortcut.result.cache=false;
+```
+Note: As of April 2024, the Result Cache feature in the Lakehouse version is in public beta and not enabled by default. It will be enabled by default in future versions.
+
+## Constraints and Limitations
+
+Cache retention period: 24 hours
+
+Maximum number of jobs supported for caching in a single workspace: 100,000
+
+Result cache size limit: Unlimited. Results less than or equal to 10MB will be cached in the control layer memory cache, while results exceeding 10MB will be persisted in the storage layer (object storage files).
+
+Non-deterministic functions and UDF query results are not supported for result caching.
+
+## Result Cache Demonstration
+
+To demonstrate query result caching, first enable the result cache and execute an SQL query in Singdata Lakehouse. Then rerun the same query to verify if the new query reuses the result cache for acceleration.
+```Python
+ --Enable result cache
+set cz.sql.enable.shortcut.result.cache=true;
+
+ select
+        l_returnflag,
+        l_linestatus,
+        sum(l_quantity) as sum_qty,
+        sum(l_extendedprice) as sum_base_price,
+        sum(l_extendedprice * (1 - l_discount)) as sum_disc_price,
+        sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge,
+        avg(l_quantity) as avg_qty,
+        avg(l_extendedprice) as avg_price,
+        avg(l_discount) as avg_disc,
+        count(*) as count_order
+from
+        tpch_100g_new.lineitem
+where
+        l_shipdate <= date '1998-12-01' - interval '85' day
+group by
+        l_returnflag,
+        l_linestatus
+order by
+        l_returnflag,
+        l_linestatus
+;
+```
+The first execution took 12.1 seconds. By checking the Job Profile, we can see that the job read a large amount of data from the disk.
+
+
+The second time this query was executed, the job reused the result cache from the previous query and returned the result within 15ms.
+
+When viewing the Job Profile of the job, you can see in the execution plan diagram on the diagnostics page that the job used "JOB RESULT REUSE", indicating that the job directly queried the result data.
+![](.topwrite/assets/result_cache_jobprofile.png)

package/bin/skills/lakehouse-doc-en/references/worksheet.md

@@ -0,0 +1,15 @@
+# Worksheet
+
+## Overview
+
+The worksheet manages or operates all task information under the current instance in the form of a list. In the worksheet, you can quickly and easily locate recently opened projects, and it supports users to create new tasks or perform batch operations on tasks within the workspace.
+
+## Interface Overview
+
+![](.topwrite/assets/image_1740136859124.png)
+
+* Recently Opened: In this tab, the last 10 records recently used in the current workspace are displayed, helping users quickly locate recently opened task nodes.
+* Mine: Lists the tasks for which I am responsible in the current workspace.
+* All: A list of all tasks in the current workspace.
+
+^

package/bin/skills/lakehouse-doc-en/references/workspace-information_schema-views.md

@@ -0,0 +1,203 @@
+# Database Metadata Views
+
+In this section, we will introduce various database metadata views to help you better understand and query your data. These views provide detailed information about databases, tables, columns, views, users, roles, and job history. Through these views, you can easily manage and monitor your data.
+
+## SCHEMAS View
+
+The SCHEMAS view provides detailed information about the database, including the WORKSPACE name, SCHEMA name, creator, type, etc.
+
+**Field Details**
+
+| COLUMN NAME | DATA TYPE | Description |
+| --- | --- | --- |
+| CATALOG\_NAME | STRING | The name of the current WORKSPACE |
+| SCHEMA\_NAME | STRING | The name of the database |
+| SCHEMA\_CREATOR | STRING | The account name of the database owner |
+| TYPE | STRING | Enum values: EXTERNAL (external), INTERNAL (internal) |
+| COMMENT | STRING | Comment information when creating the database |
+| CREATE\_TIME | TIMESTAMP | Database creation time |
+| LAST\_MODIFY\_TIME | TIMESTAMP | Database modification time |
+| PROPERTIES | MAP | PROPERTIES specified at creation, reserved field |
+
+## TABLES View
+
+The TABLES view shows detailed information about each table under the current WORKSPACE.
+
+**Field Details**
+
+| COLUMN NAME | DATA TYPE | Description |
+| --- | --- | --- |
+| TABLE\_CATALOG | STRING | The name of the current WORKSPACE |
+| TABLE\_SCHEMA | STRING | The SCHEMA to which the current TABLE belongs |
+| TABLE\_NAME | STRING | The name of the table |
+| TABLE\_TYPE | STRING | Enum values: EXTERNAL (external table), VIEW (view), MATERIALIZED VIEW (materialized view), BASE TABLE (standard table), SNAPSHOT (snapshot table) |
+| ROW\_COUNT | BIGINT | Number of rows in the table (NULL for VIEW, corresponding number for MATERIALIZED VIEW) |
+| BYTES | BIGINT | Size of the table (NULL for VIEW, corresponding size for MATERIALIZED VIEW) |
+| CREATE\_TIME | TIMESTAMP | Table creation time |
+| LAST\_MODIFY\_TIME | TIMESTAMP | Table modification time |
+| TABLE\_CREATOR | STRING | The account name of the table owner |
+| IS\_PARTITIONED | BOOLEAN | Whether it is a partitioned table (NULL for VIEW) |
+| IS\_CLUSTERED | BOOLEAN | Whether it is a clustered table (NULL for VIEW) |
+| COMMENT | STRING | Table comment information |
+| DATA\_LIFECYCLE | BIGINT | Lifecycle (in days) |
+| PROPERTIES | MAP | PROPERTIES specified at creation, reserved field |
+
+## COLUMNS View
+
+The COLUMNS view shows detailed information about each field in the table.
+
+**Field Details**
+
+| COLUMN NAME | DATA TYPE | Description |
+| --- | --- | --- |
+| TABLE\_CATALOG | STRING | The name of the current WORKSPACE |
+| TABLE\_SCHEMA | STRING | The SCHEMA to which the current TABLE belongs |
+| TABLE\_NAME | STRING | The name of the table |
+| COLUMN\_NAME | STRING | The name of the field |
+| COLUMN\_DEFAULT | STRING | Default value of the field |
+| IS\_NULLABLE | BOOLEAN | Whether it can be NULL |
+| DATA\_TYPE | STRING | Field type |
+| CREATE\_TIME | TIMESTAMP\_LTZ | Table creation time |
+| IS\_CLUSTERING\_COLUMN | BOOLEAN | Whether it is a CLUSTER field |
+| IS\_PRIMARY\_KEY | BOOLEAN | Whether it is a primary key |
+| COMMENT | STRING | Field comment information |
+
+## VIEWS View
+
+The VIEWS view shows all views and their detailed information under the current WORKSPACE.
+
+**Field Details**
+| COLUMN NAME        | DATA TYPE | DESCRIPTION     |
+| ------------------ | --------- | --------------- |
+| TABLE\_CATALOG     | STRING    | Name of the current WORKSPACE  |
+| TABLE\_SCHEMA      | STRING    | SCHEMA to which the current VIEW belongs |
+| TABLE\_NAME        | STRING    | Name of the view            |
+| TABLE\_CREATOR     | STRING    | Account name of the view owner      |
+| VIEW\_DEFINITION   | STRING    | Statement to create the view         |
+| CREATE\_TIME       | TIMESTAMP | View creation time          |
+| LAST\_MODIFY\_TIME | TIMESTAMP | View modification time          |
+| COMMENT            | STRING    | View comment information         |
+
+## USERS View
+
+Each user is displayed in a row, containing all users of the current WORKSPACE
+
+| COLUMN NAME    | DATA TYPE | DESCRIPTION           |
+| -------------- | --------- | --------------------- |
+| WORKSPACE\_NAME | STRING    | Name of the current space               |
+| USER\_NAME     | STRING    | User name                  |
+| ROLE\_NAME     | STRING    | Roles owned by the current user, multiple roles separated by commas  |
+| CREATE\_TIME   | TIMESTAMP | User join time                |
+| EMAIL          | STRING    | User email                  |
+| TELEPHONE       | STRING    | User phone number                  |
+| COMMENT        | STRING    | Description of user information                |
+| PROPERTIES     | MAP       | PROPERTIES specified at creation, reserved field |
+
+## ROLES View
+
+| COLUMN NAME    | DATA TYPE | DESCRIPTION            |
+| -------------- | --------- | ---------------------- |
+| WORKSPACE\_NAME | STRING    | Name of the current space                |
+| ROLE\_NAME     | STRING    | All roles in the space               |
+| USER\_NAMES    | STRING    | User names granted this role, multiple users separated by commas |
+| CREATE\_TIME   | TIMESTAMP | View creation time                 |
+| COMMENT        | STRING    | Description of role information                 |
+| PROPERTIES     | MAP       | PROPERTIES specified at creation, reserved field  |
+
+## JOB\_HISTORY View
+| COLUMN NAME      | DATA TYPE | DESCRIPTION                                     |
+| ---------------- | --------- | ----------------------------------------------- |
+| WORKSPACE\_NAME  | STRING    | Space where the JOB is run                                      |
+| JOB\_ID          | STRING    | Job ID                                            |
+| JOB\_NAME        | STRING    | Job name                                            |
+| JOB\_CREATOR     | STRING    | User running the job                                         |
+| STATUS           | STRING    | SCHEDULE, PROCESS, SUCCEEDED, FAILED, CANCELLED |
+| CRU              | DECIMAL   | Computing resources consumed by the task                                       |
+| ERROR\_MESSAGE   | STRING    | This information is available if an error occurs                                     |
+| JOB\_TYPE        | STRING    | Job type COPY SQL DATALAKE (file operation commands)                  |
+| JOB\_TEXT        | STRING    | Statement executing the JOB                                        |
+| QUERY\_TAG       | STRING    | User-set TAG for identifying the QUERY                              |
+| START\_TIME      | TIMESTAMP | JOB start time                                       |
+| END\_TIME        | TIMESTAMP | JOB end time                                       |
+| EXECUTION\_TIME  | DOUBLE    | Execution time in seconds, accurate to milliseconds                                 |
+| INPUT\_BYTES     | BIGINT    | Actual scanned data volume.                                       |
+| OUTPUT\_BYTES    | BIGINT    | Output bytes.                                          |
+| INPUT\_OBJECTS   | STRING    | Input table names                                           |
+| OUTPUT\_OBJECTS  | STRING    | Output table names                                           |
+| CLIENT\_INFO     | STRING    | Client information, from JDBC, client, web page                         |
+| VIRTUAL\_CLUSTER | STRING    | Computing resources used                                         |
+| ROW\_PRODUCED    | BIGINT    | Total records processed, input data                                   |
+| ROW\_INSERTED    | BIGINT    | Should have a value if it is an insert action                                     |
+| ROW\_UPDATED     | BIGINT    | Should have a value if it is an update action                                     |
+| ROW\_DELETED     | BIGINT    | Should have a value if it is a delete action                                     |
+| JOB\_CONFIG      | STRING    | Parameter information set when submitting the job                                    |
+| CACHE\_HIT       | BIGINT    | Data read from cache                                       |
+| JOB\_PRIORITY    | STRING    | Job priority                                           |
+| INPUT\_TABLES    | STRING    | Input table names                                           |
+| OUTPUT\_TABLES   | STRING    | Output table name                                           |
+
+## Materialized View Refresh History
+
+| COLUMN\_NAME             | DATA\_TYPE   | DESCRIPTION                    |
+| ------------------------ | ------------ | ------------------------------ |
+| WORKSPACE\_NAME          | STRING       | Project workspace name                         |
+| SCHEMA\_NAME             | STRING       | SCHEMA name                       |
+| MATERIALIZED\_VIEW\_NAME | STRING       | Materialized view name                         |
+| CRU                    | DECIMAL      | Cost for refreshing the materialized view                    |
+| VIRTUAL\_CLUSTER\_NAME   | STRING       | Materialized view name, this information is available for automatic refresh               |
+| STATUS                   | STRING       | PENDING\RUNNING\FINISHED\FAILED |
+| SCHEDULED\_START\_TIME   | TIMESTAMP_LTZ | Scheduled refresh time                         |
+| START\_TIME              | TIMESTAMP_LTZ | Materialized view start time                       |
+| END\_TIME                | TIMESTAMP_LTZ | Materialized view end time                       |
+| ERROR\_CODE              | STRING       |                                |
+| ERROR\_MESSAGE           | STRING       | Refresh failure information, if failed it will be here             |
+
+## AUTOMV\_REFRESH\_HISTORY View
+| COLUMN\_NAME               | DATA\_TYPE   | DESCRIPTION                                                             |
+| -------------------------- | ------------ | ----------------------------------------------------------------------- |
+| WORKSPACE_NAME            | STRING       | Project workspace name SYS                                                               |
+| SCHEMA_NAME               | STRING       | SCHEMA name, SCHEMA where AUTOMV is located                                                |
+| MATERIALIZED_VIEW\_NAME   | STRING       | Materialized view name                                                                  |
+| CRU                        | DECIMAL      | Cost for refreshing the materialized view                                                             |
+| STATUS                     | STRING       |  PROCESS: Refreshing. SUCCEEDED: Refresh completed successfully. FAILED: Refresh failed during execution. CANCELLED: Refresh was cancelled before execution.  |
+| MV\_PROCESS\_TYPE          | STRING       | BUILD: Build MV. REFRESH: Refresh                                                   |
+| START\_TIME                | TIMESTAMP_LTZ | Start time of the materialized view                                                                |
+| END\_TIME                  | TIMESTAMP_LTZ | End time of the materialized view                                                                |
+| BUILD\_FROM\_WORKSPACE     | STRING       | Source workspace for building MV                                                             |
+| JOB_ID | SRING       | Job ID for building MV                                                           |
+| ERROR\_MESSAGE             | STRING       | Error message if the refresh fails                                                      |
+
+## VOLUMES View
+
+
+| column_name       | data\_type          | description                                           |
+| ------------------ | ------------------- | ----------------------------------------------------- |
+| VOLUME\_CATALOG    | STRING              | Name of the associated Workspace                                       |
+| VOLUME\_SCHEMA     | STRING              | Name of the associated Schema                                          |
+| VOLUME\_NAME       | STRING              | Name of the Volume                                             |
+| VOLUME\_URL        | STRING              | URL bound to the Volume                                         |
+| VOLUME\_REGION     | STRING              | Region to which the Volume belongs                                           |
+| VOLUME\_TYPE       | STRING              | Type of Volume (internal means no need to specify a third-party cloud provider address when creating the volume, or external) |
+| VOLUME\_CREATOR    | STRING              | Owner of the Volume                                        |
+| CONNECTION\_NAME   | STRING              | Name of the referenced connection                                       |
+| COMMENT            | STRING              | Comment                                                    |
+| PROPERTIES         | map\<string,string> |                                                       |
+| CREATE\_TIME       | TIMESTAMP           | Creation time                                                  |
+| LAST\_MODIFY\_TIME | TIMESTAMP           | Modification time                                              |
+
+## CONNECTIONS View
+
+
+| column\_name       | data type           | description                                                                   |
+| ------------------ | ------------------- | ----------------------------------------------------------------------------- |
+| WORKSPACE\_NAME    | STRING              | The workspace where the object is located                                      |
+| CONNECTION\_NAME   | STRING              | Connection object name                                                        |
+| CONNECTION\_KIND   | STRING              | Enum value supporting connection types, STORAGE CONNECTION, API CONNECTION     |
+| TYPE               | STRING              | Specifies the type of data source connection. storage connection supports FILE\_SYSTEM, api connection supports CLOUD\_FUNCTION |
+| PROVIDER           | STRING              | When TYPE is FILE\_SYSTEM, it is OSS / COS. When TYPE is CLOUD\_FUNCTION, it is aliyun / tencent |
+| REGION             | STRING              | The region of the connection, such as ap-shanghai / cn-beijing                 |
+| SOURCE\_CREATOR    | STRING              | Creator                                                                        |
+| CREATE\_TIME       | TIMESTAMP           | Creation time                                                                  |
+| LAST\_MODIFY\_TIME | TIMESTAMP           | Last modification time                                                         |
+| COMMENT            | STRING              | Comment information                                                            |
+| PROPERTIES         | map\<string,string> |                                                                               |