@clickzetta/cz-cli-darwin-x64 0.3.95 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md

@@ -0,0 +1,185 @@
+# Lakehouse Studio Quick Tour
+
+This document helps you quickly understand the core features offered by Lakehouse Studio. In Lakehouse Studio, you can perform data analysis and engineering tasks, monitor queries, data loading/synchronization, data transformation, and workflow activities, explore your Lakehouse objects, and manage your Lakehouse, including managing costs and adding users and roles.
+
+> You can use this document to quickly understand Studio's features. We also strongly recommend referring to the [Getting Started Guide](lakehouse-studio-101.md) series to get started quickly.
+
+![](.topwrite/assets/image_1742438525073.png)
+
+^
+
+## What You Can Do in Lakehouse Studio
+
+* [Data Management](data.md), create and manage data lake/database objects such as databases, tables, dynamic tables, etc.
+* [Compute Resource Management](computation.md), create and manage compute resources such as virtual clusters, job history, etc.
+* [ELT Pipeline Development and Management](ide.md), create and manage data ELT pipeline objects such as data sources, pipeline definitions, tasks (extraction tasks, transformation tasks, etc.), workflows, alerts, etc. Tasks support the following types:
+  * [Data Synchronization Tasks](data-integration.md), ingest data from databases/data warehouses/data lakes and other data sources into Lakehouse, and export data from Lakehouse to other data sources.
+  * [SQL Tasks](task-develop.md), write SQL queries and code for data ingestion, discovery, cleaning, and transformation, leveraging auto-completion of database objects and SQL functions within worksheets.
+  * [Python Tasks](python-task.md), build, test, and deploy SQLAlchemy/Zettapark Python worksheets.
+  * [JDBC Tasks](jdbc_task.md), build, test, and deploy JDBC worksheets (manipulate data in data sources via JDBC connections).
+* [Organize Tasks](task-develop.md), organize worksheets into task folders and [task groups](task_group.md).
+* **Share Job Profiles**: Share job profiles with other users.
+* **Data Profiling**: Visualize SQL worksheet results as data profiles.
+* **Manage and Control Costs**: Manage and control costs.
+* [Job History](web-job-history.md), view query history and data loading history.
+* [Workflow DAG](task-instance-maintenance.md), view workflow graphs and run history.
+* [Backfill Tasks](backfilling_data.md), debug and re-run task graphs.
+* **Monitor Dynamic Tables**: Monitor dynamic table graphs and refresh status.
+* [User/Role Management](account_user_management.md), manage and create Lakehouse users and roles.
+* [Data Quality Management](data-quality.md), clean, optimize, and enhance massive datasets to increase their value density, thereby more effectively meeting business objectives.
+
+For details on these and other executable tasks, please refer to [Lakehouse Studio: The Web Interface for Lakehouse](studio_manual.md).
+
+## Explore and Manage Your Lakehouse Objects
+
+You can explore and manage your data lake/database objects in Lakehouse Studio as follows:
+
+* Use the data object explorer to explore data lakes/databases and objects, including tables, views, etc.
+* Create objects such as schemas and tables.
+* Search in the object explorer to browse database objects in your account.
+* Preview the contents of database objects (such as tables) and view files uploaded to volumes.
+* Load files into existing tables, or create tables from files, to get started with data in Lakehouse more quickly.
+
+For more information, refer to the following documentation:
+
+* [Search Data Objects via Data Asset Map](data_catalog.md)
+* [Load Data Using the Lakehouse Studio Web Interface](upload_data.md)
+
+## Data Synchronization Tasks
+
+Data synchronization is a seamless data integration capability built into Lakehouse, enabling data movement between various data sources. It allows users to automatically execute synchronization tasks through a powerful scheduling system. With this feature, you can easily ingest data into Lakehouse, export processed data, or coordinate data between different data sources -- without writing any code. The entire process is as simple as navigating through a user-friendly wizard.
+
+Depending on the data source, data format, loading method, and processing type (batch, streaming, or data transfer), there are multiple ways to load data into Lakehouse. Lakehouse provides various import methods, categorized by implementation approach, including: importing via user SDKs, importing via SQL commands, uploading data via clients, importing via third-party open-source tools, and importing via the Lakehouse Studio visual interface.
+
+### Import Methods Overview
+
+![](.topwrite/assets/image_1742463415615.png)
+
+### Supported Data Sources
+
+![](.topwrite/assets/image_1742437875252.png)
+
+### Real-time Multi-Table Data Synchronization Tasks (CDC)
+
+* Create a data source (Postgres)
+* Create a real-time multi-table synchronization task from Postgres to Lakehouse
+* Submit and run the task, then start the task
+* Task monitoring and maintenance (start, stop, offline)
+
+![](.topwrite/assets/image_1742437920068.png)
+
+For details, refer to the following documentation:
+
+* [Data Source Support](data-sources.md)
+* [Data Source Management](config-datasource.md)
+* [Batch Data Synchronization Tasks](batch_sync.md)
+* [Multi-Table Real-time CDC (Change Data Capture) Synchronization Tasks](multitable_realtime_sync.md)
+* [Real-time Data Synchronization from Kafka/AutoMQ](realtime_sync.md)
+
+## Write SQL and SQLAlchemy/Python Code in Worksheets and Workflow Orchestration
+
+Worksheets provide a simple way to write SQL jobs (DML and DDL), view results, and schedule them as tasks. With worksheets, you can:
+
+* Run ad hoc queries and other DDL/DML operations.
+* Write SQLAlchemy/Zettapark Python code in Python worksheets.
+* View query history and results of executed queries.
+* View multiple worksheets simultaneously, each with its own independent session.
+* Export results of selected query statements while results are still available.
+* Submit jobs and schedule them as tasks.
+
+If you select Worksheets in the navigation menu, you will see the worksheet list and can select one to view and update worksheet content.
+
+![](.topwrite/assets/image_1742437938827.png)
+
+For details, refer to the following documentation:
+
+* [Lakehouse Studio Task Development Overview](task_development.md)
+* [Workflow Orchestration with Lakehouse Studio Task Development and Scheduling](task-develop.md)
+* [Lakehouse Studio Task Groups](task_group.md)
+* [Writing Python Code in Python Worksheets](python-task.md)
+
+## Visualize Query Results as Data Profiles
+
+When running queries in Lakehouse Studio, you can choose to visualize the data profile of the results.
+
+![](.topwrite/assets/image_1742437950526.png)
+
+## Share Data
+
+Achieve collaboration by sharing data with users of other Lakehouse accounts. When sharing data, you can use the automatic delivery (or auto-fulfillment) feature to easily provide data within the same cloud region. As a data consumer, you can access datasets shared with your account, gaining real-time data insights without setting up data pipelines or writing any code.
+
+![](.topwrite/assets/image_1742437962089.png)
+
+For details, refer to the following documentation:
+
+* [Lakehouse Data Sharing](data-sharing.md)
+
+## Monitor Activities in Lakehouse Studio
+
+### Monitor Query Activity Using Query History
+
+You can monitor and view query details, explore the performance of executed queries, monitor data loading status and errors, view task graphs, and debug and re-run as needed. You can also monitor the refresh status of dynamic tables and view various tags and security policies created for maintaining data governance.
+
+![](.topwrite/assets/image_1742438014544.png)
+
+For details, refer to the following documentation:
+
+* [Monitor Query Activity Using Query History](web-job-history.md)
+
+### Monitor Workflow Tasks Using the Operations Center
+
+The Operations Center provides management operations for tasks and instances. Workflow tasks managed by the Operations Center include manually triggered tasks, periodically scheduled tasks, and their corresponding instances, for centralized management.
+
+![](.topwrite/assets/image_1742438023150.png)
+
+Data backfilling involves supplementing historical or future data within a specific time period and writing it to the corresponding time partitions. If the code contains scheduling parameters, those parameters will be automatically filled with appropriate values based on the selected data backfill business time. Combined with business logic, this ensures that data for the corresponding time period is written to the specified partitions. The partition to write to and the code logic executed are determined by the task definitions in the code.
+
+![](.topwrite/assets/image_1742438034252.png)
+
+Monitoring functionality allows you to leverage built-in rules or custom configurations to keep a close watch on anomalies (such as task execution failures) and send alert notifications when needed.
+
+![](.topwrite/assets/image_1742438047787.png)
+
+For details, refer to the following documentation:
+
+* [Task Instance Maintenance](task-instance-maintenance.md)
+* [Data Backfilling](backfilling_data.md)
+* [Monitoring and Alerting](monitoring_and_alerting.md)
+
+## Perform Compute Resource Management and Manage Lakehouse Studio
+
+These pages help you understand Lakehouse data usage, manage virtual clusters, monitor task queues in virtual clusters, manage users and roles, manage Lakehouse accounts, and more.
+
+You can manage and monitor virtual clusters.
+
+![](.topwrite/assets/image_1742438058433.png)
+
+Access users and roles.
+
+![](.topwrite/assets/image_1742438325404.png)
+
+Perform cost management.
+
+![](.topwrite/assets/image_1742438412611.png)
+
+^
+
+When you log in as a user with the **account administrator role**, you can view the account's usage records under the **Billing** feature in the Account Center (\<your\_account\_name>.[accounts.clickzetta.com/billing](http://accounts.clickzetta.com/billing)). You can also query detailed charges by SKU type (such as compute, storage, network, or specific SKU).
+
+For details, refer to the following documentation:
+
+* [Understanding Workspaces](workspace-introduction.md)
+* [Managing Accounts](manage-accounts.md)
+* [Managing Users](account_user_management.md)
+* [Managing Service Instances](managing-instance.md)
+* [User and Permission Management](authority-management.md)
+
+## Related Videos
+
+Achieve real-time data ingestion to real-time data analysis using Lakehouse Studio.
+
+```[bilibili]
+https://www.bilibili.com/video/BV1TfYYeEEYg/
+```
+
+^

package/bin/skills/lakehouse-doc-en/references/LakehouseStudioTour.md

@@ -0,0 +1,185 @@
+# Lakehouse Studio Quick Tour
+
+This document helps you quickly understand the core features offered by Lakehouse Studio. In Lakehouse Studio, you can perform data analysis and engineering tasks, monitor queries, data loading/synchronization, data transformation, and workflow activities, explore your Lakehouse objects, and manage your Lakehouse, including managing costs and adding users and roles.
+
+> You can use this document to quickly understand Studio's features. We also strongly recommend referring to the [Getting Started Guide](lakehouse-studio-101.md) series to get started quickly.
+
+![](.topwrite/assets/image_1742438525073.png)
+
+^
+
+## What You Can Do in Lakehouse Studio
+
+* [Data Management](data.md), create and manage data lake/database objects such as databases, tables, dynamic tables, etc.
+* [Compute Resource Management](computation.md), create and manage compute resources such as virtual clusters, job history, etc.
+* [ELT Pipeline Development and Management](ide.md), create and manage data ELT pipeline objects such as data sources, pipeline definitions, tasks (extraction tasks, transformation tasks, etc.), workflows, alerts, etc. Tasks support the following types:
+  * [Data Synchronization Tasks](data-integration.md), ingest data from databases/data warehouses/data lakes and other data sources into Lakehouse, and export data from Lakehouse to other data sources.
+  * [SQL Tasks](task-develop.md), write SQL queries and code for data ingestion, discovery, cleaning, and transformation, leveraging auto-completion of database objects and SQL functions within worksheets.
+  * [Python Tasks](python-task.md), build, test, and deploy SQLAlchemy/Zettapark Python worksheets.
+  * [JDBC Tasks](jdbc_task.md), build, test, and deploy JDBC worksheets (manipulate data in data sources via JDBC connections).
+* [Organize Tasks](task-develop.md), organize worksheets into task folders and [task groups](task_group.md).
+* **Share Job Profiles**: Share job profiles with other users.
+* **Data Profiling**: Visualize SQL worksheet results as data profiles.
+* **Manage and Control Costs**: Manage and control costs.
+* [Job History](web-job-history.md), view query history and data loading history.
+* [Workflow DAG](task-instance-maintenance.md), view workflow graphs and run history.
+* [Backfill Tasks](backfilling_data.md), debug and re-run task graphs.
+* **Monitor Dynamic Tables**: Monitor dynamic table graphs and refresh status.
+* [User/Role Management](account_user_management.md), manage and create Lakehouse users and roles.
+* [Data Quality Management](data-quality.md), clean, optimize, and enhance massive datasets to increase their value density, thereby more effectively meeting business objectives.
+
+For details on these and other executable tasks, please refer to [Lakehouse Studio: The Web Interface for Lakehouse](studio_manual.md).
+
+## Explore and Manage Your Lakehouse Objects
+
+You can explore and manage your data lake/database objects in Lakehouse Studio as follows:
+
+* Use the data object explorer to explore data lakes/databases and objects, including tables, views, etc.
+* Create objects such as schemas and tables.
+* Search in the object explorer to browse database objects in your account.
+* Preview the contents of database objects (such as tables) and view files uploaded to volumes.
+* Load files into existing tables, or create tables from files, to get started with data in Lakehouse more quickly.
+
+For more information, refer to the following documentation:
+
+* [Search Data Objects via Data Asset Map](data_catalog.md)
+* [Load Data Using the Lakehouse Studio Web Interface](upload_data.md)
+
+## Data Synchronization Tasks
+
+Data synchronization is a seamless data integration capability built into Lakehouse, enabling data movement between various data sources. It allows users to automatically execute synchronization tasks through a powerful scheduling system. With this feature, you can easily ingest data into Lakehouse, export processed data, or coordinate data between different data sources -- without writing any code. The entire process is as simple as navigating through a user-friendly wizard.
+
+Depending on the data source, data format, loading method, and processing type (batch, streaming, or data transfer), there are multiple ways to load data into Lakehouse. Lakehouse provides various import methods, categorized by implementation approach, including: importing via user SDKs, importing via SQL commands, uploading data via clients, importing via third-party open-source tools, and importing via the Lakehouse Studio visual interface.
+
+### Import Methods Overview
+
+![](.topwrite/assets/image_1742463415615.png)
+
+### Supported Data Sources
+
+![](.topwrite/assets/image_1742437875252.png)
+
+### Real-time Multi-Table Data Synchronization Tasks (CDC)
+
+* Create a data source (Postgres)
+* Create a real-time multi-table synchronization task from Postgres to Lakehouse
+* Submit and run the task, then start the task
+* Task monitoring and maintenance (start, stop, offline)
+
+![](.topwrite/assets/image_1742437920068.png)
+
+For details, refer to the following documentation:
+
+* [Data Source Support](data-sources.md)
+* [Data Source Management](config-datasource.md)
+* [Batch Data Synchronization Tasks](batch_sync.md)
+* [Multi-Table Real-time CDC (Change Data Capture) Synchronization Tasks](multitable_realtime_sync.md)
+* [Real-time Data Synchronization from Kafka/AutoMQ](realtime_sync.md)
+
+## Write SQL and SQLAlchemy/Python Code in Worksheets and Workflow Orchestration
+
+Worksheets provide a simple way to write SQL jobs (DML and DDL), view results, and schedule them as tasks. With worksheets, you can:
+
+* Run ad hoc queries and other DDL/DML operations.
+* Write SQLAlchemy/Zettapark Python code in Python worksheets.
+* View query history and results of executed queries.
+* View multiple worksheets simultaneously, each with its own independent session.
+* Export results of selected query statements while results are still available.
+* Submit jobs and schedule them as tasks.
+
+If you select Worksheets in the navigation menu, you will see the worksheet list and can select one to view and update worksheet content.
+
+![](.topwrite/assets/image_1742437938827.png)
+
+For details, refer to the following documentation:
+
+* [Lakehouse Studio Task Development Overview](task_development.md)
+* [Workflow Orchestration with Lakehouse Studio Task Development and Scheduling](task-develop.md)
+* [Lakehouse Studio Task Groups](task_group.md)
+* [Writing Python Code in Python Worksheets](python-task.md)
+
+## Visualize Query Results as Data Profiles
+
+When running queries in Lakehouse Studio, you can choose to visualize the data profile of the results.
+
+![](.topwrite/assets/image_1742437950526.png)
+
+## Share Data
+
+Achieve collaboration by sharing data with users of other Lakehouse accounts. When sharing data, you can use the automatic delivery (or auto-fulfillment) feature to easily provide data within the same cloud region. As a data consumer, you can access datasets shared with your account, gaining real-time data insights without setting up data pipelines or writing any code.
+
+![](.topwrite/assets/image_1742437962089.png)
+
+For details, refer to the following documentation:
+
+* [Lakehouse Data Sharing](data-sharing.md)
+
+## Monitor Activities in Lakehouse Studio
+
+### Monitor Query Activity Using Query History
+
+You can monitor and view query details, explore the performance of executed queries, monitor data loading status and errors, view task graphs, and debug and re-run as needed. You can also monitor the refresh status of dynamic tables and view various tags and security policies created for maintaining data governance.
+
+![](.topwrite/assets/image_1742438014544.png)
+
+For details, refer to the following documentation:
+
+* [Monitor Query Activity Using Query History](web-job-history.md)
+
+### Monitor Workflow Tasks Using the Operations Center
+
+The Operations Center provides management operations for tasks and instances. Workflow tasks managed by the Operations Center include manually triggered tasks, periodically scheduled tasks, and their corresponding instances, for centralized management.
+
+![](.topwrite/assets/image_1742438023150.png)
+
+Data backfilling involves supplementing historical or future data within a specific time period and writing it to the corresponding time partitions. If the code contains scheduling parameters, those parameters will be automatically filled with appropriate values based on the selected data backfill business time. Combined with business logic, this ensures that data for the corresponding time period is written to the specified partitions. The partition to write to and the code logic executed are determined by the task definitions in the code.
+
+![](.topwrite/assets/image_1742438034252.png)
+
+Monitoring functionality allows you to leverage built-in rules or custom configurations to keep a close watch on anomalies (such as task execution failures) and send alert notifications when needed.
+
+![](.topwrite/assets/image_1742438047787.png)
+
+For details, refer to the following documentation:
+
+* [Task Instance Maintenance](task-instance-maintenance.md)
+* [Data Backfilling](backfilling_data.md)
+* [Monitoring and Alerting](monitoring_and_alerting.md)
+
+## Perform Compute Resource Management and Manage Lakehouse Studio
+
+These pages help you understand Lakehouse data usage, manage virtual clusters, monitor task queues in virtual clusters, manage users and roles, manage Lakehouse accounts, and more.
+
+You can manage and monitor virtual clusters.
+
+![](.topwrite/assets/image_1742438058433.png)
+
+Access users and roles.
+
+![](.topwrite/assets/image_1742438325404.png)
+
+Perform cost management.
+
+![](.topwrite/assets/image_1742438412611.png)
+
+^
+
+When you log in as a user with the **account administrator role**, you can view the account's usage records under the **Billing** feature in the Account Center (\<your\_account\_name>.[accounts.clickzetta.com/billing](http://accounts.clickzetta.com/billing)). You can also query detailed charges by SKU type (such as compute, storage, network, or specific SKU).
+
+For details, refer to the following documentation:
+
+* [Understanding Workspaces](workspace-introduction.md)
+* [Managing Accounts](manage-accounts.md)
+* [Managing Users](account_user_management.md)
+* [Managing Service Instances](managing-instance.md)
+* [User and Permission Management](authority-management.md)
+
+## Related Videos
+
+Achieve real-time data ingestion to real-time data analysis using Lakehouse Studio.
+
+```[bilibili]
+https://www.bilibili.com/video/BV1TfYYeEEYg/
+```
+
+^

package/bin/skills/lakehouse-doc-en/references/Lakehouse_Insight.md

@@ -0,0 +1,98 @@
+# Visual Data Analysis Platform
+
+### 1. Singdata Insight Product Overview
+
+【**Preview Release】This feature is currently in an invitation-only preview phase. If access is required, please contact our technical support team for assistance**. 
+
+^
+
+Singdata Insight is a powerful visual data analysis tool designed for data analysis and visualization after data integration and data development. By using Insight, users can easily create data reports and analysis dashboards, thereby understanding data more intuitively and supporting business decisions. The main use cases of Insight are as follows:　　
+
+* **Create Daily Data Reports**: Quickly build the data reports needed for daily business, reducing the complexity of traditional data portal construction, allowing data developers to focus on underlying data development, while business analysts can build various business analysis reports based on data sets.
+* **Self-Service Analysis**: Business personnel can directly use the data sets prepared by data developers to perform self-service analysis through simple drag-and-drop operations, improving the efficiency of data analysis and avoiding delays in viewing data due to scheduling issues.
+
+### 2. User Operation Guide
+
+Before using Singdata Insight, please ensure you have a Singdata account. The detailed operation steps are as follows:
+
+#### 2.1 Singdata User Management
+
+* **Log in to the System Management Console**: Log in to the system management console using your Singdata account.
+* **Enter Insight**: Find and click Insight on the management console to enter the product.
+
+#### 2.2 Singdata Insight Platform Operations
+
+##### 2.2.1 Create Data Connection
+
+* **Create a New Data Link**: On the data connection page, click the "Create New Data Link" button.
+  ![](.topwrite/assets/image_1740572013979.png)
+
+* **Select Data Source Type**: Choose the Singdata Lakehouse data source under Cloud.
+  ![](.topwrite/assets/image_1740572033841.png)
+
+* **Fill in JDBC Information**: Enter the necessary JDBC link information, such as database URL, username, and password.
+  ![](.topwrite/assets/image_1740572056582.png)
+
+* Name: Required, used as the management name of the data connection
+
+* service: No need to fill in.
+
+* username: Required, the username of the Singdata platform
+
+* password: Required, the password to connect to the Singdata platform
+
+* schema: Optional, the specific schema under the designated workspace of the Singdata account. Note that this database must be consistent with the schema in the JDBC string
+
+* Data Gateway: None
+
+* URL: jdbc\:clickzetta://「instance\_name」.api.singdata.com/「workspace under this account」?schema=specific schema\&virtualCluster=「cluster name under the current space」
+
+* vcluster: Optional
+
+* instance: Optional
+
+* workspace: Workspace in the URL
+
+##### 2.2.2 Create Data Set
+
+* **Create a New Data Package**: On the data marketplace page, click "Create New Data Package" and select "Blank Data Package".
+* **Create a New Data Set**: Click the "Create New Data Set" button within the data package.
+* **Select Data Source**: Select the data source type and specific data table just created.
+  ![](.topwrite/assets/image_1740572084178.png)
+
+![](.topwrite/assets/image_1740572105464.png)
+
+![](.topwrite/assets/image_1740572119991.png)
+
+![](.topwrite/assets/image_1740572153093.png)
+
+##### 2.2.3 Create BI Report
+
+* **Create a New Analysis Application**: On the application creation page, click "Create New Analysis Application".
+* **Create Dashboard**: Click "Create New Dashboard" to enter the dashboard building page.
+* **Publish Dashboard**: After creation, click "Publish" to publish the dashboard to the application marketplace.
+
+##### 2.2.4 View Report
+
+* **View Data**: On the application marketplace page, click the corresponding report to view the data.
+
+### 3. Usage Examples
+
+Here are some specific examples of using Insight:
+
+* **Sales Analysis**: Create a monthly sales report that includes sales revenue, sales volume, and sales growth rate to analyze the sales performance of various product lines.
+* **Customer Behavior Analysis**: Build a dashboard that displays customer visits, dwell time, and conversion rates to analyze the effectiveness of different marketing activities.
+* **Inventory Management**: Set up a real-time updated inventory report that includes inventory quantity, inventory turnover rate, and safety stock levels to help managers stay informed about inventory status.
+
+### 4. Frequently Asked Questions (FAQ)
+
+1. Why can't I see features like Data Marketplace and Application Marketplace?
+   If you cannot see features like Data Marketplace and Application Marketplace, please confirm whether your user role has the corresponding permissions. If necessary, contact the system administrator to enable them.
+
+2. Why do I have data analysis permissions but cannot create datasets?
+   If you have data analysis permissions but cannot create datasets, please ensure you are operating under a data package. Only system administrators have the authority to create data packages.
+
+3. How do I add users to the Insight product?
+   To add users to the Insight product, please operate on the Singdata management console. Note that data synchronization may have a delay.
+
+^

package/bin/skills/lakehouse-doc-en/references/Lakehouse_Platform_Release_Note.md

@@ -0,0 +1 @@
+^

package/bin/skills/lakehouse-doc-en/references/Lakehouse_Studio_101.md

@@ -0,0 +1 @@
+^

package/bin/skills/lakehouse-doc-en/references/Lakehouse_Studio_Release_Note.md

@@ -0,0 +1 @@
+^