@clickzetta/cz-cli-darwin-arm64 0.3.95 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/dynamic_table_task.md

@@ -0,0 +1,68 @@
+# Develop and Deploy Dynamic Tables
+
+【**Preview Release】This feature is currently in an invitation-only preview phase. If access is required, please contact our technical support team for assistance**. 
+
+^
+
+Web-IDE provides a wizard-based dynamic table task template that supports collaborative development, improving the efficiency of dynamic table tasks. It also integrates with task operations and monitoring alarm functions, enhancing production-oriented operation and monitoring capabilities.
+
+The development process of dynamic tables in Web-IDE is assisted by a visual wizard to help users write the necessary content for creating dynamic tables. For example: the schema where the dynamic table is located, the name of the dynamic table, the refresh cycle, the query logic of the dynamic table, etc. After the dynamic table script is completed, when the task is "submitted," the system will create the dynamic table at the target location based on the script content, and the refresh strategy configuration of the dynamic table will be automatically scheduled, thereby achieving the deployment and launch of the dynamic table task.
+
+### Step 1: Create a new dynamic table task
+
+You can add a "dynamic table" type task by creating a new task in the Studio IDE development environment.
+
+![](.topwrite/assets/image_1740139414407.png =566)
+
+Specify the task name and save location for the task code when creating a new task.
+
+![](.topwrite/assets/image_1740139423884.png =347)
+
+### Step 2: Name the dynamic table and write the SELECT query statement for the dynamic table
+
+First, fill in the schema location where the dynamic table will be saved after submission and the name of the dynamic table.
+
+![](.topwrite/assets/image_1740139467191.png)
+
+Next, write and test the query statement for the dynamic table in the SQL code area.
+
+Write the SELECT statement for processing and transformation in the SQL code area, set the cluster name for running the SQL (when the dynamic table is deployed and running, this cluster will be used by default to run the refresh task), and you can also test the SELECT statement by running it to ensure the correctness of the processing logic.
+
+![](.topwrite/assets/image_1740139478134.png)
+
+Finally, verify the syntax correctness of the query statement through the Explain button and save the model definition of the dynamic table.
+
+Click the button to check the syntax correctness of the SELECT query, check whether the field names and data types meet expectations. After confirming correctness, click the "Confirm and Continue" button in the pop-up window. The system will save the definition of the dynamic table and exit the SQL code editing state.
+
+![](.topwrite/assets/image_1740139494269.png)
+
+### Step 3: Modify and adjust the dynamic table configuration and submit for deployment
+
+After completing the development of the dynamic table SQL code and exiting the SQL editing mode, you will see the default configuration including basic information of the dynamic table, running parameters, SQL code, dynamic table fields, dynamic table partitions, and buckets.
+
+![](.topwrite/assets/image_1740139511970.png)
+
+Before submitting the dynamic table model definition for deployment to the Lakehouse data environment, you need to check and modify the following configuration information according to business needs to meet production operation requirements. The following are the configuration items that support adjustment and modification for dynamic tables. Please configure them as needed.
+
+| Configuration Item        | Default Value                                 | Configuration Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | Example                                  |
+| ------------------------- | --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------- |
+| Dynamic Table Lifecycle   | Permanent                                     | Optional. You can choose to specify the data lifecycle of the dynamic table.                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | ![](.topwrite/assets/7_dt_lifecycle.png) |
+| Responsible Person        | Task script creator                           | Optional. You can adjust the overall responsible person for the task.                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |                                          |
+| Dynamic Table Description | None                                          | Optional. Add comments for the dynamic table.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |                                          |
+| Running Cluster           | Cluster name specified during IDE development | Optional. You can choose other computing clusters to run the dynamic table refresh task according to the cluster planning of the deployment environment.                                                                                                                                                                                                                                                                                                                                                                                            | ![](.topwrite/assets/8_dt_vc.png)        |
+| Refresh Method            | Manual Refresh                                | Optional. ***It is recommended to choose the automatic refresh method for production use***. Currently supports scheduling policies set at minute, hour, and day intervals, with a minimum scheduling interval of 1 minute.                                                                                                                                                                                                                                                                                                                         | ![](.topwrite/assets/9_dt_vc.png)        |
+| Parameter Configuration   | None                                          | Optional. Advanced parameter settings for dynamic table operation, only needed in specific optimization scenarios (usually provided by the platform for specific scenarios), no need to set by default.                                                                                                                                                                                                                                                                                                                                             | ![](.topwrite/assets/10_dt_para.png)     |
+| SQL Code                  | Last saved code content                       | Optional. When you want to adjust the SQL code after saving, you can modify and save it through the edit mode.                                                                                                                                                                                                                                                                                                                                                                                                                                      | ![](.topwrite/assets/11_dt_code.png)     |
+| Edit Fields               | None                                          | When editing fields, you can set: \* Field comments. Add comment information to the field：

Set partition fields. Add specific fields as partition fields, supporting regular partition fields and Transform partitions.

Set dynamic table bucket fields and quantity. Add specific fields as bucket fields and set the number of buckets. After enabling bucket settings, the default number of buckets is 256.

Set sorting fields. You can specify one or more fields as sorting fields and specify the sorting method. | ![](.topwrite/assets/12_dt_field.png)    |
+
+### Step 4: Submit and deploy the dynamic table model to the target environment
+
+After completing SQL development and dynamic table configuration, you can view the complete DDL definition of the dynamic table model through "DDL Preview".
+
+![](.topwrite/assets/image_1740139597231.png =728)
+
+After confirming that it meets expectations, you can click the "Submit" button on the right side of the dynamic table task. The system will pop up a comparison with the last saved code version:
+
+![](.topwrite/assets/image_1740139604254.png)
+
+After confirming the submission, your model development content will ultimately be executed and create dynamic table data objects in the Lakehouse target data environment through DDL commands. The Lakehouse system will automatically schedule execution according to the refresh strategy of the dynamic table.

package/bin/skills/lakehouse-doc-en/references/dynamic_table_using_studio.md

@@ -0,0 +1,163 @@
+# Developing and Monitoring Dynamic Tables Using Studio
+> ⚠️ **Note**: **[Preview Release] This feature is currently in an invitation-only preview stage. To request access, contact Singdata Technology through the official website.**
+
+Lakehouse Studio provides an integrated web environment for data development, job run monitoring, and anomaly alert notifications. Using the Web-IDE, you can develop dynamic table models and submit them to the target environment for execution. Through the task operations module, you can monitor the running status of dynamic tables and start or pause them as needed. You can also use the monitoring and alert module to configure monitoring rules and alert notifications for dynamic tables, so you can detect task anomalies promptly.
+
+## Developing and Deploying Dynamic Tables Using Web-IDE
+
+The dynamic table development process in Web-IDE uses a visual wizard to guide you through writing the content needed to create a dynamic table — for example, the schema, the dynamic table name, the refresh interval, and the query logic. After you finish developing the dynamic table script and submit the task, the system creates the dynamic table at the target location based on the script content and automatically schedules it according to the refresh strategy, completing the deployment.
+
+* Step 1: Create a new dynamic table task
+
+In the Studio IDE development environment, create a new task and select "Dynamic Table" as the task type.
+
+![](.topwrite/assets/image_1722413586664.png)
+
+When creating the task, specify a task name and a save location for the task code.
+
+![](.topwrite/assets/image_1722413608121.png =280)
+
+* Step 2: Name the dynamic table and write its SELECT query
+
+First, fill in the schema location where the dynamic table will be saved after submission, and enter the dynamic table name.
+
+![](.topwrite/assets/image_1722413623690.png)
+
+Next, write and test the dynamic table query in the SQL code area.
+
+Write the SELECT statement for your data transformation in the SQL code area, and set the cluster name to use when running the SQL (this cluster will be used by default when the dynamic table runs its refresh tasks). You can also click the run button to test the SELECT statement and verify the logic is correct.
+
+![](.topwrite/assets/image_1722413647339.png)
+
+Finally, use the Explain button to validate the query syntax and save the dynamic table model definition.
+
+Click the button to check the syntax of the SELECT query and verify that field names and data types are as expected. After confirming everything looks correct, click "OK and Continue" in the pop-up window. The system saves the dynamic table definition and exits the SQL code editing state.
+
+![](.topwrite/assets/image_1722413661168.png)
+
+* Step 3: Review and adjust the dynamic table configuration, then submit for deployment
+
+After completing the SQL code and exiting SQL editing mode, you will see the default configuration, including basic information, runtime parameters, SQL code, fields, partitions, and bucketing settings.
+
+![](.topwrite/assets/image_1722413674413.png)
+
+Before submitting the dynamic table model to the Lakehouse data environment, review and update the following configuration items to meet your production requirements.
+
+| Configuration Item | Default Value | Description | Example |
+| ------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------- |
+| Dynamic Table Lifecycle | Permanent | Optional. Specify a data lifecycle for the dynamic table. | ![](.topwrite/assets/image_1722413761168.png) |
+| Owner | Task script creator | Optional. Reassign the task owner. | |
+| Dynamic Table Description | None | Optional. Add a comment for the dynamic table. | |
+| Running Cluster | Cluster specified during IDE development | Optional. Select a different compute cluster to run the dynamic table refresh task, based on your deployment environment's cluster plan. | ![](.topwrite/assets/image_1722413783296.png) |
+| Refresh Method | Manual refresh | Optional. ***For production use, automatic refresh is recommended.*** Supports scheduling intervals in minutes, hours, or days, with a minimum interval of 1 minute. | ![](.topwrite/assets/image_1722413797691.png) |
+| Parameter Configuration | None | Optional. Advanced runtime parameters for the dynamic table. These are only needed in specific optimization scenarios (typically provided by the platform team for particular use cases) and do not need to be set by default. | ![](.topwrite/assets/image_1722413810954.png) |
+| SQL Code | Last saved code | Optional. If you want to adjust the SQL code after saving, you can edit and save it again. | ![](.topwrite/assets/image_1722413830727.png) |
+| Edit Fields | None | When editing fields, you can: set field comments; designate partition fields (supporting regular partitions and Transform partitions); set bucket fields and bucket count (default is 256 buckets when bucketing is enabled); and set sort fields (one or more fields, with a specified sort order). | ![](.topwrite/assets/image_1722413842446.png) |
+
+* Step 4: Submit and deploy the dynamic table model to the target environment
+
+After completing SQL development and dynamic table configuration, you can click "DDL Preview" to view the complete DDL definition of the dynamic table model.
+
+![](.topwrite/assets/image_1722413857683.png)
+
+Once you confirm it looks correct, click the "Submit" button on the right side of the dynamic table task. The system displays a diff comparing the current content with the last saved version:
+
+![](.topwrite/assets/image_1722413868003.png)
+
+After confirming the submission, the system executes the DDL commands in the Lakehouse target data environment to create the dynamic table object. Lakehouse will then automatically schedule and run the dynamic table according to its refresh strategy.
+
+## Monitoring Dynamic Table Refreshes and Managing Task Start/Stop
+
+### Dynamic Table Monitoring List
+
+Dynamic table tasks successfully submitted through the Studio Web-IDE can be viewed on the "Dynamic Table" tab under "Task Operations" in the "Operations Monitoring" section. Note that **the operations module only supports managing tasks created through the Web-IDE dynamic table node. It does not support viewing dynamic tables created manually with SQL commands (for those objects, you can still monitor refresh tasks using `SHOW DYNAMIC TABLE REFRESH HISTORY`)**.
+
+The following information is available on the dynamic table operations monitoring list page.
+
+![](.topwrite/assets/image_1722416553405.png)
+
+**Column Descriptions**
+
+* Task Name: The task name specified when the dynamic table node was created
+* Dynamic Table Name: The name of the dynamic table
+* Task Status: The scheduling status of the dynamic table. Statuses include: normally scheduled, paused, and no schedule set
+* Schema: The schema of the dynamic table
+* Action Buttons: Supports pausing scheduling, manual refresh, and taking the dynamic table offline. Note that taking a table offline will delete the dynamic table
+
+**Filter Options**
+
+* Search by task name, filter by scheduling status, or filter by dynamic table schema
+* The "More Filters" panel supports filtering by owner and dynamic table creation time
+
+Clicking a dynamic table task name lets you view its upstream dependencies, helping you understand data flow and how tasks interact. You can also view the refresh history to track execution status and see whether incremental refreshes were performed.
+
+### Dynamic Table Task Execution Details
+
+#### Task Details
+
+* View the upstream dependencies of the dynamic table. Currently only upstream dependencies are shown; downstream is not available. Click to expand the upstream view.
+* The Edit button at the top navigates to the data development view for editing.
+* The Pause button at the top pauses the dynamic table's scheduling.
+* The Manual Refresh button at the top manually triggers a refresh of the dynamic table.
+* The Offline button at the top takes the dynamic table offline, which deletes it.
+
+![](.topwrite/assets/image_1722416571859.png)
+
+#### Node Code
+
+View the creation statement for the dynamic table.
+
+![](.topwrite/assets/image_1722416583152.png)
+
+#### Operation Log
+
+Records the operation audit events throughout the dynamic table task lifecycle, from deployment to going offline. This includes task submission, pause scheduling, resume scheduling, manual refresh, and offline operations.
+
+![](.topwrite/assets/image_1722416592033.png)
+
+#### Refresh History
+
+The refresh history provides a real-time record of dynamic table refresh executions. From the refresh history, you can view the success status of each refresh and key processing metrics.
+
+**Column Descriptions**
+
+* Start Time: The start time of each dynamic table refresh execution
+* Refresh Duration: The time taken to complete the refresh
+* Job Status: The execution status of the dynamic table
+* Job ID: The job ID for the dynamic table execution. Click the ID to navigate to the job details and view the execution plan.
+* Rows Added: The number of rows added by the incremental refresh
+* Rows Deleted: The number of rows deleted during the incremental refresh, for example when data is deleted from a base table
+* Dependent Tables: The tables the dynamic table depended on during the refresh
+
+**Filter Options**
+
+* Search by job ID
+* Filter by duration, job status, execution time, and cluster
+
+![](.topwrite/assets/image_1722416609512.png)
+
+# Monitoring Alerts
+
+**Monitoring alerts only support tasks created through the dynamic table node. Dynamic tables submitted via ordinary SQL scripts or the SQL command line cannot be monitored here.**
+
+* Supports refresh timeout alerts by configuring a runtime duration threshold to detect when a dynamic table refresh takes too long
+
+  * Workspace: Monitors all tasks in the entire workspace that were **created through a dynamic table node** and have been submitted.
+  * Task Name: Monitors a single dynamic table node.
+  * Owner: Monitors dynamic tables assigned to a specific owner.
+
+* Supports refresh failure alerts
+
+  * Checks the number of dynamic table refresh failures at a specified time. If the count reaches the configured threshold, an alert is triggered.
+
+![](.topwrite/assets/image_1722416626731.png)
+
+# Relationship Between Studio Dynamic Table Tasks and Dynamic Table Objects
+
+Studio's development module provides a task type called "Dynamic Table," which supports developing, operating, and monitoring dynamic table tasks. When you submit a dynamic table task in the "Development" module, the system simultaneously creates the corresponding Lakehouse dynamic table object based on the code definition. When you take a dynamic table task offline in the "Operations" module, the system automatically deletes the corresponding Lakehouse dynamic table object.
+
+> ⚠️ **Note**: When creating a dynamic table using a dynamic table task in the Studio development module, the system checks whether a Lakehouse dynamic table object with the same name already exists at the target location. If one is found, the creation will fail with a duplicate name error. In this case, either rename the new dynamic table task or manually delete the existing Lakehouse dynamic table object before creating the Studio dynamic table task.
+
+# Constraints and Limitations
+
+The Studio monitoring and alert module does not support configuring alerts for dynamic tables created directly with the `CREATE DYNAMIC TABLE` statement. To use Studio monitoring alerts for a dynamic table, create and submit it through the "Dynamic Table" task type in the Web-IDE development module.

package/bin/skills/lakehouse-doc-en/references/dynamictable.md

@@ -0,0 +1,53 @@
+# Dynamic Table
+
+**[Preview Release] This feature is currently in an invitation-only preview stage. If you would like to try it, please contact Singdata Technology through the contact information on the official website.**
+Dynamic Table is an innovative feature in the Singdata Lakehouse solution, providing users with a flexible and efficient method for data processing and storage. Unlike traditional static tables, Dynamic Tables do not store data directly but dynamically generate data by defining query statements. This approach shows significant advantages in handling large-scale datasets and achieving real-time data updates.
+
+## Core Features of Dynamic Table
+
+1. **Query Results as Table**: Dynamic Tables dynamically generate datasets based on executed query statements, and each query may produce a completely new dataset, i.e., the content of the Dynamic Table.
+2. **Incremental Refresh Mechanism**: Dynamic Tables support incremental refresh, triggering query execution and content updates only when the underlying data source changes, greatly improving data processing efficiency, especially for large-scale data.
+3. **Operation Restrictions**: Since data is generated by queries, Dynamic Tables do not support traditional `UPDATE`, `DELETE`, `INSERT`, and other direct data operations, ensuring data consistency and accuracy.
+
+## Application Scenarios of Dynamic Table
+
+* **Data Warehouse**: Dynamic Tables can be used to store processed and transformed data, facilitating subsequent data analysis and report generation.
+* **Real-time Data Applications**: For scenarios requiring real-time data updates, Dynamic Tables can provide rapid data refresh capabilities, maintaining data timeliness.
+* **Data ETL Process**: During data extraction, transformation, and loading processes, Dynamic Tables can serve as intermediate storage, holding transformed datasets.
+
+## Differences Between Materialized View and Dynamic Table
+
+From the implementation mechanism of Lakehouse, Dynamic Tables evolve from traditional materialized views. Although they share some commonalities, there are significant differences in their positioning.
+
+**Materialized View: A Performance Optimization Tool** Materialized views are special views that precompute and store query results, significantly improving query efficiency. Unlike traditional views, materialized views store the computation results in a table-like structure, allowing subsequent similar queries to be automatically recognized and directly use the pre-stored results, reducing redundant computations and speeding up query performance.
+
+**Dynamic Table: An Efficient Data Processing Tool** Dynamic Tables focus on data processing. While they can also improve query performance, they do not rely on the query optimizer's automatic rewrite. Materialized views require data to be up-to-date for query rewriting, whereas Dynamic Tables may not always have the latest data in processing scenarios.
+
+**Traditional materialized views have some limitations that Lakehouse currently does not have**:
+
+1. Only a single base table can be used. Materialized views cannot be based on complex queries (i.e., queries with joins or nested views).
+
+2. Materialized views do not necessarily support incremental refresh; each system implements it differently. Some systems implement incremental computation for certain operators to save computational resources and speed up processing.
+
+**Advanced Features of Dynamic Table**:
+
+1. **Incremental Identification and Optimization**: Supports file-level incremental identification in big data environments, achieving real-time incremental reading and computation optimization, as well as write optimization.
+2. **Intelligent Decision Making**: Uses a cost-based optimizer (COST-BASED OPTIMIZER) to intelligently choose between full, incremental, or partial incremental computation to meet different query needs.
+3. **Real-time Task Operations**: Provides mechanisms for data version management, rollback, and rerun, offering comprehensive support for real-time task operations.
+
+**Data Timeliness Considerations**:
+
+* Materialized views require immediate data updates to ensure the accuracy of query rewriting. When base table data changes, Lakehouse uses a scheduled refresh mechanism to ensure the materialized view data is always up-to-date.
+* Dynamic Tables focus more on the flexibility of data processing, adjusting data latency based on business definitions without enforcing real-time data updates.
+
+## Dynamic Table Management Operation Guide
+
+* **Create Dynamic Table**: Users can define new Dynamic Tables through specific syntax and specify SQL query statements for data transformation. For detailed syntax and operation steps, refer to the [Create Dynamic Table](create-dynamic-table.md) document.
+* **Delete Dynamic Table**: If a Dynamic Table is no longer needed, use the `DROP TABLE` command to delete it. Before performing the delete operation, ensure the data is backed up or confirmed to be no longer needed, as this operation will remove the Dynamic Table definition and its data. For detailed syntax and operation steps, refer to the [Delete Dynamic Table](drop-dynamic-table.md) document.
+* **View Dynamic Table Refresh History**: Use the `SHOW DYNAMIC TABLE REFRESH HISTORY` command to view the refresh history of a specified Dynamic Table, monitoring the success status of refresh jobs, runtime, incremental/full refresh mode, the number of records processed, and other information.
+
+## Data Processing and Incremental Computation
+
+Dynamic Tables play an important role in data processing. By defining query statements, users can achieve data extraction, transformation, and loading (ETL). At the same time, Dynamic Tables support incremental computation, triggering data updates only when the data source changes, effectively reducing data processing time and resource consumption. For detailed information on incremental computation and data processing with Dynamic Tables, refer to the [Data Processing and Incremental Computation](dynamic_table_summary.md) document.
+
+^

package/bin/skills/lakehouse-doc-en/references/eco_integration/Zeppelin.md

@@ -0,0 +1,79 @@
+# How to Use Zeppelin to Operate Singdata Lakehouse
+
+## Introduction
+
+**Zeppelin** is a web-based notebook tool that supports multiple data processing backends and languages, allowing you to easily perform interactive data analysis. By using Zeppelin, you can connect to Singdata Lakehouse, execute SQL queries, and create data-driven interactive documents.
+
+## Preparation
+
+1. **Install Zeppelin**:
+   Please visit the [Zeppelin official website](https://zeppelin.apache.org/) and follow the instructions to complete the installation.
+
+2. **Configure Zeppelin to connect to Singdata Lakehouse**:
+   You need to complete the following steps to configure Zeppelin's JDBC interpreter to connect to Singdata Lakehouse.
+
+   2.1 First, download the [JDBC driver](../jdbc-driver.md) for Singdata Lakehouse.
+
+   2.2 Next, click on "anonymous" at the top right of the Zeppelin page, and select "Interpreter" from the dropdown list that appears.
+
+   2.3 Click the "+Create" button at the top right to create a new interpreter. Enter "clickzetta" in the Interpreter Name field, and select "jdbc" from the Interpreter Type dropdown menu.
+
+   2.4 On the configuration page, fill in the following information:
+      - **default.url**: Enter the Singdata Lakehouse JDBC connection string in the following format:
+        `jdbc:clickzetta://<instanceid>.api.singdata.com/<workspace_name>?virtualCluster=<vcluster>&schema=<schema>`
+      - **default.driver**: Enter the class name of the Singdata Lakehouse JDBC driver:
+        `com.clickzetta.client.jdbc.ClickZettaDriver`
+      - In the Artifact section, click the "Add" button and upload the previously downloaded JDBC driver JAR file.
+
+   2.5 After completing the configuration, click the "Save" button to save the settings, and restart the interpreter named "clickzetta".
+
+## Create a Notebook and Access Singdata Lakehouse
+
+1. Create a new notebook named "Data Exploration" and set the Default Interpreter to "clickzetta".
+
+   ![Create Notebook](../.topwrite/assets/image_1699930695948.png)
+
+2. After saving the notebook, you can enter SQL commands for Singdata Lakehouse in the cells and click the "Run" button to execute the operations.
+
+   ![Execute SQL Commands](../.topwrite/assets/image_1699930908479.png)
+
+   You can refer to the following commands for operations:
+   - [show tables](../show-tables.md): List all tables in the current database.
+   - [select](../query-syntax.md): Query data from the table.
+
+## View the Computing Resources (vcluster) and Database Schema in the Current Workspace
+
+1. Use the following command to view the list of computing resources (vcluster) in the current workspace:
+   ```
+   show vclusters;
+   ```
+![View vclusters](../.topwrite/assets/image_1699931749633.png)
+
+2. Use the following command to view the list of database schemas in the current workspace:
+   ```
+   show schemas;
+   ```
+## Switch the Computing Resources (vcluster) and Data Schema for Job Execution
+
+1. Use the following command to switch the computing resources (vcluster) for job execution:
+
+   ```
+   use vcluster <vcluster_name>;
+   ```
+
+   For example:
+   ```
+   use vcluster qiliang_ap;
+   ```
+
+2. Use the following command to switch the data schema to be accessed:
+
+   ```
+   use schema <schema_name>;
+   ```
+
+   For example:
+   ```
+   use schema nyc_taxi_data;
+   ```
+Now, you have successfully configured Zeppelin and started using it to operate Singdata Lakehouse. You can continue to explore more SQL commands and data visualization features to analyze and present data more effectively.

package/bin/skills/lakehouse-doc-en/references/eco_integration/airbyte.md

@@ -0,0 +1,74 @@
+# Airbyte Clickzetta Destination Plugin User Guide
+
+## Introduction
+
+Airbyte is an open-source data synchronization tool that efficiently connects various data sources with data warehouses. With Airbyte, users can easily perform data extraction, transformation, and loading (ETL) to meet data integration needs.
+
+This document aims to help users understand how to configure and use the Clickzetta Lakehouse destination plugin in Airbyte to achieve efficient data synchronization. Before reading this document, you may need to have a basic understanding of Airbyte. If you are not familiar with Airbyte, it is recommended to refer to its [official documentation](https://docs.airbyte.com/category/getting-started).
+
+## Plugin Features
+
+The Clickzetta Lakehouse destination plugin supports the following synchronization modes:
+
+1. Full Refresh Sync
+2. Incremental - Append Sync
+3. Incremental - Append + Deduped Sync
+
+## Plugin Installation
+
+Airbyte supports multiple deployment methods, including standalone and Kubernetes (k8s) clusters. After deploying Airbyte, follow these steps:
+
+1. Log in to the Airbyte console.
+2. Go to the Settings page and select Destinations.
+3. Click the New connector button to enter the new destination connector interface.
+4. Fill in the relevant information:
+   - Connector display name: Set an easily recognizable name for your connector, such as "Clickzetta Lakehouse".
+   - Docker repository name: Enter "clickzetta/clickzetta-airbyte".
+   - Docker image tag: Query and enter the latest image version number from [Docker Hub](https://hub.docker.com/r/clickzetta/clickzetta-airbyte/tags).
+   - Connector documentation URL: You can fill in the link to this document here for easy reference.
+5. Click the Add button, wait a moment, and the plugin will be configured.
+
+## Plugin Configuration
+
+To create Source, Destination, and Connection objects in Airbyte, refer to the [official documentation](https://docs.airbyte.com/quickstart/set-up-a-connection). The following are the configuration instructions for the Clickzetta Lakehouse destination connector:
+
+1. In the Airbyte console, go to the Destination page.
+2. Click the New destination button to create a new destination object.
+3. Select the Clickzetta Lakehouse connector.
+4. Fill in the configuration information:
+   - Username: Your Clickzetta Lakehouse username.
+   - Password: The corresponding user password.
+   - Service: Service address, such as "api.clickzetta.com".
+   - Instance: Instance name.
+   - Workspace: Workspace name, such as "quickstart_ws".
+   - VirtualCluster: Cluster name, such as "default". Airbyte will use the computing resources of this cluster to execute the ELT Transform SQL.
+   - Schema (optional): Default is "public", can be modified if needed.
+   - Normalization: Select "Normalized tabular data" to restore the data to the schema of the source table.
+   - Split size: Set the slice size for Normalization SQL execution. It is recommended to keep the default value to avoid affecting performance.
+
+## Usage Examples
+
+Here are some typical use cases to help you better understand how to use the Clickzetta Lakehouse destination plugin:
+
+### Example 1: Full Data Synchronization
+
+1. Create a Source object to connect to your data source.
+2. Create a Destination object and configure the Clickzetta Lakehouse connector.
+3. Create a Connection object to associate the Source and Destination.
+4. In the Connection configuration, select the Full Refresh Sync synchronization mode.
+5. Start the Connection, and Airbyte will fully synchronize the data to Clickzetta Lakehouse.
+
+### Example 2: Incremental Synchronization with Deduplication
+1. Follow the steps in Example 1 to configure the Source, Destination, and Connection objects.
+2. In the Connection configuration, select the Incremental - Append + Deduped sync mode.
+3. Set the Incremental trigger conditions, such as based on timestamps or record IDs.
+4. Start the Connection, and Airbyte will incrementally sync the data and automatically deduplicate it.
+
+## Summary
+
+This article provides a detailed introduction on how to configure and use the Clickzetta Lakehouse destination plugin in Airbyte. By following the above steps and examples, you can easily achieve efficient data synchronization.
+
+## References
+
+- [Airbyte Official Documentation](https://docs.airbyte.com/category/getting-started)
+- [Airbyte ELT Implementation and Analysis of Various Sync Modes](https://airbyte.com/tutorials/incremental-data-synchronization)

package/bin/skills/lakehouse-doc-en/references/eco_integration/datagrip-lakehouse.md

@@ -0,0 +1,55 @@
+# DataGrip Connects to Singdata Lakehouse
+
+This document will guide you on how to use the database management tool DataGrip to connect to Singdata Lakehouse for efficient data management operations.
+
+## Background Information
+
+DataGrip is a powerful commercial database management tool designed to meet the specific needs of professional SQL developers. It supports multiple databases, including Singdata Lakehouse. For more information about DataGrip, please visit the [official website](https://www.jetbrains.com/datagrip/).
+
+## Prerequisites
+
+Before starting, please ensure you meet the following conditions:
+
+1. Successfully activated the Lakehouse service.
+2. Downloaded the JDBC driver for Lakehouse. Please refer to the [official documentation](../version-update) to obtain the driver.
+3. Downloaded and installed DataGrip.
+
+## Step 1: Add Singdata Lakehouse Driver
+
+1. Open DataGrip and enter the main interface.
+2. Click "New Project" to create a new project. In the new project interface, click "Driver" and select the "+" button to add a new driver. Name the driver Lakehouse.
+![Add Lakehouse Driver](.topwrite/assets/image_1693482390581.png)
+
+## Step 2: Connect to Singdata Lakehouse Using DataGrip
+
+1. In the "Data Sources" window, select the Lakehouse data source you just created.
+2. Click the "Test Connection" button to ensure the connection is normal. If the connection fails, please check your Lakehouse service and JDBC driver settings.
+3. Click the "OK" button to complete the data source configuration.
+4. From the Lakehouse service homepage, copy the JDBC connection string. Please refer to the following example:
+
+![Lakehouse Homepage JDBC Address](.topwrite/assets/image_1693482409292.png)
+
+## Step 3: Manage Singdata Lakehouse Using DataGrip
+
+1. In the "Database" panel on the left side of DataGrip, you can see all the schemas and tables.
+
+![View Schemas and Tables](.topwrite/assets/image_1693482418469.png)
+
+2. Right-click any schema or table and select "SQL Script" to open the SQL editor.
+3. In the SQL editor, you can write and execute SQL queries. For example, you can execute the following query to get the first 10 rows of a table:
+
+```sql
+SELECT * FROM your_table LIMIT 10; -- Get the first 10 rows of the table
+```
+```sql
+SELECT * FROM your_table_name LIMIT 10;
+```
+4. After executing the query, the results will be displayed in the "Results" panel below. You can sort, filter, and export the results.
+
+# Frequently Asked Questions
+Due to different versions of DataGrip having various changes, some versions may send SQL with quotes in the schema when previewing tables. You can bypass this by editing the data source and adding the following to the startup script in the options, and checking Single session mode.
+```
+set  cz.sql.double.quoted.identifiers=true;
+```
+
+![](../.topwrite/assets/image_1733814698645.png)