@clickzetta/cz-cli-darwin-arm64 0.3.94 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md

@@ -0,0 +1,109 @@
+# Dashboard Version Management - User Guide
+
+## **Feature Overview**
+
+When using Analytics Agent, users often iterate on dashboard charts through ASK AI conversations. However, after multiple rounds of modifications, charts may deviate from expectations. Previously, the only option was to describe requirements again and have the Agent redo the work.
+
+Now, the system automatically saves a version each time AI modifies a dashboard chart. Users can view historical versions at any time and restore with one click, completely solving the problem of "can't go back after making bad changes."
+
+## Core Value
+
+* **Recover from mistakes**: Not satisfied after multiple iterations? Restore to a previously satisfactory version with one click
+
+* **Explore with confidence**: No need to worry about losing the current effect; freely try new analytical perspectives
+
+* **Traceable collaboration**: Modifications on shared dashboards are recorded, so you know who changed what and when
+
+## How to Use
+
+### 1. Open the Version Panel
+
+Click the **Version History** button in the upper right corner of the dashboard page
+
+![](/.topwrite/assets/image_1779185094088.png =555)
+
+### 2. View Historical Versions
+
+Each version card in the version panel contains:
+
+![](/.topwrite/assets/image_1779185134029.png =311)
+
+* Version number (V1, V2, V3...)
+
+* Modifier's avatar and name
+
+* Modification time
+
+* Change detail description
+
+### 3. Preview Historical Versions
+
+Click on a version card to view a snapshot of the dashboard at that version (read-only, not editable).
+
+### 4. Restore a Historical Version
+
+Hover over a version card and click the **Restore** button:
+
+![](/.topwrite/assets/image_1779185185822.png)
+
+* The system will show a confirmation prompt: "The rollback operation will create a new version based on this version"
+
+* After clicking **Confirm**, the system will create a new version based on the selected historical version (e.g., restoring from V3 will generate V7 with the note "This version is derived from V3")
+
+* Existing version records will not be lost
+
+## Version Generation Rules
+
+### What operations automatically generate versions?
+
+* Modifying dashboard charts through ASK AI conversations (adding, modifying, or deleting charts)
+
+### What operations do NOT generate versions?
+
+* Pure text conversations (no chart changes produced)
+
+### Version Retention Limit
+
+* The system retains up to 100 recent versions
+
+* Early versions exceeding the limit will be marked as "Expired"
+
+## Dashboard Visibility and Version Permissions
+
+### Private Dashboard
+
+* All versions are visible only to the creator
+
+### Shared Dashboard
+
+* All versions are visible to users within the same tenant
+
+### Private to Shared Transition
+
+![](/.topwrite/assets/image_1779185254845.png =393)
+
+* Versions before the sharing time point are visible only to the creator
+
+* Versions created after sharing are visible to all users
+
+* The version panel displays a divider at the sharing time point, marked "Dashboard was converted to shared at XX time. Versions below are visible only to you."
+
+### Example Scenario
+
+1\. User A creates a private dashboard, generating V1 through V3
+
+2\. At V4, converts the dashboard to shared
+
+3\. V5 is modified by User B, V6 is modified by User A
+
+4\. User B opens version history: can only see V4, V5, V6 (V1 through V3 are not visible)
+
+5\. User A opens version history: can see all V1 through V6
+
+## Notes
+
+1\. Only chart changes produced through ASK AI conversations automatically generate versions; manual drag-and-drop editing does not trigger version creation
+
+2\. Restoring does not delete intermediate versions; instead, it creates a new version based on the target version
+
+3\. The version retention limit is 100; it is recommended to periodically confirm the status of important versions

package/bin/skills/lakehouse-doc-en/references/data-catalog.md

@@ -0,0 +1 @@
+^

package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md

@@ -0,0 +1,58 @@
+# Overview
+
+Data synchronization is an efficient data integration service built into Singdata Lakehouse, supporting data synchronization between various data sources and creating automated synchronization tasks using the scheduling system. With the data synchronization feature, users can quickly import data into Lakehouse, export processed data, or synchronize data between different data sources without writing code, simply through a wizard-like operation.
+
+^
+
+## Basic Concepts
+
+### Data Synchronization Task
+
+A data synchronization task is a type of task used to collect data from a data source and write it to a target data source. Based on the timeliness of data synchronization, data synchronization tasks can be divided into offline synchronization tasks and real-time synchronization tasks. In Lakehouse Studio, data synchronization tasks are defined and scheduled through interface configuration without writing code.
+
+Lakehouse Studio currently supports two main types of synchronization tasks: offline periodic synchronization and real-time synchronization. In the "Development -> Tasks" section, use the new button to select the appropriate task type as needed.
+
+![](.topwrite/assets/image_1740136964041.png =600)
+
+^
+
+A data synchronization task includes the following core components:
+
+1. Data Source: The data source for the synchronization task, including databases, file systems, SaaS/applications, Lakehouse space data sources, etc. Data sources are defined and managed by the data source.
+2. Data Object: The data object under the data source that needs to be processed by the synchronization task, such as database tables, message queue topics, file system files, etc.
+3. Object and Schema Mapping: Defines the mapping relationship between the source data object and the target data object and their schemas.
+4. Data Target: The target data source where the data is written, including Lakehouse and other external data sources.
+5. Task Settings: Rule settings that affect task execution, such as fault tolerance rules, concurrency settings, traffic control, etc.
+6. Scheduling Configuration: Rules and strategies for task scheduling and execution.
+
+### Data Source
+
+A data source is an object that defines the connection information of an external service, including service address, authentication information, connection method, etc. Defined data sources can be used as data sources or data targets in data synchronization tasks. Data sources are located under the "Management -> Data Sources" function menu.
+
+^
+
+## Usage Example
+
+### Example: Synchronize Data from MySQL Database to Lakehouse
+
+Suppose you have a MySQL database containing customer order data. You want to synchronize this data to Lakehouse for data analysis and processing.
+
+1. Create a new data synchronization task in Lakehouse.
+2. Select the MySQL database as the data source and specify the corresponding data source connection information.
+3. Select the database table to be synchronized as the data object.
+4. Define the table structure and field mapping rules.
+5. Select Lakehouse as the data target and specify the target table.
+6. Configure task settings and scheduling configuration as needed.
+7. Start the synchronization task, and the data will be synchronized from the MySQL database to Lakehouse according to the set rules.
+
+Through the above example, you can see that the Lakehouse data synchronization feature can help you easily achieve data synchronization between different data sources without writing complex code. This will greatly improve data processing efficiency, allowing you to focus on data analysis and business decision-making.
+
+^
+
+For detailed guides on creating and configuring data synchronization tasks, please refer to the following help documents:
+
+* [Offline Synchronization Tasks](batch_sync.md)
+* [Real-time Synchronization Tasks](realtime_sync.md)
+* [Multi-table Real-time Synchronization Tasks](multitable_realtime_sync.md)
+
+^

package/bin/skills/lakehouse-doc-en/references/data-integration.md

@@ -0,0 +1,51 @@
+# Data Integration
+
+Singdata Lakehouse Studio includes a full-featured data integration capability that supports 40+ data sources with visual configuration — no coding required. It covers real-time CDC sync, offline batch sync, full-database migration, and more.
+
+## Choosing a Sync Approach
+
+| My scenario | Recommended approach | Documentation |
+|-------------|----------------------|---------------|
+| MySQL / PG / Oracle or similar — single-table real-time CDC | Real-time sync task | [Real-time Sync Task](realtime_sync.md) |
+| Full-database migration, syncing multiple tables together | Multi-table real-time sync | [Multi-table Real-time Sync Complete Guide](multitable_realtime_sync_sop.md) |
+| Scheduled batch sync (T+1 or hourly) | Offline sync task | [Offline Sync Task](batch_sync.md) |
+| Full-database offline migration, multiple tables at once | Multi-table offline sync | [Multi-table Offline Sync Task](multitable_batch_sync.md) |
+| Not sure which to use | Check supported sources | [Data Source Support](data-sources.md) |
+
+## Quick Start
+
+**Step 1** — [Configure a data source](config-datasource.md): add the connection details for your source database
+
+**Step 2** — Create a sync task based on your scenario (see the table above)
+
+**Step 3** — Configure a scheduling policy and publish the task
+
+**Step 4** — Monitor run status and configure alerting
+
+> 💡 **Tip**: Configuring a data source for the first time? Check the [Data Source Configuration Guide](datasource-config-guide.md) to find your database type and follow the step-by-step setup.
+
+## Scheduling, Deployment, and Operations
+
+| Scenario | Documentation |
+|----------|---------------|
+| Configure scheduled runs and task dependencies | [Task Scheduling](task_scheduling.md) · [Task Scheduling Dependencies](task_scheduling_dependency.md) |
+| Publish / unpublish a task | [Task and Instance Operations](task-instance-maintenance.md) |
+| View run history, logs, and failure details | [Task and Instance Operations](task-instance-maintenance.md) |
+| Configure alert notifications (DingTalk / Lark / email) | [Monitoring and Alerting](monitoring_and_alerting.md) |
+| Understand monitoring metric definitions | [Monitoring Metric Specifications](monitoring_item_specification.md) |
+| Backfill historical data | [Data Backfill Tasks](backfilling_data.md) |
+
+## Contents of This Section
+
+| Page | Description |
+|------|-------------|
+| [Overview](data-integration-intro.md) | Basic concepts, task types, and core components of data sync |
+| [Data Source Management](config-datasource.md) | Add and manage data source connections |
+| [Supported Data Sources](data-sources.md) | List of 40+ supported data sources |
+| [Data Source Configuration Guide](datasource-config-guide.md) | Detailed setup steps for each database type |
+| [Offline Sync Task](batch_sync.md) | Scheduled batch sync with full and incremental modes |
+| [Multi-table Offline Sync Task](multitable_batch_sync.md) | Full-database offline migration |
+| [Real-time Sync Task](realtime_sync.md) | Single-table real-time CDC sync |
+| [Multi-table Real-time Sync Task](multitable_realtime_sync.md) | Full-database real-time CDC sync |
+| [Multi-table Real-time Sync Complete Guide](multitable_realtime_sync_sop.md) | End-to-end SOP for full-database migration |
+| [Offline Sync FAQ](batch_sync_sop.md) | Common questions and usage guide |

package/bin/skills/lakehouse-doc-en/references/data-lifecycle.md

@@ -0,0 +1,38 @@
+# Data Lifecycle
+
+The data lifecycle (TTL) measures the length of time from the last update of tables and table partitions in the Singdata Lakehouse. If the data does not change within the specified period, the system will automatically reclaim the data. The lifecycle unit is days, and the value is an integer.
+
+## How the Data Lifecycle Works
+
+The expiration of the data lifecycle relies on the last modification time (last\_modified\_time). You can view the last\_modified\_time using the `desc extended` command. The last\_modified\_time changes when DDL (Data Definition Language) and DML (Data Manipulation Language) operations occur.
+
+It is important to note that expired data is not immediately reclaimed. You can still query this data until the background process performs the deletion operation. Typically, the background process will delete this data within 24 hours. Additionally, reclaimed tables still adhere to the [data retention period](timetravel.md), and you can still query using the [time travel](timetravel.md) feature. If you need to restore deleted objects, you can use the [undrop table](undrop-table.md) command.
+
+## Setting the Data Lifecycle
+
+The lifecycle unit is days, and the value is a positive integer. Setting it to '-1' means the lifecycle is not enabled, and the data will be retained permanently. You can specify the lifecycle when creating a table or modify it after the table is created.
+
+### Specific SQL Operations
+
+**Setting the lifecycle when creating a table**
+```SQL
+-- Create a table and set the lifecycle to 7 days
+CREATE TABLE tname (col1 datatype1, col2 datatype2) PROPERTIES('data_lifecycle'='7');
+-- Create a table and set the lifecycle to 7 days, and delete the table structure when the lifecycle expires
+CREATE TABLE tname (col1 datatype1, col2 datatype2) PROPERTIES('data_lifecycle'='7', 'data_lifecycle_delete_meta'='true');
+```
+**Modify the Lifecycle of an Existing Table**
+```SQL
+-- Modify the table's lifecycle to 10 days
+ALTER TABLE tname SET PROPERTIES ('data_lifecycle'='10');
+-- Modify the table's lifecycle to 10 days and delete the table structure when the lifecycle expires
+ALTER TABLE tname SET PROPERTIES ('data_lifecycle'='10', 'data_lifecycle_delete_meta'='true');
+```
+## Precautions
+
+1. Lifecycle recycling is initiated daily at a scheduled time and will scan all partitions. Data will only be recycled when the `last_modified_time` exceeds the time specified by the lifecycle.
+2. Lifecycle recycling mainly targets tables or partitions and is performed daily based on the service's busyness. It cannot ensure immediate recycling upon expiration.
+3. Upon lifecycle expiration, the default behavior is not to delete the table structure but to clear the data. If you wish to delete the table structure, please add the parameter `ALTER TABLE tname SET PROPERTIES ('data_lifecycle_delete_meta'='true')` when setting the lifecycle.
+4. When modifying the lifecycle, for example when you change the lifecycle policy from 15 days to 30 days, the new policy takes effect immediately. However, since the lifecycle recycling task starts at an unscheduled time each day and typically polls every 12 hours, in rare cases, if the recycling task happens to start and reads the old policy at the moment of the policy change, it may recycle some data according to the old policy (15 days). This behavior is a normal mechanism of the system design and does not affect the safety of data that has not yet expired.
+
+By setting a reasonable data lifecycle, you can effectively manage storage space, ensuring timely data updates and cleaning.

package/bin/skills/lakehouse-doc-en/references/data-load-summary.md

@@ -0,0 +1,57 @@
+# Data Import Overview
+
+There are various methods to load data into Lakehouse, depending on the data source, data format, loading method, and whether it is batch, streaming, or data transfer. Lakehouse provides multiple import methods, categorized by import method including: supporting user SDK import, supporting SQL command import, supporting client upload data, supporting third-party open-source tools, and supporting Lakehouse Studio visual interface import.
+
+## Import Methods Overview
+
+:-: ![](.topwrite/assets/image_1739968872514.png)
+
+## Categorized by Scenario
+
+| Data Source                                                                                                       | Specific Case                                                                                                                                                                                                                        | Applicable Scenario                                                                                                                                                                                                                                                                                                          |
+| ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Quick Upload of Local Files                                                                                       | Use the put command to upload to the internal Volume, then use the COPY command to import.                                                                                                                                           | Suitable for source file upload and requires SQL for conversion. It is recommended to use SQL for conversion or handling abnormal data during the upload process.                                                                                                                                                            |
+|                                                                                                                   | [Use the visual upload interface of Lakehouse Studio](load-data-local.md)                                                                                                                                                            | 1. Suitable for users who need to upload small data files, especially when the files are stored locally. 2. User-friendly interface simplifies the upload process. 3. Supports multiple file formats, including CSV, PARQUET, AVRO, to meet different data format needs.                                                     |
+|                                                                                                                   | [Use Jdbc client](load-data-local.md) In versions after JDBC 2.0.0, the local COPY command has been deprecated. We recommend using the PUT method to upload data to the volume, and then use the server-side COPY command to import. | 1. Small data volume without a visual interface, suitable for technical users, especially those familiar with command line operations and need to batch process small data volumes. 2. Quickly process CSV files, suitable for scripting and automation operations.                                                          |
+| Batch Import Data from Object Storage                                                                             | [Use Volume to batch load data from object storage](load-data-oss.md)                                                                                                                                                                | 1. Suitable for data stored in object storage, needing to leverage SQL performance advantages to quickly read large amounts of data. 2. Efficiently handle large-scale datasets. 3. Supports direct SQL data conversion, simplifying the data import process. 4. Supports multiple file formats, including CSV, PARQUET, ORC |
+|                                                                                                                   | [Import using Copy command](load-data-oss.md)                                                                                                                                                                                        | 1. Has the advantages of Volume, with the difference being that the Copy command provides more fault tolerance parameters and data export support. It also supports exporting to object storage data.                                                                                                                        |
+| Use JavaSDK to read Kafka and write to Lakehouse in real-time                                                     | [Import data using Java SDK](use-java-sdk-realtime-uploaddata.md)                                                                                                                                                                   | 1. Suitable for business scenarios requiring real-time data stream processing, especially for developers familiar with Java. 2. Real-time data reading and writing ensure the immediate availability of data. 3. Suitable for custom data import, providing high flexibility.                                                |
+| Use custom SDK to read files and write to Lakehouse                                                               | [Import data using Java SDK](use-java-sdk-upload-data-local.md)                                                                                                                                                                       | 1. Suitable for one-time bulk data import with low data frequency (time interval greater than five minutes). 2. Supports custom data sources, providing flexibility in data import.                                                                                                                                          |
+|                                                                                                                   | [Import data using Python SDK](use-python-sdk-upload-data.md)                                                                                                                                                                        | 1. Suitable for developers familiar with Python who need custom data import, especially when the data is not in object storage or the integration does not support the data source.                                                                                                                                          |
+| Users needing fully managed services and visual operations, especially for synchronizing third-party data sources | Use Lakehouse Studio data integration                                                                                                                                                                                                | 1. Supports a wide range of data sources and various import methods. 2. Provides real-time synchronization, CDC mirroring synchronization, and offline periodic scheduling synchronization. 3. Visual monitoring improves data management transparency.                                                                      |
+
+# By Import Method
+
+| Import Method Name                        | Usage                                                                                                                               | Usage Notes                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| COPY Command                                   | [COPY INTO Command](copy-into-table.md)                                                                                                  | 1. Currently supported object storage locations are Tencent Cloud COS and Alibaba Cloud OSS. 2. Cross-cloud vendor import is not supported (will be supported later). For example, your Lakehouse is hosted on Alibaba Cloud but the object storage is on Tencent Cloud.                                                                                                                                                                                                 |
+| Volume                                    | [Use ISNERT INTO... SELECT FORM VOLUME to import](datalake_volume.md)                                                               | 1. Currently supported object storage locations are Tencent Cloud COS and Alibaba Cloud OSS. 2. Cross-cloud vendor import is not supported (will be supported later). For example, your Lakehouse is hosted on Alibaba Cloud but the object storage is on Tencent Cloud.                                                                                                                                                                                                 |
+| Data Integration                          | [Use Data Integration to import](data-integration-intro.md)                                                                         | 1. Supports a wide range of data sources, performance is slightly worse than COPY command and Volume.                                                                                                                                                                                                                                                                                                                                                                    |
+| SDK Realtime Data Stream (RealtimeStream) | [Upload data in real-time](java_reference/realtime-upload.md)                                                                       | 1. Data written in real-time can be queried in seconds. 2. When changing the table structure, you need to stop the real-time writing task first, then restart the task after a period of time (about 90 minutes) after the table structure change. 3. Table stream, materialized view, and dynamic table can only display committed data. Data written by real-time tasks needs to wait 1 minute to be confirmed, so table stream also needs to wait 1 minute to see it. |
+| SDK Batch Upload (BulkloadStream)         | [Batch upload data Java SDK](java_reference/bulkload-upload.md)、[Batch upload data Python SDK](python_reference/bulkload-upload.md) |                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+| Use Open Source Tools to Import Data      | [AIRBYTE](airbyte.md)、[DATAX](eco_integration/datax.md)     [ FLINK CONNECTOR ](flink-write-connector.md)                           |                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+
+# Import Instructions
+
+Lakehouse fundamentally provides two main methods for data ingestion to accommodate different data processing needs and performance considerations.
+
+### **Direct Pull by SQL Engine**
+
+The SQL engine of Lakehouse can directly pull data from object storage, which is particularly suitable for scenarios where the data source is already located in object storage. In the future, Lakehouse plans to expand support for more types of data sources to further enhance data ingestion flexibility.
+
+* **Advantages**: The SQL engine directly pulls data, leveraging its performance advantages to efficiently process data stored in object storage. Using the COPY command and Volume support for data transformation during import.
+* **Compute Resources**: Executing SQL queries consumes compute resources (VirtualCluster), but this consumption is worthwhile as it provides powerful data processing capabilities.
+* **File Formats**: Currently supported file formats include CSV, PARQUET, and ORC, which are commonly used formats in data analysis and processing.
+* **Import Methods**: Supports various data import methods, such as the COPY command, Volume, offline data integration import, and SDK bulk upload (BulkloadStream).
+
+### **Push Data to Ingestion Service**
+
+The Ingestion Service of Lakehouse provides a solution for scenarios requiring real-time data ingestion. Clients push data to the server, which is received and committed to the table by the Ingestion Service, including SDK real-time data streams (RealtimeStream) and CDC real-time writes.
+
+* **Advantages**: The advantage of this method is that it improves the timeliness of data ingestion, achieving real-time writes and real-time readability.
+* **Resource Consumption**: Currently, the Ingestion Service is in preview and using this service is temporarily free of charge. However, please note that charges will be applied in the future.
+* **Applicable Scenarios**: Suitable for business scenarios requiring real-time data stream processing, such as real-time analysis and instant data updates.
+
+# Batch Loading Recommendations
+
+Batch ingestion involves loading large bounded datasets that do not require real-time processing. They are typically ingested at specific rule-based frequencies, and not all data arrives immediately. For batch loading, it is recommended to use object storage to store incoming data. Lakehouse supports multiple file formats, including CSV, PARQUET, and ORC file formats.

package/bin/skills/lakehouse-doc-en/references/data-mamager-tool.md

@@ -0,0 +1 @@
+^

package/bin/skills/lakehouse-doc-en/references/data-manager-tool.md

@@ -0,0 +1 @@
+# Data Management Tools

package/bin/skills/lakehouse-doc-en/references/data-quality.md

@@ -0,0 +1,99 @@
+# Data Quality Management
+
+In the era of big data, data quality management is a critical step to ensure data correctness, validity, and consistency. Through data quality management, we can clean, process, and optimize massive datasets, thereby enhancing data value density to better serve business needs. The data quality module provides you with comprehensive monitoring and evaluation of data quality across six dimensions: completeness, uniqueness, consistency, accuracy, validity, and timeliness. With the data quality module, you can achieve continuous improvement and optimization of data quality.
+
+## Overview
+
+The data quality overview page provides an intuitive data quality monitoring dashboard, making it easy to view the overall status of quality rules and validation runs. Below are detailed explanations of some indicators:
+
+![](.topwrite/assets/image_1760004305684.png)
+
+The definitions of some indicators are as follows:
+
+* Quality rule count: The total number of quality rules configured across all workspaces of the service instance, including disabled rules.
+* Total covered tables: The number of tables with quality rules configured across all workspaces of the service instance.
+* Detection run count: The number of quality rule runs in the past month.
+* Validation pass rate: Number of passes / total number of validations in the past month.
+* Quality rule distribution: The distribution of quality rules across all workspaces of the service instance, grouped by owner or workspace.
+* Validation result distribution: The distribution of validation result statuses for quality rules across all workspaces.
+* High-quality tables: Tables with quality rules configured that have passed validation for the past 7 consecutive days.
+* Results needing attention today: Quality rules that failed validation.
+
+## Quality Rules
+
+The quality rules page displays all your configured quality rules in a list format. Use the filter area at the top to quickly find the rules you need.
+
+![](.topwrite/assets/image_1760004360956.png)
+
+Use the top filter area to perform detailed searches.
+
+### Create a New Quality Rule
+
+1. In the quality rules list, or in the rule list of a validation object, click the "Create Rule" button to enter the new quality rule page.
+2. Fill in the required configuration items, such as data source, workspace, validation object, owner, description, parameter configuration, value filtering, validation method, expected result, trigger method, execution cluster, and timeout duration.
+ ![](.topwrite/assets/image_1760004384816.png)
+
+| **Configuration Item**       | **Configuration Description**                                                                                                                                                                                                                                |
+| ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Data Source           | The type of data source. Currently only Lakehouse data sources are supported.                                                                                                                                                                                                                |
+| Workspace          | The workspace to which the validation object belongs.                                                                                                                                                                                                                             |
+| Validation Object          | When the validation object is a Lakehouse table, select its Schema and name (table name, view name, etc.).                                                                                                                                                                                              |
+| Owner           | The person responsible for the quality rule, affecting who receives alerts.                                                                                                                                                                                                                        |
+| Description            | A description defined for the quality rule.                                                                                                                                                                                                                              |
+| Parameter Configuration          | In the quality rule, when using value filtering and custom SQL to compute metric values, you can reference predefined dynamic parameter values. For example, define a parameter: partition = $[yyyyMMdd].                                                                                                                                                           |
+| Value Filtering          | Used to filter the range of objects to validate, such as filtering by partition. Supports parameter references: dt = ${partition}.                                                                                                                                                                                       |
+| Validation Method - Built-in Metrics   | System built-in validation metrics. Select as needed.                                                                                                                                                                                                                        |
+| Validation Method - Custom SQL | If the built-in metrics do not meet your needs, you can compute metric values using custom SQL. Important: The result of custom SQL must be a single numeric value to enable comparison.                                                                                                                                                               |
+| Expected Result          | Define the expected result for the metric value.                                                                                                                                                                                                                              |
+| Trigger Method          | Configures how the quality rule is triggered to run. 1. Scheduled trigger: The system triggers a single validation run at the specified time. 2. Periodic task trigger: Triggered by an associated periodic task instance after the instance runs successfully. For periodic scheduling triggers, there are two scheduling blocking options: A. Strong blocking: If validation fails, the associated task instance is marked as failed, blocking downstream instances. B. Non-blocking: The quality rule runs as a bypass and does not affect the task instance's run status. 3. Manual trigger: Manually triggered as needed. |
+| Execution Cluster          | Specifies the compute cluster within the workspace for running the quality rule.                                                                                                                                                                                                                     |
+| Timeout Duration          | If the quality rule validation does not complete within the set timeout, it will be automatically canceled by the system.                                                                                                                                                                                                       |
+
+After filling in the required configuration items, click the "Confirm" button to create the rule.
+
+3. Example: If you want to check whether the record count of a table meets expectations, you can select "Record Count" as the built-in metric, set value filtering to a specific partition, set the expected result to a specific numeric value, trigger method to scheduled trigger, and execution cluster to your workspace's compute cluster.
+
+### Test Run a Quality Rule
+
+After creating a quality rule, it is recommended to use the "Test Run" feature to verify the correctness of the configuration. After a successful test run, you can view the results to adjust the rule.
+
+![](.topwrite/assets/image_1760004576132.png)
+
+### View Test Run Results
+
+After clicking "Test Run", follow the prompts to click "View Results" to see the test run validation results.
+
+![](.topwrite/assets/image_1699875850066.png)
+
+### Configure Monitoring Alerts
+
+To ensure data quality issues are addressed promptly, you can configure monitoring alerts for quality rules. There are two ways:
+
+1. **Enable global quality monitoring alerts**: In the monitoring and alerting module, search for "Data Quality Check Failed" and enable the system's built-in global quality validation monitoring rule.
+   ![](.topwrite/assets/image_1699875920709.png)
+2. **Configure custom quality monitoring alerts**: Create custom monitoring rules, select "Quality Rule Validation Failed" as the monitoring message, and set filter conditions such as workspace or specific validation object.
+   ![](.topwrite/assets/image_1699875932007.png)
+
+## Validation Objects
+
+On the validation objects page, you can manage all quality rules by the validation object (table) dimension. Use the search filter area to quickly locate specific rules.
+
+![](.topwrite/assets/image_1699875683208.png)
+
+## Validation Results
+
+### Validation Results List
+
+On the validation results list page, you can view the run status of all quality rules. Use the search and filter area to precisely find the required validation results.
+
+![](.topwrite/assets/image_1699875971234.png)
+
+### Operations on Validation Results
+
+For each validation result, you can perform the following operations:
+
+* **Terminate**: Cancel the current validation run.
+* **Set Success/Set Failure**: Manually set the validation result to success or failure.
+* **Revalidate**: Trigger the quality validation to run again. Note: If a rule that previously failed validation passes upon revalidation, no new monitoring alert will be sent.
+
+With the above features, you can effectively manage and monitor data quality, ensuring that the value of data in business applications is fully realized.

package/bin/skills/lakehouse-doc-en/references/data-recover.md

@@ -0,0 +1,52 @@
+# Lakehouse Time Travel Feature Usage Guide
+
+## Introduction to Time Travel Feature
+
+The Time Travel feature of Lakehouse allows you to easily access data at any point within the time travel window. This means you can view data at different points in time and even restore to a previous data state.
+
+With Time Travel, you can query updated or deleted data, restore deleted tables, or recover expired tables.
+![](.topwrite/assets/image_1708936936399.png =740)
+
+As shown in the figure above, the table originally had one piece of data, and operations such as insert, update, and delete were performed on the table, resulting in a total of four versions. When the specified version is between v0 and v1, it will return the data of v0. By specifying the version through Time Travel, you can view historical version data.
+
+## Accessing Data at a Specific Point in Time
+
+[TIME TRAVEL](timetravel.md)
+
+## Setting Data Retention Period
+
+You can configure the data retention period for Time Travel to determine how long data is retained within the time travel window. Depending on your needs, you can set different retention periods.
+
+The Time Travel retention period determines how far back you can access data. For example, if the Time Travel retention period is set to 7 days, you can access data from any point within the past 7 days. If it exceeds 7 days, you will no longer be able to use Time Travel to access expired data, and the data will be physically deleted.
+
+Currently, in the Preview version, the default retention period is 7 days, allowing you to query data within 7 days. In the future, Lakehouse will charge storage fees separately for Time Travel.
+
+### Setting Data Retention Period on a Table
+
+You can set the data retention period for a table using the following SQL statement:
+
+```SQL
+ALTER TABLE orders SET PROPERTIES ('data_retention_days'='num');
+```
+
+The setting range for `num` is 0-90, indicating the number of days the data is retained.
+
+## Restore Deleted Objects
+
+If you accidentally delete an object, you can use the `UNDROP` command to restore it. Here is an example of restoring a deleted table:
+
+```SQL
+UNDROP TABLE orders;
+```
+
+This command will restore the deleted table named `orders`.
+
+## Restore Data to a Specific Version
+
+Using the `RESTORE` command, you can restore data to a specific version. This is useful for rolling back or fixing erroneous data changes. Below is an example of restoring data to a specified version:
+
+```SQL
+RESTORE TABLE orders TO TIMESTAMP AS OF '2023-03-01 10:00:00';
+```
+
+This command restores the data of the `orders` table to the version as of 10:00:00 on March 1, 2023.