@clickzetta/cz-cli-darwin-x64 0.5.16 → 0.5.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/cz-cli +0 -0
- package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
- package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
- package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
- package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
- package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
- package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
- package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
- package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
- package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
- package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
- package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
- package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
- package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
- package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
- package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
- package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
- package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
- package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
- package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
- package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
- package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
- package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
- package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
- package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
- package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
- package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
- package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
- package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
- package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
- package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
- package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
- package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
- package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
- package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
- package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
- package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
- package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
- package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
- package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
- package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
- package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
- package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
- package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
- package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
- package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
- package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
- package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
- package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
- package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
- package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
- package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
- package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
- package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
- package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
- package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
- package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
- package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
- package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
- package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
- package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
- package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
- package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
- package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
- package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
- package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
- package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
- package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
- package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
- package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
- package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
- package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
- package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
- package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
- package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
- package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
- package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
- package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
- package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
- package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
- package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
- package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
- package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
- package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
- package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
- package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
- package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
- package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
- package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
- package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
- package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
- package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
- package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
- package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
- package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
- package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
- package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
- package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
- package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
- package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
- package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
- package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
- package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
- package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
- package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
- package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
- package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
- package/package.json +1 -1
- package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
- package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
|
@@ -6,7 +6,9 @@ In the era of big data, data quality management is a critical step to ensure dat
|
|
|
6
6
|
|
|
7
7
|
The data quality overview page provides an intuitive data quality monitoring dashboard, making it easy to view the overall status of quality rules and validation runs. Below are detailed explanations of some indicators:
|
|
8
8
|
|
|
9
|
-

|
|
10
|
+
|
|
11
|
+
^
|
|
10
12
|
|
|
11
13
|
The definitions of some indicators are as follows:
|
|
12
14
|
|
|
@@ -23,7 +25,7 @@ The definitions of some indicators are as follows:
|
|
|
23
25
|
|
|
24
26
|
The quality rules page displays all your configured quality rules in a list format. Use the filter area at the top to quickly find the rules you need.
|
|
25
27
|
|
|
26
|
-
|
|
28
|
+
^
|
|
27
29
|
|
|
28
30
|
Use the top filter area to perform detailed searches.
|
|
29
31
|
|
|
@@ -31,23 +33,22 @@ Use the top filter area to perform detailed searches.
|
|
|
31
33
|
|
|
32
34
|
1. In the quality rules list, or in the rule list of a validation object, click the "Create Rule" button to enter the new quality rule page.
|
|
33
35
|
2. Fill in the required configuration items, such as data source, workspace, validation object, owner, description, parameter configuration, value filtering, validation method, expected result, trigger method, execution cluster, and timeout duration.
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
|
37
|
-
|
|
|
38
|
-
|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
|
43
|
-
|
|
|
44
|
-
|
|
|
45
|
-
| Validation Method -
|
|
46
|
-
|
|
|
47
|
-
|
|
|
48
|
-
|
|
|
49
|
-
|
|
|
50
|
-
| Timeout Duration | If the quality rule validation does not complete within the set timeout, it will be automatically canceled by the system. |
|
|
36
|
+
|
|
37
|
+
| **Configuration Item** | **Configuration Description** |
|
|
38
|
+
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
39
|
+
| Data Source | The type of data source. Currently only Lakehouse data sources are supported. |
|
|
40
|
+
| Workspace | The workspace to which the validation object belongs. |
|
|
41
|
+
| Validation Object | When the validation object is a Lakehouse table, select its Schema and name (table name, view name, etc.). |
|
|
42
|
+
| Owner | The person responsible for the quality rule, affecting who receives alerts. |
|
|
43
|
+
| Description | A description defined for the quality rule. |
|
|
44
|
+
| Parameter Configuration | In the quality rule, when using value filtering and custom SQL to compute metric values, you can reference predefined dynamic parameter values. For example, define a parameter: partition = $\[yyyyMMdd]. |
|
|
45
|
+
| Value Filtering | Used to filter the range of objects to validate, such as filtering by partition. Supports parameter references: dt = ${partition}. |
|
|
46
|
+
| Validation Method - Built-in Metrics | System built-in validation metrics. Select as needed. |
|
|
47
|
+
| Validation Method - Custom SQL | If the built-in metrics do not meet your needs, you can compute metric values using custom SQL. Important: The result of custom SQL must be a single numeric value to enable comparison. |
|
|
48
|
+
| Expected Result | Define the expected result for the metric value. |
|
|
49
|
+
| Trigger Method | Configures how the quality rule is triggered to run. 1. Scheduled trigger: The system triggers a single validation run at the specified time. 2. Periodic task trigger: Triggered by an associated periodic task instance after the instance runs successfully. For periodic scheduling triggers, there are two scheduling blocking options: A. Strong blocking: If validation fails, the associated task instance is marked as failed, blocking downstream instances. B. Non-blocking: The quality rule runs as a bypass and does not affect the task instance's run status. 3. Manual trigger: Manually triggered as needed. |
|
|
50
|
+
| Execution Cluster | Specifies the compute cluster within the workspace for running the quality rule. |
|
|
51
|
+
| Timeout Duration | If the quality rule validation does not complete within the set timeout, it will be automatically canceled by the system. |
|
|
51
52
|
|
|
52
53
|
After filling in the required configuration items, click the "Confirm" button to create the rule.
|
|
53
54
|
|
|
@@ -57,28 +58,27 @@ After filling in the required configuration items, click the "Confirm" button to
|
|
|
57
58
|
|
|
58
59
|
After creating a quality rule, it is recommended to use the "Test Run" feature to verify the correctness of the configuration. After a successful test run, you can view the results to adjust the rule.
|
|
59
60
|
|
|
60
|
-
|
|
61
|
+
^
|
|
61
62
|
|
|
62
63
|
### View Test Run Results
|
|
63
64
|
|
|
64
65
|
After clicking "Test Run", follow the prompts to click "View Results" to see the test run validation results.
|
|
65
66
|
|
|
66
|
-
|
|
67
|
+
^
|
|
67
68
|
|
|
68
69
|
### Configure Monitoring Alerts
|
|
69
70
|
|
|
70
71
|
To ensure data quality issues are addressed promptly, you can configure monitoring alerts for quality rules. There are two ways:
|
|
71
72
|
|
|
72
73
|
1. **Enable global quality monitoring alerts**: In the monitoring and alerting module, search for "Data Quality Check Failed" and enable the system's built-in global quality validation monitoring rule.
|
|
73
|
-
|
|
74
|
+
|
|
74
75
|
2. **Configure custom quality monitoring alerts**: Create custom monitoring rules, select "Quality Rule Validation Failed" as the monitoring message, and set filter conditions such as workspace or specific validation object.
|
|
75
|
-

|
|
76
76
|
|
|
77
77
|
## Validation Objects
|
|
78
78
|
|
|
79
79
|
On the validation objects page, you can manage all quality rules by the validation object (table) dimension. Use the search filter area to quickly locate specific rules.
|
|
80
80
|
|
|
81
|
-

|
|
82
82
|
|
|
83
83
|
## Validation Results
|
|
84
84
|
|
|
@@ -86,7 +86,7 @@ On the validation objects page, you can manage all quality rules by the validati
|
|
|
86
86
|
|
|
87
87
|
On the validation results list page, you can view the run status of all quality rules. Use the search and filter area to precisely find the required validation results.
|
|
88
88
|
|
|
89
|
-
|
|
89
|
+
^
|
|
90
90
|
|
|
91
91
|
### Operations on Validation Results
|
|
92
92
|
|
|
@@ -53,40 +53,32 @@ You can use `all tables in schema <schema_name>` and `all views in schema <schem
|
|
|
53
53
|
|
|
54
54
|
**1. Create a share object**
|
|
55
55
|
|
|
56
|
-
To ensure data security, share objects must be created by users with the "Instance Administrator" (
|
|
56
|
+
To ensure data security, share objects must be created by users with the "Instance Administrator" (instance\_admin) role. Click "Data Management" - "Data Sharing" in the left menu to enter the data sharing list page.
|
|
57
57
|
|
|
58
58
|
Click "+ New Share" to open the new data sharing window:
|
|
59
59
|
|
|
60
|
-
|
|
60
|
+
1. Fill in the "Share Name" for the data share.
|
|
61
61
|
|
|
62
|
-
|
|
62
|
+
2. Select "Workspace," which refers to the workspace containing the tables or views to be shared. A single data share object can only contain data from one workspace.
|
|
63
63
|
|
|
64
|
-
|
|
64
|
+
3. In the data object row, click "Add." Select the specific tables or views to include in the data share. Multiple selections are allowed. Note that tables and views originating from other shares cannot be selected, because data sharing authorization does not permit secondary sharing. Data shares created via the web interface only support including existing tables and views; specifying "future" tables or views under an entire schema is not yet supported.
|
|
65
65
|
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
4) In the recipient instance row, click "Add." Enter the name of the service instance that needs to receive the data. Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
|
|
69
|
-
|
|
70
|
-
:-: 
|
|
66
|
+
4. In the recipient instance row, click "Add." Enter the name of the service instance that needs to receive the data. Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
|
|
71
67
|
|
|
72
68
|
^
|
|
73
69
|
|
|
74
70
|
*Note*:
|
|
75
71
|
|
|
76
|
-
1. *To ensure data security with workspace-level isolation, only users with the Workspace Administrator role can add data objects from their managed workspace to a data share
|
|
77
|
-
2. *In the create share dialog, "Create" and "Add Data" are two separate operations. Even if adding shared data fails partially or completely, it does not affect the creation of the share object. When any operation fails, an error message is returned. You can click on an already-created share in the data sharing list to continue adding data objects or recipient service instances
|
|
72
|
+
1. *To ensure data security with workspace-level isolation, only users with the Workspace Administrator role can add data objects from their managed workspace to a data share*.
|
|
73
|
+
2. *In the create share dialog, "Create" and "Add Data" are two separate operations. Even if adding shared data fails partially or completely, it does not affect the creation of the share object. When any operation fails, an error message is returned. You can click on an already-created share in the data sharing list to continue adding data objects or recipient service instances*.
|
|
78
74
|
|
|
79
75
|
**2. Query created share objects**
|
|
80
76
|
|
|
81
|
-
All share objects created within the current service instance are displayed on the "Data Sharing" list page. Users with the Instance Administrator role (
|
|
82
|
-
|
|
83
|
-
:-: 
|
|
77
|
+
All share objects created within the current service instance are displayed on the "Data Sharing" list page. Users with the Instance Administrator role (instance\_admin) or Workspace Administrator role (workspace\_admin) can view the list of share objects.
|
|
84
78
|
|
|
85
79
|
**3. View share details**
|
|
86
80
|
|
|
87
|
-
On the "Data Sharing" list page, click a specific share name to view detailed information about the share, including basic information, recipient service instances, and shared data objects. Users with the Instance Administrator role (
|
|
88
|
-
|
|
89
|
-
:-: 
|
|
81
|
+
On the "Data Sharing" list page, click a specific share name to view detailed information about the share, including basic information, recipient service instances, and shared data objects. Users with the Instance Administrator role (instance\_admin) or Workspace Administrator role (workspace\_admin) can view share object details.
|
|
90
82
|
|
|
91
83
|
**4. Add/Remove shared data objects**
|
|
92
84
|
|
|
@@ -96,19 +88,16 @@ On the data sharing list page, click the name of the share object to which you w
|
|
|
96
88
|
|
|
97
89
|
Added data objects become immediately visible to share consumers and appear in their schema for extracting data.
|
|
98
90
|
|
|
99
|
-
:-: 
|
|
100
|
-
|
|
101
91
|
**5. Configure share targets**
|
|
102
92
|
|
|
103
|
-
You must configure the recipient service instance name (
|
|
93
|
+
You must configure the recipient service instance name (instance\_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
|
|
104
94
|
|
|
105
|
-
:-: 
|
|
106
95
|
|
|
107
96
|
^
|
|
108
97
|
|
|
109
98
|
**6. Delete a share object**
|
|
110
99
|
|
|
111
|
-
Users with the Instance Administrator role (
|
|
100
|
+
Users with the Instance Administrator role (instance\_admin) or the Workspace Administrator role (workspace\_admin) for the workspace to which the share belongs can delete share objects.
|
|
112
101
|
|
|
113
102
|
On the data sharing list page or in the share details, click the "Delete" button under "Actions" to delete the share object. Once a share object is deleted, it cannot be recovered. Data consumers immediately lose access to the shared data.
|
|
114
103
|
|
|
@@ -116,7 +105,7 @@ On the data sharing list page or in the share details, click the "Delete" button
|
|
|
116
105
|
|
|
117
106
|
**1. Create a share object**
|
|
118
107
|
|
|
119
|
-
To ensure data security, share objects must be created by users with the "Instance Administrator" (
|
|
108
|
+
To ensure data security, share objects must be created by users with the "Instance Administrator" (instance\_admin) role. When creating a share object, execute the following statement in the workspace where you plan to share data:
|
|
120
109
|
|
|
121
110
|
```SQL
|
|
122
111
|
CREATE SHARE <share_name>;
|
|
@@ -145,7 +134,7 @@ Grant select, read metadata on ALL tables in SCHEMA share_demo_schema TO SHARE s
|
|
|
145
134
|
Grant select, read metadata on ALL views in SCHEMA share_demo_schema TO SHARE share_demo;
|
|
146
135
|
```
|
|
147
136
|
|
|
148
|
-
When adding data objects to a share, the user executing the operation must have SELECT (query) and READ METADATA (view metadata) privileges on the involved tables or views, and must be able to grant them onward (with grant option). The Workspace Administrator role (
|
|
137
|
+
When adding data objects to a share, the user executing the operation must have SELECT (query) and READ METADATA (view metadata) privileges on the involved tables or views, and must be able to grant them onward (with grant option). The Workspace Administrator role (workspace\_admin) inherently has the above privileges.
|
|
149
138
|
|
|
150
139
|
To remove shared data objects from a share, use the REVOKE statement:
|
|
151
140
|
|
|
@@ -159,11 +148,10 @@ Note: When sharing a table or view, since the table or view always exists under
|
|
|
159
148
|
|
|
160
149
|
**3. Configure share targets**
|
|
161
150
|
|
|
162
|
-
You must configure the recipient service instance name (
|
|
151
|
+
You must configure the recipient service instance name (instance\_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
|
|
163
152
|
|
|
164
153
|
Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
|
|
165
154
|
|
|
166
|
-
:-: 
|
|
167
155
|
|
|
168
156
|
The syntax for the data provider to configure share targets is:
|
|
169
157
|
|
|
@@ -185,17 +173,13 @@ You can execute the SHOW command to query created share objects. The syntax is:
|
|
|
185
173
|
SHOW SHARES;
|
|
186
174
|
```
|
|
187
175
|
|
|
188
|
-
Example result:
|
|
189
|
-
|
|
190
|
-
:-: 
|
|
191
|
-
|
|
192
176
|
Where:
|
|
193
177
|
|
|
194
178
|
* provider is the tenant name of the share provider;
|
|
195
|
-
*
|
|
196
|
-
*
|
|
179
|
+
* provider\_instance is the service instance name of the share provider;
|
|
180
|
+
* provider\_workspace is the workspace to which the share belongs;
|
|
197
181
|
* scope is the sharing scope of the share. Currently only PRIVATE is supported -- sharing to specified instances;
|
|
198
|
-
*
|
|
182
|
+
* to\_instance is the service instance name(s) to which the share is shared. Multiple instance names are separated by commas (,);
|
|
199
183
|
* kind is the share type. OUTBOUND indicates data shared from the current service instance; INBOUND indicates data shared from other service instances to the current service instance.
|
|
200
184
|
|
|
201
185
|
^
|
|
@@ -208,9 +192,6 @@ Execute the following statement to query the data objects granted in a share:
|
|
|
208
192
|
DESC SHARE <share_name>;
|
|
209
193
|
```
|
|
210
194
|
|
|
211
|
-
Example result:
|
|
212
|
-
|
|
213
|
-
:-: 
|
|
214
195
|
|
|
215
196
|
^
|
|
216
197
|
|
|
@@ -230,11 +211,10 @@ Once a share object is deleted, it cannot be recovered. Data consumers immediate
|
|
|
230
211
|
|
|
231
212
|
**1. View shared data**
|
|
232
213
|
|
|
233
|
-
Users with the Instance Administrator (
|
|
214
|
+
Users with the Instance Administrator (instance\_admin) role or Workspace Administrator (workspace\_admin) role can view all share objects shared to the current service instance in the "Shared With Me" tab under the "Data Sharing" menu.
|
|
234
215
|
|
|
235
216
|
Click a share object name to view details including the source, reception time, and the data (tables or views) currently contained in the share.
|
|
236
217
|
|
|
237
|
-
:-: 
|
|
238
218
|
|
|
239
219
|
^
|
|
240
220
|
|
|
@@ -242,11 +222,9 @@ Click a share object name to view details including the source, reception time,
|
|
|
242
222
|
|
|
243
223
|
On the "Shared With Me" list page or the share details page, click the "Extract" button to extract data from the share.
|
|
244
224
|
|
|
245
|
-
:-: 
|
|
246
225
|
|
|
247
226
|
When extracting data, extraction can only be done at the schema level. Individual table or view extraction is not yet supported.
|
|
248
227
|
|
|
249
|
-
:-: 
|
|
250
228
|
|
|
251
229
|
First, select "Source Schema," which is the schema from which to extract data. If the share contains multiple schemas, you need to perform multiple extractions.
|
|
252
230
|
|
|
@@ -260,7 +238,6 @@ After completing data extraction, you can see all tables and views shared under
|
|
|
260
238
|
|
|
261
239
|
Shared data is marked with a special icon in the "Data" panel for easy identification.
|
|
262
240
|
|
|
263
|
-
:-: 
|
|
264
241
|
|
|
265
242
|
### Using SQL
|
|
266
243
|
|
|
@@ -284,7 +261,7 @@ Where `<provider_instance>` and `<share_name>` can be obtained from the `SHOW SH
|
|
|
284
261
|
|
|
285
262
|
The schema name in `CREATE SCHEMA <schema_name>` can be customized and does not need to match the schema name in the share.
|
|
286
263
|
|
|
287
|
-
When executing the above command, the operator must have permission to create schemas in the target workspace. The Workspace Administrator role (
|
|
264
|
+
When executing the above command, the operator must have permission to create schemas in the target workspace. The Workspace Administrator role (workspace\_admin) inherently has this permission.
|
|
288
265
|
|
|
289
266
|
^
|
|
290
267
|
|
|
@@ -296,15 +273,15 @@ After executing `CREATE SCHEMA FROM SHARE`, you can see all tables and views sha
|
|
|
296
273
|
|
|
297
274
|
**1. Provider permissions**
|
|
298
275
|
|
|
299
|
-
Only users with the Instance Administrator (
|
|
276
|
+
Only users with the Instance Administrator (instance\_admin) role can create share objects. The permission points for share operations are as follows. Granting share object permissions to other roles or users via the GRANT statement is not currently supported.
|
|
300
277
|
|
|
301
|
-
| **Permission**
|
|
302
|
-
|
|
|
303
|
-
| Create share
|
|
304
|
-
| Alter share
|
|
305
|
-
| Drop share
|
|
306
|
-
| Read metadata
|
|
307
|
-
| Grant objectprivilege to share / Revoke objectprivilege from share | Add or remove data objects from a share: GRANT objectPrivilege TO; REVOKE objectPrivilege FROM.
|
|
278
|
+
| **Permission** | **Description** |
|
|
279
|
+
| ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
|
|
280
|
+
| Create share | Permission to create share objects. |
|
|
281
|
+
| Alter share | Permission to modify share objects, add or remove shared instance IDs. |
|
|
282
|
+
| Drop share | Permission to delete share instances. |
|
|
283
|
+
| Read metadata | Permission for SHOW SHARES and DESC SHARE. DESC SHARE returns the objects and granted object privileges within the share. |
|
|
284
|
+
| Grant objectprivilege to share / Revoke objectprivilege from share | Add or remove data objects from a share: GRANT objectPrivilege TO; REVOKE objectPrivilege FROM. |
|
|
308
285
|
|
|
309
286
|
^
|
|
310
287
|
|
|
@@ -312,9 +289,9 @@ Only users with the Instance Administrator (instance_admin) role can create shar
|
|
|
312
289
|
|
|
313
290
|
All members in a workspace have USE and READ METADATA permissions on share objects. However, the CREATE SCHEMA permission in the workspace is required to create a schema from the data in a share.
|
|
314
291
|
|
|
315
|
-
| **Permission**
|
|
316
|
-
|
|
|
317
|
-
| Use
|
|
318
|
-
| Read metadata
|
|
292
|
+
| **Permission** | **Description** |
|
|
293
|
+
| -------------- | ----------------------------------------------------------------------------------------------------- |
|
|
294
|
+
| Use | Permission to use the share. |
|
|
295
|
+
| Read metadata | Permission to query share metadata. After obtaining this, SHOW SHARES and DESC SHARE can be executed. |
|
|
319
296
|
|
|
320
297
|
^
|
|
@@ -10,53 +10,53 @@ The data source types and versions currently supported by each task type are as
|
|
|
10
10
|
|
|
11
11
|
### Offline Sync
|
|
12
12
|
|
|
13
|
-
| Data Source
|
|
14
|
-
|
|
|
15
|
-
| Singdata Lakehouse
|
|
16
|
-
| ADB MySQL | GA (Beta)
|
|
17
|
-
| ADB PostgreSQL | GA (Beta)
|
|
18
|
-
| AMQP | GA (Beta)
|
|
19
|
-
| AutoMQ | GA
|
|
20
|
-
| ClickHouse
|
|
21
|
-
| COS | GA
|
|
22
|
-
| Databricks | GA
|
|
23
|
-
| DB2 | GA (Beta)
|
|
24
|
-
| DM | GA (Beta)
|
|
25
|
-
| Doris | GA (Beta)
|
|
26
|
-
| DynamoDB | GA
|
|
27
|
-
| Elasticsearch | GA
|
|
28
|
-
| Greenplum
|
|
29
|
-
| HBase
|
|
30
|
-
| Hive | GA
|
|
31
|
-
| Hologres | GA (Beta)
|
|
32
|
-
| Kafka
|
|
33
|
-
| MaxCompute | GA
|
|
34
|
-
| MariaDB | GA (Beta)
|
|
35
|
-
| MongoDB
|
|
36
|
-
| MySQL | GA
|
|
37
|
-
| Oracle | GA (Beta)
|
|
38
|
-
| OSS | GA
|
|
39
|
-
| PolarDB for MySQL | GA
|
|
40
|
-
| PolarDB for PostgreSQL | GA
|
|
41
|
-
| PostgreSQL
|
|
42
|
-
| Redis | GA
|
|
43
|
-
| Redshift | GA (Beta)
|
|
44
|
-
| RestApi | GA
|
|
45
|
-
| SLS (LogHub)
|
|
46
|
-
| SQL Server
|
|
47
|
-
| StarRocks | GA (Beta)
|
|
48
|
-
| TiDB | GA (Beta)
|
|
13
|
+
| Data Source | Release Level | Offline Single-Table Read | Offline Single-Table Write | Supported Versions |
|
|
14
|
+
| ---------------------- | ------------- | ------------------------- | -------------------------- | ------------------------- |
|
|
15
|
+
| Singdata Lakehouse | GA | Y | Y | - |
|
|
16
|
+
| ADB MySQL | GA (Beta) | Y | Y | 3.x |
|
|
17
|
+
| ADB PostgreSQL | GA (Beta) | Y | Y | 6.x |
|
|
18
|
+
| AMQP | GA (Beta) | Y | N | - |
|
|
19
|
+
| AutoMQ | GA | Y | Y | - |
|
|
20
|
+
| ClickHouse | GA | Y | Y | 19.x and above |
|
|
21
|
+
| COS | GA | Y | Y | - |
|
|
22
|
+
| Databricks | GA | Y | Y | - |
|
|
23
|
+
| DB2 | GA (Beta) | Y | Y | 11.x |
|
|
24
|
+
| DM | GA (Beta) | Y | Y | 8.x |
|
|
25
|
+
| Doris | GA (Beta) | Y | Y | 1.x |
|
|
26
|
+
| DynamoDB | GA | Y | Y | - |
|
|
27
|
+
| Elasticsearch | GA | Y | Y | 7.x |
|
|
28
|
+
| Greenplum | GA (Beta) | Y | Y | 5.x and above |
|
|
29
|
+
| HBase | GA | N | Y | 1.4+ |
|
|
30
|
+
| Hive | GA | Y | Y | 2.x |
|
|
31
|
+
| Hologres | GA (Beta) | Y | Y | - |
|
|
32
|
+
| Kafka | GA | Y | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
|
|
33
|
+
| MaxCompute | GA | Y | N | - |
|
|
34
|
+
| MariaDB | GA (Beta) | Y | Y | - |
|
|
35
|
+
| MongoDB | GA | Y | N | 3.4 and above |
|
|
36
|
+
| MySQL | GA | Y | Y | 5.x, 8.x |
|
|
37
|
+
| Oracle | GA (Beta) | Y | Y | - |
|
|
38
|
+
| OSS | GA | Y | Y | - |
|
|
39
|
+
| PolarDB for MySQL | GA | Y | Y | - |
|
|
40
|
+
| PolarDB for PostgreSQL | GA | Y | Y | - |
|
|
41
|
+
| PostgreSQL | GA | Y | Y | 9.4 and above |
|
|
42
|
+
| Redis | GA | N | Y | - |
|
|
43
|
+
| Redshift | GA (Beta) | Y | Y | - |
|
|
44
|
+
| RestApi | GA | Y | N | - |
|
|
45
|
+
| SLS (LogHub) | GA | Y | N | - |
|
|
46
|
+
| SQL Server | GA | Y | Y | 2012 and above |
|
|
47
|
+
| StarRocks | GA (Beta) | Y | Y | 2.5 |
|
|
48
|
+
| TiDB | GA (Beta) | Y | Y | - |
|
|
49
49
|
|
|
50
50
|
### Real-Time Sync
|
|
51
51
|
|
|
52
|
-
| Data Source
|
|
53
|
-
|
|
|
54
|
-
| Singdata Lakehouse
|
|
55
|
-
| AutoMQ | GA
|
|
56
|
-
| Kafka
|
|
57
|
-
| MySQL
|
|
58
|
-
| PolarDB for MySQL | GA
|
|
59
|
-
| PolarDB for PostgreSQL | GA
|
|
60
|
-
| SQL Server
|
|
52
|
+
| Data Source | Release Level | Real-Time Single-Table Read | Real-Time Multi-Table Read | Real-Time Write | Supported Versions |
|
|
53
|
+
| ---------------------- | ------------- | --------------------------- | -------------------------- | --------------- | ---------------------------------------------- |
|
|
54
|
+
| Singdata Lakehouse | GA | N | N | Y | - |
|
|
55
|
+
| AutoMQ | GA | Y | N | N | - |
|
|
56
|
+
| Kafka | GA | Y | N | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
|
|
57
|
+
| MySQL | GA | Y | Y | N | 5.x, 8.x |
|
|
58
|
+
| PolarDB for MySQL | GA | Y | Y | N | - |
|
|
59
|
+
| PolarDB for PostgreSQL | GA | Y | Y | N | - |
|
|
60
|
+
| SQL Server | GA | Y | Y | N | SQL Server 2016 Service Pack 1 (SP1) and above |
|
|
61
61
|
|
|
62
62
|
^
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
Data Catalog, also known as the data asset map, provides powerful data search and display capabilities, designed to help users more easily find, understand, and use data. It supports two modes: Data Search mode and Data Management mode. Data Search mode allows users to search data directly, while Data Management mode allows users to browse and manage data tables through a directory structure. With Data Catalog, you can manage and utilize data resources more efficiently.
|
|
6
6
|
|
|
7
|
-

|
|
8
8
|
|
|
9
9
|
* Asset Details: Displays the total data assets of the current tenant.
|
|
10
10
|
* Search Box: Click to enter the data search results page.
|
|
@@ -16,11 +16,11 @@ Data Catalog, also known as the data asset map, provides powerful data search an
|
|
|
16
16
|
|
|
17
17
|
Enter keywords in the search box on the Data Catalog homepage to enter the search results page. The search function supports searching by the name and description of object types (Table, View, Materialized View). The search results page provides various filtering options, including object type, workspace/Schema, creation time, owner, etc., as well as sorting options by name, creation time, update time, etc.
|
|
18
18
|
|
|
19
|
-

|
|
20
20
|
|
|
21
21
|
On the search results filtering: supports filtering by object type, workspace/schema, creation time, and owner; also supports filtering by name, creation time, and update time.
|
|
22
22
|
|
|
23
|
-

|
|
24
24
|
|
|
25
25
|
Click the table name on the search results page to navigate to the corresponding table details page.
|
|
26
26
|
|
|
@@ -28,55 +28,53 @@ Click the table name on the search results page to navigate to the corresponding
|
|
|
28
28
|
|
|
29
29
|
The Data Management page displays all data assets that the current user has permission to manage under the service instance in a directory format. Click **Data Management** from the homepage to enter the Data Management page, which displays the list of workspaces that the current user has permissions to under the service.
|
|
30
30
|
|
|
31
|
-

|
|
32
|
+
|
|
33
|
+
^
|
|
32
34
|
|
|
33
35
|
### Workspace Level
|
|
34
36
|
|
|
35
37
|
At the workspace level, you can see the list of all Schemas under that workspace. Click the directory tree on the left to expand and view the hierarchy. The top area displays basic workspace information, such as source, creation time/modification time, owner, etc. The information area presents specific Schema information, including name, creation time, update time, owner, etc.
|
|
36
38
|
|
|
37
|
-
Each workspace has a default
|
|
38
|
-
|
|
39
|
-
Schemas under a workspace include two categories: internally created and externally created.
|
|
39
|
+
Each workspace has a default information\_schema.
|
|
40
40
|
|
|
41
|
-
|
|
41
|
+
Schemas under a workspace include two categories: internally created and externally created.
|
|
42
42
|
|
|
43
43
|
Users can create schema information under a workspace through a visual interface. Click the question mark next to the title to view help documentation for creating objects.
|
|
44
44
|
|
|
45
|
-

|
|
46
46
|
|
|
47
47
|
### Schema Level
|
|
48
48
|
|
|
49
|
-
After clicking a specific Schema, you can drill down to the Schema's details page. At the Schema level, you can view all object information managed under the Schema, with specific objects organized by object type. The system currently supports three object types: Table, View, and Materialized View.
|
|
50
|
-
|
|
51
|
-

|
|
49
|
+
After clicking a specific Schema, you can drill down to the Schema's details page. At the Schema level, you can view all object information managed under the Schema, with specific objects organized by object type. The system currently supports three object types: Table, View, and Materialized View.
|
|
52
50
|
|
|
53
51
|
Objects can be created directly under a Schema, supporting both script-based creation and visual creation. The system has built-in templates for different objects to help users better understand Lakehouse syntax.
|
|
54
52
|
|
|
55
|
-

|
|
56
54
|
|
|
57
55
|
## Table Details Page
|
|
58
56
|
|
|
59
|
-
The table details page displays all detailed information for tables in the Singdata Lakehouse library, including field information, data preview, data lineage, job history, uploads, etc. The details page is divided into two parts: the top area displays the table name, owner, and some quick table operations; the information area contains six major sections: Details, Fields, Preview, Lineage, Jobs, and Uploads.
|
|
60
|
-
|
|
61
|
-

|
|
57
|
+
The table details page displays all detailed information for tables in the Singdata Lakehouse library, including field information, data preview, data lineage, job history, uploads, etc. The details page is divided into two parts: the top area displays the table name, owner, and some quick table operations; the information area contains six major sections: Details, Fields, Preview, Lineage, Jobs, and Uploads.
|
|
62
58
|
|
|
63
59
|
### Top Functional Area
|
|
64
60
|
|
|
65
61
|
The top area displays the full structure of the current table (i.e., the corresponding workspace/schema/table name information), row count/storage size, creation time/update time, owner, and other basic information. The top area also provides an **Upload** function, supporting the uploading of local files to the Singdata Lakehouse platform.
|
|
66
62
|
|
|
67
|
-
|
|
63
|
+
^
|
|
64
|
+
|
|
65
|
+

|
|
68
66
|
|
|
69
|
-
|
|
67
|
+
DDL preview:
|
|
70
68
|
|
|
71
69
|
The Details tab displays the DDL statement of the current table, supporting one-click copy. It also supports one-click navigation to the permission management page for permission authorization.
|
|
72
70
|
|
|
73
|
-
|
|
71
|
+
^
|
|
74
72
|
|
|
75
73
|
### Fields
|
|
76
74
|
|
|
77
75
|
The **Fields** tab displays the field names, types, descriptions, and other information for the current table. If there is primary key information, it can also display the primary key and standardized labels for the fields.
|
|
78
76
|
|
|
79
|
-

|
|
80
78
|
|
|
81
79
|
### Preview
|
|
82
80
|
|
|
@@ -84,26 +82,26 @@ In the Data Preview tab, you can preview the first 100 rows of data in the curre
|
|
|
84
82
|
|
|
85
83
|
Data preview requires the user to manually select a compute cluster participating in the query to display the preview data.
|
|
86
84
|
|
|
87
|
-
|
|
85
|
+
^
|
|
88
86
|
|
|
89
87
|
### Lineage
|
|
90
88
|
|
|
91
89
|
In the Data Lineage tab, you can view the upstream and downstream dependencies of the current table. The lineage relationships here are generated based on code parsing of the corresponding task that produces this table, and belong to relationships from the "definition perspective."
|
|
92
90
|
|
|
93
|
-
|
|
91
|
+
^
|
|
94
92
|
|
|
95
93
|
Click on a specific upstream or downstream table node to view more detailed information:
|
|
96
94
|
|
|
97
|
-
|
|
95
|
+
^
|
|
98
96
|
|
|
99
97
|
### Jobs
|
|
100
98
|
|
|
101
99
|
Under the Jobs tab, you can view the SQL jobs of the tasks that produce the table. By clicking on the job ID, you can further view the job details.
|
|
102
100
|
|
|
103
|
-
|
|
101
|
+
^
|
|
104
102
|
|
|
105
103
|
### Uploads
|
|
106
104
|
|
|
107
105
|
Under the Uploads tab, all historical records uploaded to the current table via the **Upload** function are displayed.
|
|
108
106
|
|
|
109
|
-
|
|
107
|
+
^
|
|
@@ -13,11 +13,14 @@ When the system processes your data queries, the following data will be sent to
|
|
|
13
13
|
|
|
14
14
|
**Data**:
|
|
15
15
|
|
|
16
|
-
* Column value data with indexing enabled (You can prevent specific column data from being sent to the large language model by disabling column value indexing)
|
|
16
|
+
* Column value data with indexing enabled (You can prevent specific column data from being sent to the large language model by disabling column value indexing)
|
|
17
|
+
|
|
18
|
+

|
|
17
19
|
|
|
18
20
|
**Document Segmentation**:
|
|
19
21
|
|
|
20
|
-
The system segments
|
|
22
|
+
* The system segments document content into chunks
|
|
23
|
+
* These document segments are sent to the Large Language Model to enable intelligent question-answering
|
|
21
24
|
|
|
22
25
|
## Data Protection Measures
|
|
23
26
|
|