@clickzetta/cz-cli-darwin-arm64 0.5.16 → 0.5.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (243) hide show
  1. package/bin/cz-cli +0 -0
  2. package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
  3. package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
  4. package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
  5. package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
  6. package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
  7. package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
  8. package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
  9. package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
  10. package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
  11. package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
  12. package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
  13. package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
  14. package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
  15. package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
  16. package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
  17. package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
  18. package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
  19. package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
  20. package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
  21. package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
  22. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
  23. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
  24. package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
  25. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
  26. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
  27. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
  28. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
  29. package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
  30. package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
  31. package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
  32. package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
  33. package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
  34. package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
  35. package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
  36. package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
  37. package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
  38. package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
  39. package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
  40. package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
  41. package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
  42. package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
  43. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
  44. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
  45. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
  46. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
  47. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
  48. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
  49. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
  50. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
  51. package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
  52. package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
  53. package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
  54. package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
  55. package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
  56. package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
  57. package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
  58. package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
  59. package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
  60. package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
  61. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
  62. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
  63. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
  64. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
  65. package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
  66. package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
  67. package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
  68. package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
  69. package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
  70. package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
  71. package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
  72. package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
  73. package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
  74. package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
  75. package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
  76. package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
  77. package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
  78. package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
  79. package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
  80. package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
  81. package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
  82. package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
  83. package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
  84. package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
  85. package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
  86. package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
  87. package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
  88. package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
  89. package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
  90. package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
  91. package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
  92. package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
  93. package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
  94. package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
  95. package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
  96. package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
  97. package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
  98. package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
  99. package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
  100. package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
  101. package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
  102. package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
  103. package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
  104. package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
  105. package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
  106. package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
  107. package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
  108. package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
  109. package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
  110. package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
  111. package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
  112. package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
  113. package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
  114. package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
  115. package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
  116. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
  117. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
  118. package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
  119. package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
  120. package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
  121. package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
  122. package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
  123. package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
  124. package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
  125. package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
  126. package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
  127. package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
  128. package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
  129. package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
  130. package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
  131. package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
  132. package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
  133. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
  134. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
  135. package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
  136. package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
  137. package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
  138. package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
  139. package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
  140. package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
  141. package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
  142. package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
  143. package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
  144. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
  145. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
  146. package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
  147. package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
  148. package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
  149. package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
  150. package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
  151. package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
  152. package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
  153. package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
  154. package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
  155. package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
  156. package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
  157. package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
  158. package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
  159. package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
  160. package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
  161. package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
  162. package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
  163. package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
  164. package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
  165. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
  166. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
  167. package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
  168. package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
  169. package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
  170. package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
  171. package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
  172. package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
  173. package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
  174. package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
  175. package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
  176. package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
  177. package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
  178. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
  179. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
  180. package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
  181. package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
  182. package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
  183. package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
  184. package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
  185. package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
  186. package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
  187. package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
  188. package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
  189. package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
  190. package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
  191. package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
  192. package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
  193. package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
  194. package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
  195. package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
  196. package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
  197. package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
  198. package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
  199. package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
  200. package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
  201. package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
  202. package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
  203. package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
  204. package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
  205. package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
  206. package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
  207. package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
  208. package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
  209. package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
  210. package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
  211. package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
  212. package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
  213. package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
  214. package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
  215. package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
  216. package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
  217. package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
  218. package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
  219. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
  220. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
  221. package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
  222. package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
  223. package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
  224. package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
  225. package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
  226. package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
  227. package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
  228. package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
  229. package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
  230. package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
  231. package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
  232. package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
  233. package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
  234. package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
  235. package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
  236. package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
  237. package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
  238. package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
  239. package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
  240. package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
  241. package/package.json +1 -1
  242. package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
  243. package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
@@ -6,7 +6,9 @@ In the era of big data, data quality management is a critical step to ensure dat
6
6
 
7
7
  The data quality overview page provides an intuitive data quality monitoring dashboard, making it easy to view the overall status of quality rules and validation runs. Below are detailed explanations of some indicators:
8
8
 
9
- ![](.topwrite/assets/image_1760004305684.png)
9
+ ![](/.topwrite/assets/image_1780673989475.png =740)
10
+
11
+ ^
10
12
 
11
13
  The definitions of some indicators are as follows:
12
14
 
@@ -23,7 +25,7 @@ The definitions of some indicators are as follows:
23
25
 
24
26
  The quality rules page displays all your configured quality rules in a list format. Use the filter area at the top to quickly find the rules you need.
25
27
 
26
- ![](.topwrite/assets/image_1760004360956.png)
28
+ ^
27
29
 
28
30
  Use the top filter area to perform detailed searches.
29
31
 
@@ -31,23 +33,22 @@ Use the top filter area to perform detailed searches.
31
33
 
32
34
  1. In the quality rules list, or in the rule list of a validation object, click the "Create Rule" button to enter the new quality rule page.
33
35
  2. Fill in the required configuration items, such as data source, workspace, validation object, owner, description, parameter configuration, value filtering, validation method, expected result, trigger method, execution cluster, and timeout duration.
34
- ![](.topwrite/assets/image_1760004384816.png)
35
-
36
- | **Configuration Item** | **Configuration Description** |
37
- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
38
- | Data Source | The type of data source. Currently only Lakehouse data sources are supported. |
39
- | Workspace | The workspace to which the validation object belongs. |
40
- | Validation Object | When the validation object is a Lakehouse table, select its Schema and name (table name, view name, etc.). |
41
- | Owner | The person responsible for the quality rule, affecting who receives alerts. |
42
- | Description | A description defined for the quality rule. |
43
- | Parameter Configuration | In the quality rule, when using value filtering and custom SQL to compute metric values, you can reference predefined dynamic parameter values. For example, define a parameter: partition = $[yyyyMMdd]. |
44
- | Value Filtering | Used to filter the range of objects to validate, such as filtering by partition. Supports parameter references: dt = ${partition}. |
45
- | Validation Method - Built-in Metrics | System built-in validation metrics. Select as needed. |
46
- | Validation Method - Custom SQL | If the built-in metrics do not meet your needs, you can compute metric values using custom SQL. Important: The result of custom SQL must be a single numeric value to enable comparison. |
47
- | Expected Result | Define the expected result for the metric value. |
48
- | Trigger Method | Configures how the quality rule is triggered to run. 1. Scheduled trigger: The system triggers a single validation run at the specified time. 2. Periodic task trigger: Triggered by an associated periodic task instance after the instance runs successfully. For periodic scheduling triggers, there are two scheduling blocking options: A. Strong blocking: If validation fails, the associated task instance is marked as failed, blocking downstream instances. B. Non-blocking: The quality rule runs as a bypass and does not affect the task instance's run status. 3. Manual trigger: Manually triggered as needed. |
49
- | Execution Cluster | Specifies the compute cluster within the workspace for running the quality rule. |
50
- | Timeout Duration | If the quality rule validation does not complete within the set timeout, it will be automatically canceled by the system. |
36
+
37
+ | **Configuration Item** | **Configuration Description** |
38
+ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
39
+ | Data Source | The type of data source. Currently only Lakehouse data sources are supported. |
40
+ | Workspace | The workspace to which the validation object belongs. |
41
+ | Validation Object | When the validation object is a Lakehouse table, select its Schema and name (table name, view name, etc.). |
42
+ | Owner | The person responsible for the quality rule, affecting who receives alerts. |
43
+ | Description | A description defined for the quality rule. |
44
+ | Parameter Configuration | In the quality rule, when using value filtering and custom SQL to compute metric values, you can reference predefined dynamic parameter values. For example, define a parameter: partition = $\[yyyyMMdd]. |
45
+ | Value Filtering | Used to filter the range of objects to validate, such as filtering by partition. Supports parameter references: dt = ${partition}. |
46
+ | Validation Method - Built-in Metrics | System built-in validation metrics. Select as needed. |
47
+ | Validation Method - Custom SQL | If the built-in metrics do not meet your needs, you can compute metric values using custom SQL. Important: The result of custom SQL must be a single numeric value to enable comparison. |
48
+ | Expected Result | Define the expected result for the metric value. |
49
+ | Trigger Method | Configures how the quality rule is triggered to run. 1. Scheduled trigger: The system triggers a single validation run at the specified time. 2. Periodic task trigger: Triggered by an associated periodic task instance after the instance runs successfully. For periodic scheduling triggers, there are two scheduling blocking options: A. Strong blocking: If validation fails, the associated task instance is marked as failed, blocking downstream instances. B. Non-blocking: The quality rule runs as a bypass and does not affect the task instance's run status. 3. Manual trigger: Manually triggered as needed. |
50
+ | Execution Cluster | Specifies the compute cluster within the workspace for running the quality rule. |
51
+ | Timeout Duration | If the quality rule validation does not complete within the set timeout, it will be automatically canceled by the system. |
51
52
 
52
53
  After filling in the required configuration items, click the "Confirm" button to create the rule.
53
54
 
@@ -57,28 +58,27 @@ After filling in the required configuration items, click the "Confirm" button to
57
58
 
58
59
  After creating a quality rule, it is recommended to use the "Test Run" feature to verify the correctness of the configuration. After a successful test run, you can view the results to adjust the rule.
59
60
 
60
- ![](.topwrite/assets/image_1760004576132.png)
61
+ ^
61
62
 
62
63
  ### View Test Run Results
63
64
 
64
65
  After clicking "Test Run", follow the prompts to click "View Results" to see the test run validation results.
65
66
 
66
- ![](.topwrite/assets/image_1699875850066.png)
67
+ ^
67
68
 
68
69
  ### Configure Monitoring Alerts
69
70
 
70
71
  To ensure data quality issues are addressed promptly, you can configure monitoring alerts for quality rules. There are two ways:
71
72
 
72
73
  1. **Enable global quality monitoring alerts**: In the monitoring and alerting module, search for "Data Quality Check Failed" and enable the system's built-in global quality validation monitoring rule.
73
- ![](.topwrite/assets/image_1699875920709.png)
74
+
74
75
  2. **Configure custom quality monitoring alerts**: Create custom monitoring rules, select "Quality Rule Validation Failed" as the monitoring message, and set filter conditions such as workspace or specific validation object.
75
- ![](.topwrite/assets/image_1699875932007.png)
76
76
 
77
77
  ## Validation Objects
78
78
 
79
79
  On the validation objects page, you can manage all quality rules by the validation object (table) dimension. Use the search filter area to quickly locate specific rules.
80
80
 
81
- ![](.topwrite/assets/image_1699875683208.png)
81
+ ![](/.topwrite/assets/image_1780674038464.png)
82
82
 
83
83
  ## Validation Results
84
84
 
@@ -86,7 +86,7 @@ On the validation objects page, you can manage all quality rules by the validati
86
86
 
87
87
  On the validation results list page, you can view the run status of all quality rules. Use the search and filter area to precisely find the required validation results.
88
88
 
89
- ![](.topwrite/assets/image_1699875971234.png)
89
+ ^
90
90
 
91
91
  ### Operations on Validation Results
92
92
 
@@ -53,40 +53,32 @@ You can use `all tables in schema <schema_name>` and `all views in schema <schem
53
53
 
54
54
  **1. Create a share object**
55
55
 
56
- To ensure data security, share objects must be created by users with the "Instance Administrator" (instance_admin) role. Click "Data Management" - "Data Sharing" in the left menu to enter the data sharing list page.
56
+ To ensure data security, share objects must be created by users with the "Instance Administrator" (instance\_admin) role. Click "Data Management" - "Data Sharing" in the left menu to enter the data sharing list page.
57
57
 
58
58
  Click "+ New Share" to open the new data sharing window:
59
59
 
60
- :-: ![](.topwrite/assets/image_1734492957409.png =507)
60
+ 1. Fill in the "Share Name" for the data share.
61
61
 
62
- 1) Fill in the "Share Name" for the data share.
62
+ 2. Select "Workspace," which refers to the workspace containing the tables or views to be shared. A single data share object can only contain data from one workspace.
63
63
 
64
- 2) Select "Workspace," which refers to the workspace containing the tables or views to be shared. A single data share object can only contain data from one workspace.
64
+ 3. In the data object row, click "Add." Select the specific tables or views to include in the data share. Multiple selections are allowed. Note that tables and views originating from other shares cannot be selected, because data sharing authorization does not permit secondary sharing. Data shares created via the web interface only support including existing tables and views; specifying "future" tables or views under an entire schema is not yet supported.
65
65
 
66
- 3) In the data object row, click "Add." Select the specific tables or views to include in the data share. Multiple selections are allowed. Note that tables and views originating from other shares cannot be selected, because data sharing authorization does not permit secondary sharing. Data shares created via the web interface only support including existing tables and views; specifying "future" tables or views under an entire schema is not yet supported.
67
-
68
- 4) In the recipient instance row, click "Add." Enter the name of the service instance that needs to receive the data. Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
69
-
70
- :-: ![](.topwrite/assets/share_img_instancename.png =786)
66
+ 4. In the recipient instance row, click "Add." Enter the name of the service instance that needs to receive the data. Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
71
67
 
72
68
  ^
73
69
 
74
70
  *Note*:
75
71
 
76
- 1. *To ensure data security with workspace-level isolation, only users with the Workspace Administrator role can add data objects from their managed workspace to a data share.*
77
- 2. *In the create share dialog, "Create" and "Add Data" are two separate operations. Even if adding shared data fails partially or completely, it does not affect the creation of the share object. When any operation fails, an error message is returned. You can click on an already-created share in the data sharing list to continue adding data objects or recipient service instances.*
72
+ 1. *To ensure data security with workspace-level isolation, only users with the Workspace Administrator role can add data objects from their managed workspace to a data share*.
73
+ 2. *In the create share dialog, "Create" and "Add Data" are two separate operations. Even if adding shared data fails partially or completely, it does not affect the creation of the share object. When any operation fails, an error message is returned. You can click on an already-created share in the data sharing list to continue adding data objects or recipient service instances*.
78
74
 
79
75
  **2. Query created share objects**
80
76
 
81
- All share objects created within the current service instance are displayed on the "Data Sharing" list page. Users with the Instance Administrator role (instance_admin) or Workspace Administrator role (workspace_admin) can view the list of share objects.
82
-
83
- :-: ![](.topwrite/assets/image_1734493289109.png =737)
77
+ All share objects created within the current service instance are displayed on the "Data Sharing" list page. Users with the Instance Administrator role (instance\_admin) or Workspace Administrator role (workspace\_admin) can view the list of share objects.
84
78
 
85
79
  **3. View share details**
86
80
 
87
- On the "Data Sharing" list page, click a specific share name to view detailed information about the share, including basic information, recipient service instances, and shared data objects. Users with the Instance Administrator role (instance_admin) or Workspace Administrator role (workspace_admin) can view share object details.
88
-
89
- :-: ![](.topwrite/assets/image_1734493331354.png =735)
81
+ On the "Data Sharing" list page, click a specific share name to view detailed information about the share, including basic information, recipient service instances, and shared data objects. Users with the Instance Administrator role (instance\_admin) or Workspace Administrator role (workspace\_admin) can view share object details.
90
82
 
91
83
  **4. Add/Remove shared data objects**
92
84
 
@@ -96,19 +88,16 @@ On the data sharing list page, click the name of the share object to which you w
96
88
 
97
89
  Added data objects become immediately visible to share consumers and appear in their schema for extracting data.
98
90
 
99
- :-: ![](.topwrite/assets/image_1734493367553.png =721)
100
-
101
91
  **5. Configure share targets**
102
92
 
103
- You must configure the recipient service instance name (instance_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
93
+ You must configure the recipient service instance name (instance\_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
104
94
 
105
- :-: ![](.topwrite/assets/image_1734493417202.png =771)
106
95
 
107
96
  ^
108
97
 
109
98
  **6. Delete a share object**
110
99
 
111
- Users with the Instance Administrator role (instance_admin) or the Workspace Administrator role (workspace_admin) for the workspace to which the share belongs can delete share objects.
100
+ Users with the Instance Administrator role (instance\_admin) or the Workspace Administrator role (workspace\_admin) for the workspace to which the share belongs can delete share objects.
112
101
 
113
102
  On the data sharing list page or in the share details, click the "Delete" button under "Actions" to delete the share object. Once a share object is deleted, it cannot be recovered. Data consumers immediately lose access to the shared data.
114
103
 
@@ -116,7 +105,7 @@ On the data sharing list page or in the share details, click the "Delete" button
116
105
 
117
106
  **1. Create a share object**
118
107
 
119
- To ensure data security, share objects must be created by users with the "Instance Administrator" (instance_admin) role. When creating a share object, execute the following statement in the workspace where you plan to share data:
108
+ To ensure data security, share objects must be created by users with the "Instance Administrator" (instance\_admin) role. When creating a share object, execute the following statement in the workspace where you plan to share data:
120
109
 
121
110
  ```SQL
122
111
  CREATE SHARE <share_name>;
@@ -145,7 +134,7 @@ Grant select, read metadata on ALL tables in SCHEMA share_demo_schema TO SHARE s
145
134
  Grant select, read metadata on ALL views in SCHEMA share_demo_schema TO SHARE share_demo;
146
135
  ```
147
136
 
148
- When adding data objects to a share, the user executing the operation must have SELECT (query) and READ METADATA (view metadata) privileges on the involved tables or views, and must be able to grant them onward (with grant option). The Workspace Administrator role (workspace_admin) inherently has the above privileges.
137
+ When adding data objects to a share, the user executing the operation must have SELECT (query) and READ METADATA (view metadata) privileges on the involved tables or views, and must be able to grant them onward (with grant option). The Workspace Administrator role (workspace\_admin) inherently has the above privileges.
149
138
 
150
139
  To remove shared data objects from a share, use the REVOKE statement:
151
140
 
@@ -159,11 +148,10 @@ Note: When sharing a table or view, since the table or view always exists under
159
148
 
160
149
  **3. Configure share targets**
161
150
 
162
- You must configure the recipient service instance name (instance_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
151
+ You must configure the recipient service instance name (instance\_name) for the share object to complete data sharing. The service instance name needs to be provided by the data consumer.
163
152
 
164
153
  Service instance names are globally unique within the Lakehouse service. Consumer users can find their service instance name on the right side of the homepage or in the service instance URL and provide it to the sharer for configuration.
165
154
 
166
- :-: ![](.topwrite/assets/share_img_instancename.png =786)
167
155
 
168
156
  The syntax for the data provider to configure share targets is:
169
157
 
@@ -185,17 +173,13 @@ You can execute the SHOW command to query created share objects. The syntax is:
185
173
  SHOW SHARES;
186
174
  ```
187
175
 
188
- Example result:
189
-
190
- :-: ![](.topwrite/assets/share_img_showshares.png =803)
191
-
192
176
  Where:
193
177
 
194
178
  * provider is the tenant name of the share provider;
195
- * provider_instance is the service instance name of the share provider;
196
- * provider_workspace is the workspace to which the share belongs;
179
+ * provider\_instance is the service instance name of the share provider;
180
+ * provider\_workspace is the workspace to which the share belongs;
197
181
  * scope is the sharing scope of the share. Currently only PRIVATE is supported -- sharing to specified instances;
198
- * to_instance is the service instance name(s) to which the share is shared. Multiple instance names are separated by commas (,);
182
+ * to\_instance is the service instance name(s) to which the share is shared. Multiple instance names are separated by commas (,);
199
183
  * kind is the share type. OUTBOUND indicates data shared from the current service instance; INBOUND indicates data shared from other service instances to the current service instance.
200
184
 
201
185
  ^
@@ -208,9 +192,6 @@ Execute the following statement to query the data objects granted in a share:
208
192
  DESC SHARE <share_name>;
209
193
  ```
210
194
 
211
- Example result:
212
-
213
- :-: ![](.topwrite/assets/share_img_descshare.png =798)
214
195
 
215
196
  ^
216
197
 
@@ -230,11 +211,10 @@ Once a share object is deleted, it cannot be recovered. Data consumers immediate
230
211
 
231
212
  **1. View shared data**
232
213
 
233
- Users with the Instance Administrator (instance_admin) role or Workspace Administrator (workspace_admin) role can view all share objects shared to the current service instance in the "Shared With Me" tab under the "Data Sharing" menu.
214
+ Users with the Instance Administrator (instance\_admin) role or Workspace Administrator (workspace\_admin) role can view all share objects shared to the current service instance in the "Shared With Me" tab under the "Data Sharing" menu.
234
215
 
235
216
  Click a share object name to view details including the source, reception time, and the data (tables or views) currently contained in the share.
236
217
 
237
- :-: ![](.topwrite/assets/image_1734493496334.png =777)
238
218
 
239
219
  ^
240
220
 
@@ -242,11 +222,9 @@ Click a share object name to view details including the source, reception time,
242
222
 
243
223
  On the "Shared With Me" list page or the share details page, click the "Extract" button to extract data from the share.
244
224
 
245
- :-: ![](.topwrite/assets/image_1734493713534.png =793)
246
225
 
247
226
  When extracting data, extraction can only be done at the schema level. Individual table or view extraction is not yet supported.
248
227
 
249
- :-: ![](.topwrite/assets/image_1734493642759.png =541)
250
228
 
251
229
  First, select "Source Schema," which is the schema from which to extract data. If the share contains multiple schemas, you need to perform multiple extractions.
252
230
 
@@ -260,7 +238,6 @@ After completing data extraction, you can see all tables and views shared under
260
238
 
261
239
  Shared data is marked with a special icon in the "Data" panel for easy identification.
262
240
 
263
- :-: ![](.topwrite/assets/image_1734493884869.png =279)
264
241
 
265
242
  ### Using SQL
266
243
 
@@ -284,7 +261,7 @@ Where `<provider_instance>` and `<share_name>` can be obtained from the `SHOW SH
284
261
 
285
262
  The schema name in `CREATE SCHEMA <schema_name>` can be customized and does not need to match the schema name in the share.
286
263
 
287
- When executing the above command, the operator must have permission to create schemas in the target workspace. The Workspace Administrator role (workspace_admin) inherently has this permission.
264
+ When executing the above command, the operator must have permission to create schemas in the target workspace. The Workspace Administrator role (workspace\_admin) inherently has this permission.
288
265
 
289
266
  ^
290
267
 
@@ -296,15 +273,15 @@ After executing `CREATE SCHEMA FROM SHARE`, you can see all tables and views sha
296
273
 
297
274
  **1. Provider permissions**
298
275
 
299
- Only users with the Instance Administrator (instance_admin) role can create share objects. The permission points for share operations are as follows. Granting share object permissions to other roles or users via the GRANT statement is not currently supported.
276
+ Only users with the Instance Administrator (instance\_admin) role can create share objects. The permission points for share operations are as follows. Granting share object permissions to other roles or users via the GRANT statement is not currently supported.
300
277
 
301
- | **Permission** | **Description** |
302
- | --------------------------------------------------------------- | ----------------------------------------------------------------------------- |
303
- | Create share | Permission to create share objects. |
304
- | Alter share | Permission to modify share objects, add or remove shared instance IDs. |
305
- | Drop share | Permission to delete share instances. |
306
- | Read metadata | Permission for SHOW SHARES and DESC SHARE. DESC SHARE returns the objects and granted object privileges within the share. |
307
- | Grant objectprivilege to share / Revoke objectprivilege from share | Add or remove data objects from a share: GRANT objectPrivilege TO; REVOKE objectPrivilege FROM. |
278
+ | **Permission** | **Description** |
279
+ | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- |
280
+ | Create share | Permission to create share objects. |
281
+ | Alter share | Permission to modify share objects, add or remove shared instance IDs. |
282
+ | Drop share | Permission to delete share instances. |
283
+ | Read metadata | Permission for SHOW SHARES and DESC SHARE. DESC SHARE returns the objects and granted object privileges within the share. |
284
+ | Grant objectprivilege to share / Revoke objectprivilege from share | Add or remove data objects from a share: GRANT objectPrivilege TO; REVOKE objectPrivilege FROM. |
308
285
 
309
286
  ^
310
287
 
@@ -312,9 +289,9 @@ Only users with the Instance Administrator (instance_admin) role can create shar
312
289
 
313
290
  All members in a workspace have USE and READ METADATA permissions on share objects. However, the CREATE SCHEMA permission in the workspace is required to create a schema from the data in a share.
314
291
 
315
- | **Permission** | **Description** |
316
- | ------------- | --------------------------------------------- |
317
- | Use | Permission to use the share. |
318
- | Read metadata | Permission to query share metadata. After obtaining this, SHOW SHARES and DESC SHARE can be executed. |
292
+ | **Permission** | **Description** |
293
+ | -------------- | ----------------------------------------------------------------------------------------------------- |
294
+ | Use | Permission to use the share. |
295
+ | Read metadata | Permission to query share metadata. After obtaining this, SHOW SHARES and DESC SHARE can be executed. |
319
296
 
320
297
  ^
@@ -10,53 +10,53 @@ The data source types and versions currently supported by each task type are as
10
10
 
11
11
  ### Offline Sync
12
12
 
13
- | Data Source | Release Level | Offline Single-Table Read | Offline Single-Table Write | Supported Versions |
14
- | --------------------- | ------- | ---- | ---- | --------------------- |
15
- | Singdata Lakehouse | GA | Y | Y | - |
16
- | ADB MySQL | GA (Beta) | Y | Y | 3.x |
17
- | ADB PostgreSQL | GA (Beta) | Y | Y | 6.x |
18
- | AMQP | GA (Beta) | Y | N | - |
19
- | AutoMQ | GA | Y | Y | - |
20
- | ClickHouse | GA | Y | Y | 19.x and above |
21
- | COS | GA | Y | Y | - |
22
- | Databricks | GA | Y | Y | - |
23
- | DB2 | GA (Beta) | Y | Y | 11.x |
24
- | DM | GA (Beta) | Y | Y | 8.x |
25
- | Doris | GA (Beta) | Y | Y | 1.x |
26
- | DynamoDB | GA | Y | Y | - |
27
- | Elasticsearch | GA | Y | Y | 7.x |
28
- | Greenplum | GA (Beta) | Y | Y | 5.x and above |
29
- | HBase | GA | N | Y | 1.4+ |
30
- | Hive | GA | Y | Y | 2.x |
31
- | Hologres | GA (Beta) | Y | Y | - |
32
- | Kafka | GA | Y | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
33
- | MaxCompute | GA | Y | N | - |
34
- | MariaDB | GA (Beta) | Y | Y | - |
35
- | MongoDB | GA | Y | N | 3.4 and above |
36
- | MySQL | GA | Y | Y | 5.x, 8.x |
37
- | Oracle | GA (Beta) | Y | Y | - |
38
- | OSS | GA | Y | Y | - |
39
- | PolarDB for MySQL | GA | Y | Y | - |
40
- | PolarDB for PostgreSQL | GA | Y | Y | - |
41
- | PostgreSQL | GA | Y | Y | 9.4 and above |
42
- | Redis | GA | N | Y | - |
43
- | Redshift | GA (Beta) | Y | Y | - |
44
- | RestApi | GA | Y | N | - |
45
- | SLS (LogHub) | GA | Y | N | - |
46
- | SQL Server | GA | Y | Y | 2012 and above |
47
- | StarRocks | GA (Beta) | Y | Y | 2.5 |
48
- | TiDB | GA (Beta) | Y | Y | - |
13
+ | Data Source | Release Level | Offline Single-Table Read | Offline Single-Table Write | Supported Versions |
14
+ | ---------------------- | ------------- | ------------------------- | -------------------------- | ------------------------- |
15
+ | Singdata Lakehouse | GA | Y | Y | - |
16
+ | ADB MySQL | GA (Beta) | Y | Y | 3.x |
17
+ | ADB PostgreSQL | GA (Beta) | Y | Y | 6.x |
18
+ | AMQP | GA (Beta) | Y | N | - |
19
+ | AutoMQ | GA | Y | Y | - |
20
+ | ClickHouse | GA | Y | Y | 19.x and above |
21
+ | COS | GA | Y | Y | - |
22
+ | Databricks | GA | Y | Y | - |
23
+ | DB2 | GA (Beta) | Y | Y | 11.x |
24
+ | DM | GA (Beta) | Y | Y | 8.x |
25
+ | Doris | GA (Beta) | Y | Y | 1.x |
26
+ | DynamoDB | GA | Y | Y | - |
27
+ | Elasticsearch | GA | Y | Y | 7.x |
28
+ | Greenplum | GA (Beta) | Y | Y | 5.x and above |
29
+ | HBase | GA | N | Y | 1.4+ |
30
+ | Hive | GA | Y | Y | 2.x |
31
+ | Hologres | GA (Beta) | Y | Y | - |
32
+ | Kafka | GA | Y | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
33
+ | MaxCompute | GA | Y | N | - |
34
+ | MariaDB | GA (Beta) | Y | Y | - |
35
+ | MongoDB | GA | Y | N | 3.4 and above |
36
+ | MySQL | GA | Y | Y | 5.x, 8.x |
37
+ | Oracle | GA (Beta) | Y | Y | - |
38
+ | OSS | GA | Y | Y | - |
39
+ | PolarDB for MySQL | GA | Y | Y | - |
40
+ | PolarDB for PostgreSQL | GA | Y | Y | - |
41
+ | PostgreSQL | GA | Y | Y | 9.4 and above |
42
+ | Redis | GA | N | Y | - |
43
+ | Redshift | GA (Beta) | Y | Y | - |
44
+ | RestApi | GA | Y | N | - |
45
+ | SLS (LogHub) | GA | Y | N | - |
46
+ | SQL Server | GA | Y | Y | 2012 and above |
47
+ | StarRocks | GA (Beta) | Y | Y | 2.5 |
48
+ | TiDB | GA (Beta) | Y | Y | - |
49
49
 
50
50
  ### Real-Time Sync
51
51
 
52
- | Data Source | Release Level | Real-Time Single-Table Read | Real-Time Multi-Table Read | Real-Time Write | Supported Versions |
53
- | --------------- | --- | ---- | ---- | ---- | ---- |
54
- | Singdata Lakehouse | GA | N | N | Y | - |
55
- | AutoMQ | GA | Y | N | N | - |
56
- | Kafka | GA | Y | N | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
57
- | MySQL | GA | Y | Y | N | 5.x, 8.x |
58
- | PolarDB for MySQL | GA | Y | Y | N | - |
59
- | PolarDB for PostgreSQL | GA | Y | Y | N | - |
60
- | SQL Server | GA | Y | Y | N | SQL Server 2016 Service Pack 1 (SP1) and above |
52
+ | Data Source | Release Level | Real-Time Single-Table Read | Real-Time Multi-Table Read | Real-Time Write | Supported Versions |
53
+ | ---------------------- | ------------- | --------------------------- | -------------------------- | --------------- | ---------------------------------------------- |
54
+ | Singdata Lakehouse | GA | N | N | Y | - |
55
+ | AutoMQ | GA | Y | N | N | - |
56
+ | Kafka | GA | Y | N | Y | 0.8.x, 0.9.x, 0.10.x, 2.x |
57
+ | MySQL | GA | Y | Y | N | 5.x, 8.x |
58
+ | PolarDB for MySQL | GA | Y | Y | N | - |
59
+ | PolarDB for PostgreSQL | GA | Y | Y | N | - |
60
+ | SQL Server | GA | Y | Y | N | SQL Server 2016 Service Pack 1 (SP1) and above |
61
61
 
62
62
  ^
@@ -4,7 +4,7 @@
4
4
 
5
5
  Data Catalog, also known as the data asset map, provides powerful data search and display capabilities, designed to help users more easily find, understand, and use data. It supports two modes: Data Search mode and Data Management mode. Data Search mode allows users to search data directly, while Data Management mode allows users to browse and manage data tables through a directory structure. With Data Catalog, you can manage and utilize data resources more efficiently.
6
6
 
7
- ![](.topwrite/assets/image_1735621816126.png)
7
+ ![](/.topwrite/assets/image_1780673609913.png)
8
8
 
9
9
  * Asset Details: Displays the total data assets of the current tenant.
10
10
  * Search Box: Click to enter the data search results page.
@@ -16,11 +16,11 @@ Data Catalog, also known as the data asset map, provides powerful data search an
16
16
 
17
17
  Enter keywords in the search box on the Data Catalog homepage to enter the search results page. The search function supports searching by the name and description of object types (Table, View, Materialized View). The search results page provides various filtering options, including object type, workspace/Schema, creation time, owner, etc., as well as sorting options by name, creation time, update time, etc.
18
18
 
19
- ![](.topwrite/assets/image_1735638515501.png)
19
+ ![](/.topwrite/assets/image_1780673622966.png)
20
20
 
21
21
  On the search results filtering: supports filtering by object type, workspace/schema, creation time, and owner; also supports filtering by name, creation time, and update time.
22
22
 
23
- ![](.topwrite/assets/image_1735638559254.png)
23
+ ![](/.topwrite/assets/image_1780673651359.png)
24
24
 
25
25
  Click the table name on the search results page to navigate to the corresponding table details page.
26
26
 
@@ -28,55 +28,53 @@ Click the table name on the search results page to navigate to the corresponding
28
28
 
29
29
  The Data Management page displays all data assets that the current user has permission to manage under the service instance in a directory format. Click **Data Management** from the homepage to enter the Data Management page, which displays the list of workspaces that the current user has permissions to under the service.
30
30
 
31
- ![](.topwrite/assets/536c830f76/6d452f0b5d750f3a42ac150d3d2f2a46a1b06216.png)
31
+ ![](/.topwrite/assets/image_1780673677696.png)
32
+
33
+ ^
32
34
 
33
35
  ### Workspace Level
34
36
 
35
37
  At the workspace level, you can see the list of all Schemas under that workspace. Click the directory tree on the left to expand and view the hierarchy. The top area displays basic workspace information, such as source, creation time/modification time, owner, etc. The information area presents specific Schema information, including name, creation time, update time, owner, etc.
36
38
 
37
- Each workspace has a default information_schema.
38
-
39
- Schemas under a workspace include two categories: internally created and externally created.
39
+ Each workspace has a default information\_schema.
40
40
 
41
- ![](.topwrite/assets/536c830f76/13c0c5b0de5cdb544d9dfb30208ecd7897706983.png)
41
+ Schemas under a workspace include two categories: internally created and externally created.![](/.topwrite/assets/image_1780673705918.png)
42
42
 
43
43
  Users can create schema information under a workspace through a visual interface. Click the question mark next to the title to view help documentation for creating objects.
44
44
 
45
- ![](.topwrite/assets/536c830f76/ca874a7a019814f4b59811207177035046a8f740.png =297)
45
+ ![](/.topwrite/assets/image_1780673717394.png =372)
46
46
 
47
47
  ### Schema Level
48
48
 
49
- After clicking a specific Schema, you can drill down to the Schema's details page. At the Schema level, you can view all object information managed under the Schema, with specific objects organized by object type. The system currently supports three object types: Table, View, and Materialized View.
50
-
51
- ![Schema Level](.topwrite/assets/536c830f76/1c37e9e34e320d58f7d975ad833c5686162e1159.png)
49
+ After clicking a specific Schema, you can drill down to the Schema's details page. At the Schema level, you can view all object information managed under the Schema, with specific objects organized by object type. The system currently supports three object types: Table, View, and Materialized View.![](/.topwrite/assets/image_1780673754518.png =728)
52
50
 
53
51
  Objects can be created directly under a Schema, supporting both script-based creation and visual creation. The system has built-in templates for different objects to help users better understand Lakehouse syntax.
54
52
 
55
- ![Create Object Template](.topwrite/assets/536c830f76/79ef0ae4339fac60c009a9508b495ac8f3cbd090.png =529)
53
+ ![](/.topwrite/assets/image_1780673779657.png =646)
56
54
 
57
55
  ## Table Details Page
58
56
 
59
- The table details page displays all detailed information for tables in the Singdata Lakehouse library, including field information, data preview, data lineage, job history, uploads, etc. The details page is divided into two parts: the top area displays the table name, owner, and some quick table operations; the information area contains six major sections: Details, Fields, Preview, Lineage, Jobs, and Uploads.
60
-
61
- ![](.topwrite/assets/536c830f76/441e6d94ae7efacb852c089f4a4aa0871cc04c3f.png)
57
+ The table details page displays all detailed information for tables in the Singdata Lakehouse library, including field information, data preview, data lineage, job history, uploads, etc. The details page is divided into two parts: the top area displays the table name, owner, and some quick table operations; the information area contains six major sections: Details, Fields, Preview, Lineage, Jobs, and Uploads.![](/.topwrite/assets/image_1780673802254.png)
62
58
 
63
59
  ### Top Functional Area
64
60
 
65
61
  The top area displays the full structure of the current table (i.e., the corresponding workspace/schema/table name information), row count/storage size, creation time/update time, owner, and other basic information. The top area also provides an **Upload** function, supporting the uploading of local files to the Singdata Lakehouse platform.
66
62
 
67
- ![Top Functional Area](.topwrite/assets/536c830f76/67ea57ab34dd3b76f3ab553faf2a258f582864bc.png)
63
+ ^
64
+
65
+ ![](/.topwrite/assets/image_1780673850037.png =695)
68
66
 
69
- ### Details
67
+ DDL preview:
70
68
 
71
69
  The Details tab displays the DDL statement of the current table, supporting one-click copy. It also supports one-click navigation to the permission management page for permission authorization.
72
70
 
73
- ![](.topwrite/assets/536c830f76/f081a3d6f3b91bec4de88b3ee67a10bd42003355.png)
71
+ ^
74
72
 
75
73
  ### Fields
76
74
 
77
75
  The **Fields** tab displays the field names, types, descriptions, and other information for the current table. If there is primary key information, it can also display the primary key and standardized labels for the fields.
78
76
 
79
- ![](.topwrite/assets/536c830f76/47b33b650c6c77e2de4ff2df037836580e4be390.png)
77
+ ![](/.topwrite/assets/image_1780673897761.png =652)
80
78
 
81
79
  ### Preview
82
80
 
@@ -84,26 +82,26 @@ In the Data Preview tab, you can preview the first 100 rows of data in the curre
84
82
 
85
83
  Data preview requires the user to manually select a compute cluster participating in the query to display the preview data.
86
84
 
87
- ![](.topwrite/assets/536c830f76/c5efa313984239360d57373d1912d37f030c7345.png)
85
+ ^
88
86
 
89
87
  ### Lineage
90
88
 
91
89
  In the Data Lineage tab, you can view the upstream and downstream dependencies of the current table. The lineage relationships here are generated based on code parsing of the corresponding task that produces this table, and belong to relationships from the "definition perspective."
92
90
 
93
- ![](.topwrite/assets/536c830f76/3e8ffc557a0d11d362aef2a576257b5b023f4e78.png)
91
+ ^
94
92
 
95
93
  Click on a specific upstream or downstream table node to view more detailed information:
96
94
 
97
- ![](.topwrite/assets/536c830f76/0820ab62e0868ea1f873defc635eb4577142a084.png)
95
+ ^
98
96
 
99
97
  ### Jobs
100
98
 
101
99
  Under the Jobs tab, you can view the SQL jobs of the tasks that produce the table. By clicking on the job ID, you can further view the job details.
102
100
 
103
- ![](.topwrite/assets/536c830f76/b18ee57b085317472a1abfadb1777bda2fab3c36.png)
101
+ ^
104
102
 
105
103
  ### Uploads
106
104
 
107
105
  Under the Uploads tab, all historical records uploaded to the current table via the **Upload** function are displayed.
108
106
 
109
- ![](.topwrite/assets/536c830f76/89ad928925aedfc753ac5eca1b07411286234857.png)
107
+ ^
@@ -13,11 +13,14 @@ When the system processes your data queries, the following data will be sent to
13
13
 
14
14
  **Data**:
15
15
 
16
- * Column value data with indexing enabled (You can prevent specific column data from being sent to the large language model by disabling column value indexing) ![](.topwrite/assets/20250219-130449.jpeg)
16
+ * Column value data with indexing enabled (You can prevent specific column data from being sent to the large language model by disabling column value indexing)
17
+
18
+ ![](.topwrite/assets/20250115-094842.jpeg)
17
19
 
18
20
  **Document Segmentation**:
19
21
 
20
- The system segments the document into vectorized chunks through semantic parsing. These vectorized segments are then processed by the Large Language Model (LLM) to enable intelligent question-answering capabilities.
22
+ * The system segments document content into chunks
23
+ * These document segments are sent to the Large Language Model to enable intelligent question-answering
21
24
 
22
25
  ## Data Protection Measures
23
26