@clickzetta/cz-cli-darwin-arm64 0.5.16 → 0.5.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (243) hide show
  1. package/bin/cz-cli +0 -0
  2. package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
  3. package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
  4. package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
  5. package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
  6. package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
  7. package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
  8. package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
  9. package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
  10. package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
  11. package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
  12. package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
  13. package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
  14. package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
  15. package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
  16. package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
  17. package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
  18. package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
  19. package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
  20. package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
  21. package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
  22. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
  23. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
  24. package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
  25. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
  26. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
  27. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
  28. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
  29. package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
  30. package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
  31. package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
  32. package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
  33. package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
  34. package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
  35. package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
  36. package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
  37. package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
  38. package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
  39. package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
  40. package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
  41. package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
  42. package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
  43. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
  44. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
  45. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
  46. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
  47. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
  48. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
  49. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
  50. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
  51. package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
  52. package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
  53. package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
  54. package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
  55. package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
  56. package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
  57. package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
  58. package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
  59. package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
  60. package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
  61. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
  62. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
  63. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
  64. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
  65. package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
  66. package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
  67. package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
  68. package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
  69. package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
  70. package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
  71. package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
  72. package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
  73. package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
  74. package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
  75. package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
  76. package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
  77. package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
  78. package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
  79. package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
  80. package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
  81. package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
  82. package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
  83. package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
  84. package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
  85. package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
  86. package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
  87. package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
  88. package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
  89. package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
  90. package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
  91. package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
  92. package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
  93. package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
  94. package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
  95. package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
  96. package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
  97. package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
  98. package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
  99. package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
  100. package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
  101. package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
  102. package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
  103. package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
  104. package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
  105. package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
  106. package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
  107. package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
  108. package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
  109. package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
  110. package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
  111. package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
  112. package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
  113. package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
  114. package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
  115. package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
  116. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
  117. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
  118. package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
  119. package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
  120. package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
  121. package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
  122. package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
  123. package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
  124. package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
  125. package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
  126. package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
  127. package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
  128. package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
  129. package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
  130. package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
  131. package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
  132. package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
  133. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
  134. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
  135. package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
  136. package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
  137. package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
  138. package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
  139. package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
  140. package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
  141. package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
  142. package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
  143. package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
  144. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
  145. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
  146. package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
  147. package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
  148. package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
  149. package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
  150. package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
  151. package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
  152. package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
  153. package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
  154. package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
  155. package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
  156. package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
  157. package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
  158. package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
  159. package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
  160. package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
  161. package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
  162. package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
  163. package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
  164. package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
  165. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
  166. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
  167. package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
  168. package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
  169. package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
  170. package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
  171. package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
  172. package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
  173. package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
  174. package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
  175. package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
  176. package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
  177. package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
  178. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
  179. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
  180. package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
  181. package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
  182. package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
  183. package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
  184. package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
  185. package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
  186. package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
  187. package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
  188. package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
  189. package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
  190. package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
  191. package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
  192. package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
  193. package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
  194. package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
  195. package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
  196. package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
  197. package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
  198. package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
  199. package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
  200. package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
  201. package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
  202. package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
  203. package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
  204. package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
  205. package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
  206. package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
  207. package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
  208. package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
  209. package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
  210. package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
  211. package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
  212. package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
  213. package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
  214. package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
  215. package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
  216. package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
  217. package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
  218. package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
  219. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
  220. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
  221. package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
  222. package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
  223. package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
  224. package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
  225. package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
  226. package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
  227. package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
  228. package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
  229. package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
  230. package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
  231. package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
  232. package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
  233. package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
  234. package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
  235. package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
  236. package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
  237. package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
  238. package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
  239. package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
  240. package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
  241. package/package.json +1 -1
  242. package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
  243. package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
@@ -23,7 +23,7 @@ pip install -e ".[dev]"
23
23
  ## Dependency Requirements
24
24
 
25
25
  ### Python Version
26
- - Python 3.9 or higher
26
+ - Python 3.10 or higher
27
27
 
28
28
  ### Core Dependencies
29
29
  The following dependencies are installed automatically with the package:
@@ -10,29 +10,28 @@ In Singdata Lakehouse, every account has a unique identifier used to distinguish
10
10
 
11
11
  The account name is your unique identifier within Singdata Lakehouse and cannot be changed once generated. You can find your account name in the prompt shown after registration or on the Management Center home page.
12
12
 
13
- :-: ![](.topwrite/assets/image\_1739329828430.png =780)
13
+
14
14
 
15
15
  ^
16
16
 
17
- :-: ![](.topwrite/assets/image\_1739329908616.png =783)
18
17
 
19
18
  ### Login URL
20
19
 
21
20
  In Singdata products, each account has its own dedicated login URL. The login URL follows the format: `<your_account_name>.accounts.singdata.com`.
22
21
 
23
- :-: ![](.topwrite/assets/image\_1739329970639.png =799)
22
+
24
23
 
25
24
  ### Service Instance Name
26
25
 
27
26
  Singdata Lakehouse is a multi-cloud big data platform that supports creating service instances across different regions of multiple cloud providers. Each service instance provisioned generates a unique identifier within that region and carves out a logical area within the infrastructure resources to provide Lakehouse services. The service instance name (`instance_name`) is the unique identifier for a service instance. It is automatically generated when you provision the instance and consists of an 8-character random string. You can find your service instance name in the service instance URL or on the right side of the service instance home page.
28
27
 
29
- :-: ![](.topwrite/assets/image\_1740540520113.png =807)
28
+
30
29
 
31
30
  ### Account Registration Phone Number
32
31
 
33
32
  The account registration phone number is the phone number you provided when creating the account and adding the first user. The initial account phone number is the same as the phone number of the first user under the account. Account administrators can change the registration phone number in the Management Center. Keep your registration phone number safe — in Singdata products, each registration phone number corresponds to only one account, and it is the sole credential for account recovery.
34
33
 
35
- :-: ![](.topwrite/assets/image\_1739330026163.png =806)
34
+
36
35
 
37
36
  ^ ^
38
37
 
@@ -127,7 +126,6 @@ Example:
127
126
 
128
127
  In some SQL clients (such as DBeaver), JDBC drivers, or Python/Java library configuration files, you need to configure connection parameters. For example, when configuring the DBeaver driver:
129
128
 
130
- :-: ![](.topwrite/assets/image\_1740549280083.png =361)
131
129
 
132
130
  In the JDBC connection string, use the service instance name (`instance_name`), for example:
133
131
 
@@ -161,16 +159,13 @@ When configuring the Python SDK, use the service instance name (`instance_name`)
161
159
 
162
160
  When you log in to Singdata with a user that has the account administrator role, you can manage your account in the Management Center. On the "Account Home" page, you can view basic account information and the account login URL. You can also click the "Modify" button next to the registration phone number to change it. Note that on the "Account Center" page you will see two separate phone number fields: "Registration Phone" under "Account Info" — this is the phone number bound to the account and can be used to retrieve the account name; and "Registration Phone" under "User Info" — this is the phone number bound to the currently logged-in user and can be used for MFA verification, username recovery, and password recovery for that user. Be sure to distinguish between the two.
163
161
 
164
- ![](.topwrite/assets/image\_1740550879753.png)
165
162
 
166
163
 
167
164
  ### Changing the Account Registration Phone Number
168
165
 
169
166
  To change the account registration phone number, you must be logged in as a user with the account administrator (`account_admin`) role. In the "Account Center", click the modify button next to "Registration Phone" under "Account Info". After verifying the old phone number, you can enter and save the new phone number.
170
167
 
171
- :-: ![](.topwrite/assets/image\_1740551806201.png =765)
172
168
 
173
- :-: ![](.topwrite/assets/image\_1740551835302.png =438)
174
169
 
175
170
  ### Changing the Account Name or Service Instance Name
176
171
 
@@ -716,5 +716,6 @@ With 20/22 validations passing (the 2 warnings are source data quality issues, n
716
716
  - Spark SQL syntax migration: [Spark SQL Syntax Migration Guide](migration-spark-sql.md)
717
717
  - Volume usage guide: [Volume Usage Guide](zettapark-volume-guide.md)
718
718
  - Data type compatibility: [Data Type Compatibility Reference](migration-sql-compatibility.md)
719
- - Custom functions: [SQL Function](create-sql-function.md) · [External Function](om-external-function.md)
719
+ - Custom functions: [SQL Function](create-sql-function.md) · [External Function](create_external_function.md)
720
+ - Spark Connector: [Spark Connector Usage](spark-connector-use.md)
720
721
  - Spark Connector: [Using Spark Connector](spark-connector-use.md)
@@ -1,45 +1,82 @@
1
- # Metrics and Answer Builder Configuration
1
+ # Metrics and Answer Builder
2
2
 
3
- ## Constructing Metrics:&#x20;
3
+ When AI answers questions involving calculation logic, without pre-defined formulas, the model may generate different SQL each time, resulting in inconsistent metric definitions. Metrics and Answer Builder solve exactly this problem — they lock in the calculation logic so the model can reference it directly instead of inferring it on the fly.
4
4
 
5
- In **Left Navigation Bar: Data** -> **Metrics**, you can click "+ **New Metric**" to add new metrics. Metrics can be generated using aggregation or custom code methods.
5
+ **When to use Metrics vs Answer Builder**?
6
+
7
+ | Scenario | Recommended | Reason |
8
+ | ------------------------------------------------------------- | -------------- | ----------------------------------------------------------------------- |
9
+ | Single-table aggregation, e.g., SUM, AVG, COUNT | Metrics | Simple to configure; supports aliases and period-over-period comparison |
10
+ | Requires multi-table JOIN | Answer Builder | Metrics do not support multiple tables; a full SQL template is needed |
11
+ | Has complex filter conditions (e.g., count only valid orders) | Answer Builder | Filter logic goes in the WHERE clause of the SQL template |
12
+ | Detail query (non-aggregate, returns row-level data) | Answer Builder | Metrics only support aggregate calculations |
13
+
14
+ ***
15
+
16
+ ## Creating Metrics
17
+
18
+ In the left navigation bar, go to **Data** -> **Metrics** and click + **New Metric**. Both aggregation and custom code methods are supported.
6
19
 
7
20
  ![](.topwrite/assets/b.jpg =558)
8
21
 
9
22
  ^
10
23
 
11
- ## Creating Answer Builder:&#x20;
24
+ A single metric template can define multiple metrics, and each metric can have multiple aliases, allowing the model to understand different names for the same concept (e.g., "average order value" and "spend per customer").
25
+
26
+ > ⚠️ **Note**: In the Default Domain, duplicate metric names and aliases are allowed. In other user-created Domains, metric names (including aliases) must be unique.
27
+
28
+ ### Analysis Method
12
29
 
13
- The Answer Builder is used to define metrics with complex calculation logic and to provide SQL templates for detailed queries. In **Left Navigation Bar: Data** -> **Answer Builder**, you can click "+ **New Answer Builder**" and follow the steps to fill in the code.
30
+ Each metric must specify an analysis method, which affects the period-over-period calculation logic:
14
31
 
15
- * SQL Template for Answer Builder: `${dims}` and `${filters}` are fixed syntax representing dimension and filter condition variables. The selectable columns for these variables are defined in the **Filters** and **Dims** sections below the template.
32
+ | Analysis Method | Period-over-Period Algorithm | Applicable Scenario |
33
+ | ------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------- |
34
+ | Additive metric | (Current period value - Previous period value) / \|Previous period value\| | Absolute numeric metrics, e.g., revenue, order count |
35
+ | Proportional metric | Current period value - Previous period value | Percentage metrics, e.g., market share, conversion rate |
16
36
 
17
- ```SQL
37
+ Example: If market share is 30% this year and 20% last year, the proportional method gives a year-over-year result of +10% (not +50%).
38
+
39
+ ***
40
+
41
+ ## Creating an Answer Builder
42
+
43
+ In the left navigation bar, go to **Data** -> **Answer Builder** and click + **New Answer Builder**.
44
+
45
+ The core of an Answer Builder is a SQL template. `${dims}` and `${filters}` are fixed placeholders representing dimensions and filter conditions respectively; they are dynamically filled in by the AI when answering questions.
46
+
47
+ ```sql
18
48
  SELECT
19
49
  ${dims},
20
50
  sum(op.payment_value) as total_sales_bz,
21
51
  avg(op.payment_value) as avg_sales_bz
22
52
  FROM datagpt_ws.public.v_gpt_orders AS o
23
53
  LEFT JOIN datagpt_ws.public.v_gpt_order_items AS oi
24
- ON o.order_id = oi.order_id LEFT JOIN datagpt_ws.public.v_gpt_products AS p
25
- ON oi.product_id = p.product_id LEFT JOIN datagpt_ws.public.v_gpt_customers AS c
26
- ON o.customer_id = c.customer_id LEFT JOIN datagpt_ws.public.v_gpt_payments AS op
27
- ON o.order_id = op.order_id LEFT JOIN datagpt_ws.public.v_gpt_sellers AS os ON oi.seller_id = os.seller_id
28
- where ${filters}
29
- group by ${dims}
54
+ ON o.order_id = oi.order_id
55
+ LEFT JOIN datagpt_ws.public.v_gpt_products AS p
56
+ ON oi.product_id = p.product_id
57
+ LEFT JOIN datagpt_ws.public.v_gpt_customers AS c
58
+ ON o.customer_id = c.customer_id
59
+ LEFT JOIN datagpt_ws.public.v_gpt_payments AS op
60
+ ON o.order_id = op.order_id
61
+ LEFT JOIN datagpt_ws.public.v_gpt_sellers AS os
62
+ ON oi.seller_id = os.seller_id
63
+ WHERE ${filters}
64
+ GROUP BY ${dims}
30
65
  ```
31
66
 
32
- * **Filters and Dims Variable Definition**: Note: Columns not appearing in Filter and Dims here will not be used as filter conditions and dimensions for the issue. (This is related to the **Usage** field configuration of the table columns in **Data** -> **Tables**: columns configured as FILTER and DIM in the **Usage** field will be selected by default here, and the final configuration will be based on this page)![](.topwrite/assets/20250219-115158.jpeg =736)
67
+ ### Configuring Filters and Dims
33
68
 
34
- ### Configure Metric Aliases and Analysis Methods:
69
+ Below the SQL template, you need to define which columns can be used as filter conditions (Filters) and dimensions (Dims). Columns not selected here will not be used as filter conditions or dimensions for questions.
35
70
 
36
- In a metric template, multiple metrics can be defined. Each metric can specify multiple aliases.
71
+ :-: ![](/.topwrite/assets/image_1780906414361.png =686)
37
72
 
38
- > ⚠️ **Note**: In the system Default Domain, duplicate metrics and aliases are allowed; in other user-created Domains, duplicate metric names (including aliases) are not allowed.
73
+ > 💡 **Tip**: In the column configuration under **Data** -> **Tables**, setting a column's **Usage** field to FILTER or DIM will cause it to be selected by default here. The final configuration is governed by the settings on the Answer Builder page.
39
74
 
40
- ### **Analysis Methods**:
75
+ ## Related Documentation
41
76
 
42
- * **Additive Metrics**: The algorithm for period-over-period comparison is: (current period value - previous period value) / absolute value of previous period value.
43
- * **Proportional Metrics**: The algorithm for period-over-period comparison is: current period value - previous period value. For example, if this year's market share is 30% and last year's was 20%, then the year-over-year growth is 10%.
77
+ * [Answer Accuracy Improvement](answer-accuracy-improve.md) The role of Metrics and Answer Builder in the overall accuracy improvement strategy
78
+ * [Data Source Management](datagpt_data_source.md) Configure the data table sources that metrics depend on
79
+ * [Row-Level Permissions](row_level_permission.md) — Control the data range accessible to different users
80
+ * [Conversational Data Analytics (Analytics Agent)](datagpt_introduction.md) — Return to feature overview
44
81
 
45
- ![](.topwrite/assets/20250219-115256.jpeg)
82
+ ^
@@ -188,7 +188,7 @@ Set the value for each parameter:
188
188
 
189
189
  :-: ![](.topwrite/assets/image_1737534633567.png =562)
190
190
 
191
- For how to obtain parameter values, please refer to [this article](https://uat-doc.singdata.com/JDBC-Driver).
191
+ For how to obtain parameter values, please refer to [this article](jdbc-driver.md).
192
192
 
193
193
  ^
194
194
 
@@ -1,4 +1,4 @@
1
- # Introduction to MindsDB
1
+ # Connecting MindsDB to Singdata Lakehouse
2
2
 
3
3
  > ⚠️ **Note:** If your goal with MindsDB is to call AI models on your data, Singdata Lakehouse provides native AI capabilities — no need to deploy or maintain a MindsDB service:
4
4
  >
@@ -6,55 +6,57 @@ The monitoring and alerting system is a powerful feature set that allows users t
6
6
 
7
7
  ## Core Concepts
8
8
 
9
- | Concept | Description |
10
- | ---- | ----------------------------------------------------------------------------- |
11
- | Monitoring Rule | A set of configuration information containing key attributes such as specific objects and message types, informing the system which key messages to watch for. Rules configured for specific objects and specific conditions generate alert events when conditions are met. |
12
- | Alert Event | Based on monitoring rules, a potential event record that needs to trigger an alert notification when monitoring conditions are satisfied. |
13
- | Notification History | The actual notification records pushed to users after alert events are generated. Alert notification delivery is influenced by the following three strategies. |
14
- | Notification Strategy | Defines which notification channels to use, sending frequency, etc., when pushing alerts to recipients. |
9
+ | Concept | Description |
10
+ | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
11
+ | Monitoring Rule | A set of configuration information containing key attributes such as specific objects and message types, informing the system which key messages to watch for. Rules configured for specific objects and specific conditions generate alert events when conditions are met. |
12
+ | Alert Event | Based on monitoring rules, a potential event record that needs to trigger an alert notification when monitoring conditions are satisfied. |
13
+ | Notification History | The actual notification records pushed to users after alert events are generated. Alert notification delivery is influenced by the following three strategies. |
14
+ | Notification Strategy | Defines which notification channels to use, sending frequency, etc., when pushing alerts to recipients. |
15
15
 
16
16
  ## Monitoring Rules
17
17
 
18
18
  The monitoring rules list displays all currently configured rules. Users can filter to quickly find the needed rules.
19
19
 
20
- ![Monitoring Rules List](.topwrite/assets/226957dd01/8d11a86d9d484c752427e010555ea2d1efbf0b50.png)
20
+ ![](/.topwrite/assets/image_1780674124352.png =715)
21
21
 
22
22
  For individual rules, the following operations are available:
23
23
 
24
- | Operation Name | Behavior Definition | Authorized Personnel |
25
- | ------- | ------------------------ | ------------ |
26
- | View Details | Open the monitoring alert rule details page to view complete information. | Open to all instance members |
27
- | Enable/Disable | Enable or disable the alert rule. | Instance administrator, instance operations role |
28
- | Copy | Copy the configuration attributes of the current rule to create a new rule. | Instance administrator, instance operations role |
29
- | Edit | Modify the attributes of the monitoring rule through appropriate interactive methods. | Instance administrator, instance operations role |
30
- | Subscribe/Unsubscribe | Add or remove the operator from the alert recipient list. | Open to all instance members |
24
+ | Operation Name | Behavior Definition | Authorized Personnel |
25
+ | --------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------ |
26
+ | View Details | Open the monitoring alert rule details page to view complete information. | Open to all instance members |
27
+ | Enable/Disable | Enable or disable the alert rule. | Instance administrator, instance operations role |
28
+ | Copy | Copy the configuration attributes of the current rule to create a new rule. | Instance administrator, instance operations role |
29
+ | Edit | Modify the attributes of the monitoring rule through appropriate interactive methods. | Instance administrator, instance operations role |
30
+ | Subscribe/Unsubscribe | Add or remove the operator from the alert recipient list. | Open to all instance members |
31
31
 
32
32
  ### System Built-in Rules
33
33
 
34
34
  The system provides some preset global monitoring rules that users can enable as needed.
35
35
 
36
- | Rule Name | Rule Function | Default Status |
37
- | ---------- | ------------- | ------ |
38
- | General Rule - Monitor Task Failure | Default rule for monitoring task instance failures. | Disabled by default |
36
+ | Rule Name | Rule Function | Default Status |
37
+ | ----------------------------------- | --------------------------------------------------- | ------------------- |
38
+ | General Rule - Monitor Task Failure | Default rule for monitoring task instance failures. | Disabled by default |
39
39
 
40
40
  ### Create a New Monitoring Rule
41
41
 
42
42
  Click the "Create Rule" button to create custom monitoring rules based on your needs.
43
43
 
44
- ![Create Monitoring Rule Interface](.topwrite/assets/226957dd01/d2242b08605d2a3ad9b3713d8e6e6d7ab191e066.png =493)
45
-
46
- | Category | Parameter | Description |
47
- | --------- | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
48
- | Basic Information | Name | Enter the name of the new custom rule. |
49
- | Description | Optional. You can add a description of the current rule or note the handling procedures after receiving an alert. | |
50
- | Trigger Condition | Monitoring Items | The specific monitoring objects. The system currently supports "Event Monitoring" and "Metric Monitoring" two modes. |
51
- | Filter Condition | Filter conditions for messages. Multiple conditions are combined with "AND". | |
52
- | Alert Level | Alert Level | The alert level configuration in the universal template is as follows, and users can also customize notification methods for different levels in the notification strategy. Critical: Send via all alert channels, including phone. Severe: Send via all alert channels, including phone. Warning: In-system, email, SMS, Webhook, excluding phone. Info: In-system, email, Webhook, excluding phone and SMS. Clicking the alert level will link to the information displayed in the notification strategy list below. |
53
- | Monitoring Notification | Notification Strategy | Click the dropdown to select a notification strategy managed in "Notification Strategy", or click the "+" button to create a new one. For configuration details, see Notification Strategy. |
54
- | Alert Subscription | Dropdown to select the specific recipients to be notified for this rule. | |
55
- | Webhook Notification | Select notification method. Currently supported types: DingTalk, Feishu. | |
56
- | Notification Start Time | The start time for sending notifications after the monitoring rule is triggered. | |
57
- | Notification End Time | The end time for sending notifications after the monitoring rule is triggered. | |
44
+ ![](/.topwrite/assets/image_1780674143790.png =677)
45
+
46
+ ^
47
+
48
+ | Category | Parameter | Description |
49
+ | ----------------------- | ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
50
+ | Basic Information | Name | Enter the name of the new custom rule. |
51
+ | Description | Optional. You can add a description of the current rule or note the handling procedures after receiving an alert. | |
52
+ | Trigger Condition | Monitoring Items | The specific monitoring objects. The system currently supports "Event Monitoring" and "Metric Monitoring" two modes. |
53
+ | Filter Condition | Filter conditions for messages. Multiple conditions are combined with "AND". | |
54
+ | Alert Level | Alert Level | The alert level configuration in the universal template is as follows, and users can also customize notification methods for different levels in the notification strategy. Critical: Send via all alert channels, including phone. Severe: Send via all alert channels, including phone. Warning: In-system, email, SMS, Webhook, excluding phone. Info: In-system, email, Webhook, excluding phone and SMS. Clicking the alert level will link to the information displayed in the notification strategy list below. |
55
+ | Monitoring Notification | Notification Strategy | Click the dropdown to select a notification strategy managed in "Notification Strategy", or click the "+" button to create a new one. For configuration details, see Notification Strategy. |
56
+ | Alert Subscription | Dropdown to select the specific recipients to be notified for this rule. | |
57
+ | Webhook Notification | Select notification method. Currently supported types: DingTalk, Feishu. | |
58
+ | Notification Start Time | The start time for sending notifications after the monitoring rule is triggered. | |
59
+ | Notification End Time | The end time for sending notifications after the monitoring rule is triggered. | |
58
60
 
59
61
  #### Trigger Condition Rules
60
62
 
@@ -62,7 +64,9 @@ Trigger conditions consist of monitoring items, metric calculation methods, thre
62
64
 
63
65
  **Metric-Based Monitoring Configuration**
64
66
 
65
- ![](.topwrite/assets/226957dd01/0d872f1cb6d11f16e4a48c9a6466ac9d43eb8606.png =548)
67
+ \*\*\*\*![](/.topwrite/assets/image_1780674178425.png =577)
68
+
69
+ ^
66
70
 
67
71
  For metric monitoring, after defining the calculation method and threshold, two trigger methods are supported.
68
72
 
@@ -72,7 +76,7 @@ Check Interval: Users define that when the threshold is touched a cumulative N t
72
76
 
73
77
  Metric monitoring configuration: Users can define calculation methods and thresholds, and select trigger methods. For example, configure the full-incremental integrated sync task delay metric: delay time >= 50 seconds, continuous for 3 data points, alert frequency limited to once every 30 minutes.
74
78
 
75
- ![](.topwrite/assets/226957dd01/4f091acaf0cca77057e17882d353011c0853801e.png =600)
79
+ ^
76
80
 
77
81
  At 00:40, when the threshold is exceeded for 3 consecutive times, the first alert is triggered. From then until 01:50, it remains in the first alert stage.
78
82
 
@@ -99,48 +103,49 @@ The alert event list displays all alert information generated after monitoring r
99
103
  Suppress: Set the current alert event to not send messages for a specified number of minutes.
100
104
 
101
105
  Close: Close the current alert event and stop receiving such messages.
102
- ![Alert Event List](.topwrite/assets/226957dd01/c8922b31a22c405c99d2ca37c0c04661f5fd9524.png =680)
106
+
107
+ ![](/.topwrite/assets/image_1780674232157.png =701)
103
108
 
104
109
  #### Alert Event Handling Operations
105
110
 
106
- | Operation | Definition | Applicable Scenario | Scope |
107
- | ----------------------- | ------------------ | ------------------- | ------ |
108
- | Suppress | Stop sending messages for the alert event within a specified time period. | A known issue is being addressed and repeated alerts are not needed temporarily. | Current alert event |
109
- | Close | Close the alert event and stop receiving such messages. | Issue has been resolved or confirmed as a false alarm. | Current alert event |
110
- | Auto Close (not triggered by clicking on the alert event) | The system automatically closes the alert after detecting issue recovery. | Task instance rerun succeeds, manually set to success. | Related alert events |
111
+ | Operation | Definition | Applicable Scenario | Scope |
112
+ | ---------------------------------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | -------------------- |
113
+ | Suppress | Stop sending messages for the alert event within a specified time period. | A known issue is being addressed and repeated alerts are not needed temporarily. | Current alert event |
114
+ | Close | Close the alert event and stop receiving such messages. | Issue has been resolved or confirmed as a false alarm. | Current alert event |
115
+ | Auto Close (not triggered by clicking on the alert event) | The system automatically closes the alert after detecting issue recovery. | Task instance rerun succeeds, manually set to success. | Related alert events |
111
116
 
112
117
  ## Notification History
113
118
 
114
119
  Notification history records all notification messages actually delivered based on notification strategies.
115
120
 
116
- ![Notification History List](.topwrite/assets/226957dd01/df5b643e1c5d0b495e559e67d30d3dbe9d699179.png =680)
121
+ ^
117
122
 
118
123
  ## Notification Strategy
119
124
 
120
125
  The notification strategy list displays all defined notification strategies. Users can search and filter.
121
126
 
122
- ![Notification Strategy List](.topwrite/assets/226957dd01/34595b2465ace2f704ee53ff5906a5a81e4f85d2.png =680)
127
+ ![](/.topwrite/assets/image_1780674263546.png =700)
123
128
 
124
- ![](.topwrite/assets/226957dd01/d7725ff84c1b3f1871e83dc4e23b576edad6d103.png =680)
129
+ ^
125
130
 
126
131
  ### Create a New Notification Strategy
127
132
 
128
133
  Users can click the "Create Strategy" button to create a new notification strategy based on their needs.
129
134
 
130
- ![Create Notification Strategy Interface](.topwrite/assets/226957dd01/a17beff395adf70cdc56ab189d7dbb09e5263f8d.png =360)
131
-
132
- | Category | Parameter | Description |
133
- | ---- | -------- | ------------------------------------------------------------------------------------------------------------------------ |
134
- | Basic Information | Name | The name of the notification strategy. |
135
- | | Description | Optional. You can add a description of the current rule. |
136
- | Notification Method | Critical Alert | Set the specific notification method for different alert levels. Supported methods: Webhook, SMS, Phone. |
137
- | | Severe Alert | |
138
- | | Warning Alert | |
139
- | | Info Alert | |
140
- | Notification Time | Send Interval (minutes) | The time interval between two alerts. |
141
- | | Max Send Count | The maximum number of alerts. After exceeding this count, no more alerts will be generated. |
142
- | | Do Not Disturb Start Time | During the do-not-disturb period, the system will not send alerts. For example, if a task failure alert is configured and the do-not-disturb time is set from 00:00 to 08:00, no alert will be sent during that period. If at 08:00 the task is still in an abnormal state, an alert will be sent. |
143
- | | Do Not Disturb End Time | |
135
+ ![](/.topwrite/assets/image_1780674286569.png =677)
136
+
137
+ | Category | Parameter | Description |
138
+ | ------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
139
+ | Basic Information | Name | The name of the notification strategy. |
140
+ | | Description | Optional. You can add a description of the current rule. |
141
+ | Notification Method | Critical Alert | Set the specific notification method for different alert levels. Supported methods: Webhook, SMS, Phone. |
142
+ | | Severe Alert | |
143
+ | | Warning Alert | |
144
+ | | Info Alert | |
145
+ | Notification Time | Send Interval (minutes) | The time interval between two alerts. |
146
+ | | Max Send Count | The maximum number of alerts. After exceeding this count, no more alerts will be generated. |
147
+ | | Do Not Disturb Start Time | During the do-not-disturb period, the system will not send alerts. For example, if a task failure alert is configured and the do-not-disturb time is set from 00:00 to 08:00, no alert will be sent during that period. If at 08:00 the task is still in an abnormal state, an alert will be sent. |
148
+ | | Do Not Disturb End Time | |
144
149
 
145
150
  ## Configuration Management
146
151
 
@@ -150,19 +155,19 @@ Configuration management allows users to configure personal information and Webh
150
155
 
151
156
  Users can modify their phone number and email address used for receiving alerts, and set do-not-disturb periods in personal configuration.
152
157
 
153
- ![Personal Configuration Interface](.topwrite/assets/226957dd01/d7725ff84c1b3f1871e83dc4e23b576edad6d103.png =680)
158
+ ^
154
159
 
155
160
  ### Webhook Configuration
156
161
 
157
162
  Webhook configuration is used to define Webhook channels for alert delivery. Currently supports Feishu and DingTalk.
158
163
 
159
- ![Webhook Configuration Interface](.topwrite/assets/226957dd01/6a8bc6fc74fb8f046f7169d91fac4169aab3b968.png =680)
164
+ ^
160
165
 
161
166
  #### Create a New Webhook Configuration
162
167
 
163
168
  Users can click the "Create Configuration" button and fill in the required parameters to create a new Webhook configuration. It is recommended to test before saving to ensure the configuration is correct.
164
169
 
165
- ![](.topwrite/assets/226957dd01/4ce77ae787a0c4c3c8d908b7c6feb5f5beaf7241.png =300)
170
+ ^
166
171
 
167
172
  ## Other
168
173
 
@@ -174,4 +179,4 @@ For monitoring alerts on task instance run failures, after the Operations Center
174
179
 
175
180
  For IM platforms such as DingTalk, Webhook alert delivery has certain security settings. Add "Singdata" as a custom keyword in the security settings.
176
181
 
177
- ![](.topwrite/assets/image_1748598090100.png =300)
182
+ ^
@@ -4,41 +4,41 @@ This document describes the specific definitions and scope of "Event" and "Metri
4
4
 
5
5
  ## Event-Based Monitoring
6
6
 
7
- | Event Name | Target Objects | Scope Description |
8
- | ------------------ | -------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
9
- | Scheduled Task Instance Run Failure | Includes: Periodically scheduled task instances, backfill task instances, and manual reruns of scheduled instances. Excludes: Manually run temporary instances, real-time sync task instances. | An alert event is generated when a scheduled task instance fails due to various reasons (code logic errors, timeout, system exceptions, etc.). If the scheduled task is configured with automatic retry on error, no alert events are generated during the automatic retry process; only one alert is generated upon final failure. |
10
- | Scheduled Task Instance Completion Time | Includes: Periodically scheduled task instances, manually rerun scheduled task instances, etc. Excludes: Manually run temporary instances, backfill task instances. | An alert event is generated when the completion time of a scheduled task exceeds the expected time point (does not finish running by the specified time). |
11
- | Scheduled Task Instance Scheduling Duration (Including Wait Time) | Includes: Periodically scheduled task instances. Excludes: Manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the total duration from the planned time to the end time of a scheduled task instance exceeds the threshold. This includes the time the task instance waits for upstream tasks to complete. |
12
- | Scheduled Task Run Time Delay | Includes: Periodically scheduled task instances. Excludes: Manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the actual start time of a scheduled task instance (when it enters running state) is delayed beyond the threshold compared to the planned time. |
13
- | Task Instance Execution Duration (Excluding Wait Time) | All task instances. Includes periodically scheduled task instances, manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the pure execution time of the instance (excluding the time waiting for upstream completion) exceeds the threshold. |
14
- | Dynamic Table Refresh Timeout | Dynamic Tables | An alert event is generated when a dynamic table refresh operation exceeds the set timeout period. |
15
- | Streaming SQL Task Execution Failure | Streaming SQL (Continuous Job) tasks | An alert event is generated when the running status of a streaming SQL task changes from "Running" to "Failed". |
16
- | Quality Rule Validation Failure | Quality Rules + Target Table: Select tables configured with quality rules. When a rule validation fails on that table, an alert history is generated. | An alert event is generated when the execution result of a data quality validation rule is "Failed". |
17
- | Quality Rule Validation Timeout | Quality Rules | An alert event is generated when the execution result of a data quality validation rule is "Timeout". |
18
- | Multi-Table Real-Time Sync Task Run Failure | Multi-table real-time sync tasks | An alert event is generated when the status of a multi-table real-time sync task changes from "Running" to "Failed". |
19
- | Multi-Table Real-Time Sync Task Table Enters Blacklist | Multi-table real-time sync tasks | An alert event is generated when a table in the task is added to the blacklist due to consecutive sync failures. |
20
- | Multi-Table Real-Time Sync Task Single Table Full Data Sync Exception | Multi-table real-time sync tasks | An alert event is generated when an exception occurs during the full data synchronization of a single table in a multi-table real-time sync task. |
21
- | Multi-Table Real-Time Sync Task Single Table Incremental Data Sync Exception | Multi-table real-time sync tasks | An alert event is generated when an exception occurs during the incremental data synchronization of a single table in a multi-table real-time sync task. |
22
- | Full & Incremental Integrated Sync Single Table Full Data Sync Exception | Full & incremental integrated sync tasks | An alert event is generated when an exception occurs during the full data synchronization of a single table in a full & incremental integrated sync task. |
23
- | Full & Incremental Integrated Sync Single Table Incremental Data Sync Exception | Full & incremental integrated sync tasks | An alert event is generated when an exception occurs during the incremental data synchronization of a single table in a full & incremental integrated sync task. |
24
- | Full & Incremental Integrated Sync Task Run Failure | Full & incremental integrated sync tasks | An alert event is generated when the status of a full & incremental integrated sync task changes from "Running" to "Failed". |
25
- | Full & Incremental Integrated Sync Task Target Table Schema Change Failure | Full & incremental integrated sync tasks | An alert event is generated when automatic schema evolution of the target table encounters an error. |
7
+ | Event Name | Target Objects | Scope Description |
8
+ | ------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
9
+ | Scheduled Task Instance Run Failure | Includes: Periodically scheduled task instances, backfill task instances, and manual reruns of scheduled instances. Excludes: Manually run temporary instances, real-time sync task instances. | An alert event is generated when a scheduled task instance fails due to various reasons (code logic errors, timeout, system exceptions, etc.). If the scheduled task is configured with automatic retry on error, no alert events are generated during the automatic retry process; only one alert is generated upon final failure. |
10
+ | Scheduled Task Instance Completion Time | Includes: Periodically scheduled task instances, manually rerun scheduled task instances, etc. Excludes: Manually run temporary instances, backfill task instances. | An alert event is generated when the completion time of a scheduled task exceeds the expected time point (does not finish running by the specified time). |
11
+ | Scheduled Task Instance Scheduling Duration (Including Wait Time) | Includes: Periodically scheduled task instances. Excludes: Manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the total duration from the planned time to the end time of a scheduled task instance exceeds the threshold. This includes the time the task instance waits for upstream tasks to complete. |
12
+ | Scheduled Task Run Time Delay | Includes: Periodically scheduled task instances. Excludes: Manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the actual start time of a scheduled task instance (when it enters running state) is delayed beyond the threshold compared to the planned time. |
13
+ | Task Instance Execution Duration (Excluding Wait Time) | All task instances. Includes periodically scheduled task instances, manually run temporary instances, backfill task instances, manually rerun scheduled task instances, etc. | An alert event is generated when the pure execution time of the instance (excluding the time waiting for upstream completion) exceeds the threshold. |
14
+ | Dynamic Table Refresh Timeout | Dynamic Tables | An alert event is generated when a dynamic table refresh operation exceeds the set timeout period. |
15
+ | Streaming SQL Task Execution Failure | Streaming SQL (Continuous Job) tasks | An alert event is generated when the running status of a streaming SQL task changes from "Running" to "Failed". |
16
+ | Quality Rule Validation Failure | Quality Rules + Target Table: Select tables configured with quality rules. When a rule validation fails on that table, an alert history is generated. | An alert event is generated when the execution result of a data quality validation rule is "Failed". |
17
+ | Quality Rule Validation Timeout | Quality Rules | An alert event is generated when the execution result of a data quality validation rule is "Timeout". |
18
+ | Multi-Table Real-Time Sync Task Run Failure | Multi-table real-time sync tasks | An alert event is generated when the status of a multi-table real-time sync task changes from "Running" to "Failed". |
19
+ | Multi-Table Real-Time Sync Task Table Enters Blacklist | Multi-table real-time sync tasks | An alert event is generated when a table in the task is added to the blacklist due to consecutive sync failures. |
20
+ | Multi-Table Real-Time Sync Task Single Table Full Data Sync Exception | Multi-table real-time sync tasks | An alert event is generated when an exception occurs during the full data synchronization of a single table in a multi-table real-time sync task. |
21
+ | Multi-Table Real-Time Sync Task Single Table Incremental Data Sync Exception | Multi-table real-time sync tasks | An alert event is generated when an exception occurs during the incremental data synchronization of a single table in a multi-table real-time sync task. |
22
+ | Full & Incremental Integrated Sync Single Table Full Data Sync Exception | Full & incremental integrated sync tasks | An alert event is generated when an exception occurs during the full data synchronization of a single table in a full & incremental integrated sync task. |
23
+ | Full & Incremental Integrated Sync Single Table Incremental Data Sync Exception | Full & incremental integrated sync tasks | An alert event is generated when an exception occurs during the incremental data synchronization of a single table in a full & incremental integrated sync task. |
24
+ | Full & Incremental Integrated Sync Task Run Failure | Full & incremental integrated sync tasks | An alert event is generated when the status of a full & incremental integrated sync task changes from "Running" to "Failed". |
25
+ | Full & Incremental Integrated Sync Task Target Table Schema Change Failure | Full & incremental integrated sync tasks | An alert event is generated when automatic schema evolution of the target table encounters an error. |
26
26
 
27
27
  ## Metric-Based Monitoring
28
28
 
29
- | Metric Name | Target Objects | Scope Description |
30
- | ------------------ | ----------- | ------------------------------------------------------------ |
31
- | Dynamic Table Refresh Failure | Dynamic Tables | Number of dynamic table refresh operation failures. |
32
- | Full & Incremental Integrated Sync Task Delay | Full & incremental integrated real-time sync tasks | Time delay of data synchronization. |
33
- | Full & Incremental Integrated Sync Task Single Table Sync Failure | Full & incremental integrated real-time sync tasks | Statistics on single table synchronization failure count. |
34
- | Multi-Table Real-Time Sync Task Delay | Multi-table real-time sync tasks | Overall delay of multi-table real-time synchronization. |
35
- | Multi-Table Real-Time Sync Task Job Failover | Multi-table real-time sync tasks | Number of failover occurrences in multi-table sync tasks. |
36
- | Multi-Table Real-Time Sync Task Read Position Delay | Multi-table real-time sync tasks | Delay of data read position relative to the source. |
37
- | Multi-Table Real-Time Sync Task Source Database Read Delay | Multi-table real-time sync tasks | Difference between source database data generation rate and synchronization rate. |
38
- | Multi-Table Real-Time Sync Task Data Sync Status Exception | Multi-table real-time sync tasks | Monitoring of abnormal conditions in data synchronization status. |
39
- | Queued Jobs Count | Single SQL Job | Sampled every 5 seconds. Counts all SQL jobs whose status is "queued" at each sampling point. |
40
- | Average Job Queue Time | Single SQL Job | Sampled every 5 seconds. Calculates the TP90 job queue time within each 5-second window. |
41
- | General-Purpose / Integration Vcluster Load | VCluster | At each sampling point, computes the ratio of compute resources currently processing tasks to the maximum compute resources supported by that vcluster size. |
42
- | Analytics Vcluster Load | VCluster | At each sampling point, computes the ratio of current job concurrency to the vcluster's maximum supported concurrency (calculated as max replicas x max concurrency per replica). |
29
+ | Metric Name | Target Objects | Scope Description |
30
+ | ----------------------------------------------------------------- | -------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
31
+ | Dynamic Table Refresh Failure | Dynamic Tables | Number of dynamic table refresh operation failures. |
32
+ | Full & Incremental Integrated Sync Task Delay | Full & incremental integrated real-time sync tasks | Time delay of data synchronization. |
33
+ | Full & Incremental Integrated Sync Task Single Table Sync Failure | Full & incremental integrated real-time sync tasks | Statistics on single table synchronization failure count. |
34
+ | Multi-Table Real-Time Sync Task Delay | Multi-table real-time sync tasks | Overall delay of multi-table real-time synchronization. |
35
+ | Multi-Table Real-Time Sync Task Job Failover | Multi-table real-time sync tasks | Number of failover occurrences in multi-table sync tasks. |
36
+ | Multi-Table Real-Time Sync Task Read Position Delay | Multi-table real-time sync tasks | Delay of data read position relative to the source. |
37
+ | Multi-Table Real-Time Sync Task Source Database Read Delay | Multi-table real-time sync tasks | Difference between source database data generation rate and synchronization rate. |
38
+ | Multi-Table Real-Time Sync Task Data Sync Status Exception | Multi-table real-time sync tasks | Monitoring of abnormal conditions in data synchronization status. |
39
+ | Queued Jobs Count | Single SQL Job | Sampled every 5 seconds. Counts all SQL jobs whose status is "queued" at each sampling point. |
40
+ | Average Job Queue Time | Single SQL Job | Sampled every 5 seconds. Calculates the TP90 job queue time within each 5-second window. |
41
+ | General-Purpose / Integration Vcluster Load | VCluster | At each sampling point, computes the ratio of compute resources currently processing tasks to the maximum compute resources supported by that vcluster size. |
42
+ | Analytics Vcluster Load | VCluster | At each sampling point, computes the ratio of current job concurrency to the vcluster's maximum supported concurrency (calculated as max replicas x max concurrency per replica). |
43
43
 
44
44
  ^