@clickzetta/cz-cli-darwin-arm64 0.5.16 → 0.5.18

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (243) hide show
  1. package/bin/cz-cli +0 -0
  2. package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
  3. package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
  4. package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
  5. package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
  6. package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
  7. package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
  8. package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
  9. package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
  10. package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
  11. package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
  12. package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
  13. package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
  14. package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
  15. package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
  16. package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
  17. package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
  18. package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
  19. package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
  20. package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
  21. package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
  22. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
  23. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
  24. package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
  25. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
  26. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
  27. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
  28. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
  29. package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
  30. package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
  31. package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
  32. package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
  33. package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
  34. package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
  35. package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
  36. package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
  37. package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
  38. package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
  39. package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
  40. package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
  41. package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
  42. package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
  43. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
  44. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
  45. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
  46. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
  47. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
  48. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
  49. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
  50. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
  51. package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
  52. package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
  53. package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
  54. package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
  55. package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
  56. package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
  57. package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
  58. package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
  59. package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
  60. package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
  61. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
  62. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
  63. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
  64. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
  65. package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
  66. package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
  67. package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
  68. package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
  69. package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
  70. package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
  71. package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
  72. package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
  73. package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
  74. package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
  75. package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
  76. package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
  77. package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
  78. package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
  79. package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
  80. package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
  81. package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
  82. package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
  83. package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
  84. package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
  85. package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
  86. package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
  87. package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
  88. package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
  89. package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
  90. package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
  91. package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
  92. package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
  93. package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
  94. package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
  95. package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
  96. package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
  97. package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
  98. package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
  99. package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
  100. package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
  101. package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
  102. package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
  103. package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
  104. package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
  105. package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
  106. package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
  107. package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
  108. package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
  109. package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
  110. package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
  111. package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
  112. package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
  113. package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
  114. package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
  115. package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
  116. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
  117. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
  118. package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
  119. package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
  120. package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
  121. package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
  122. package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
  123. package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
  124. package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
  125. package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
  126. package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
  127. package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
  128. package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
  129. package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
  130. package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
  131. package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
  132. package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
  133. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
  134. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
  135. package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
  136. package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
  137. package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
  138. package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
  139. package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
  140. package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
  141. package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
  142. package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
  143. package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
  144. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
  145. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
  146. package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
  147. package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
  148. package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
  149. package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
  150. package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
  151. package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
  152. package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
  153. package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
  154. package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
  155. package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
  156. package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
  157. package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
  158. package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
  159. package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
  160. package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
  161. package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
  162. package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
  163. package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
  164. package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
  165. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
  166. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
  167. package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
  168. package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
  169. package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
  170. package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
  171. package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
  172. package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
  173. package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
  174. package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
  175. package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
  176. package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
  177. package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
  178. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
  179. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
  180. package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
  181. package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
  182. package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
  183. package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
  184. package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
  185. package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
  186. package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
  187. package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
  188. package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
  189. package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
  190. package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
  191. package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
  192. package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
  193. package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
  194. package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
  195. package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
  196. package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
  197. package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
  198. package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
  199. package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
  200. package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
  201. package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
  202. package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
  203. package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
  204. package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
  205. package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
  206. package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
  207. package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
  208. package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
  209. package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
  210. package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
  211. package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
  212. package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
  213. package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
  214. package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
  215. package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
  216. package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
  217. package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
  218. package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
  219. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
  220. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
  221. package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
  222. package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
  223. package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
  224. package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
  225. package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
  226. package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
  227. package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
  228. package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
  229. package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
  230. package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
  231. package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
  232. package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
  233. package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
  234. package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
  235. package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
  236. package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
  237. package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
  238. package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
  239. package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
  240. package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
  241. package/package.json +1 -1
  242. package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
  243. package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
@@ -8,18 +8,18 @@ The specific adjustments to the specifications are as follows:
8
8
 
9
9
  From XS to 5XLarge, each level increase doubles the computing resources and also doubles the resource consumption:
10
10
 
11
- | | | |
12
- | ------- | ------ | ------------------------- |
13
- | **Specification** | **Abbreviation** | **Hourly Consumption** (**CRU**\***hour**) |
14
- | XSMALL | XS | 1 |
15
- | SMALL | S | 2 |
16
- | MEDIUM | M | 4 |
17
- | LARGE | L | 8 |
18
- | XLARGE | XL | 16 |
19
- | 2XLARGE | 2XL | 32 |
20
- | 3XLARGE | 3XL | 64 |
21
- | 4XLARGE | 4XL | 128 |
22
- | 5XLARGE | 5XL | 256 |
11
+ | | | |
12
+ | ----------------- | ---------------- | ------------------------------------------ |
13
+ | **Specification** | **Abbreviation** | **Hourly Consumption** (**CRU**\***hour**) |
14
+ | XSMALL | XS | 1 |
15
+ | SMALL | S | 2 |
16
+ | MEDIUM | M | 4 |
17
+ | LARGE | L | 8 |
18
+ | XLARGE | XL | 16 |
19
+ | 2XLARGE | 2XL | 32 |
20
+ | 3XLARGE | 3XL | 64 |
21
+ | 4XLARGE | 4XL | 128 |
22
+ | 5XLARGE | 5XL | 256 |
23
23
 
24
24
  ## **New Calculation Cluster Specifications**
25
25
 
@@ -27,11 +27,11 @@ Each type of calculation cluster is represented by CRU (Compute Resource Unit).
27
27
 
28
28
  The detailed specifications of various calculation clusters are as follows:
29
29
 
30
- | **Type** | **Minimum Specification** (**CRU**) | **Maximum Specification** (**CRU**) | **Default Value** (**CRU**) | **Step Size**
 (Minimum unit of specification increase/decrease) |
31
- | ------------------------------ | ----------------- | ----------------- | ---------------- | ----------------------------------------------------------------------------- |
32
- | General Purpose
 (GP Vcluster) | 1 | 256 | 1 | 1 (Specification examples: 1, 2, 3, 4, 5, 6...256) |
33
- | Analytical
 (AP Vcluster) | 1 | 256 | 1 | Powers of 2 (Specification examples: 1, 2, 4, 8, 16, 32, 64, 128, 256) |
34
- | Integration
 (Integration Vcluster) | 0.25 | 256 | 0.5 | 1 (Specification sequence: there are two special small specifications of 0.25 CRU and 0.5 CRU, the step size for other specifications is 1. Specification examples: 0.25, 0.5, 1, 2, 3, 4, 5, 6...256) |
30
+ | **Type** | **Minimum Specification** (**CRU**) | **Maximum Specification** (**CRU**) | **Default Value** (**CRU**) | **Step Size**
 (Minimum unit of specification increase/decrease) |
31
+ | --------------------------------------- | ----------------------------------- | ----------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
32
+ | General Purpose
 (GP Vcluster) | 1 | 256 | 1 | 1 (Specification examples: 1, 2, 3, 4, 5, 6...256) |
33
+ | Analytical
 (AP Vcluster) | 1 | 256 | 1 | Powers of 2 (Specification examples: 1, 2, 4, 8, 16, 32, 64, 128, 256) |
34
+ | Integration
 (Integration Vcluster) | 0.25 | 256 | 0.5 | 1 (Specification sequence: there are two special small specifications of 0.25 CRU and 0.5 CRU, the step size for other specifications is 1. Specification examples: 0.25, 0.5, 1, 2, 3, 4, 5, 6...256) |
35
35
 
36
36
  The amount of computing resources consumed by each cluster per hour is its CRU specification\*1 hour, i.e., the unit is CRU\*hour. For example, an analytical calculation cluster with a specification of 3 CRU running for 1 hour consumes 3 CRU \* 1 hour = 3 CRU\*hour. The cost is also calculated according to the "CRU\*hour" unit price in the region where the calculation cluster is located.
37
37
 
@@ -41,24 +41,25 @@ The amount of computing resources consumed by each cluster per hour is its CRU s
41
41
 
42
42
  The specifications of existing calculation clusters will be automatically converted to the new specifications, and the computing resource consumption and costs per unit time will not change. The specific correspondence of specifications is as follows:
43
43
 
44
- | | | |
45
- | ------- | ------- | -------- |
44
+ | | | |
45
+ | ------------------ | ------------- | -------------------- |
46
46
  | **Original Value** | **New Value** | **Applicable Scope** |
47
- | XSMALL | 1 | All types of VC |
48
- | SMALL | 2 | All types of VC |
49
- | MEDIUM | 4 | All types of VC |
50
- | LARGE | 8 | All types of VC |
51
- | XLARGE | 16 | All types of VC |
52
- | 2XLARGE | 32 | All types of VC |
53
- | 3XLARGE | 64 | All types of VC |
54
- | 4XLARGE | 128 | All types of VC |
55
- | 5XLARGE | 256 | All types of VC |
47
+ | XSMALL | 1 | All types of VC |
48
+ | SMALL | 2 | All types of VC |
49
+ | MEDIUM | 4 | All types of VC |
50
+ | LARGE | 8 | All types of VC |
51
+ | XLARGE | 16 | All types of VC |
52
+ | 2XLARGE | 32 | All types of VC |
53
+ | 3XLARGE | 64 | All types of VC |
54
+ | 4XLARGE | 128 | All types of VC |
55
+ | 5XLARGE | 256 | All types of VC |
56
56
 
57
57
  ### SQL Create New Compute Cluster
58
58
 
59
59
  After the new specification code is launched, the SQL statements for creating and modifying compute clusters will temporarily be compatible with the old specification codes. That is, the Vcluster\_size parameter can support both numerical expressions and code expressions like 'XSMALL, SMALL' for compute cluster specifications. Please migrate to using numerical expressions for compute cluster specifications as soon as possible to avoid affecting the normal creation of compute clusters when switching to an incompatible version.
60
60
 
61
61
  Example code for creating a new cluster is as follows:
62
+
62
63
  ```SQL
63
64
  -- Create a compute cluster
64
65
  CREATE VCLUSTER [IF NOT EXISTS] vc_name
@@ -69,11 +70,12 @@ objectProperties
69
70
  objectProperties ::=
70
71
  VCLUSTER_SIZE = NUM --0.25-256. Compatible with XSMALL | SMALL | MEDIUM | LARGE | XLARGE | 2XLARGE | 3XLARGE | 4XLARGE | 5XLARGE
71
72
  ```
73
+
72
74
  ### Create a New Compute Cluster on the Web
73
75
 
74
76
  When creating and modifying compute clusters on the web, the specifications of the compute cluster are now expressed in numerical terms. An example is shown below:
75
77
 
76
- :-: ![](.topwrite/assets/image_1733751304880.png =420)
78
+ ^
77
79
 
78
80
  ### Query Compute Cluster via SQL
79
81
 
@@ -83,7 +85,7 @@ When querying compute cluster specifications via SQL, all fields representing co
83
85
 
84
86
  The specifications of the compute cluster on the web are now expressed in numerical terms.
85
87
 
86
- :-: ![](.topwrite/assets/image_1733751321636.png)
88
+ ^
87
89
 
88
90
  ## Billing Instructions
89
91
 
@@ -93,4 +95,4 @@ After the change, estimating the hourly resource consumption and cost of a clust
93
95
 
94
96
  There is no need to remember that the "S" specification corresponds to 2 CRU per hour, and then multiply by the unit price to estimate the cost.
95
97
 
96
- A compute cluster of 2 CRU consumes "2 CRU\*hour" per hour, and you only need to match the unit price to calculate the hourly cost.
98
+ A compute cluster of 2 CRU consumes "2 CRU\*hour" per hour, and you only need to match the unit price to calculate the hourly cost.
@@ -14,11 +14,11 @@ A virtual cluster's specification defines the amount of compute resources availa
14
14
 
15
15
  > ⚠️ **Note**: Starting from December 2024, the old specification codes for virtual clusters (XS through 5XLarge) have been replaced with numeric specifications. For details, see [Virtual Cluster Specification Code Change Description](vcluster_size_description.md).
16
16
 
17
- | **Type** | **Minimum Spec (CRU)** | **Maximum Spec (CRU)** | **Default (CRU)** | **Step Size** (minimum increment/decrement unit) |
18
- | ------------------------------------- | ---------------------- | ---------------------- | ----------------- | ----------------------------------------------------------------------------------------- |
19
- | General Purpose (GP VCluster) | 1 | 256 | 1 | 1 CRU per step. Examples: 1, 2, 3, 4, 5, 6...256 |
20
- | Analytics (AP VCluster) | 1 | 256 | 1 | Powers of 2. Examples: 1, 2, 4, 8, 16, 32, 64, 128, 256 |
21
- | Integration (Integration VCluster) | 0.25 | 256 | 0.5 | Supports any decimal specification. Examples: 0.25, 0.5, 0.75, 1, 1.25...256 |
17
+ | **Type** | **Minimum Spec (CRU**) | **Maximum Spec (CRU**) | **Default (CRU**) | **Step Size** (minimum increment/decrement unit) |
18
+ | ---------------------------------- | ---------------------- | ---------------------- | ----------------- | ---------------------------------------------------------------------------- |
19
+ | General Purpose (GP VCluster) | 1 | 256 | 1 | 1 CRU per step. Examples: 1, 2, 3, 4, 5, 6...256 |
20
+ | Analytics (AP VCluster) | 1 | 256 | 1 | Powers of 2. Examples: 1, 2, 4, 8, 16, 32, 64, 128, 256 |
21
+ | Integration (Integration VCluster) | 0.25 | 256 | 0.5 | Supports any decimal specification. Examples: 0.25, 0.5, 0.75, 1, 1.25...256 |
22
22
 
23
23
  > ⚠️ **Note**: Analytics cluster specifications must be a power of 2 (1, 2, 4, 8, 16, 32, 64, 128, 256). Setting any other value will result in an error.
24
24
 
@@ -28,29 +28,28 @@ The compute resources consumed by a cluster per hour equal its CRU specification
28
28
 
29
29
  Virtual clusters come in three types: General Purpose, Analytics, and Integration.
30
30
 
31
- * **General Purpose (GENERAL)**: Suitable for offline jobs. Jobs share compute resources, and new and existing jobs are allocated resources using fair scheduling.
32
- * **Analytics (ANALYTICS)**: Supports multiple compute instances and auto-scaling. Suitable for online and high-concurrency workloads. When the concurrency limit is reached, subsequent jobs queue up, ensuring first-submitted jobs are executed first.
33
- * **Integration (INTEGRATION)**: Suitable for offline and real-time integration tasks. Multiple integration tasks can share a single Integration cluster instance. Tasks that exceed the cluster's capacity enter a queue.
31
+ * **General Purpose (GENERAL**): Suitable for offline jobs. Jobs share compute resources, and new and existing jobs are allocated resources using fair scheduling.
32
+ * **Analytics (ANALYTICS**): Supports multiple compute instances and auto-scaling. Suitable for online and high-concurrency workloads. When the concurrency limit is reached, subsequent jobs queue up, ensuring first-submitted jobs are executed first.
33
+ * **Integration (INTEGRATION**): Suitable for offline and real-time integration tasks. Multiple integration tasks can share a single Integration cluster instance. Tasks that exceed the cluster's capacity enter a queue.
34
34
 
35
35
  The relationship between cluster types and task types is shown below:
36
36
 
37
- | **Task Type** | **Supported Cluster Types** | **Recommended Cluster Type** |
38
- | ----------------------- | ---------------------------- | ----------------------------------------------------------------------------- |
39
- | Offline Sync | Integration | Integration |
40
- | SQL | General Purpose, Analytics | **ETL tasks**: General Purpose<br>**Ad-hoc queries**: Analytics |
41
- | Python | No cluster required | / |
42
- | Shell | No cluster required | / |
43
- | JDBC | No cluster required | / |
44
- | Virtual Node | No cluster required | / |
45
- | Databricks SQL | No cluster required | / |
46
- | Databricks Notebook | No cluster required | / |
47
- | Real-time Sync | Integration | Integration |
48
- | Multi-table Real-time Sync | Integration | Integration |
49
- | Dynamic Table | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose<br>**High-frequency, small data volume**: Analytics |
50
- | Streaming SQL | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose<br>**High-frequency, small data volume**: Analytics |
51
-
52
- :-: ![](.topwrite/assets/image_1743421840284.png =782)
37
+ | **Task Type** | **Supported Cluster Types** | **Recommended Cluster Type** |
38
+ | -------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------- |
39
+ | Offline Sync | Integration | Integration |
40
+ | SQL | General Purpose, Analytics | **ETL tasks**: General Purpose**Ad-hoc queries**: Analytics |
41
+ | Python | No cluster required | / |
42
+ | Shell | No cluster required | / |
43
+ | JDBC | No cluster required | / |
44
+ | Virtual Node | No cluster required | / |
45
+ | Databricks SQL | No cluster required | / |
46
+ | Databricks Notebook | No cluster required | / |
47
+ | Real-time Sync | Integration | Integration |
48
+ | Multi-table Real-time Sync | Integration | Integration |
49
+ | Dynamic Table | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose**High-frequency, small data volume**: Analytics |
50
+ | Streaming SQL | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose**High-frequency, small data volume**: Analytics |
53
51
 
52
+ ^
54
53
  ^
55
54
 
56
55
  ### Minimum Instances
@@ -81,23 +80,22 @@ Singdata Lakehouse provides two ways to manage virtual clusters: 1) Web UI; 2) S
81
80
 
82
81
  Log in to the Lakehouse platform and click "Compute" in the left menu to open the "Virtual Clusters" list page. Here you can view all virtual clusters in the current workspace that you have access to.
83
82
 
84
- ![](.topwrite/assets/image_1741611037239.png)
83
+ ^
85
84
 
86
85
  Click any cluster name to open the details page, where you can view job execution status, configuration details, and your permissions for that cluster.
87
86
 
88
- :-: ![](.topwrite/assets/image_1728532541352.png =805)
87
+ ^
89
88
 
90
89
  #### Creating a Virtual Cluster
91
90
 
92
91
  On the "Virtual Clusters" list page, click "New Cluster" to open the creation interface. Fill in the cluster name, select the cluster type, configure the specification, and click "Create Cluster." The new cluster will appear in the list and can be used once its status changes to "Running."
93
92
 
94
- :-: ![](.topwrite/assets/image_1742214451076.png =813)
95
-
93
+ ^
96
94
  ^
97
95
 
98
96
  **Creating a General Purpose Cluster**
99
97
 
100
- :-: ![](.topwrite/assets/image_1742216967064.png =414)
98
+ ^
101
99
 
102
100
  If you need to run ETL or other batch workloads, create a General Purpose cluster:
103
101
 
@@ -114,8 +112,7 @@ After completing the configuration, click "OK" to finish creating the General Pu
114
112
 
115
113
  **Creating an Analytics Cluster**
116
114
 
117
- :-: ![](.topwrite/assets/image_1742216902917.png =404)
118
-
115
+ ^
119
116
  ^
120
117
 
121
118
  If you need compute resources for ad-hoc queries or BI dashboards, create an Analytics cluster:
@@ -130,7 +127,7 @@ If you need compute resources for ad-hoc queries or BI dashboards, create an Ana
130
127
 
131
128
  **Creating an Integration Cluster**
132
129
 
133
- :-: ![](.topwrite/assets/image_1742217991159.png =411)
130
+ ^
134
131
 
135
132
  If you need to run offline or real-time integration tasks, create an Integration cluster:
136
133
 
@@ -141,7 +138,7 @@ If you need to run offline or real-time integration tasks, create an Integration
141
138
 
142
139
  * Integration clusters provide a "Specification Estimation" feature to help you choose an appropriate size. Enter the "Number of Tasks" and "Average Task Concurrency," and the system calculates an estimated specification using the formula: Specification = Number of Tasks × Average Task Concurrency × 0.05.
143
140
 
144
- :-: ![](.topwrite/assets/image_1742218209994.png =441)
141
+ ^
145
142
 
146
143
  * Choose whether to enable "Auto Resume" and "Auto Suspend," and configure a suitable "Auto Suspend" time. We recommend enabling "Auto Resume" to handle jobs promptly, and setting "Auto Suspend" to 1 minute so the cluster shuts down quickly after jobs complete, saving costs.
147
144
 
@@ -151,21 +148,20 @@ If you need to run offline or real-time integration tasks, create an Integration
151
148
 
152
149
  In the cluster list, select the cluster and click "Modify" in the "Actions" column to adjust its configuration. Configuration changes may affect cluster status. When modifying key attributes such as specification or instance count, make sure there are no jobs currently running on the cluster.
153
150
 
154
- :-: ![](.topwrite/assets/image_1733812147311.png =813)
151
+ ^
155
152
 
156
153
  #### Starting and Stopping a Virtual Cluster
157
154
 
158
155
  In the cluster list, click "Start" or "Stop" in the "Actions" column to start or stop the cluster. When stopping a cluster, it waits for all running jobs to complete before fully stopping.
159
156
 
160
- :-: ![](.topwrite/assets/image_1733812217836.png =813)
161
-
157
+ ^
162
158
  ^
163
159
 
164
160
  #### Deleting a Virtual Cluster
165
161
 
166
162
  In the cluster list, click "Delete" in the "Actions" column to delete the specified cluster. The cluster must be stopped and all jobs must be complete before deletion can proceed.
167
163
 
168
- :-: ![](.topwrite/assets/image_1733812348735.png =810)
164
+ ^
169
165
 
170
166
  ### SQL
171
167
 
@@ -193,14 +189,14 @@ In practice, it is recommended to test queries with different cluster sizes (for
193
189
 
194
190
  The following table provides sample specifications across several dimensions — business load type, execution frequency, job concurrency, data volume, and SLA requirements — for reference:
195
191
 
196
- | Business Scenario | Load Type | Execution Frequency | Job Concurrency | Data Volume | VCluster Type | Job Latency SLA | VCluster Size |
197
- | --------------------------------------- | -------------------------- | ------------------- | --------------- | ----------- | --------------- | ---------------------------- | ------------- |
198
- | ETL Scheduling Jobs | Near real-time offline | Hourly | 1 | 1 TB | General Purpose | 15 min | 4 |
199
- | | T+1 offline | Daily | 1 | 10 TB | General Purpose | 4 hours | 8 |
200
- | Tableau/FineBI | Ad-hoc analytics | Ad-hoc | 8 | 1 TB | General Purpose | <1 min, TP90 <5s | 16 |
201
- | Data Application Products | Applications | On demand | 8 | 100 GB | Analytics | <1s | 4 |
202
- | | | On demand | 96 | 100 MB | Analytics | <1s | 4 |
203
- | Singdata Web UI (Data Development/Test) | Ad-hoc analytics | Ad-hoc | 8 | 3 TB | General Purpose | <1 min, TP90 <15s | 16 |
192
+ | Business Scenario | Load Type | Execution Frequency | Job Concurrency | Data Volume | VCluster Type | Job Latency SLA | VCluster Size |
193
+ | --------------------------------------- | ---------------------- | ------------------- | --------------- | ----------- | --------------- | ----------------- | ------------- |
194
+ | ETL Scheduling Jobs | Near real-time offline | Hourly | 1 | 1 TB | General Purpose | 15 min | 4 |
195
+ | | T+1 offline | Daily | 1 | 10 TB | General Purpose | 4 hours | 8 |
196
+ | Tableau/FineBI | Ad-hoc analytics | Ad-hoc | 8 | 1 TB | General Purpose | <1 min, TP90 <5s | 16 |
197
+ | Data Application Products | Applications | On demand | 8 | 100 GB | Analytics | <1s | 4 |
198
+ | | | On demand | 96 | 100 MB | Analytics | <1s | 4 |
199
+ | Singdata Web UI (Data Development/Test) | Ad-hoc analytics | Ad-hoc | 8 | 3 TB | General Purpose | <1 min, TP90 <15s | 16 |
204
200
 
205
201
  #### Load Isolation with Multiple Clusters
206
202
 
@@ -211,3 +207,5 @@ The following table provides sample specifications across several dimensions —
211
207
  * Periodic ELT tasks and online business report queries should use separate clusters for isolation. Use General Purpose for ELT and Analytics for reports to take full advantage of each type's performance characteristics. This also makes it easy to configure different auto-suspend times — set a short auto-suspend (e.g., 1 minute) for the ELT cluster, and a slightly longer one (e.g., 5 minutes) for the reporting cluster.
212
208
 
213
209
  * Isolate clusters running large jobs from those running small jobs. Configure a larger cluster for large jobs to ensure sufficient runtime, and a smaller cluster for small jobs to avoid underutilization and resource waste.
210
+
211
+ ^
@@ -1,121 +1,106 @@
1
1
  # Lakehouse Job History
2
2
 
3
- Through the job history page, you can view detailed information about queries executed in the Lakehouse account over the past 7 days. This page displays a list of historical jobs, including tasks submitted using your own jdbc jar, studio temporary queries, and studio periodic scheduling tasks.
3
+ Through the job history page, you can view detailed information about queries executed in the Lakehouse account over the past 7 days. This page displays a list of historical jobs, including tasks submitted using a JDBC jar, Studio ad-hoc queries, and Studio scheduled tasks.
4
4
 
5
5
  ## Job History
6
6
 
7
- ### View History
8
-
9
- Click on Compute in the left navigation bar, then click on Job History to enter the job history page:
10
-
11
- ![](.topwrite/assets/image_1740557896004.png)
12
-
13
- ## Job History Search
14
- - Supports searching by job ID generated by SQL jobs
15
- - Supports searching by status
16
- - setup: SQL initialization phase, including operations such as compilation and optimization
17
- - resuming_cluster: Waiting for vc to start, if it takes a long time, please contact technical support
18
- - queued: Job is queued waiting for resources, if it takes a long time, check if vc resources are fully occupied, consider expanding vc
19
- - running: SQL data processing time, you can click on diagnostics to view the time taken by each operator
20
- - finish: Job has ended, some jobs may have a cleanup process in this stage, which can take time
21
- - Supports filtering by duration
22
- - Supports filtering by time, currently up to 10,000 jobs from the past seven days can be queried
23
- - Supports filtering by the name of the cluster where the job ran
24
- - Supports filtering by the job submitter
25
- - Supports filtering by queryTag
26
- Users can use the command set query_tag="" to indicate the source of the job and tag the job when submitting SQL. This makes it easy to filter in this interface
27
- - Set query_tag in the studio interface, select two days of SQL to execute simultaneously
28
- ```SQL
29
- set query_tag="test";
30
- select 1;
31
- ```
32
- - Add `query_tag` in the JDBC URL, so each submitted SQL will carry the `query_tag` to indicate the source, which can be filtered on the page.
33
- ```text
34
- jdbc:clickzetta://xxxx.api.singdata.com/default?schema=public&query_tag=test
35
-
36
- ```
37
- - Set query_tag in python code
38
- ```python
39
- # Establish connection
40
- conn = connect(username='',
41
- password='',
42
- service='api.singdata.com',
43
- instance='jnsxwfyr',
44
- workspace='', schema='public', vcluster='default')
45
- # Execute SQL
46
- cursor = conn.cursor()
47
- my_param["hints"]["query_tag"] ="test"
48
- cursor.execute('select 1;', parameters=my_param)
49
-
50
- ```
51
- - The default Schema for the job, the context in which the submitted SQL runs, can be obtained through select current_schema().
7
+ ## View History
8
+
9
+ Click "Compute" in the left navigation bar, then click "Job History" to enter the job history page:
10
+
11
+ ^
12
+
13
+ ## Interface
14
+
15
+ * Supports searching by job ID generated by SQL jobs
16
+ * Supports searching by status
17
+ * Initializing: SQL initialization phase, including operations such as compilation and optimization
18
+ * Cluster Starting: Waiting for the Virtual Cluster to start. If it takes a long time to start, please contact technical support
19
+ * Queued: Job is queued waiting for resources. If it takes a long time, check whether the Virtual Cluster resources are fully occupied and consider scaling up the Virtual Cluster
20
+ * Running: SQL data processing time. You can click into diagnostics to view the time taken by each operator
21
+ * Succeeded: Job has ended. Some jobs may have a cleanup process at this stage, which can take time
22
+ * Supports filtering by duration
23
+ * Supports filtering by time. Currently, up to 10,000 jobs from the past seven days can be queried
24
+ * Supports filtering by the name of the cluster where the job ran
25
+ * Supports filtering by the job submitter
26
+ * Supports filtering by queryTag
27
+ Users can use the command `set query_tag=""` when submitting SQL to indicate the job source and tag the job, making it easy to filter in this interface.
28
+ * Set query\_tag in the Studio interface, select two SQL statements and execute simultaneously:
29
+ ```SQL
30
+ set query_tag="test";
31
+ select 1;
32
+ ```
33
+ * Add query\_tag in the JDBC URL so that every submitted SQL carries the query\_tag to indicate the source, which can be filtered on the page:
34
+ ```text
35
+ jdbc:clickzetta://demo_instance.cn-shanghai-alicloud.api.clickzetta.com/default?schema=public&query_tag=test
36
+
37
+ ```
38
+ * Set query\_tag in Python code:
39
+ ```python
40
+ # Establish connection
41
+ conn = connect(username='',
42
+ password='',
43
+ service='cn-shanghai-alicloud.api.clickzetta.com',
44
+ instance='demo_instance',
45
+ workspace='', schema='public', vcluster='DEFAULT')
46
+ # Execute SQL
47
+ cursor = conn.cursor()
48
+ my_param["hints"]["query_tag"] ="test"
49
+ cursor.execute('select 1;', parameters=my_param)
50
+
51
+ ```
52
+ * The default Schema for the job is the context in which the submitted SQL runs, which can be obtained through `select current_schema()`.
52
53
 
53
54
  ## Job Details
54
55
 
55
56
  Enter the job details to view detailed information about the job.
56
57
 
57
- ![](.topwrite/assets/image_1740557994319.png)
58
-
59
- **Field Information Explained as Follows**
60
-
61
- | | |
62
- | ------ | -------------------------------------------------------------------------------------------------------------------- |
63
- | Field | Description |
64
- | Status | Indicates the status of the job, currently includes&#xA;Ready to execute: The job is being compiled and other operations&#xA;Waiting to execute: Waiting for resources&#xA;Executing: Running, the job is processing data&#xA;Execution successful: The job ran successfully&#xA;Execution terminated: The job was terminated&#xA;Execution failed: The job failed to run |
65
- | Submit Time | The time the server received the request to start |
66
- | End Time | The time the job ended |
67
- | Duration | Includes compilation optimization, starting computing resources, queuing, and job running time |
68
- | Owner | The user who submitted the job |
69
- | Spill to Disk | The size of the data spilled to disk, this parameter can help understand if the performance is slow due to insufficient cluster memory resources |
70
- | Read from Cache | In the analytical cluster, the already read data will be cached first to speed up computation |
71
- | Output Records | The number of output records of the job |
72
- | Consumption | Refers to the computing resources used by the job, excluding the computing resources while the job is waiting for IO. |
73
- | Cluster | The cluster used to run the job |
58
+ ^
74
59
 
75
60
  ### Stage Interface
76
61
 
77
- SQL execution is divided into each stage, and stages are executed from bottom to top. Clicking on a specific stage will show the specific time consumption of each stage.
62
+ SQL execution is divided into stages, which are executed from bottom to top. Clicking on a specific stage will show the specific time consumption of each stage.
78
63
 
79
- ![](.topwrite/assets/image_1740558003358.png)
64
+ ^
80
65
 
81
66
  **Field information is explained as follows**
82
67
 
83
- | | |
84
- | ---------- | --------------------------------------------------------------------------------- |
85
- | Field | Description |
86
- | Status | Indicates the status of the job, currently includes: &#xA;Waiting for execution: waiting for resources&#xA;Executing: running, the job is processing data&#xA;Execution successful: job ran successfully&#xA;Execution failed: job ran unsuccessfully |
87
- | Start Time | The start time of the stage, stages in the analytical cluster will run simultaneously |
88
- | End Time | The end time of the job |
89
- | Duration | The duration of the stage running |
90
- | Spilled to Disk | The size of the data spilled to disk, this parameter can help understand if the slow performance is due to insufficient memory resources in the cluster |
91
- | Read from Cache | In the analytical cluster, the already read data will be cached first. This speeds up computation |
92
- | Contains Operator | Stage |
68
+ | | |
69
+ | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
70
+ | Field | Description |
71
+ | Status | Indicates the status of the job. Currently includes:&#xA;Queued: Waiting for resources&#xA;Running: Running, the job is processing data&#xA;Succeeded: Job ran successfully&#xA;Failed: Job ran unsuccessfully |
72
+ | Start Time | The start time of the stage. In AP clusters, stages run in parallel |
73
+ | End Time | The end time of the job |
74
+ | Duration | The duration of the stage running |
75
+ | Spill to Disk | The size of the data spilled to disk. This parameter indicates whether slow performance is due to insufficient cluster memory resources |
76
+ | Read from Cache | In AP clusters, data that has already been read is cached first to accelerate computation |
77
+ | Contains Operators | Stage |
93
78
 
94
79
  ### Operator Interface
95
80
 
96
- In the operator interface, you can view the type of each operator and the upstream and downstream dependencies. An operator is a concept in the query plan that represents the specific steps and strategies for executing a query, including data access, transformation, and update.
81
+ In the operator interface, you can view the type of each operator and its upstream and downstream dependencies. An operator is a concept in the query plan that represents the specific steps and strategies for executing a query, including data access, transformation, and updates.
97
82
 
98
- ![](.topwrite/assets/image_1740558011631.png)
83
+ ^
99
84
 
100
85
  **Field information is explained as follows**
101
86
 
102
- | | |
103
- | ------- | ------------------------------------- |
104
- | Belongs to Stage | The stage corresponding to the operator |
105
- | Spilled to Disk | The size of the data spilled to disk, this parameter can help understand if the slow performance is due to insufficient memory resources in the cluster |
106
- | Read from Cache | In the analytical cluster, the already read data will be cached first. This speeds up computation |
107
- | Total Rows | The data currently processed by the operator |
108
- | Time Consumed | operate will run concurrently, currently showing the average time of operate |
109
- | Field | Corresponds to the SQL script, corresponding field |
87
+ | Field | Description |
88
+ | :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------- |
89
+ | Belongs to Stage | The stage corresponding to the operator |
90
+ | Spill to Disk | The size of the data spilled to disk. This parameter indicates whether slow performance is due to insufficient cluster memory resources |
91
+ | Read from Cache | In AP clusters, data that has already been read is cached first to accelerate computation |
92
+ | Total Rows | The data currently processed by the operator |
93
+ | Duration | Operators run concurrently. The displayed value is the average time of the operator |
94
+ | Field | Corresponds to the fields in the SQL script |
110
95
 
111
96
  ### Table Mode
112
97
 
113
- Supports stage switching table mode
98
+ Supports switching stages to table mode
99
+
100
+ ^
114
101
 
115
- ![](.topwrite/assets/image_1740558022227.png)
102
+ Supports switching operators to table mode
116
103
 
117
- Supports operator switching table mode
118
- ![](.topwrite/assets/image_1740558027758.png)
119
104
 
120
105
  ## Analyzing Failed Jobs
121
106
 
@@ -124,55 +109,56 @@ When a job fails to run, you can view the reason for the failure through the det
124
109
  ### SQL Syntax Error
125
110
 
126
111
  **Error Example**:
112
+
127
113
  ```SQL
128
114
  Execute select * from ttt;
129
115
  Error: CZLH-42000:[1,15] Semantic analysis exception - table or view not found - uat_demo_project.`public`.ttt
130
116
  ```
131
- **Error Code CZLH-42000** indicates an error related to SQL language parsing. [1,15] indicates that the error occurred on line 1, column 15 of the SQL. In this example, the reason is that the `ttt` table does not exist. Please note that if you submit tasks through Studio development, the job editor in development will include descriptive information, which is not submitted to the SQL engine. Therefore, the correct line number should subtract the preceding descriptive information. You can view the correct positional relationship through the job content in the job details or find the error location in the SQL editor.
132
117
 
118
+ **Error code CZLH-42000** indicates an error related to SQL language parsing. `[1,15]` means the error occurred at line 1, column 15 of the SQL. In this example, the reason is that the `ttt` table does not exist.
133
119
 
134
120
  ## Analyzing Slow Running Jobs
135
121
 
136
- Slow running jobs may encounter issues during the compilation phase, the virtual cluster startup phase, or the execution phase. Below are the analyses and solutions for these phases:
122
+ Slow running jobs may encounter issues during the compilation phase, the Virtual Cluster startup phase, or the execution phase. Below are analyses and solutions for these phases:
137
123
 
138
124
  ### Compilation Phase
139
125
 
140
- **Issue**: Job status shows "Preparing to execute".
126
+ **Issue**: Job status shows "Initializing".
141
127
 
142
128
  **Possible Causes and Solutions**:
143
129
 
144
- 1. **Complex execution plan**: The job may require a long time to optimize. Please be patient; it should not take more than 10 minutes under normal circumstances.
145
- 2. **Compilation resources are fully occupied**: If you submit a large number of SQLs at once, it may lead to insufficient compilation resources. Please reduce the frequency of SQL submissions. If you have a large number of high-concurrency query requirements, please submit a ticket for assistance.
130
+ 1. **Complex execution plan**: The job may require a long time to optimize. Please be patient it should not take more than 10 minutes under normal circumstances.
131
+ 2. **Compilation resources are fully occupied**: If you submit a large number of SQL statements at once, it may lead to insufficient compilation resources. Please reduce the frequency of SQL submissions. If you have a large number of high-concurrency query requirements, please submit a ticket for assistance.
146
132
 
147
133
  ### Virtual Cluster Startup Phase
148
134
 
149
- **Issue**: Job status shows "Cluster starting". The current status is that the computing cluster is being started.
135
+ **Issue**: Job status shows "Cluster Starting". (The current status is that the compute cluster is being started)
150
136
 
151
- ### Waiting Execution Phase
137
+ ### Queued Phase
152
138
 
153
- **Issue**: Job status shows "Waiting to execute".
139
+ **Issue**: Job status shows "Queued".
154
140
 
155
141
  **Possible Causes and Solutions**:
156
142
 
157
- **Insufficient computing resources in the current cluster**: You can filter and view the currently running jobs through the interface. A large number of running jobs may be causing resource waiting. You can expand resources or terminate large running jobs to free up resources.
143
+ 1. **Insufficient computing resources in the current cluster**: You can filter and view the currently running jobs through the interface. A large number of running jobs may be causing resource queuing. You can scale up resources, or terminate large running jobs to free up resources.
158
144
 
159
145
  ### Execution Phase
160
146
 
161
- **Issue**: Job status shows "Running". At this stage, you can click to view the execution status of each operator.
147
+ **Issue**: Job status shows "Running". At this stage you can click in to view the execution status of each operator.
162
148
 
163
149
  **Possible Causes and Solutions**:
164
150
 
165
151
  1. **Waiting for resources**: The job may be waiting for resource allocation. Please check the resource usage. If necessary, please submit a ticket for assistance.
166
- 2. **Slow data reading speed**: The job may be executing slowly due to slow data reading speed. Please check the data storage and reading strategy. If necessary, optimize the data storage structure or adjust the reading strategy.
152
+ 2. **Slow data reading speed**: The job may be executing slowly due to slow data reading. Please check the data storage and reading strategy. If necessary, optimize the data storage structure or adjust the reading strategy.
167
153
 
168
- ## Viewing Execution Plan and Operator Details
154
+ ### Share for Diagnosis
169
155
 
170
- During the execution phase, you can click on the executing operator to query specific steps and strategies. For example, moving the mouse over a specific operator can correspond to the specific operator executed in the SQL. This helps you identify which specific operator is causing the job to run slowly.
156
+ After entering the job details page, click the "Share" button to share the job with a third party. Recipients can view the content without logging in.
157
+ \:-: ![](.topwrite/assets/image_1740557522715.png =774)
158
+ If you have any questions or need further assistance, please copy the link above and contact the Lakehouse team. We will diagnose and resolve the issue for you.
159
+
160
+ ## Viewing Execution Plan and Operator Details
171
161
 
172
- ![](.topwrite/assets/image_1740558276614.png)
173
- ## Share for Diagnosis
162
+ During the execution phase, you can click on a specific operator to query its execution steps and strategies. For example, moving the mouse over a specific operator will highlight the corresponding operator in the SQL. This helps you identify which specific operator is causing the job to run slowly.
174
163
 
175
- After entering the assignment details page, click the “Share” button to share the assignment with others. Recipients can view the content without logging in.
176
- ![](.topwrite/assets/image_1740558353576.png)
177
- If you need to stop sharing, you can revoke access to the link at any time.
178
- If you have any questions or need further assistance, please copy the link above and contact the Lakehouse team. We will diagnose and resolve the issue for you.
164
+ ^