@clickzetta/cz-cli-darwin-x64 0.5.16 → 0.5.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (243) hide show
  1. package/bin/cz-cli +0 -0
  2. package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
  3. package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
  4. package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
  5. package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
  6. package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
  7. package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
  8. package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
  9. package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
  10. package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
  11. package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
  12. package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
  13. package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
  14. package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
  15. package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
  16. package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
  17. package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
  18. package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
  19. package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
  20. package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
  21. package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
  22. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
  23. package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
  24. package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
  25. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
  26. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
  27. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
  28. package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
  29. package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
  30. package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
  31. package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
  32. package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
  33. package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
  34. package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
  35. package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
  36. package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
  37. package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
  38. package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
  39. package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
  40. package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
  41. package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
  42. package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
  43. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
  44. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
  45. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
  46. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
  47. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
  48. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
  49. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
  50. package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
  51. package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
  52. package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
  53. package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
  54. package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
  55. package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
  56. package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
  57. package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
  58. package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
  59. package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
  60. package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
  61. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
  62. package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
  63. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
  64. package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
  65. package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
  66. package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
  67. package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
  68. package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
  69. package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
  70. package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
  71. package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
  72. package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
  73. package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
  74. package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
  75. package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
  76. package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
  77. package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
  78. package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
  79. package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
  80. package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
  81. package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
  82. package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
  83. package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
  84. package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
  85. package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
  86. package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
  87. package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
  88. package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
  89. package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
  90. package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
  91. package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
  92. package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
  93. package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
  94. package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
  95. package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
  96. package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
  97. package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
  98. package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
  99. package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
  100. package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
  101. package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
  102. package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
  103. package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
  104. package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
  105. package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
  106. package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
  107. package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
  108. package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
  109. package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
  110. package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
  111. package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
  112. package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
  113. package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
  114. package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
  115. package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
  116. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
  117. package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
  118. package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
  119. package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
  120. package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
  121. package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
  122. package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
  123. package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
  124. package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
  125. package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
  126. package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
  127. package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
  128. package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
  129. package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
  130. package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
  131. package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
  132. package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
  133. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
  134. package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
  135. package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
  136. package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
  137. package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
  138. package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
  139. package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
  140. package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
  141. package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
  142. package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
  143. package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
  144. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
  145. package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
  146. package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
  147. package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
  148. package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
  149. package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
  150. package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
  151. package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
  152. package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
  153. package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
  154. package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
  155. package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
  156. package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
  157. package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
  158. package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
  159. package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
  160. package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
  161. package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
  162. package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
  163. package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
  164. package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
  165. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
  166. package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
  167. package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
  168. package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
  169. package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
  170. package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
  171. package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
  172. package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
  173. package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
  174. package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
  175. package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
  176. package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
  177. package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
  178. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
  179. package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
  180. package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
  181. package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
  182. package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
  183. package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
  184. package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
  185. package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
  186. package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
  187. package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
  188. package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
  189. package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
  190. package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
  191. package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
  192. package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
  193. package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
  194. package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
  195. package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
  196. package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
  197. package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
  198. package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
  199. package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
  200. package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
  201. package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
  202. package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
  203. package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
  204. package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
  205. package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
  206. package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
  207. package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
  208. package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
  209. package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
  210. package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
  211. package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
  212. package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
  213. package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
  214. package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
  215. package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
  216. package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
  217. package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
  218. package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
  219. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
  220. package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
  221. package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
  222. package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
  223. package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
  224. package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
  225. package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
  226. package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
  227. package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
  228. package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
  229. package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
  230. package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
  231. package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
  232. package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
  233. package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
  234. package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
  235. package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
  236. package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
  237. package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
  238. package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
  239. package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
  240. package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
  241. package/package.json +1 -1
  242. package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
  243. package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
@@ -41,24 +41,24 @@ The source database requires certain parameter settings and permission grants to
41
41
 
42
42
  Note: Modifying the following parameters requires restarting the PostgreSQL server to take effect.
43
43
 
44
- | | | |
45
- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
46
- | Configuration | Description | Default Value (Unit) |
47
- | wal\_level | WAL level. `logical` is required for real-time sync. `replica` supports WAL archiving and replication. `minimal` keeps only what is needed to recover from a crash. | replica |
48
- | max\_replication\_slots | Maximum number of replication slots allowed on the server. | 10 |
49
- | max\_wal\_senders | Maximum number of WAL sender processes that can run simultaneously, corresponding to the number of concurrent real-time sync tasks. | 10 |
50
- | max\_slot\_wal\_keep\_size | Size of WAL retained per slot. `-1` means unlimited. | -1 (MB) |
51
- | wal\_sender\_timeout | Replication connections idle longer than this value will be terminated. | 60000 (ms) |
44
+ | | | |
45
+ | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
46
+ | Configuration | Description | Default Value (Unit) |
47
+ | wal\_level | WAL level. `logical` is required for real-time sync. `replica` supports WAL archiving and replication. `minimal` keeps only what is needed to recover from a crash. | replica |
48
+ | max\_replication\_slots | Maximum number of replication slots allowed on the server. | 10 |
49
+ | max\_wal\_senders | Maximum number of WAL sender processes that can run simultaneously, corresponding to the number of concurrent real-time sync tasks. | 10 |
50
+ | max\_slot\_wal\_keep\_size | Size of WAL retained per slot. `-1` means unlimited. | -1 (MB) |
51
+ | wal\_sender\_timeout | Replication connections idle longer than this value will be terminated. | 60000 (ms) |
52
52
 
53
53
  **MySQL**
54
54
 
55
- | | | | |
56
- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- | ------------------------------------------------ |
57
- | Attribute | Description | Required Setting | Query Method |
58
- | log\_bin | Whether binlog is enabled. | ON | SHOW GLOBAL VARIABLES LIKE 'log\_bin' |
59
- | binlog\_format | Binlog format. `statement` records SQL (compact but can cause sync errors with non-deterministic functions). `row` records full before/after row images (accurate but higher volume). `mixed` lets MySQL choose automatically. | ROW | SHOW GLOBAL VARIABLES LIKE 'binlog\_format' |
60
- | binlog\_row\_image | Whether full before/after row images are recorded. | FULL (record all fields in both images) | SHOW GLOBAL VARIABLES LIKE 'binlog\_row\_image' |
61
- | binlog\_expire\_logs\_seconds | Binlog automatic cleanup interval. | Configure based on business needs; 86400 seconds (1 day) or more recommended. | |
55
+ | | | | |
56
+ | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | ----------------------------------------------- |
57
+ | Attribute | Description | Required Setting | Query Method |
58
+ | log\_bin | Whether binlog is enabled. | ON | SHOW GLOBAL VARIABLES LIKE 'log\_bin' |
59
+ | binlog\_format | Binlog format. `statement` records SQL (compact but can cause sync errors with non-deterministic functions). `row` records full before/after row images (accurate but higher volume). `mixed` lets MySQL choose automatically. | ROW | SHOW GLOBAL VARIABLES LIKE 'binlog\_format' |
60
+ | binlog\_row\_image | Whether full before/after row images are recorded. | FULL (record all fields in both images) | SHOW GLOBAL VARIABLES LIKE 'binlog\_row\_image' |
61
+ | binlog\_expire\_logs\_seconds | Binlog automatic cleanup interval. | Configure based on business needs; 86400 seconds (1 day) or more recommended. | |
62
62
 
63
63
  #### Database Permission Configuration
64
64
 
@@ -68,11 +68,11 @@ Appropriate permissions must be configured on each source database to ensure cha
68
68
 
69
69
  When executing grant SQL statements, ensure the executing account itself has the ability to grant those permissions — using an administrator account is recommended. Execute the grants for all scenarios listed below to ensure the task runs smoothly.
70
70
 
71
- **Scenario: Task configuration (fetching metadata: schema list, table list, field list)**
71
+ **Scenario: Task configuration (fetching metadata: schema list, table list, field list**)
72
72
 
73
73
  Required permissions:
74
74
 
75
- > SELECT (on information_schema and the tables to be inspected)
75
+ > SELECT (on information\_schema and the tables to be inspected)
76
76
 
77
77
  Grant statements:
78
78
 
@@ -100,7 +100,7 @@ Grant statements:
100
100
  CREATE ROLE <name> REPLICATION LOGIN;
101
101
  ```
102
102
 
103
- **Scenario: Sync historical full data (optional)**
103
+ **Scenario: Sync historical full data (optional**)
104
104
 
105
105
  * Required permissions:
106
106
 
@@ -138,7 +138,7 @@ Grant statement:
138
138
 
139
139
  When executing grant SQL statements, ensure the executing account has the GRANT OPTION privilege — using a superuser such as root is recommended. Execute the grants for all scenarios listed below.
140
140
 
141
- **Scenario: Task configuration (fetching metadata: database list, table list, field list)**
141
+ **Scenario: Task configuration (fetching metadata: database list, table list, field list**)
142
142
 
143
143
  Required permissions:
144
144
 
@@ -210,13 +210,13 @@ Follow these steps in order:
210
210
 
211
211
  * Extended fields currently support: source server name, database name, schema name, and table name.
212
212
 
213
- ![](.topwrite/assets/image_1740314767939.png =680)
213
+ ^
214
214
 
215
215
  ### General: If the same primary key exists in different sharded source tables, how do I avoid write conflicts in the target table?
216
216
 
217
217
  * Enable the extended fields feature in the task configuration, and include those extended fields in the composite primary key of the target table.
218
218
 
219
- ![](.topwrite/assets/image_1740314789823.png =680)
219
+ ^
220
220
 
221
221
  ### General: If the sharded source tables have mostly — but not exactly — the same field structure, how do I configure the sync task?
222
222
 
@@ -314,7 +314,7 @@ There are several options:
314
314
 
315
315
  * `step1.taskmanager.memory.task.off-heap.size` — for example, set to `500M` (default: `256M`)
316
316
 
317
- ![](.topwrite/assets/image_1740314863947.png =680)
317
+ ^
318
318
 
319
319
  ### Emergency Maintenance: The task has failed and I cannot resolve it on my own — how do I get help from Singdata?
320
320
 
@@ -334,31 +334,31 @@ There are several options:
334
334
 
335
335
  After a task starts, it goes through three phases: initialization, full sync, and incremental sync. You can view the status of these phases in the instance monitoring area.
336
336
 
337
- ![](.topwrite/assets/image_1740314893402.png =600)
337
+ ^
338
338
 
339
339
  #### Metric Monitoring
340
340
 
341
- | | |
342
- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
343
- | Metric | Description |
344
- | Data read | Number of records read from the source during the measurement period. |
345
- | Data written | Number of records written to the target during the measurement period. |
346
- | Avg. read rate | Average read rate during the measurement period (total records read / period duration). |
347
- | Avg. write rate | Average write rate during the measurement period (total records written / period duration). |
348
- | Failover count | Number of failovers during the measurement period. This reflects the operational stability of the sync service itself. |
341
+ | | |
342
+ | --------------- | ---------------------------------------------------------------------------------------------------------------------- |
343
+ | Metric | Description |
344
+ | Data read | Number of records read from the source during the measurement period. |
345
+ | Data written | Number of records written to the target during the measurement period. |
346
+ | Avg. read rate | Average read rate during the measurement period (total records read / period duration). |
347
+ | Avg. write rate | Average write rate during the measurement period (total records written / period duration). |
348
+ | Failover count | Number of failovers during the measurement period. This reflects the operational stability of the sync service itself. |
349
349
 
350
- ![](.topwrite/assets/image_1740314904374.png =600)
350
+ ^
351
351
 
352
352
  #### Per-Table Sync Progress
353
353
 
354
- | | |
355
- | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
356
- | Metric | Description |
357
- | Latest read position | The write time of the most recent record written to the target table, used as a proxy for the current read position. |
358
- | Latest update time | The last time a record was written to the target table. |
359
- | Data latency | The time between a transaction committing on the source and the data becoming visible on the target. |
354
+ | | |
355
+ | -------------------- | -------------------------------------------------------------------------------------------------------------------- |
356
+ | Metric | Description |
357
+ | Latest read position | The write time of the most recent record written to the target table, used as a proxy for the current read position. |
358
+ | Latest update time | The last time a record was written to the target table. |
359
+ | Data latency | The time between a transaction committing on the source and the data becoming visible on the target. |
360
360
 
361
- ![](.topwrite/assets/image_1740314933658.png =600)
361
+ ^
362
362
 
363
363
  ### Configuring Task Monitoring and Alerts
364
364
 
@@ -381,25 +381,25 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
381
381
 
382
382
  3. Add a new webhook configuration in the product — select Lark or WeCom as the channel and enter the bot's webhook URL.
383
383
 
384
- ![](.topwrite/assets/image_1740315003712.png =680)
384
+ ^
385
385
 
386
386
  4. Enable the webhook in the notification policy.
387
387
 
388
- ![](.topwrite/assets/image_1740315034077.png =680)
388
+ ^
389
389
 
390
390
  5. In the monitoring rule, select the notification policy with the webhook enabled, then choose the webhook configuration you just created.
391
391
 
392
- ![](.topwrite/assets/image_1740315043022.png =680)
392
+ ^
393
393
 
394
394
  #### Configuring Exception Monitoring for Real-Time Data Integration Tasks
395
395
 
396
396
  1. **Task failover alert**: create a new monitoring rule and select "Multi-table real-time sync job failover" as the monitoring event. You can add filter attributes such as workspace or task name. If no filter is added, all multi-table real-time sync tasks in the instance are monitored by default.
397
397
 
398
- ![](.topwrite/assets/image_1740315064690.png =680)
398
+ ^
399
399
 
400
400
  2. **Task stop alert**: create a new monitoring rule and select "Multi-table real-time sync task run failure" as the monitoring event. You can add filter attributes such as workspace or task name. If no filter is added, all multi-table real-time sync tasks in the instance are monitored by default.
401
401
 
402
- ![](.topwrite/assets/image_1740315072098.png =460)
402
+ ^
403
403
 
404
404
  #### Configuring In-Task Exception Monitoring
405
405
 
@@ -409,21 +409,21 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
409
409
 
410
410
  * You can configure an alert for when a table is added to the blacklist, covering two scenarios: Schema Evolution failure and a single field value exceeding the 10 MB size limit.
411
411
 
412
- ![](.topwrite/assets/image_1740315081002.png =460)
412
+ ^
413
413
 
414
414
  * Schema Evolution failure alert: create a new monitoring rule and select "Multi-table real-time sync target table change failure."
415
415
 
416
- ![](.topwrite/assets/image_1740315087204.png =460)
416
+ ^
417
417
 
418
418
  #### Configuring Maximum Latency Monitoring for Real-Time Data Integration Tasks
419
419
 
420
420
  1. **End-to-end sync latency**: create a new monitoring rule and select "Multi-table real-time sync latency." You can add filter attributes such as workspace or task name. If no filter is added, all multi-table real-time sync tasks in the instance are monitored by default.
421
421
 
422
- ![](.topwrite/assets/image_1740315096380.png =680)
422
+ ^
423
423
 
424
424
  2. **Read checkpoint latency**: create a new monitoring rule and select "Multi-table real-time sync read checkpoint latency." You can add filter attributes such as workspace or task name. If no filter is added, all multi-table real-time sync tasks in the instance are monitored by default.
425
425
 
426
- ![](.topwrite/assets/image_1740315103745.png =680)
426
+ ^
427
427
 
428
428
  ## Common Error Troubleshooting
429
429
 
@@ -443,11 +443,11 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
443
443
 
444
444
  * Query the current binlog file and position with `SHOW MASTER STATUS`:
445
445
 
446
- ![](.topwrite/assets/image_1740315126085.png =460)
446
+ ^
447
447
 
448
448
  * Restart the sync task using the file and position from the result.
449
449
 
450
- ![](.topwrite/assets/image_1740315139248.png =460)
450
+ ^
451
451
 
452
452
  * If you need to recover the data that was missed, resync the affected tables.
453
453
 
@@ -507,7 +507,7 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
507
507
  * Cause
508
508
 
509
509
  * The total encoded length of the primary key fields in the source table exceeds the default limit of 128 bytes.
510
- * In a merge scenario, multiple extended fields (e.g., server_id, database, schema, table) are included in the composite primary key to prevent conflicts across sharded tables — and the total encoded key length exceeds 128 bytes.
510
+ * In a merge scenario, multiple extended fields (e.g., server\_id, database, schema, table) are included in the composite primary key to prevent conflicts across sharded tables — and the total encoded key length exceeds 128 bytes.
511
511
 
512
512
  * Resolution
513
513
 
@@ -567,7 +567,7 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
567
567
 
568
568
  * Problem
569
569
 
570
- * A table's status automatically changes to "sync stopped." The tooltip on the table object shows errors such as `pk column different`, `pk column type mismatch`, or `invalid modify column`.
570
+ * A table's status automatically changes to "sync stopped." The tooltip on the table object shows errors such as `pk column different`, `pk column type mismatch`, or `invalid modify column`.
571
571
 
572
572
  * Cause
573
573
 
@@ -586,3 +586,5 @@ In the monitoring and alerting module, you can configure monitoring rules to tra
586
586
  * To prevent write conflicts on the target, only source tables with primary key (PK) fields are supported. Tables without a primary key cannot be synced.
587
587
  * The sync task will automatically create target tables. To ensure stable task execution and data correctness, avoid manually creating, modifying, or deleting target tables unless absolutely necessary.
588
588
  * Schema Evolution supports adding and removing columns on the source. Changing column types and automatically adding new tables are not currently supported.
589
+
590
+ ^
@@ -1,3 +1,5 @@
1
+ # Navicat Connection Guide
2
+
1
3
  Navicat (full name Navicat Premium) is a powerful database management tool widely used for database development, management, and maintenance. This article describes how to use Navicat to connect to Lakehouse via the MySQL protocol.
2
4
 
3
5
  # Prerequisites
@@ -8,26 +8,26 @@ Think of a Dynamic Table as an "automatically running data processing pipeline"
8
8
 
9
9
  ## Comparison with Other Table Types
10
10
 
11
- | Aspect | Dynamic Table | Materialized View | View | Table |
12
- |--------|--------|----------|----------|--------|
13
- | Definition | Efficient tool focused on data processing | Special view that pre-computes and stores query results | Stores no data; only saves the query definition | General-purpose storage object |
14
- | Data Storage | Yes | Yes | No | Yes |
15
- | Data Freshness | Adjustable; emphasizes processing flexibility | Requires near-real-time updates (ensures accurate rewriting) | Always latest data on each query | Determined by write timing |
16
- | Update Mechanism | Incremental computation; only processes changed data | Periodic full/incremental refresh | Real-time computation on each query | Manual DML |
17
- | Query Optimization | Does not rely on optimizer auto-rewriting | **Optimizer automatically identifies and uses pre-stored results** | Recomputed on every query | — |
18
- | DML Support | No | No | No | Yes |
19
- | Primary Use | Data processing pipelines (ODS → DWD → DWS) | Query acceleration, result reuse | Logic encapsulation for simple queries | Raw data storage |
20
- | Operations | Supports adding columns, version rollback | No complex operations | No complex operations | Flexible |
11
+ | Aspect | Dynamic Table | Materialized View | View | Table |
12
+ | ------------------ | ---------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------- | ------------------------------ |
13
+ | Definition | Efficient tool focused on data processing | Special view that pre-computes and stores query results | Stores no data; only saves the query definition | General-purpose storage object |
14
+ | Data Storage | Yes | Yes | No | Yes |
15
+ | Data Freshness | Adjustable; emphasizes processing flexibility | Requires near-real-time updates (ensures accurate rewriting) | Always latest data on each query | Determined by write timing |
16
+ | Update Mechanism | Incremental computation; only processes changed data | Periodic full/incremental refresh | Real-time computation on each query | Manual DML |
17
+ | Query Optimization | Does not rely on optimizer auto-rewriting | **Optimizer automatically identifies and uses pre-stored results** | Recomputed on every query | — |
18
+ | DML Support | No | No | No | Yes |
19
+ | Primary Use | Data processing pipelines (ODS → DWD → DWS) | Query acceleration, result reuse | Logic encapsulation for simple queries | Raw data storage |
20
+ | Operations | Supports adding columns, version rollback | No complex operations | No complex operations | Flexible |
21
21
 
22
22
  **When to use a Dynamic Table**: When you need to automatically compute and store results based on upstream tables. The typical scenario is the ODS → DWD → DWS data processing pipeline. Incremental computation only processes changed data, saving significant compute resources compared to full refreshes.
23
23
 
24
24
  **When not to use a Dynamic Table**:
25
25
 
26
- - Only need transparent acceleration of existing queries → use a Materialized View (the optimizer auto-rewrites queries to use pre-computed results)
27
- - Only need logic encapsulation without storing data → use a View
28
- - Data source is in an external database → use a sync job to write into a regular table; Dynamic Tables can only consume Lakehouse-internal tables
29
- - Need precise minute-level cron scheduling → use a Studio SQL job
30
- - Query contains heavy ORDER BY or complex window functions → incremental computation is limited; use a regular View + scheduling
26
+ * Only need transparent acceleration of existing queries → use a Materialized View (the optimizer auto-rewrites queries to use pre-computed results)
27
+ * Only need logic encapsulation without storing data → use a View
28
+ * Data source is in an external database → use a sync job to write into a regular table; Dynamic Tables can only consume Lakehouse-internal tables
29
+ * Need precise minute-level cron scheduling → use a Studio SQL job
30
+ * Query contains heavy ORDER BY or complex window functions → incremental computation is limited; use a regular View + scheduling
31
31
 
32
32
  ## Incremental Computation Principles
33
33
 
@@ -36,9 +36,9 @@ Dynamic Tables work based on the Lakehouse's MVCC version management mechanism:
36
36
  1. **Version awareness**: On each refresh, the system records the source table's last version offset
37
37
  2. **Delta capture**: Compares the current version with the last version to identify INSERT/UPDATE/DELETE changes
38
38
  3. **Incremental execution**: Executes computation only on changed data; different operators handle changes differently:
39
- - Filter/Project: processes only changed rows
40
- - Join: joins changed rows with historical data from the right table
41
- - Aggregate: merges changed rows with historical aggregation results
39
+ * Filter/Project: processes only changed rows
40
+ * Join: joins changed rows with historical data from the right table
41
+ * Aggregate: merges changed rows with historical aggregation results
42
42
  4. **Result merge**: Merges incremental results into the Dynamic Table via MERGE INTO
43
43
 
44
44
  The system automatically selects incremental or full refresh mode. When the SQL contains operators that don't support incremental processing (e.g., ORDER BY), when the source table change volume is too large, or on the first refresh, the system automatically falls back to full computation.
@@ -47,11 +47,11 @@ The system automatically selects incremental or full refresh mode. When the SQL
47
47
 
48
48
  Dynamic Tables support three scheduling approaches:
49
49
 
50
- | Scheduling Approach | Suitable Scenario | Advantages | Disadvantages |
51
- |---------|---------|------|------|
52
- | DDL-defined refresh interval (`REFRESH INTERVAL`) | Simple scenarios, quick deployment | Simple to use; no external tools needed | No upstream/downstream dependency support; minimum interval is 1 minute |
53
- | Lakehouse Studio scheduling | Multi-layer DT pipelines requiring dependency control | Visual configuration; supports task dependencies (trigger B after A completes); failure/timeout alerts | Minimum interval is 1 minute |
54
- | Third-party scheduling engine | Existing scheduling infrastructure requiring flexible control | No interval restrictions; integrates with existing scheduling systems | Introduces external dependencies; requires self-maintenance |
50
+ | Scheduling Approach | Suitable Scenario | Advantages | Disadvantages |
51
+ | ------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- |
52
+ | DDL-defined refresh interval (`REFRESH INTERVAL`) | Simple scenarios, quick deployment | Simple to use; no external tools needed | No upstream/downstream dependency support; minimum interval is 1 minute |
53
+ | Lakehouse Studio scheduling | Multi-layer DT pipelines requiring dependency control | Visual configuration; supports task dependencies (trigger B after A completes); failure/timeout alerts | Minimum interval is 1 minute |
54
+ | Third-party scheduling engine | Existing scheduling infrastructure requiring flexible control | No interval restrictions; integrates with existing scheduling systems | Introduces external dependencies; requires self-maintenance |
55
55
 
56
56
  > ⚠️ **Note**: The refresh interval must be greater than the duration of a single refresh run; otherwise jobs will queue up. Use `SHOW DYNAMIC TABLE REFRESH HISTORY` to check refresh durations.
57
57
 
@@ -109,9 +109,10 @@ SELECT * FROM dws_category_sales;
109
109
  **Symptom**: Refresh status shows `QUEUED`; data lag keeps growing.
110
110
 
111
111
  **Solution**:
112
- - Use `SHOW DYNAMIC TABLE REFRESH HISTORY` to check refresh durations
113
- - The refresh interval should be 1.5–2x the single refresh duration
114
- - If refresh duration keeps growing, incremental computation may have degraded to full computation
112
+
113
+ * Use `SHOW DYNAMIC TABLE REFRESH HISTORY` to check refresh durations
114
+ * The refresh interval should be 1.5–2x the single refresh duration
115
+ * If refresh duration keeps growing, incremental computation may have degraded to full computation
115
116
 
116
117
  ### Issue 2: Non-Deterministic Functions Cause Inconsistent Results
117
118
 
@@ -120,42 +121,44 @@ SELECT * FROM dws_category_sales;
120
121
  **Symptom**: Function return values differ on each refresh, potentially causing incremental computation results to deviate from expectations.
121
122
 
122
123
  **Solution**:
123
- - Avoid non-deterministic functions in DT definitions
124
- - For day-based filtering, use parameterized DT + `SESSION_CONFIGS()['dt.args.bizdate']`; see [Dynamic Table Parameterized Definitions](dynamic-table-parameters.md)
125
- - If you need to record refresh time, use `CURRENT_TIMESTAMP()` in downstream queries rather than in the DT definition
124
+
125
+ * Avoid non-deterministic functions in DT definitions
126
+ * For day-based filtering, use parameterized DT + `SESSION_CONFIGS()['dt.args.bizdate']`; see [Dynamic Table Parameterized Definitions](dynamic-table-parameters.md)
127
+ * If you need to record refresh time, use `CURRENT_TIMESTAMP()` in downstream queries rather than in the DT definition
126
128
 
127
129
  ### Issue 3: Multi-Layer Pipeline Latency Accumulation
128
130
 
129
- **Problem**: DT_A (5-minute refresh) → DT_B (1-minute refresh); expected DT_B latency is 1 minute.
131
+ **Problem**: DT\_A (5-minute refresh) → DT\_B (1-minute refresh); expected DT\_B latency is 1 minute.
130
132
 
131
- **Symptom**: DT_B actual latency is at least 5 minutes.
133
+ **Symptom**: DT\_B actual latency is at least 5 minutes.
132
134
 
133
135
  **Solution**:
134
- - The upstream DT's refresh frequency determines the minimum latency achievable downstream
135
- - Use Studio task dependencies (trigger B after A completes) to avoid polling waits
136
- - Total pipeline latency = sum of refresh intervals across all layers
136
+
137
+ * The upstream DT's refresh frequency determines the minimum latency achievable downstream
138
+ * Use Studio task dependencies (trigger B after A completes) to avoid polling waits
139
+ * Total pipeline latency = sum of refresh intervals across all layers
137
140
 
138
141
  ## Usage Restrictions
139
142
 
140
- - **Non-deterministic functions not supported**: DT definitions cannot use `RANDOM()`, `CURRENT_TIMESTAMP()`, `CURRENT_DATE()`, or other non-deterministic functions; otherwise incremental refresh results are unpredictable. For time-based filtering, use parameterized DT (`SESSION_CONFIGS()['dt.args.bizdate']`); see [Dynamic Table Parameterized Definitions](dynamic-table-parameters.md).
141
- - **Direct data modification not supported**: You cannot run `UPDATE`, `DELETE`, or `TRUNCATE` on a Dynamic Table. Data in a Dynamic Table can only be updated through the refresh mechanism.
143
+ * **Non-deterministic functions not supported**: DT definitions cannot use `RANDOM()`, `CURRENT_TIMESTAMP()`, `CURRENT_DATE()`, or other non-deterministic functions; otherwise incremental refresh results are unpredictable. For time-based filtering, use parameterized DT (`SESSION_CONFIGS()['dt.args.bizdate']`); see [Dynamic Table Parameterized Definitions](dynamic-table-parameters.md).
144
+ * **Direct data modification not supported**: You cannot run `UPDATE`, `DELETE`, or `TRUNCATE` on a Dynamic Table. Data in a Dynamic Table can only be updated through the refresh mechanism.
142
145
 
143
146
  ## Cost Implications
144
147
 
145
148
  ### Compute Cost
146
149
 
147
- - Each refresh consumes VCluster CRU resources
148
- - Higher refresh frequency means greater CRU consumption:
149
- - `1 DAY`: 1 refresh per day — lowest cost
150
- - `1 HOUR`: 24 refreshes per day
151
- - `1 MINUTE`: 1440 refreshes per day — evaluate carefully
152
- - Incremental refresh saves significantly more resources than full refresh (verify with actual testing)
150
+ * Each refresh consumes VCluster CRU resources
151
+ * Higher refresh frequency means greater CRU consumption:
152
+ * `1 DAY`: 1 refresh per day — lowest cost
153
+ * `1 HOUR`: 24 refreshes per day
154
+ * `1 MINUTE`: 1440 refreshes per day — evaluate carefully
155
+ * Incremental refresh saves significantly more resources than full refresh (verify with actual testing)
153
156
 
154
157
  ### Storage Cost
155
158
 
156
- - Dynamic Tables store computed results and consume storage space
157
- - Time Travel is supported; the default retention is 1 day of historical versions, which increases storage
158
- - Time Travel retention can be adjusted with the following command (range: 0–90 days):
159
+ * Dynamic Tables store computed results and consume storage space
160
+ * Time Travel is supported; the default retention is 1 day of historical versions, which increases storage
161
+ * Time Travel retention can be adjusted with the following command (range: 0–90 days):
159
162
 
160
163
  ```sql
161
164
  ALTER TABLE dws_category_sales SET PROPERTIES ('data_retention_days'='7');
@@ -182,23 +185,23 @@ SHOW DYNAMIC TABLE REFRESH HISTORY WHERE name = 'dws_category_sales' LIMIT 10;
182
185
 
183
186
  Field descriptions:
184
187
 
185
- | Field | Description |
186
- |------|------|
187
- | `workspace_name` | Workspace name |
188
- | `schema_name` | Schema name |
189
- | `name` | Dynamic Table name |
190
- | `virtual_cluster` | Cluster used for the refresh |
191
- | `start_time` | Refresh start time |
192
- | `end_time` | Refresh end time |
193
- | `duration` | Refresh duration |
194
- | `state` | Job state: `setup` / `resuming cluster` / `queued` / `running` / `SUCCEED` / `FAILED` |
195
- | `refresh_trigger` | Trigger type: `MANUAL` (user-triggered, including Studio scheduling) / `LH_SCHEDULED` (Lakehouse auto-scheduling) |
196
- | `suspended_reason` | Reason scheduling is suspended (null when not suspended) |
197
- | `refresh_mode` | Refresh mode: `INCREMENTAL` / `FULL` / `NO_DATA` (no changes) |
198
- | `error_message` | Error message on failure |
199
- | `source_tables` | List of source tables the Dynamic Table depends on (JSON format) |
200
- | `stats` | Incremental refresh stats: `rows_inserted` / `rows_deleted` (values are string type) |
201
- | `job_id` | Job ID; click to view the Job Profile and incremental execution plan |
188
+ | Field | Description |
189
+ | ------------------ | ----------------------------------------------------------------------------------------------------------------- |
190
+ | `workspace_name` | Workspace name |
191
+ | `schema_name` | Schema name |
192
+ | `name` | Dynamic Table name |
193
+ | `virtual_cluster` | Cluster used for the refresh |
194
+ | `start_time` | Refresh start time |
195
+ | `end_time` | Refresh end time |
196
+ | `duration` | Refresh duration |
197
+ | `state` | Job state: `setup` / `resuming cluster` / `queued` / `running` / `SUCCEED` / `FAILED` |
198
+ | `refresh_trigger` | Trigger type: `MANUAL` (user-triggered, including Studio scheduling) / `LH_SCHEDULED` (Lakehouse auto-scheduling) |
199
+ | `suspended_reason` | Reason scheduling is suspended (null when not suspended) |
200
+ | `refresh_mode` | Refresh mode: `INCREMENTAL` / `FULL` / `NO_DATA` (no changes) |
201
+ | `error_message` | Error message on failure |
202
+ | `source_tables` | List of source tables the Dynamic Table depends on (JSON format) |
203
+ | `stats` | Incremental refresh stats: `rows_inserted` / `rows_deleted` (values are string type) |
204
+ | `job_id` | Job ID; click to view the Job Profile and incremental execution plan |
202
205
 
203
206
  ### Modifying and Dropping
204
207
 
@@ -217,7 +220,9 @@ UNDROP TABLE dws_category_sales;
217
220
 
218
221
  ## Related Documentation
219
222
 
220
- - [Dynamic Table Overview](dynamic_table_summary.md) — Core concepts and working principles
221
- - [CREATE DYNAMIC TABLE](create-dynamic-table.md) — Complete syntax
222
- - [Incremental Computation](incremental-computing.md) — Incremental refresh principles
223
- - [Real-Time Pipeline Selection Guide](realtime-pipeline-selection-guide.md) — Choosing between Pipe, Stream, and Dynamic Table
223
+ * [Dynamic Table Overview](dynamic_table_summary.md) — Core concepts and working principles
224
+ * [CREATE DYNAMIC TABLE](create-dynamic-table.md) — Complete syntax
225
+ * [Incremental Computation](incremental-computing.md) — Incremental refresh principles
226
+ * [Real-Time Pipeline Selection Guide](realtime-pipeline-selection-guide.md) — Choosing between Pipe, Stream, and Dynamic Table
227
+
228
+ ^
@@ -63,6 +63,8 @@ ALTER VCLUSTER my_gp_cluster SUSPEND;
63
63
  SHOW VCLUSTERS;
64
64
  ```
65
65
 
66
+ > ⚠️ **Note**: VCluster names are automatically converted to **uppercase** when stored (unlike table names and schema names, which are lowercased). For example, a cluster created as `my_gp_cluster` will have the actual name `MY_GP_CLUSTER`. References are case-insensitive, but `SHOW VCLUSTERS` always displays uppercase names.
67
+
66
68
  ## Cost Implications
67
69
 
68
70
  ### Compute Cost
@@ -0,0 +1,79 @@
1
+ # CreateSession - Create a Conversation Session
2
+
3
+ Create a new conversation session (Session) to organize one or more question-and-answer interactions.
4
+
5
+ ## Interface Description
6
+
7
+ A Session represents a conversation context. After creation, a `sessionId` is returned. Subsequent question requests must be associated with a Session.
8
+
9
+ If the client already has a reusable `sessionId`, this step can be skipped and a question can be submitted directly.
10
+
11
+ ### Usage Notes
12
+
13
+ - A single Session can contain multiple questions.
14
+ - A Session can be reused multiple times; you do not need to create a new one for every question.
15
+
16
+ ## Request Method
17
+
18
+ ```
19
+ POST /open/session/safe_new?tenantId={tenantId}&userId={userId}&loginToken={loginToken}
20
+ ```
21
+
22
+ > 💡 **Note**: `tenantId`, `userId`, and `loginToken` must appear **both** in the URL query parameters and in the request body. Missing query parameters will cause the request to fail.
23
+
24
+ ## Request Parameters
25
+
26
+ | Parameter | Location | Type | Required | Description |
27
+ |-----------|----------|------|----------|-------------|
28
+ | tenantId | Query + Body | Integer | Yes | Tenant ID |
29
+ | userId | Query + Body | Integer | Yes | User ID |
30
+ | domainId | Body | Integer | Yes | Data domain ID, corresponding to a dataset |
31
+ | title | Body | String | No | Session title, used to identify the Session |
32
+ | loginToken | Query + Body | String | Yes | Authentication Token obtained via GenerateAuthToken |
33
+
34
+ ## Response Parameters
35
+
36
+ | Parameter | Type | Description |
37
+ |-----------|------|-------------|
38
+ | success | Boolean | Whether the request was successful |
39
+ | data | Integer | The sessionId of the newly created session |
40
+
41
+ ## Request Example
42
+
43
+ ```http
44
+ POST /open/session/safe_new
45
+ Content-Type: application/json
46
+
47
+ {
48
+ "tenantId": 10,
49
+ "userId": 1,
50
+ "domainId": 106,
51
+ "title": "Q1 Sales Analysis",
52
+ "loginToken": "eyJhbGciOiJIUzI1NiJ9..."
53
+ }
54
+ ```
55
+
56
+ ## Response Example
57
+
58
+ Successful response:
59
+
60
+ ```json
61
+ {
62
+ "success": true,
63
+ "data": 4729
64
+ }
65
+ ```
66
+
67
+ The value `4729` in `data` is the newly created `sessionId`.
68
+
69
+ ## Error Codes
70
+
71
+ | Error Code | Description |
72
+ |------------|-------------|
73
+ | success=false | Token is invalid, parameters are missing, or insufficient permissions |
74
+
75
+ ## Related Documentation
76
+
77
+ - [GenerateAuthToken](open-api-generate-auth-token) — Previous step: obtain authentication Token
78
+ - [Text2InsightQuery](open-api-text2insight-query) — Next step: submit a data analysis request
79
+ - [Quick Start](open-api-quick-start) — Complete end-to-end example
@@ -0,0 +1,63 @@
1
+ # GenerateAuthToken - Obtain Authentication Token
2
+
3
+ Obtain an authentication Token for subsequent API calls using an application secret key (appSecretKey).
4
+
5
+ ## Interface Description
6
+
7
+ Before calling any business interface in Open Text2Insight, you must first use this interface to obtain a login Token. This Token serves as the identity credential for all subsequent requests.
8
+
9
+ ### Usage Notes
10
+
11
+ - The appSecretKey is generated and distributed by the platform administrator in the backend.
12
+ - The Token has an expiration time; you must obtain a new one after it expires.
13
+ - Keep your appSecretKey secure to prevent leakage.
14
+
15
+ ## Request Method
16
+
17
+ ```
18
+ GET /open/api/v1/appSecretKey/generateAuthToken
19
+ ```
20
+
21
+ ## Request Parameters
22
+
23
+ | Parameter | Location | Type | Required | Description |
24
+ |-----------|----------|------|----------|-------------|
25
+ | appSecretKey | Query | String | Yes | Application secret key, assigned by the administrator |
26
+
27
+ ## Response Parameters
28
+
29
+ | Parameter | Type | Description |
30
+ |-----------|------|-------------|
31
+ | success | Boolean | Whether the request was successful |
32
+ | data.token | String | Authentication Token, used for subsequent interface calls |
33
+
34
+ ## Request Example
35
+
36
+ ```http
37
+ GET /open/api/v1/appSecretKey/generateAuthToken?appSecretKey=your_secret_key_here
38
+ ```
39
+
40
+ ## Response Example
41
+
42
+ Successful response:
43
+
44
+ ```json
45
+ {
46
+ "success": true,
47
+ "data": {
48
+ "token": "eyJhbGciOiJIUzI1NiJ9..."
49
+ }
50
+ }
51
+ ```
52
+
53
+ ## Error Codes
54
+
55
+ | Error Code | Description |
56
+ |------------|-------------|
57
+ | success=false | appSecretKey is invalid or has expired |
58
+
59
+ ## Related Documentation
60
+
61
+ - [Open API Overview](open-api-overview) — Interface list and general description
62
+ - [Quick Start](open-api-quick-start) — Complete call flow after obtaining the Token
63
+ - [CreateSession](open-api-create-session) — Next step: create a session using the Token