@clickzetta/cz-cli-darwin-arm64 0.5.16 → 0.5.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/cz-cli +0 -0
- package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
- package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
- package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
- package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
- package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
- package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
- package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
- package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
- package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
- package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
- package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
- package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
- package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
- package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
- package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
- package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
- package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
- package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
- package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
- package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
- package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
- package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
- package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
- package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
- package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
- package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
- package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
- package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
- package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
- package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
- package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
- package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
- package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
- package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
- package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
- package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
- package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
- package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
- package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
- package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
- package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
- package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
- package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
- package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
- package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
- package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
- package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
- package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
- package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
- package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
- package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
- package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
- package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
- package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
- package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
- package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
- package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
- package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
- package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
- package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
- package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
- package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
- package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
- package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
- package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
- package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
- package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
- package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
- package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
- package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
- package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
- package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
- package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
- package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
- package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
- package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
- package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
- package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
- package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
- package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
- package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
- package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
- package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
- package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
- package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
- package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
- package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
- package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
- package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
- package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
- package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
- package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
- package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
- package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
- package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
- package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
- package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
- package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
- package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
- package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
- package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
- package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
- package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
- package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
- package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
- package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
- package/package.json +1 -1
- package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
- package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
|
@@ -8,18 +8,18 @@ The specific adjustments to the specifications are as follows:
|
|
|
8
8
|
|
|
9
9
|
From XS to 5XLarge, each level increase doubles the computing resources and also doubles the resource consumption:
|
|
10
10
|
|
|
11
|
-
|
|
|
12
|
-
|
|
|
13
|
-
| **Specification**
|
|
14
|
-
| XSMALL
|
|
15
|
-
| SMALL
|
|
16
|
-
| MEDIUM
|
|
17
|
-
| LARGE
|
|
18
|
-
| XLARGE
|
|
19
|
-
| 2XLARGE
|
|
20
|
-
| 3XLARGE
|
|
21
|
-
| 4XLARGE
|
|
22
|
-
| 5XLARGE
|
|
11
|
+
| | | |
|
|
12
|
+
| ----------------- | ---------------- | ------------------------------------------ |
|
|
13
|
+
| **Specification** | **Abbreviation** | **Hourly Consumption** (**CRU**\***hour**) |
|
|
14
|
+
| XSMALL | XS | 1 |
|
|
15
|
+
| SMALL | S | 2 |
|
|
16
|
+
| MEDIUM | M | 4 |
|
|
17
|
+
| LARGE | L | 8 |
|
|
18
|
+
| XLARGE | XL | 16 |
|
|
19
|
+
| 2XLARGE | 2XL | 32 |
|
|
20
|
+
| 3XLARGE | 3XL | 64 |
|
|
21
|
+
| 4XLARGE | 4XL | 128 |
|
|
22
|
+
| 5XLARGE | 5XL | 256 |
|
|
23
23
|
|
|
24
24
|
## **New Calculation Cluster Specifications**
|
|
25
25
|
|
|
@@ -27,11 +27,11 @@ Each type of calculation cluster is represented by CRU (Compute Resource Unit).
|
|
|
27
27
|
|
|
28
28
|
The detailed specifications of various calculation clusters are as follows:
|
|
29
29
|
|
|
30
|
-
| **Type**
|
|
31
|
-
|
|
|
32
|
-
| General Purpose
 (GP Vcluster)
|
|
33
|
-
| Analytical
 (AP Vcluster)
|
|
34
|
-
| Integration
 (Integration Vcluster) | 0.25
|
|
30
|
+
| **Type** | **Minimum Specification** (**CRU**) | **Maximum Specification** (**CRU**) | **Default Value** (**CRU**) | **Step Size**
 (Minimum unit of specification increase/decrease) |
|
|
31
|
+
| --------------------------------------- | ----------------------------------- | ----------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
32
|
+
| General Purpose
 (GP Vcluster) | 1 | 256 | 1 | 1 (Specification examples: 1, 2, 3, 4, 5, 6...256) |
|
|
33
|
+
| Analytical
 (AP Vcluster) | 1 | 256 | 1 | Powers of 2 (Specification examples: 1, 2, 4, 8, 16, 32, 64, 128, 256) |
|
|
34
|
+
| Integration
 (Integration Vcluster) | 0.25 | 256 | 0.5 | 1 (Specification sequence: there are two special small specifications of 0.25 CRU and 0.5 CRU, the step size for other specifications is 1. Specification examples: 0.25, 0.5, 1, 2, 3, 4, 5, 6...256) |
|
|
35
35
|
|
|
36
36
|
The amount of computing resources consumed by each cluster per hour is its CRU specification\*1 hour, i.e., the unit is CRU\*hour. For example, an analytical calculation cluster with a specification of 3 CRU running for 1 hour consumes 3 CRU \* 1 hour = 3 CRU\*hour. The cost is also calculated according to the "CRU\*hour" unit price in the region where the calculation cluster is located.
|
|
37
37
|
|
|
@@ -41,24 +41,25 @@ The amount of computing resources consumed by each cluster per hour is its CRU s
|
|
|
41
41
|
|
|
42
42
|
The specifications of existing calculation clusters will be automatically converted to the new specifications, and the computing resource consumption and costs per unit time will not change. The specific correspondence of specifications is as follows:
|
|
43
43
|
|
|
44
|
-
|
|
|
45
|
-
|
|
|
44
|
+
| | | |
|
|
45
|
+
| ------------------ | ------------- | -------------------- |
|
|
46
46
|
| **Original Value** | **New Value** | **Applicable Scope** |
|
|
47
|
-
| XSMALL
|
|
48
|
-
| SMALL
|
|
49
|
-
| MEDIUM
|
|
50
|
-
| LARGE
|
|
51
|
-
| XLARGE
|
|
52
|
-
| 2XLARGE
|
|
53
|
-
| 3XLARGE
|
|
54
|
-
| 4XLARGE
|
|
55
|
-
| 5XLARGE
|
|
47
|
+
| XSMALL | 1 | All types of VC |
|
|
48
|
+
| SMALL | 2 | All types of VC |
|
|
49
|
+
| MEDIUM | 4 | All types of VC |
|
|
50
|
+
| LARGE | 8 | All types of VC |
|
|
51
|
+
| XLARGE | 16 | All types of VC |
|
|
52
|
+
| 2XLARGE | 32 | All types of VC |
|
|
53
|
+
| 3XLARGE | 64 | All types of VC |
|
|
54
|
+
| 4XLARGE | 128 | All types of VC |
|
|
55
|
+
| 5XLARGE | 256 | All types of VC |
|
|
56
56
|
|
|
57
57
|
### SQL Create New Compute Cluster
|
|
58
58
|
|
|
59
59
|
After the new specification code is launched, the SQL statements for creating and modifying compute clusters will temporarily be compatible with the old specification codes. That is, the Vcluster\_size parameter can support both numerical expressions and code expressions like 'XSMALL, SMALL' for compute cluster specifications. Please migrate to using numerical expressions for compute cluster specifications as soon as possible to avoid affecting the normal creation of compute clusters when switching to an incompatible version.
|
|
60
60
|
|
|
61
61
|
Example code for creating a new cluster is as follows:
|
|
62
|
+
|
|
62
63
|
```SQL
|
|
63
64
|
-- Create a compute cluster
|
|
64
65
|
CREATE VCLUSTER [IF NOT EXISTS] vc_name
|
|
@@ -69,11 +70,12 @@ objectProperties
|
|
|
69
70
|
objectProperties ::=
|
|
70
71
|
VCLUSTER_SIZE = NUM --0.25-256. Compatible with XSMALL | SMALL | MEDIUM | LARGE | XLARGE | 2XLARGE | 3XLARGE | 4XLARGE | 5XLARGE
|
|
71
72
|
```
|
|
73
|
+
|
|
72
74
|
### Create a New Compute Cluster on the Web
|
|
73
75
|
|
|
74
76
|
When creating and modifying compute clusters on the web, the specifications of the compute cluster are now expressed in numerical terms. An example is shown below:
|
|
75
77
|
|
|
76
|
-
|
|
78
|
+
^
|
|
77
79
|
|
|
78
80
|
### Query Compute Cluster via SQL
|
|
79
81
|
|
|
@@ -83,7 +85,7 @@ When querying compute cluster specifications via SQL, all fields representing co
|
|
|
83
85
|
|
|
84
86
|
The specifications of the compute cluster on the web are now expressed in numerical terms.
|
|
85
87
|
|
|
86
|
-
|
|
88
|
+
^
|
|
87
89
|
|
|
88
90
|
## Billing Instructions
|
|
89
91
|
|
|
@@ -93,4 +95,4 @@ After the change, estimating the hourly resource consumption and cost of a clust
|
|
|
93
95
|
|
|
94
96
|
There is no need to remember that the "S" specification corresponds to 2 CRU per hour, and then multiply by the unit price to estimate the cost.
|
|
95
97
|
|
|
96
|
-
A compute cluster of 2 CRU consumes "2 CRU\*hour" per hour, and you only need to match the unit price to calculate the hourly cost.
|
|
98
|
+
A compute cluster of 2 CRU consumes "2 CRU\*hour" per hour, and you only need to match the unit price to calculate the hourly cost.
|
|
@@ -14,11 +14,11 @@ A virtual cluster's specification defines the amount of compute resources availa
|
|
|
14
14
|
|
|
15
15
|
> ⚠️ **Note**: Starting from December 2024, the old specification codes for virtual clusters (XS through 5XLarge) have been replaced with numeric specifications. For details, see [Virtual Cluster Specification Code Change Description](vcluster_size_description.md).
|
|
16
16
|
|
|
17
|
-
| **Type**
|
|
18
|
-
|
|
|
19
|
-
| General Purpose (GP VCluster)
|
|
20
|
-
| Analytics (AP VCluster)
|
|
21
|
-
| Integration (Integration VCluster)
|
|
17
|
+
| **Type** | **Minimum Spec (CRU**) | **Maximum Spec (CRU**) | **Default (CRU**) | **Step Size** (minimum increment/decrement unit) |
|
|
18
|
+
| ---------------------------------- | ---------------------- | ---------------------- | ----------------- | ---------------------------------------------------------------------------- |
|
|
19
|
+
| General Purpose (GP VCluster) | 1 | 256 | 1 | 1 CRU per step. Examples: 1, 2, 3, 4, 5, 6...256 |
|
|
20
|
+
| Analytics (AP VCluster) | 1 | 256 | 1 | Powers of 2. Examples: 1, 2, 4, 8, 16, 32, 64, 128, 256 |
|
|
21
|
+
| Integration (Integration VCluster) | 0.25 | 256 | 0.5 | Supports any decimal specification. Examples: 0.25, 0.5, 0.75, 1, 1.25...256 |
|
|
22
22
|
|
|
23
23
|
> ⚠️ **Note**: Analytics cluster specifications must be a power of 2 (1, 2, 4, 8, 16, 32, 64, 128, 256). Setting any other value will result in an error.
|
|
24
24
|
|
|
@@ -28,29 +28,28 @@ The compute resources consumed by a cluster per hour equal its CRU specification
|
|
|
28
28
|
|
|
29
29
|
Virtual clusters come in three types: General Purpose, Analytics, and Integration.
|
|
30
30
|
|
|
31
|
-
* **General Purpose (GENERAL)
|
|
32
|
-
* **Analytics (ANALYTICS)
|
|
33
|
-
* **Integration (INTEGRATION)
|
|
31
|
+
* **General Purpose (GENERAL**): Suitable for offline jobs. Jobs share compute resources, and new and existing jobs are allocated resources using fair scheduling.
|
|
32
|
+
* **Analytics (ANALYTICS**): Supports multiple compute instances and auto-scaling. Suitable for online and high-concurrency workloads. When the concurrency limit is reached, subsequent jobs queue up, ensuring first-submitted jobs are executed first.
|
|
33
|
+
* **Integration (INTEGRATION**): Suitable for offline and real-time integration tasks. Multiple integration tasks can share a single Integration cluster instance. Tasks that exceed the cluster's capacity enter a queue.
|
|
34
34
|
|
|
35
35
|
The relationship between cluster types and task types is shown below:
|
|
36
36
|
|
|
37
|
-
| **Task Type**
|
|
38
|
-
|
|
|
39
|
-
| Offline Sync
|
|
40
|
-
| SQL
|
|
41
|
-
| Python
|
|
42
|
-
| Shell
|
|
43
|
-
| JDBC
|
|
44
|
-
| Virtual Node
|
|
45
|
-
| Databricks SQL
|
|
46
|
-
| Databricks Notebook
|
|
47
|
-
| Real-time Sync
|
|
48
|
-
| Multi-table Real-time Sync | Integration
|
|
49
|
-
| Dynamic Table
|
|
50
|
-
| Streaming SQL
|
|
51
|
-
|
|
52
|
-
:-: 
|
|
37
|
+
| **Task Type** | **Supported Cluster Types** | **Recommended Cluster Type** |
|
|
38
|
+
| -------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------- |
|
|
39
|
+
| Offline Sync | Integration | Integration |
|
|
40
|
+
| SQL | General Purpose, Analytics | **ETL tasks**: General Purpose**Ad-hoc queries**: Analytics |
|
|
41
|
+
| Python | No cluster required | / |
|
|
42
|
+
| Shell | No cluster required | / |
|
|
43
|
+
| JDBC | No cluster required | / |
|
|
44
|
+
| Virtual Node | No cluster required | / |
|
|
45
|
+
| Databricks SQL | No cluster required | / |
|
|
46
|
+
| Databricks Notebook | No cluster required | / |
|
|
47
|
+
| Real-time Sync | Integration | Integration |
|
|
48
|
+
| Multi-table Real-time Sync | Integration | Integration |
|
|
49
|
+
| Dynamic Table | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose**High-frequency, small data volume**: Analytics |
|
|
50
|
+
| Streaming SQL | General Purpose, Analytics | **Low-frequency, large data volume**: General Purpose**High-frequency, small data volume**: Analytics |
|
|
53
51
|
|
|
52
|
+
^
|
|
54
53
|
^
|
|
55
54
|
|
|
56
55
|
### Minimum Instances
|
|
@@ -81,23 +80,22 @@ Singdata Lakehouse provides two ways to manage virtual clusters: 1) Web UI; 2) S
|
|
|
81
80
|
|
|
82
81
|
Log in to the Lakehouse platform and click "Compute" in the left menu to open the "Virtual Clusters" list page. Here you can view all virtual clusters in the current workspace that you have access to.
|
|
83
82
|
|
|
84
|
-
|
|
83
|
+
^
|
|
85
84
|
|
|
86
85
|
Click any cluster name to open the details page, where you can view job execution status, configuration details, and your permissions for that cluster.
|
|
87
86
|
|
|
88
|
-
|
|
87
|
+
^
|
|
89
88
|
|
|
90
89
|
#### Creating a Virtual Cluster
|
|
91
90
|
|
|
92
91
|
On the "Virtual Clusters" list page, click "New Cluster" to open the creation interface. Fill in the cluster name, select the cluster type, configure the specification, and click "Create Cluster." The new cluster will appear in the list and can be used once its status changes to "Running."
|
|
93
92
|
|
|
94
|
-
|
|
95
|
-
|
|
93
|
+
^
|
|
96
94
|
^
|
|
97
95
|
|
|
98
96
|
**Creating a General Purpose Cluster**
|
|
99
97
|
|
|
100
|
-
|
|
98
|
+
^
|
|
101
99
|
|
|
102
100
|
If you need to run ETL or other batch workloads, create a General Purpose cluster:
|
|
103
101
|
|
|
@@ -114,8 +112,7 @@ After completing the configuration, click "OK" to finish creating the General Pu
|
|
|
114
112
|
|
|
115
113
|
**Creating an Analytics Cluster**
|
|
116
114
|
|
|
117
|
-
|
|
118
|
-
|
|
115
|
+
^
|
|
119
116
|
^
|
|
120
117
|
|
|
121
118
|
If you need compute resources for ad-hoc queries or BI dashboards, create an Analytics cluster:
|
|
@@ -130,7 +127,7 @@ If you need compute resources for ad-hoc queries or BI dashboards, create an Ana
|
|
|
130
127
|
|
|
131
128
|
**Creating an Integration Cluster**
|
|
132
129
|
|
|
133
|
-
|
|
130
|
+
^
|
|
134
131
|
|
|
135
132
|
If you need to run offline or real-time integration tasks, create an Integration cluster:
|
|
136
133
|
|
|
@@ -141,7 +138,7 @@ If you need to run offline or real-time integration tasks, create an Integration
|
|
|
141
138
|
|
|
142
139
|
* Integration clusters provide a "Specification Estimation" feature to help you choose an appropriate size. Enter the "Number of Tasks" and "Average Task Concurrency," and the system calculates an estimated specification using the formula: Specification = Number of Tasks × Average Task Concurrency × 0.05.
|
|
143
140
|
|
|
144
|
-
|
|
141
|
+
^
|
|
145
142
|
|
|
146
143
|
* Choose whether to enable "Auto Resume" and "Auto Suspend," and configure a suitable "Auto Suspend" time. We recommend enabling "Auto Resume" to handle jobs promptly, and setting "Auto Suspend" to 1 minute so the cluster shuts down quickly after jobs complete, saving costs.
|
|
147
144
|
|
|
@@ -151,21 +148,20 @@ If you need to run offline or real-time integration tasks, create an Integration
|
|
|
151
148
|
|
|
152
149
|
In the cluster list, select the cluster and click "Modify" in the "Actions" column to adjust its configuration. Configuration changes may affect cluster status. When modifying key attributes such as specification or instance count, make sure there are no jobs currently running on the cluster.
|
|
153
150
|
|
|
154
|
-
|
|
151
|
+
^
|
|
155
152
|
|
|
156
153
|
#### Starting and Stopping a Virtual Cluster
|
|
157
154
|
|
|
158
155
|
In the cluster list, click "Start" or "Stop" in the "Actions" column to start or stop the cluster. When stopping a cluster, it waits for all running jobs to complete before fully stopping.
|
|
159
156
|
|
|
160
|
-
|
|
161
|
-
|
|
157
|
+
^
|
|
162
158
|
^
|
|
163
159
|
|
|
164
160
|
#### Deleting a Virtual Cluster
|
|
165
161
|
|
|
166
162
|
In the cluster list, click "Delete" in the "Actions" column to delete the specified cluster. The cluster must be stopped and all jobs must be complete before deletion can proceed.
|
|
167
163
|
|
|
168
|
-
|
|
164
|
+
^
|
|
169
165
|
|
|
170
166
|
### SQL
|
|
171
167
|
|
|
@@ -193,14 +189,14 @@ In practice, it is recommended to test queries with different cluster sizes (for
|
|
|
193
189
|
|
|
194
190
|
The following table provides sample specifications across several dimensions — business load type, execution frequency, job concurrency, data volume, and SLA requirements — for reference:
|
|
195
191
|
|
|
196
|
-
| Business Scenario | Load Type
|
|
197
|
-
| --------------------------------------- |
|
|
198
|
-
| ETL Scheduling Jobs | Near real-time offline
|
|
199
|
-
| | T+1 offline
|
|
200
|
-
| Tableau/FineBI | Ad-hoc analytics
|
|
201
|
-
| Data Application Products | Applications
|
|
202
|
-
| |
|
|
203
|
-
| Singdata Web UI (Data Development/Test) | Ad-hoc analytics
|
|
192
|
+
| Business Scenario | Load Type | Execution Frequency | Job Concurrency | Data Volume | VCluster Type | Job Latency SLA | VCluster Size |
|
|
193
|
+
| --------------------------------------- | ---------------------- | ------------------- | --------------- | ----------- | --------------- | ----------------- | ------------- |
|
|
194
|
+
| ETL Scheduling Jobs | Near real-time offline | Hourly | 1 | 1 TB | General Purpose | 15 min | 4 |
|
|
195
|
+
| | T+1 offline | Daily | 1 | 10 TB | General Purpose | 4 hours | 8 |
|
|
196
|
+
| Tableau/FineBI | Ad-hoc analytics | Ad-hoc | 8 | 1 TB | General Purpose | <1 min, TP90 <5s | 16 |
|
|
197
|
+
| Data Application Products | Applications | On demand | 8 | 100 GB | Analytics | <1s | 4 |
|
|
198
|
+
| | | On demand | 96 | 100 MB | Analytics | <1s | 4 |
|
|
199
|
+
| Singdata Web UI (Data Development/Test) | Ad-hoc analytics | Ad-hoc | 8 | 3 TB | General Purpose | <1 min, TP90 <15s | 16 |
|
|
204
200
|
|
|
205
201
|
#### Load Isolation with Multiple Clusters
|
|
206
202
|
|
|
@@ -211,3 +207,5 @@ The following table provides sample specifications across several dimensions —
|
|
|
211
207
|
* Periodic ELT tasks and online business report queries should use separate clusters for isolation. Use General Purpose for ELT and Analytics for reports to take full advantage of each type's performance characteristics. This also makes it easy to configure different auto-suspend times — set a short auto-suspend (e.g., 1 minute) for the ELT cluster, and a slightly longer one (e.g., 5 minutes) for the reporting cluster.
|
|
212
208
|
|
|
213
209
|
* Isolate clusters running large jobs from those running small jobs. Configure a larger cluster for large jobs to ensure sufficient runtime, and a smaller cluster for small jobs to avoid underutilization and resource waste.
|
|
210
|
+
|
|
211
|
+
^
|
|
@@ -1,121 +1,106 @@
|
|
|
1
1
|
# Lakehouse Job History
|
|
2
2
|
|
|
3
|
-
Through the job history page, you can view detailed information about queries executed in the Lakehouse account over the past 7 days. This page displays a list of historical jobs, including tasks submitted using
|
|
3
|
+
Through the job history page, you can view detailed information about queries executed in the Lakehouse account over the past 7 days. This page displays a list of historical jobs, including tasks submitted using a JDBC jar, Studio ad-hoc queries, and Studio scheduled tasks.
|
|
4
4
|
|
|
5
5
|
## Job History
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
Click
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
##
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
7
|
+
## View History
|
|
8
|
+
|
|
9
|
+
Click "Compute" in the left navigation bar, then click "Job History" to enter the job history page:
|
|
10
|
+
|
|
11
|
+
^
|
|
12
|
+
|
|
13
|
+
## Interface
|
|
14
|
+
|
|
15
|
+
* Supports searching by job ID generated by SQL jobs
|
|
16
|
+
* Supports searching by status
|
|
17
|
+
* Initializing: SQL initialization phase, including operations such as compilation and optimization
|
|
18
|
+
* Cluster Starting: Waiting for the Virtual Cluster to start. If it takes a long time to start, please contact technical support
|
|
19
|
+
* Queued: Job is queued waiting for resources. If it takes a long time, check whether the Virtual Cluster resources are fully occupied and consider scaling up the Virtual Cluster
|
|
20
|
+
* Running: SQL data processing time. You can click into diagnostics to view the time taken by each operator
|
|
21
|
+
* Succeeded: Job has ended. Some jobs may have a cleanup process at this stage, which can take time
|
|
22
|
+
* Supports filtering by duration
|
|
23
|
+
* Supports filtering by time. Currently, up to 10,000 jobs from the past seven days can be queried
|
|
24
|
+
* Supports filtering by the name of the cluster where the job ran
|
|
25
|
+
* Supports filtering by the job submitter
|
|
26
|
+
* Supports filtering by queryTag
|
|
27
|
+
Users can use the command `set query_tag=""` when submitting SQL to indicate the job source and tag the job, making it easy to filter in this interface.
|
|
28
|
+
* Set query\_tag in the Studio interface, select two SQL statements and execute simultaneously:
|
|
29
|
+
```SQL
|
|
30
|
+
set query_tag="test";
|
|
31
|
+
select 1;
|
|
32
|
+
```
|
|
33
|
+
* Add query\_tag in the JDBC URL so that every submitted SQL carries the query\_tag to indicate the source, which can be filtered on the page:
|
|
34
|
+
```text
|
|
35
|
+
jdbc:clickzetta://demo_instance.cn-shanghai-alicloud.api.clickzetta.com/default?schema=public&query_tag=test
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
* Set query\_tag in Python code:
|
|
39
|
+
```python
|
|
40
|
+
# Establish connection
|
|
41
|
+
conn = connect(username='',
|
|
42
|
+
password='',
|
|
43
|
+
service='cn-shanghai-alicloud.api.clickzetta.com',
|
|
44
|
+
instance='demo_instance',
|
|
45
|
+
workspace='', schema='public', vcluster='DEFAULT')
|
|
46
|
+
# Execute SQL
|
|
47
|
+
cursor = conn.cursor()
|
|
48
|
+
my_param["hints"]["query_tag"] ="test"
|
|
49
|
+
cursor.execute('select 1;', parameters=my_param)
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
* The default Schema for the job is the context in which the submitted SQL runs, which can be obtained through `select current_schema()`.
|
|
52
53
|
|
|
53
54
|
## Job Details
|
|
54
55
|
|
|
55
56
|
Enter the job details to view detailed information about the job.
|
|
56
57
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
**Field Information Explained as Follows**
|
|
60
|
-
|
|
61
|
-
| | |
|
|
62
|
-
| ------ | -------------------------------------------------------------------------------------------------------------------- |
|
|
63
|
-
| Field | Description |
|
|
64
|
-
| Status | Indicates the status of the job, currently includes
Ready to execute: The job is being compiled and other operations
Waiting to execute: Waiting for resources
Executing: Running, the job is processing data
Execution successful: The job ran successfully
Execution terminated: The job was terminated
Execution failed: The job failed to run |
|
|
65
|
-
| Submit Time | The time the server received the request to start |
|
|
66
|
-
| End Time | The time the job ended |
|
|
67
|
-
| Duration | Includes compilation optimization, starting computing resources, queuing, and job running time |
|
|
68
|
-
| Owner | The user who submitted the job |
|
|
69
|
-
| Spill to Disk | The size of the data spilled to disk, this parameter can help understand if the performance is slow due to insufficient cluster memory resources |
|
|
70
|
-
| Read from Cache | In the analytical cluster, the already read data will be cached first to speed up computation |
|
|
71
|
-
| Output Records | The number of output records of the job |
|
|
72
|
-
| Consumption | Refers to the computing resources used by the job, excluding the computing resources while the job is waiting for IO. |
|
|
73
|
-
| Cluster | The cluster used to run the job |
|
|
58
|
+
^
|
|
74
59
|
|
|
75
60
|
### Stage Interface
|
|
76
61
|
|
|
77
|
-
SQL execution is divided into
|
|
62
|
+
SQL execution is divided into stages, which are executed from bottom to top. Clicking on a specific stage will show the specific time consumption of each stage.
|
|
78
63
|
|
|
79
|
-
|
|
64
|
+
^
|
|
80
65
|
|
|
81
66
|
**Field information is explained as follows**
|
|
82
67
|
|
|
83
|
-
|
|
|
84
|
-
|
|
|
85
|
-
| Field
|
|
86
|
-
| Status
|
|
87
|
-
| Start Time
|
|
88
|
-
| End Time
|
|
89
|
-
| Duration
|
|
90
|
-
|
|
|
91
|
-
| Read from Cache
|
|
92
|
-
| Contains
|
|
68
|
+
| | |
|
|
69
|
+
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
70
|
+
| Field | Description |
|
|
71
|
+
| Status | Indicates the status of the job. Currently includes:
Queued: Waiting for resources
Running: Running, the job is processing data
Succeeded: Job ran successfully
Failed: Job ran unsuccessfully |
|
|
72
|
+
| Start Time | The start time of the stage. In AP clusters, stages run in parallel |
|
|
73
|
+
| End Time | The end time of the job |
|
|
74
|
+
| Duration | The duration of the stage running |
|
|
75
|
+
| Spill to Disk | The size of the data spilled to disk. This parameter indicates whether slow performance is due to insufficient cluster memory resources |
|
|
76
|
+
| Read from Cache | In AP clusters, data that has already been read is cached first to accelerate computation |
|
|
77
|
+
| Contains Operators | Stage |
|
|
93
78
|
|
|
94
79
|
### Operator Interface
|
|
95
80
|
|
|
96
|
-
In the operator interface, you can view the type of each operator and
|
|
81
|
+
In the operator interface, you can view the type of each operator and its upstream and downstream dependencies. An operator is a concept in the query plan that represents the specific steps and strategies for executing a query, including data access, transformation, and updates.
|
|
97
82
|
|
|
98
|
-
|
|
83
|
+
^
|
|
99
84
|
|
|
100
85
|
**Field information is explained as follows**
|
|
101
86
|
|
|
102
|
-
|
|
|
103
|
-
|
|
|
104
|
-
| Belongs to Stage | The stage corresponding to the operator
|
|
105
|
-
|
|
|
106
|
-
| Read from Cache | In
|
|
107
|
-
| Total Rows
|
|
108
|
-
|
|
|
109
|
-
| Field
|
|
87
|
+
| Field | Description |
|
|
88
|
+
| :--------------- | :-------------------------------------------------------------------------------------------------------------------------------------- |
|
|
89
|
+
| Belongs to Stage | The stage corresponding to the operator |
|
|
90
|
+
| Spill to Disk | The size of the data spilled to disk. This parameter indicates whether slow performance is due to insufficient cluster memory resources |
|
|
91
|
+
| Read from Cache | In AP clusters, data that has already been read is cached first to accelerate computation |
|
|
92
|
+
| Total Rows | The data currently processed by the operator |
|
|
93
|
+
| Duration | Operators run concurrently. The displayed value is the average time of the operator |
|
|
94
|
+
| Field | Corresponds to the fields in the SQL script |
|
|
110
95
|
|
|
111
96
|
### Table Mode
|
|
112
97
|
|
|
113
|
-
Supports
|
|
98
|
+
Supports switching stages to table mode
|
|
99
|
+
|
|
100
|
+
^
|
|
114
101
|
|
|
115
|
-
|
|
102
|
+
Supports switching operators to table mode
|
|
116
103
|
|
|
117
|
-
Supports operator switching table mode
|
|
118
|
-

|
|
119
104
|
|
|
120
105
|
## Analyzing Failed Jobs
|
|
121
106
|
|
|
@@ -124,55 +109,56 @@ When a job fails to run, you can view the reason for the failure through the det
|
|
|
124
109
|
### SQL Syntax Error
|
|
125
110
|
|
|
126
111
|
**Error Example**:
|
|
112
|
+
|
|
127
113
|
```SQL
|
|
128
114
|
Execute select * from ttt;
|
|
129
115
|
Error: CZLH-42000:[1,15] Semantic analysis exception - table or view not found - uat_demo_project.`public`.ttt
|
|
130
116
|
```
|
|
131
|
-
**Error Code CZLH-42000** indicates an error related to SQL language parsing. [1,15] indicates that the error occurred on line 1, column 15 of the SQL. In this example, the reason is that the `ttt` table does not exist. Please note that if you submit tasks through Studio development, the job editor in development will include descriptive information, which is not submitted to the SQL engine. Therefore, the correct line number should subtract the preceding descriptive information. You can view the correct positional relationship through the job content in the job details or find the error location in the SQL editor.
|
|
132
117
|
|
|
118
|
+
**Error code CZLH-42000** indicates an error related to SQL language parsing. `[1,15]` means the error occurred at line 1, column 15 of the SQL. In this example, the reason is that the `ttt` table does not exist.
|
|
133
119
|
|
|
134
120
|
## Analyzing Slow Running Jobs
|
|
135
121
|
|
|
136
|
-
Slow running jobs may encounter issues during the compilation phase, the
|
|
122
|
+
Slow running jobs may encounter issues during the compilation phase, the Virtual Cluster startup phase, or the execution phase. Below are analyses and solutions for these phases:
|
|
137
123
|
|
|
138
124
|
### Compilation Phase
|
|
139
125
|
|
|
140
|
-
**Issue**: Job status shows "
|
|
126
|
+
**Issue**: Job status shows "Initializing".
|
|
141
127
|
|
|
142
128
|
**Possible Causes and Solutions**:
|
|
143
129
|
|
|
144
|
-
1. **Complex execution plan**: The job may require a long time to optimize. Please be patient
|
|
145
|
-
2. **Compilation resources are fully occupied**: If you submit a large number of
|
|
130
|
+
1. **Complex execution plan**: The job may require a long time to optimize. Please be patient — it should not take more than 10 minutes under normal circumstances.
|
|
131
|
+
2. **Compilation resources are fully occupied**: If you submit a large number of SQL statements at once, it may lead to insufficient compilation resources. Please reduce the frequency of SQL submissions. If you have a large number of high-concurrency query requirements, please submit a ticket for assistance.
|
|
146
132
|
|
|
147
133
|
### Virtual Cluster Startup Phase
|
|
148
134
|
|
|
149
|
-
**Issue**: Job status shows "Cluster
|
|
135
|
+
**Issue**: Job status shows "Cluster Starting". (The current status is that the compute cluster is being started)
|
|
150
136
|
|
|
151
|
-
###
|
|
137
|
+
### Queued Phase
|
|
152
138
|
|
|
153
|
-
**Issue**: Job status shows "
|
|
139
|
+
**Issue**: Job status shows "Queued".
|
|
154
140
|
|
|
155
141
|
**Possible Causes and Solutions**:
|
|
156
142
|
|
|
157
|
-
**Insufficient computing resources in the current cluster**: You can filter and view the currently running jobs through the interface. A large number of running jobs may be causing resource
|
|
143
|
+
1. **Insufficient computing resources in the current cluster**: You can filter and view the currently running jobs through the interface. A large number of running jobs may be causing resource queuing. You can scale up resources, or terminate large running jobs to free up resources.
|
|
158
144
|
|
|
159
145
|
### Execution Phase
|
|
160
146
|
|
|
161
|
-
**Issue**: Job status shows "Running". At this stage
|
|
147
|
+
**Issue**: Job status shows "Running". At this stage you can click in to view the execution status of each operator.
|
|
162
148
|
|
|
163
149
|
**Possible Causes and Solutions**:
|
|
164
150
|
|
|
165
151
|
1. **Waiting for resources**: The job may be waiting for resource allocation. Please check the resource usage. If necessary, please submit a ticket for assistance.
|
|
166
|
-
2. **Slow data reading speed**: The job may be executing slowly due to slow data reading
|
|
152
|
+
2. **Slow data reading speed**: The job may be executing slowly due to slow data reading. Please check the data storage and reading strategy. If necessary, optimize the data storage structure or adjust the reading strategy.
|
|
167
153
|
|
|
168
|
-
|
|
154
|
+
### Share for Diagnosis
|
|
169
155
|
|
|
170
|
-
|
|
156
|
+
After entering the job details page, click the "Share" button to share the job with a third party. Recipients can view the content without logging in.
|
|
157
|
+
\:-: 
|
|
158
|
+
If you have any questions or need further assistance, please copy the link above and contact the Lakehouse team. We will diagnose and resolve the issue for you.
|
|
159
|
+
|
|
160
|
+
## Viewing Execution Plan and Operator Details
|
|
171
161
|
|
|
172
|
-
|
|
173
|
-
## Share for Diagnosis
|
|
162
|
+
During the execution phase, you can click on a specific operator to query its execution steps and strategies. For example, moving the mouse over a specific operator will highlight the corresponding operator in the SQL. This helps you identify which specific operator is causing the job to run slowly.
|
|
174
163
|
|
|
175
|
-
|
|
176
|
-

|
|
177
|
-
If you need to stop sharing, you can revoke access to the link at any time.
|
|
178
|
-
If you have any questions or need further assistance, please copy the link above and contact the Lakehouse team. We will diagnose and resolve the issue for you.
|
|
164
|
+
^
|