@clickzetta/cz-cli-darwin-x64 0.5.15 → 0.5.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/cz-cli +0 -0
- package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
- package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
- package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
- package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
- package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
- package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
- package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
- package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
- package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
- package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
- package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
- package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
- package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
- package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
- package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
- package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
- package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
- package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
- package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
- package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
- package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
- package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
- package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
- package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
- package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
- package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
- package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
- package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
- package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
- package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
- package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
- package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
- package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
- package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
- package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
- package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
- package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
- package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
- package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
- package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
- package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
- package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
- package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
- package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
- package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
- package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
- package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
- package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
- package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
- package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
- package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
- package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
- package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
- package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
- package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
- package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
- package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
- package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
- package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
- package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
- package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
- package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
- package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
- package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
- package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
- package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
- package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
- package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
- package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
- package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
- package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
- package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
- package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
- package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
- package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
- package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
- package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
- package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
- package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
- package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
- package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
- package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
- package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
- package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
- package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
- package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
- package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
- package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
- package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
- package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
- package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
- package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
- package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
- package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
- package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
- package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
- package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
- package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
- package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
- package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
- package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
- package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
- package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
- package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
- package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
- package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
- package/package.json +1 -1
- package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
- package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
|
@@ -6,21 +6,16 @@ This page covers cz-cli commands for SQL execution, schema and table management,
|
|
|
6
6
|
|
|
7
7
|
### Basic Usage
|
|
8
8
|
|
|
9
|
-
```bash
|
|
10
|
-
```
|
|
11
|
-
|
|
12
9
|
Synchronous execution, returns results directly (default):
|
|
13
10
|
|
|
14
11
|
```bash
|
|
15
12
|
cz-cli -p prod sql "SELECT current_timestamp()"
|
|
16
|
-
|
|
17
13
|
```
|
|
18
14
|
|
|
19
15
|
Pass SQL with -e:
|
|
20
16
|
|
|
21
17
|
```bash
|
|
22
18
|
cz-cli -p prod sql -e "SELECT * FROM public.orders LIMIT 10"
|
|
23
|
-
|
|
24
19
|
```
|
|
25
20
|
|
|
26
21
|
Read SQL from a file:
|
|
@@ -33,21 +28,17 @@ cz-cli -p prod sql -f query.sql
|
|
|
33
28
|
|
|
34
29
|
`cz-cli sql` executes synchronously by default (`--sync`), waiting for results before exiting. For long-running queries, you can submit asynchronously, get the job\_id, and then retrieve results:
|
|
35
30
|
|
|
36
|
-
```bash
|
|
37
|
-
```
|
|
38
31
|
|
|
39
32
|
Submit asynchronously, returns job\_id immediately:
|
|
40
33
|
|
|
41
34
|
```bash
|
|
42
35
|
cz-cli -p prod sql "SELECT * FROM huge_table" --async
|
|
43
|
-
|
|
44
36
|
```
|
|
45
37
|
|
|
46
38
|
Check job status:
|
|
47
39
|
|
|
48
40
|
```bash
|
|
49
41
|
cz-cli -p prod sql status <job_id>
|
|
50
|
-
|
|
51
42
|
```
|
|
52
43
|
|
|
53
44
|
Or use the job command:
|
|
@@ -110,28 +101,23 @@ cz-cli -p prod sql --dry-run -f deploy.sql
|
|
|
110
101
|
|
|
111
102
|
### Output Control
|
|
112
103
|
|
|
113
|
-
```bash
|
|
114
|
-
```
|
|
115
104
|
|
|
116
105
|
Do not truncate long fields:
|
|
117
106
|
|
|
118
107
|
```bash
|
|
119
108
|
cz-cli -p prod sql "SELECT * FROM public.orders" --no-truncate
|
|
120
|
-
|
|
121
109
|
```
|
|
122
110
|
|
|
123
111
|
Remove row limit (default 100 rows):
|
|
124
112
|
|
|
125
113
|
```bash
|
|
126
114
|
cz-cli -p prod sql "SELECT * FROM public.orders" --no-limit
|
|
127
|
-
|
|
128
115
|
```
|
|
129
116
|
|
|
130
117
|
Do not output column names:
|
|
131
118
|
|
|
132
119
|
```bash
|
|
133
120
|
cz-cli -p prod sql "SELECT id, name FROM public.orders" --no-header
|
|
134
|
-
|
|
135
121
|
```
|
|
136
122
|
|
|
137
123
|
Specify output format:
|
|
@@ -144,7 +130,7 @@ cz-cli -p prod sql "SELECT * FROM public.orders LIMIT 5" -o csv
|
|
|
144
130
|
### Full Parameter Reference
|
|
145
131
|
|
|
146
132
|
| Parameter | Description | Default |
|
|
147
|
-
|
|
133
|
+
| --------- | ----------- | ------- |
|
|
148
134
|
| `--sync` / `--no-sync` | Wait synchronously for results | `true` |
|
|
149
135
|
| `--async` | Submit asynchronously, returns job\_id immediately | `false` |
|
|
150
136
|
| `--write` | Allow write operations (DDL/DML) | Off |
|
|
@@ -166,28 +152,22 @@ cz-cli -p prod sql "SELECT * FROM public.orders LIMIT 5" -o csv
|
|
|
166
152
|
|
|
167
153
|
## cz-cli schema — Schema Management
|
|
168
154
|
|
|
169
|
-
```bash
|
|
170
|
-
```
|
|
171
|
-
|
|
172
155
|
List all schemas:
|
|
173
156
|
|
|
174
157
|
```bash
|
|
175
158
|
cz-cli -p prod schema list
|
|
176
|
-
|
|
177
159
|
```
|
|
178
160
|
|
|
179
161
|
View schema details (including table list):
|
|
180
162
|
|
|
181
163
|
```bash
|
|
182
164
|
cz-cli -p prod schema describe public
|
|
183
|
-
|
|
184
165
|
```
|
|
185
166
|
|
|
186
167
|
Create a schema:
|
|
187
168
|
|
|
188
169
|
```bash
|
|
189
170
|
cz-cli -p prod schema create dwd
|
|
190
|
-
|
|
191
171
|
```
|
|
192
172
|
|
|
193
173
|
Drop a schema (requires confirmation):
|
|
@@ -200,56 +180,47 @@ cz-cli -p prod schema drop old_schema
|
|
|
200
180
|
|
|
201
181
|
## cz-cli table — Table Management and Data Exploration
|
|
202
182
|
|
|
203
|
-
```bash
|
|
204
|
-
```
|
|
205
183
|
|
|
206
184
|
List all tables in the current schema:
|
|
207
185
|
|
|
208
186
|
```bash
|
|
209
187
|
cz-cli -p prod table list
|
|
210
|
-
|
|
211
188
|
```
|
|
212
189
|
|
|
213
190
|
List tables in a specific schema:
|
|
214
191
|
|
|
215
192
|
```bash
|
|
216
193
|
cz-cli -p prod -s dwd table list
|
|
217
|
-
|
|
218
194
|
```
|
|
219
195
|
|
|
220
196
|
View table structure (column names, types, comments):
|
|
221
197
|
|
|
222
198
|
```bash
|
|
223
199
|
cz-cli -p prod table describe public.orders
|
|
224
|
-
|
|
225
200
|
```
|
|
226
201
|
|
|
227
202
|
Preview table data (default 10 rows):
|
|
228
203
|
|
|
229
204
|
```bash
|
|
230
205
|
cz-cli -p prod table preview public.orders
|
|
231
|
-
|
|
232
206
|
```
|
|
233
207
|
|
|
234
208
|
View table row count and recent job statistics:
|
|
235
209
|
|
|
236
210
|
```bash
|
|
237
211
|
cz-cli -p prod table stats public.orders
|
|
238
|
-
|
|
239
212
|
```
|
|
240
213
|
|
|
241
214
|
View table version history (Time Travel support):
|
|
242
215
|
|
|
243
216
|
```bash
|
|
244
217
|
cz-cli -p prod table history public.orders
|
|
245
|
-
|
|
246
218
|
```
|
|
247
219
|
|
|
248
220
|
Create a table from DDL:
|
|
249
221
|
|
|
250
222
|
```bash
|
|
251
223
|
cz-cli -p prod table create --write "CREATE TABLE public.test (id INT, name STRING)"
|
|
252
|
-
|
|
253
224
|
```
|
|
254
225
|
|
|
255
226
|
Drop a table (requires confirmation):
|
|
@@ -264,21 +235,17 @@ cz-cli -p prod table drop public.test
|
|
|
264
235
|
|
|
265
236
|
Asynchronously submitted queries return a job\_id; use the `job` command to track them:
|
|
266
237
|
|
|
267
|
-
```bash
|
|
268
|
-
```
|
|
269
238
|
|
|
270
239
|
View job status and execution summary:
|
|
271
240
|
|
|
272
241
|
```bash
|
|
273
242
|
cz-cli -p prod job status <job_id>
|
|
274
|
-
|
|
275
243
|
```
|
|
276
244
|
|
|
277
245
|
Get job query results (waits if still running):
|
|
278
246
|
|
|
279
247
|
```bash
|
|
280
248
|
cz-cli -p prod job result <job_id>
|
|
281
|
-
|
|
282
249
|
```
|
|
283
250
|
|
|
284
251
|
View job execution profile (analyze performance bottlenecks):
|
|
@@ -291,28 +258,23 @@ cz-cli -p prod sql --job-profile <job_id>
|
|
|
291
258
|
|
|
292
259
|
## cz-cli workspace — Workspace Switching
|
|
293
260
|
|
|
294
|
-
```bash
|
|
295
|
-
```
|
|
296
261
|
|
|
297
262
|
View current workspace:
|
|
298
263
|
|
|
299
264
|
```bash
|
|
300
265
|
cz-cli -p prod workspace current
|
|
301
|
-
|
|
302
266
|
```
|
|
303
267
|
|
|
304
268
|
List all available workspaces:
|
|
305
269
|
|
|
306
270
|
```bash
|
|
307
271
|
cz-cli -p prod workspace list
|
|
308
|
-
|
|
309
272
|
```
|
|
310
273
|
|
|
311
274
|
Temporarily switch workspace (current command only):
|
|
312
275
|
|
|
313
276
|
```bash
|
|
314
277
|
cz-cli -p prod workspace use analytics
|
|
315
|
-
|
|
316
278
|
```
|
|
317
279
|
|
|
318
280
|
Persistent switch (saved to profile):
|
|
@@ -335,8 +297,6 @@ cz-cli -p prod table stats public.orders
|
|
|
335
297
|
|
|
336
298
|
**Scenario 2: Debugging a slow query**
|
|
337
299
|
|
|
338
|
-
```bash
|
|
339
|
-
```
|
|
340
300
|
|
|
341
301
|
Submit asynchronously first:
|
|
342
302
|
|
|
@@ -353,14 +313,11 @@ cz-cli -p prod sql --job-profile <job_id>
|
|
|
353
313
|
|
|
354
314
|
**Scenario 3: Executing DDL in CI/CD**
|
|
355
315
|
|
|
356
|
-
```bash
|
|
357
|
-
```
|
|
358
316
|
|
|
359
317
|
Validate syntax with dry-run first:
|
|
360
318
|
|
|
361
319
|
```bash
|
|
362
320
|
cz-cli -p prod sql --dry-run -f migrations/v2.sql
|
|
363
|
-
|
|
364
321
|
```
|
|
365
322
|
|
|
366
323
|
Execute after confirming no issues:
|
|
@@ -389,7 +346,7 @@ cz-cli -p prod sql \
|
|
|
389
346
|
**Lakehouse Related Documentation**
|
|
390
347
|
|
|
391
348
|
* [Workspace](workspace-introduction.md) — Workspace concepts, user management, permission system
|
|
392
|
-
* [
|
|
349
|
+
* [Virtual Cluster](virtual-cluster.md) — Virtual Cluster type selection, specification configuration
|
|
393
350
|
* [Schema](schema.md) — Schema creation and management
|
|
394
351
|
* [Time Travel](timetravel-summary.md) — Historical version queries (the underlying mechanism of the `table history` command)
|
|
395
352
|
|
|
@@ -1,36 +1,38 @@
|
|
|
1
|
-
#
|
|
1
|
+
# ClickZetta CLI (cz-cli)
|
|
2
2
|
|
|
3
|
-
cz-cli is the command-line tool for Singdata Lakehouse. For human users, it lets data developers query tables, run SQL, manage tasks, and view run logs from the terminal
|
|
3
|
+
cz-cli is the command-line operations tool for Singdata Lakehouse. For human users, it lets data developers query tables, run SQL, manage tasks, and view run logs from the terminal—no browser required. For AI Agents, it provides a high-level operation interface tailored to data warehouse scenarios, letting agents complete full operations with minimal context overhead.
|
|
4
|
+
|
|
5
|
+

|
|
4
6
|
|
|
5
7
|
## Why cz-cli?
|
|
6
8
|
|
|
7
|
-
Lakehouse already offers Studio Web UI, MCP Server, JDBC, REST API, and other access methods
|
|
9
|
+
Lakehouse already offers Studio Web UI, MCP Server, JDBC, REST API, and other access methods—but each has fundamental limitations for AI Agents. cz-cli provides differentiated value for two core scenarios:
|
|
8
10
|
|
|
9
11
|
### Scenario 1: Agent Calls Commands Directly (Tool-Call Mode)
|
|
10
12
|
|
|
11
|
-
The agent uses `cz-cli` commands to perform data warehouse operations
|
|
13
|
+
The agent uses `cz-cli` commands to perform data warehouse operations—one command per complete business action.
|
|
12
14
|
|
|
13
15
|
**Limitations of other interfaces:**
|
|
14
16
|
|
|
15
|
-
| Interface | Core
|
|
17
|
+
| Interface | Core Limitation |
|
|
16
18
|
|------|---------|
|
|
17
19
|
| **JDBC / SQL** | Agent must inject the full schema before writing SQL; large warehouse schemas can cost tens of thousands of tokens. Without schema context, hallucinations are severe (fabricated table and column names). Studio task status, run logs, and other operational actions have no SQL interface at all. |
|
|
18
|
-
| **REST API** | No unified interface discovery mechanism
|
|
19
|
-
| **MCP Server** | Each tool description costs ~300
|
|
20
|
+
| **REST API** | No unified interface discovery mechanism—agents cannot automatically know which endpoints exist. Even with an OpenAPI spec, the full spec is a context bomb. A single business action requires chaining 3-5 API calls; any intermediate failure silently breaks the flow. |
|
|
21
|
+
| **MCP Server** | Each tool description costs ~300-600 tokens; 3 MCP servers can consume 70%+ of the context window. As tool count grows, agent selection accuracy drops significantly—quality degrades noticeably beyond 50 tools. |
|
|
20
22
|
| **Studio Web UI** | Requires simulating browser interactions; many steps, unstable state, high agent operation cost, low accuracy. |
|
|
21
23
|
|
|
22
24
|
**cz-cli advantages:**
|
|
23
25
|
|
|
24
26
|
- **Self-describing and discoverable**: `cz-cli --help` and `cz-cli <subcommand> --help` are self-describing. Agents query on demand without preloading any documentation. `cz-cli ai-guide` generates a more compact task-oriented command reference.
|
|
25
|
-
- **Business-semantic encapsulation**: Each command maps to one complete business action
|
|
26
|
-
- **Covers capabilities beyond SQL**: Studio task management, run inspection, backfill, data source browsing
|
|
27
|
+
- **Business-semantic encapsulation**: Each command maps to one complete business action—agents get it done in one step, no multi-step composition, no pagination, authentication, or error code handling.
|
|
28
|
+
- **Covers capabilities beyond SQL**: Studio task management, run inspection, backfill, data source browsing—all unreachable via JDBC.
|
|
27
29
|
- **Built-in guardrails**: Write operations require `--write`; high-risk operations require `-y` confirmation, reducing agent error risk.
|
|
28
|
-
- **Structured output**: JSON by default
|
|
30
|
+
- **Structured output**: JSON by default—agents parse directly without processing natural language or HTML.
|
|
29
31
|
- **Headless-friendly**: A plain process, no persistent connection needed. Works directly in CI/CD pipelines, scheduled scripts, and enterprise bots.
|
|
30
32
|
|
|
31
33
|
### Scenario 2: As a Specialized Sub-Agent (cz-cli agent Mode)
|
|
32
34
|
|
|
33
|
-
When a primary agent (Claude Code, Cursor, Kiro, etc.) loads too many skills, two problems emerge: **context pollution
|
|
35
|
+
When a primary agent (Claude Code, Cursor, Kiro, etc.) loads too many skills, two problems emerge: **context pollution**—information accumulated from a previous task interferes with reasoning on the next—and **attention dilution**—the more tools available, the higher the probability of choosing the wrong one.
|
|
34
36
|
|
|
35
37
|
The solution is Anthropic's recommended **orchestrator-subagent pattern**: the primary agent handles planning and coordination, delegating specialized domain problems wholesale to a specialized sub-agent, which completes the work in an isolated, clean context and returns results to the primary agent.
|
|
36
38
|
|
|
@@ -44,50 +46,45 @@ Primary Agent (Claude Code)
|
|
|
44
46
|
└─ returns structured results to primary agent
|
|
45
47
|
```
|
|
46
48
|
|
|
47
|
-
Anthropic's Research feature uses the same architecture
|
|
49
|
+
Anthropic's Research feature uses the same architecture—benchmarks show a 90.2% improvement over single-agent approaches with 90% less research time. Data warehouse operations (checking task status, reading logs, backfilling, diagnosing SQL) are a natural fit for this delegation pattern—tasks are independent, boundaries are clear, and results are verifiable.
|
|
48
50
|
|
|
49
51
|
## Command Overview
|
|
50
52
|
|
|
51
|
-
| Command | Description |
|
|
52
|
-
|
|
53
|
-
| `cz-cli profile` |
|
|
54
|
-
| `cz-cli status` |
|
|
55
|
-
| `cz-cli sql` | Execute SQL queries
|
|
56
|
-
| `cz-cli schema` | View and manage
|
|
57
|
-
| `cz-cli table` | View table structure, preview data, count rows |
|
|
58
|
-
| `cz-cli workspace` | View and switch workspaces |
|
|
59
|
-
| `cz-cli task` | Create, configure, deploy,
|
|
60
|
-
| `cz-cli runs` |
|
|
61
|
-
| `cz-cli attempts` |
|
|
62
|
-
| `cz-cli job` | SQL
|
|
63
|
-
| `cz-cli datasource` |
|
|
64
|
-
| `cz-cli agent` | *(Agent)* Start
|
|
65
|
-
| `cz-cli ai-guide` | *(Agent)* Generate
|
|
53
|
+
| Command | Object | Description |
|
|
54
|
+
|------|----------|------|
|
|
55
|
+
| `cz-cli profile` | Local connection configuration | Create, switch, update Lakehouse connections |
|
|
56
|
+
| `cz-cli status` | Current connection | Verify connection is working |
|
|
57
|
+
| `cz-cli sql` | Lakehouse SQL engine | Execute SQL queries, DDL/DML |
|
|
58
|
+
| `cz-cli schema` | Schemas under Workspace | View and manage Schemas |
|
|
59
|
+
| `cz-cli table` | Lakehouse tables | View table structure, preview data, count rows |
|
|
60
|
+
| `cz-cli workspace` | Workspaces under instance | View and switch workspaces |
|
|
61
|
+
| `cz-cli task` | Studio tasks | Create, configure, deploy, execute Studio tasks |
|
|
62
|
+
| `cz-cli runs` | Studio task run records | View run logs, backfill, rerun |
|
|
63
|
+
| `cz-cli attempts` | Retry records for a single run | View retry details and logs |
|
|
64
|
+
| `cz-cli job` | SQL Jobs | SQL Job performance diagnostics |
|
|
65
|
+
| `cz-cli datasource` | Studio external data sources | Browse schemas/tables in data sources, test connectivity |
|
|
66
|
+
| `cz-cli agent` | *(Agent only)* Local AI Agent | Start AI Agent session, operate Lakehouse in natural language |
|
|
67
|
+
| `cz-cli ai-guide` | *(Agent only)* Command documentation | Generate compact command reference for agents to load |
|
|
66
68
|
|
|
67
69
|
## Quick Start
|
|
68
70
|
|
|
69
|
-
```bash
|
|
70
|
-
```
|
|
71
|
-
|
|
72
71
|
Install:
|
|
73
72
|
|
|
74
73
|
```bash
|
|
75
74
|
npm install -g @clickzetta/cz-cli
|
|
76
|
-
|
|
77
75
|
```
|
|
78
76
|
|
|
79
77
|
Configure connection:
|
|
80
78
|
|
|
81
79
|
```bash
|
|
82
|
-
cz-cli setup
|
|
83
|
-
|
|
80
|
+
cz-cli setup
|
|
84
81
|
```
|
|
82
|
+
Follow the prompts to create a new Singdata Lakehouse account connection or enter the credentials for an existing account.
|
|
85
83
|
|
|
86
84
|
Verify connection:
|
|
87
85
|
|
|
88
86
|
```bash
|
|
89
87
|
cz-cli status
|
|
90
|
-
|
|
91
88
|
```
|
|
92
89
|
|
|
93
90
|
Run a query:
|
|
@@ -100,31 +97,45 @@ For detailed installation and configuration steps, see the [Installation and Con
|
|
|
100
97
|
|
|
101
98
|
## Use Cases
|
|
102
99
|
|
|
103
|
-
| Scenario | Recommended
|
|
100
|
+
| Scenario | Recommended Approach |
|
|
104
101
|
|------|---------|
|
|
105
102
|
| Daily data queries, viewing table structure, sample data | `cz-cli sql` / `cz-cli table` |
|
|
106
103
|
| Managing and debugging Studio tasks, viewing run logs | `cz-cli task` / `cz-cli runs` |
|
|
107
104
|
| Automated DDL or task execution in CI/CD pipelines | cz-cli commands + `--write` |
|
|
108
|
-
| AI
|
|
109
|
-
| AI
|
|
105
|
+
| AI Agent performing data warehouse operations | cz-cli tool-call mode |
|
|
106
|
+
| AI Agent handling complex warehouse operations | `cz-cli agent run` sub-agent mode |
|
|
110
107
|
| Visual configuration of data sync or complex task orchestration | Studio Web UI |
|
|
111
108
|
|
|
112
|
-
##
|
|
109
|
+
## LLM Source
|
|
110
|
+
|
|
111
|
+
The LLM model used by `cz-cli agent` is provided uniformly by **[AI Gateway](AIGateway.md)**. New users are recommended to complete configuration in one step via the CLI connection string (Singdata built-in LLM, no separate API Key required). External LLMs can also be integrated via `cz-cli agent llm add`—in this case, enter the AI Gateway Endpoint as an OpenAI-compatible gateway:
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
cz-cli agent llm add my-gateway \
|
|
115
|
+
--provider openai-compatible \
|
|
116
|
+
--base-url https://<your-instance>.singdata.com/gateway/v1 \
|
|
117
|
+
--api-key <AI-Gateway-API-Key> \
|
|
118
|
+
--use
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
See [AI Agent Integration → Configure LLM](cz-cli-agent.md) for details.
|
|
122
|
+
|
|
123
|
+
## Related Documents
|
|
113
124
|
|
|
114
125
|
**cz-cli Documentation**
|
|
115
126
|
|
|
116
|
-
- [Installation and Configuration Guide](setup_cz_cli.md) — Installation,
|
|
127
|
+
- [Installation and Configuration Guide](setup_cz_cli.md) — Installation, Profile configuration, output formats, upgrades, FAQ
|
|
117
128
|
- [SQL Execution and Data Exploration](cz-cli-sql.md) — Full reference for sql, schema, table, job, workspace commands
|
|
118
129
|
- [Studio Task Development and Operations](cz-cli-studio-tasks.md) — Task creation, scheduling, runs operations, backfill, task flow
|
|
119
130
|
- [Studio External Data Source Management](cz-cli-datasource.md) — Data source browsing, connectivity testing, sample data preview
|
|
120
131
|
- [AI Agent Integration](cz-cli-agent.md) — Agent LLM configuration, natural language operations, enterprise bot scenarios
|
|
121
132
|
|
|
122
|
-
**Lakehouse
|
|
133
|
+
**Lakehouse Related Documents**
|
|
123
134
|
|
|
124
135
|
- [Studio Guide](studio_manual.md) — Complete Studio Web UI operations guide
|
|
125
136
|
- [Workspace](workspace-introduction.md) — Workspace concepts, user management, permission model
|
|
126
|
-
- [
|
|
127
|
-
- [Task Development and Scheduling](task-develop.md) — Studio SQL task development
|
|
137
|
+
- [Virtual Cluster](virtual-cluster.md) — Virtual Cluster types, specs, start/stop management
|
|
138
|
+
- [Task Development and Scheduling](task-develop.md) — Studio SQL task development, scheduling configuration
|
|
128
139
|
- [Real-time Sync Tasks](realtime_sync.md) — CDC real-time sync configuration and management
|
|
129
140
|
- [Batch Sync Tasks](batch_sync.md) — Batch offline sync configuration and management
|
|
130
141
|
- [Data Source Management](config-datasource.md) — External data source connection configuration
|
|
@@ -20,13 +20,13 @@ Now, the system automatically saves a version each time AI modifies a dashboard
|
|
|
20
20
|
|
|
21
21
|
Click the **Version History** button in the upper right corner of the dashboard page
|
|
22
22
|
|
|
23
|
-

|
|
24
24
|
|
|
25
25
|
### 2. View Historical Versions
|
|
26
26
|
|
|
27
27
|
Each version card in the version panel contains:
|
|
28
28
|
|
|
29
|
-

|
|
30
30
|
|
|
31
31
|
* Version number (V1, V2, V3...)
|
|
32
32
|
|
|
@@ -44,7 +44,7 @@ Click on a version card to view a snapshot of the dashboard at that version (rea
|
|
|
44
44
|
|
|
45
45
|
Hover over a version card and click the **Restore** button:
|
|
46
46
|
|
|
47
|
-

|
|
48
48
|
|
|
49
49
|
* The system will show a confirmation prompt: "The rollback operation will create a new version based on this version"
|
|
50
50
|
|
|
@@ -80,7 +80,7 @@ Hover over a version card and click the **Restore** button:
|
|
|
80
80
|
|
|
81
81
|
### Private to Shared Transition
|
|
82
82
|
|
|
83
|
-

|
|
84
84
|
|
|
85
85
|
* Versions before the sharing time point are visible only to the creator
|
|
86
86
|
|
|
@@ -107,3 +107,11 @@ Hover over a version card and click the **Restore** button:
|
|
|
107
107
|
2\. Restoring does not delete intermediate versions; instead, it creates a new version based on the target version
|
|
108
108
|
|
|
109
109
|
3\. The version retention limit is 100; it is recommended to periodically confirm the status of important versions
|
|
110
|
+
|
|
111
|
+
## Related Documentation
|
|
112
|
+
|
|
113
|
+
* [Chart Auto-Refresh Settings](chart-auto-refresh-guide.md) — Set automatic data updates for dashboard charts
|
|
114
|
+
* [Table Rendering](table_rendering.md) — Generate complex table layouts through natural language
|
|
115
|
+
* [Conversational Data Analytics (Analytics Agent)](datagpt_introduction.md) — Return to feature overview
|
|
116
|
+
|
|
117
|
+
^
|
|
@@ -12,7 +12,7 @@ A data synchronization task is a type of task used to collect data from a data s
|
|
|
12
12
|
|
|
13
13
|
Lakehouse Studio currently supports two main types of synchronization tasks: offline periodic synchronization and real-time synchronization. In the "Development -> Tasks" section, use the new button to select the appropriate task type as needed.
|
|
14
14
|
|
|
15
|
-

|
|
15
|
+
:-: 
|
|
16
16
|
|
|
17
17
|
^
|
|
18
18
|
|
|
@@ -4,13 +4,13 @@ Singdata Lakehouse Studio includes a full-featured data integration capability t
|
|
|
4
4
|
|
|
5
5
|
## Choosing a Sync Approach
|
|
6
6
|
|
|
7
|
-
| My scenario
|
|
8
|
-
|
|
9
|
-
| MySQL / PG / Oracle or similar — single-table real-time CDC | Real-time sync task
|
|
10
|
-
| Full-database migration, syncing multiple tables together
|
|
11
|
-
| Scheduled batch sync (T+1 or hourly)
|
|
12
|
-
| Full-database offline migration, multiple tables at once
|
|
13
|
-
| Not sure which to use
|
|
7
|
+
| My scenario | Recommended approach | Documentation |
|
|
8
|
+
| ----------------------------------------------------------- | -------------------------- | ---------------------------------------------------------------------------- |
|
|
9
|
+
| MySQL / PG / Oracle or similar — single-table real-time CDC | Real-time sync task | [Real-time Sync Task](realtime_sync.md) |
|
|
10
|
+
| Full-database migration, syncing multiple tables together | Multi-table real-time sync | [Multi-table Real-time Sync Complete Guide](multitable_realtime_sync_sop.md) |
|
|
11
|
+
| Scheduled batch sync (T+1 or hourly) | Offline sync task | [Offline Sync Task](batch_sync.md) |
|
|
12
|
+
| Full-database offline migration, multiple tables at once | Multi-table offline sync | [Multi-table Offline Sync Task](multitable_batch_sync.md) |
|
|
13
|
+
| Not sure which to use | Check supported sources | [Data Source Support](data-sources.md) |
|
|
14
14
|
|
|
15
15
|
## Quick Start
|
|
16
16
|
|
|
@@ -26,26 +26,28 @@ Singdata Lakehouse Studio includes a full-featured data integration capability t
|
|
|
26
26
|
|
|
27
27
|
## Scheduling, Deployment, and Operations
|
|
28
28
|
|
|
29
|
-
| Scenario
|
|
30
|
-
|
|
31
|
-
| Configure scheduled runs and task dependencies
|
|
32
|
-
| Publish / unpublish a task
|
|
33
|
-
| View run history, logs, and failure details
|
|
34
|
-
| Configure alert notifications (DingTalk / Lark / email) | [Monitoring and Alerting](monitoring_and_alerting.md)
|
|
35
|
-
| Understand monitoring metric definitions
|
|
36
|
-
| Backfill historical data
|
|
29
|
+
| Scenario | Documentation |
|
|
30
|
+
| ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
|
|
31
|
+
| Configure scheduled runs and task dependencies | [Task Scheduling](task_scheduling.md) · [Task Scheduling Dependencies](task_scheduling_dependency.md) |
|
|
32
|
+
| Publish / unpublish a task | [Task and Instance Operations](task-instance-maintenance.md) |
|
|
33
|
+
| View run history, logs, and failure details | [Task and Instance Operations](task-instance-maintenance.md) |
|
|
34
|
+
| Configure alert notifications (DingTalk / Lark / email) | [Monitoring and Alerting](monitoring_and_alerting.md) |
|
|
35
|
+
| Understand monitoring metric definitions | [Monitoring Metric Specifications](monitoring_item_specification.md) |
|
|
36
|
+
| Backfill historical data | [Data Backfill Tasks](backfilling_data.md) |
|
|
37
37
|
|
|
38
38
|
## Contents of This Section
|
|
39
39
|
|
|
40
|
-
| Page
|
|
41
|
-
|
|
42
|
-
| [Overview](data-integration-intro.md)
|
|
43
|
-
| [Data Source Management](config-datasource.md)
|
|
44
|
-
| [Supported Data Sources](data-sources.md)
|
|
45
|
-
| [Data Source Configuration Guide](datasource-config-guide.md)
|
|
46
|
-
| [Offline Sync Task](batch_sync.md)
|
|
47
|
-
| [Multi-table Offline Sync Task](multitable_batch_sync.md)
|
|
48
|
-
| [Real-time Sync Task](realtime_sync.md)
|
|
49
|
-
| [Multi-table Real-time Sync Task](multitable_realtime_sync.md)
|
|
50
|
-
| [Multi-table Real-time Sync Complete Guide](multitable_realtime_sync_sop.md) | End-to-end SOP for full-database migration
|
|
51
|
-
| [Offline Sync FAQ](batch_sync_sop.md)
|
|
40
|
+
| Page | Description |
|
|
41
|
+
| ---------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
|
42
|
+
| [Overview](data-integration-intro.md) | Basic concepts, task types, and core components of data sync |
|
|
43
|
+
| [Data Source Management](config-datasource.md) | Add and manage data source connections |
|
|
44
|
+
| [Supported Data Sources](data-sources.md) | List of 40+ supported data sources |
|
|
45
|
+
| [Data Source Configuration Guide](datasource-config-guide.md) | Detailed setup steps for each database type |
|
|
46
|
+
| [Offline Sync Task](batch_sync.md) | Scheduled batch sync with full and incremental modes |
|
|
47
|
+
| [Multi-table Offline Sync Task](multitable_batch_sync.md) | Full-database offline migration |
|
|
48
|
+
| [Real-time Sync Task](realtime_sync.md) | Single-table real-time CDC sync |
|
|
49
|
+
| [Multi-table Real-time Sync Task](multitable_realtime_sync.md) | Full-database real-time CDC sync |
|
|
50
|
+
| [Multi-table Real-time Sync Complete Guide](multitable_realtime_sync_sop.md) | End-to-end SOP for full-database migration |
|
|
51
|
+
| [Offline Sync FAQ](batch_sync_sop.md) | Common questions and usage guide |
|
|
52
|
+
|
|
53
|
+
^
|
|
@@ -15,8 +15,8 @@ There are various methods to load data into Lakehouse, depending on the data sou
|
|
|
15
15
|
| | [Use Jdbc client](load-data-local.md) In versions after JDBC 2.0.0, the local COPY command has been deprecated. We recommend using the PUT method to upload data to the volume, and then use the server-side COPY command to import. | 1. Small data volume without a visual interface, suitable for technical users, especially those familiar with command line operations and need to batch process small data volumes. 2. Quickly process CSV files, suitable for scripting and automation operations. |
|
|
16
16
|
| Batch Import Data from Object Storage | [Use Volume to batch load data from object storage](load-data-oss.md) | 1. Suitable for data stored in object storage, needing to leverage SQL performance advantages to quickly read large amounts of data. 2. Efficiently handle large-scale datasets. 3. Supports direct SQL data conversion, simplifying the data import process. 4. Supports multiple file formats, including CSV, PARQUET, ORC |
|
|
17
17
|
| | [Import using Copy command](load-data-oss.md) | 1. Has the advantages of Volume, with the difference being that the Copy command provides more fault tolerance parameters and data export support. It also supports exporting to object storage data. |
|
|
18
|
-
| Use JavaSDK to read Kafka and write to Lakehouse in real-time | [Import data using Java SDK](use-java-sdk-realtime-uploaddata.md)
|
|
19
|
-
| Use custom SDK to read files and write to Lakehouse | [Import data using Java SDK](use-java-sdk-upload-data-local.md)
|
|
18
|
+
| Use JavaSDK to read Kafka and write to Lakehouse in real-time | [Import data using Java SDK](use-java-sdk-realtime-uploaddata.md) | 1. Suitable for business scenarios requiring real-time data stream processing, especially for developers familiar with Java. 2. Real-time data reading and writing ensure the immediate availability of data. 3. Suitable for custom data import, providing high flexibility. |
|
|
19
|
+
| Use custom SDK to read files and write to Lakehouse | [Import data using Java SDK](use-java-sdk-upload-data-local.md) | 1. Suitable for one-time bulk data import with low data frequency (time interval greater than five minutes). 2. Supports custom data sources, providing flexibility in data import. |
|
|
20
20
|
| | [Import data using Python SDK](use-python-sdk-upload-data.md) | 1. Suitable for developers familiar with Python who need custom data import, especially when the data is not in object storage or the integration does not support the data source. |
|
|
21
21
|
| Users needing fully managed services and visual operations, especially for synchronizing third-party data sources | Use Lakehouse Studio data integration | 1. Supports a wide range of data sources and various import methods. 2. Provides real-time synchronization, CDC mirroring synchronization, and offline periodic scheduling synchronization. 3. Visual monitoring improves data management transparency. |
|
|
22
22
|
|
|
@@ -24,7 +24,7 @@ There are various methods to load data into Lakehouse, depending on the data sou
|
|
|
24
24
|
|
|
25
25
|
| Import Method Name | Usage | Usage Notes |
|
|
26
26
|
| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
27
|
-
| COPY Command
|
|
27
|
+
| COPY Command | [COPY INTO Command](copy-into-table.md) | 1. Currently supported object storage locations are Tencent Cloud COS and Alibaba Cloud OSS. 2. Cross-cloud vendor import is not supported (will be supported later). For example, your Lakehouse is hosted on Alibaba Cloud but the object storage is on Tencent Cloud. |
|
|
28
28
|
| Volume | [Use ISNERT INTO... SELECT FORM VOLUME to import](datalake_volume.md) | 1. Currently supported object storage locations are Tencent Cloud COS and Alibaba Cloud OSS. 2. Cross-cloud vendor import is not supported (will be supported later). For example, your Lakehouse is hosted on Alibaba Cloud but the object storage is on Tencent Cloud. |
|
|
29
29
|
| Data Integration | [Use Data Integration to import](data-integration-intro.md) | 1. Supports a wide range of data sources, performance is slightly worse than COPY command and Volume. |
|
|
30
30
|
| SDK Realtime Data Stream (RealtimeStream) | [Upload data in real-time](java_reference/realtime-upload.md) | 1. Data written in real-time can be queried in seconds. 2. When changing the table structure, you need to stop the real-time writing task first, then restart the task after a period of time (about 90 minutes) after the table structure change. 3. Table stream, materialized view, and dynamic table can only display committed data. Data written by real-time tasks needs to wait 1 minute to be confirmed, so table stream also needs to wait 1 minute to see it. |
|