@clickzetta/cz-cli-darwin-x64 0.5.16 → 0.5.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/cz-cli +0 -0
- package/bin/skills/lakehouse-doc-en/SKILL.md +6 -11
- package/bin/skills/lakehouse-doc-en/references/AIGateway.md +58 -13
- package/bin/skills/lakehouse-doc-en/references/Computation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/DataSource_Amazon_DocumentDB.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/Foreach.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/JDBC-Driver.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/LakehouseAI-overview.md +21 -8
- package/bin/skills/lakehouse-doc-en/references/LakehouseDataGPT-tour.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/LakehouseStudio-tour.md +14 -19
- package/bin/skills/lakehouse-doc-en/references/Lakehouse_Zilliz_MakeDataReadyforBIandAI.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Logstash.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/Migrate_Spark_DataEngineeringBestPractices_Project_to_Lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/Notebook.md +17 -17
- package/bin/skills/lakehouse-doc-en/references/RemoteFunction-as-udf.md +14 -14
- package/bin/skills/lakehouse-doc-en/references/SQL_External_Catalog_Guide.md +1 -9
- package/bin/skills/lakehouse-doc-en/references/SUMMARY.md +59 -29
- package/bin/skills/lakehouse-doc-en/references/WINDOWFUNCTION.md +99 -57
- package/bin/skills/lakehouse-doc-en/references/Zettapark_Data_Engineering_Demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/access-control-configuration.md +1 -8
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-2-5-1.0.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-29-1.0.2.md +14 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-3-8-1.0.1.md +16 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-4-28-1.1.md +29 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-12-1.1.1.md +18 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-15-1.2.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-21-1.3.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-5-28-1.4.md +10 -0
- package/bin/skills/lakehouse-doc-en/references/aigw-2026-6-3-1.5.md +9 -0
- package/bin/skills/lakehouse-doc-en/references/alicloud-arn-externalid.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/answer-accuracy-improve.md +120 -103
- package/bin/skills/lakehouse-doc-en/references/application-list.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/approval-list.md +16 -17
- package/bin/skills/lakehouse-doc-en/references/batch-load-parquet-file-into-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/batch_sync.md +9 -9
- package/bin/skills/lakehouse-doc-en/references/batch_sync_Sop.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/batchloadparquetfileintoLakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/bulkloadv1-python-sdk.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/chart-auto-refresh-guide.md +12 -6
- package/bin/skills/lakehouse-doc-en/references/clickzetta-sample-data.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/code_approval.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/composite_task.md +31 -42
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_environment_and_data_generate.md +6 -9
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_javasdk_bulkload_realtime.md +4 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_kafka_realtime_sync.md +1 -10
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_local_file_into_table_by_studio.md +0 -6
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_batchload_public_network.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_python_node.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_realtime_cdc_public_network.md +13 -18
- package/bin/skills/lakehouse-doc-en/references/comprehensive_guide_to_ingesting_studio_sql_insert.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/concepts.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/config-datasource.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/connect-with-cli.md +116 -72
- package/bin/skills/lakehouse-doc-en/references/connect-with-cz-cli.md +151 -0
- package/bin/skills/lakehouse-doc-en/references/continue-job.md +9 -17
- package/bin/skills/lakehouse-doc-en/references/create-api-connection.md +315 -286
- package/bin/skills/lakehouse-doc-en/references/create-catalog-connection.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/create-dynamic-table.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/create-external-catalog.md +85 -22
- package/bin/skills/lakehouse-doc-en/references/create-table-ddl.md +45 -0
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkendpoint.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/creating_alicloud_privatelinkservice.md +4 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkendpoint.md +2 -7
- package/bin/skills/lakehouse-doc-en/references/creating_tencentcloud_privatelinkservice.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/cz-cli-agent.md +15 -10
- package/bin/skills/lakehouse-doc-en/references/cz-cli-datasource.md +0 -8
- package/bin/skills/lakehouse-doc-en/references/cz-cli-sql.md +2 -45
- package/bin/skills/lakehouse-doc-en/references/cz-cli.md +53 -42
- package/bin/skills/lakehouse-doc-en/references/dashboard-version-management-guide.md +12 -4
- package/bin/skills/lakehouse-doc-en/references/data-integration-intro.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/data-integration.md +29 -27
- package/bin/skills/lakehouse-doc-en/references/data-load-summary.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/data-quality.md +25 -25
- package/bin/skills/lakehouse-doc-en/references/data-sharing.md +31 -54
- package/bin/skills/lakehouse-doc-en/references/data-sources.md +45 -45
- package/bin/skills/lakehouse-doc-en/references/data_catalog.md +23 -25
- package/bin/skills/lakehouse-doc-en/references/data_privacy.md +5 -2
- package/bin/skills/lakehouse-doc-en/references/data_sharing_between_accounts_guide.md +0 -4
- package/bin/skills/lakehouse-doc-en/references/data_visualization.md +4 -15
- package/bin/skills/lakehouse-doc-en/references/dataagent.md +39 -7
- package/bin/skills/lakehouse-doc-en/references/databricks-delta-to-lakehouse-migration.md +168 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-dlt-to-lakehouse-migration.md +331 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-external-catalog-practice.md +367 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-jobs-to-studio-migration.md +199 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-notebook-to-studio-migration.md +350 -0
- package/bin/skills/lakehouse-doc-en/references/databricks-uc-governance-to-lakehouse-migration.md +327 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt-model-config.md +34 -0
- package/bin/skills/lakehouse-doc-en/references/datagpt_data_source.md +50 -37
- package/bin/skills/lakehouse-doc-en/references/datagpt_introduction.md +55 -79
- package/bin/skills/lakehouse-doc-en/references/datagpt_quickstart.md +50 -64
- package/bin/skills/lakehouse-doc-en/references/datalake-acceleration.md +75 -2
- package/bin/skills/lakehouse-doc-en/references/dbt-databricks-to-clickzetta-migration.md +242 -0
- package/bin/skills/lakehouse-doc-en/references/dynamic-mask.md +30 -30
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-bestpractice.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic-table-introduce.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/dynamic_table_summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/streamlit.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/eco_integration/superset.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/ecosystem-all.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/ecosystem.md +145 -0
- package/bin/skills/lakehouse-doc-en/references/external-catalog-summary.md +33 -38
- package/bin/skills/lakehouse-doc-en/references/external-function-combo-practice.md +466 -0
- package/bin/skills/lakehouse-doc-en/references/f6fc6447ee.md +7 -9
- package/bin/skills/lakehouse-doc-en/references/federation-query.md +56 -6
- package/bin/skills/lakehouse-doc-en/references/finebi-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/get-started-with-sample-data.md +10 -11
- package/bin/skills/lakehouse-doc-en/references/gitfolder.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/grant-privileges.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/iceberg-rest-catalog-databricks.md +166 -0
- package/bin/skills/lakehouse-doc-en/references/ide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/if_else_task.md +59 -57
- package/bin/skills/lakehouse-doc-en/references/input_output.md +10 -7
- package/bin/skills/lakehouse-doc-en/references/jobprofile-bestpractices.md +60 -64
- package/bin/skills/lakehouse-doc-en/references/kafka-connection.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/key-concepts.md +146 -117
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-gateway-cz-cli.md +317 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-ai-sql-analysis.md +345 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-dqc-guide.md +300 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-medallion-sql-dt-guide.md +543 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multi-cloud-acceleration.md +274 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-multimodal-ai-pipeline.md +198 -0
- package/bin/skills/lakehouse-doc-en/references/lakehouse-quick-experience_guide.md +49 -52
- package/bin/skills/lakehouse-doc-en/references/lakehouse-volume-pipe-acceleration-guide.md +380 -0
- package/bin/skills/lakehouse-doc-en/references/langchain-plug-installation.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/management.md +4 -9
- package/bin/skills/lakehouse-doc-en/references/medallion-lakehouse-from-scratch.md +2 -1
- package/bin/skills/lakehouse-doc-en/references/metrics_answer_build.md +58 -21
- package/bin/skills/lakehouse-doc-en/references/migrate-spark-data-engineering-best-practices-to-lakehouse.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/mindsdb.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/monitoring_and_alerting.md +65 -60
- package/bin/skills/lakehouse-doc-en/references/monitoring_item_specification.md +33 -33
- package/bin/skills/lakehouse-doc-en/references/multitable_batch_sync.md +16 -16
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync.md +65 -72
- package/bin/skills/lakehouse-doc-en/references/multitable_realtime_sync_sop.md +54 -52
- package/bin/skills/lakehouse-doc-en/references/navicat-mysql.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/om-dynamic-table.md +71 -66
- package/bin/skills/lakehouse-doc-en/references/om-vcluster.md +2 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-create-session.md +79 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-generate-auth-token.md +63 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-overview.md +96 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-quick-start.md +286 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-response-guide.md +264 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-safe-question-poll.md +201 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-query.md +99 -0
- package/bin/skills/lakehouse-doc-en/references/open-api-text2insight-stop.md +74 -0
- package/bin/skills/lakehouse-doc-en/references/overview.md +6 -7
- package/bin/skills/lakehouse-doc-en/references/permission-application.md +5 -5
- package/bin/skills/lakehouse-doc-en/references/pipe-introduction.md +1 -0
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka-table-stream.md +72 -70
- package/bin/skills/lakehouse-doc-en/references/pipe-kafka.md +105 -110
- package/bin/skills/lakehouse-doc-en/references/pipe-overview.md +40 -40
- package/bin/skills/lakehouse-doc-en/references/pipe-storage-object.md +43 -48
- package/bin/skills/lakehouse-doc-en/references/pipe-summary.md +14 -4
- package/bin/skills/lakehouse-doc-en/references/pipe-syntax.md +58 -151
- package/bin/skills/lakehouse-doc-en/references/practice_python_task.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/pricing-ai-gateway.md +181 -0
- package/bin/skills/lakehouse-doc-en/references/pricing-lakehouse.md +316 -0
- package/bin/skills/lakehouse-doc-en/references/pricing.md +44 -288
- package/bin/skills/lakehouse-doc-en/references/private-link-general.md +0 -2
- package/bin/skills/lakehouse-doc-en/references/pyspark-to-zettapark-migration-f1.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-igs.md +7 -3
- package/bin/skills/lakehouse-doc-en/references/python-sample-put-github-rt-events.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_advanced.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_reference/connector_examples.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/python_sdk_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/python_shell_datasource.md +11 -9
- package/bin/skills/lakehouse-doc-en/references/quick_start_batch_sync_data.md +9 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_bi_analysis.md +8 -25
- package/bin/skills/lakehouse-doc-en/references/quick_start_create_workspace.md +4 -6
- package/bin/skills/lakehouse-doc-en/references/quick_start_data_quality.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_etl.md +16 -20
- package/bin/skills/lakehouse-doc-en/references/quick_start_monitoring_and_alerting.md +10 -18
- package/bin/skills/lakehouse-doc-en/references/quick_start_sql_query.md +7 -10
- package/bin/skills/lakehouse-doc-en/references/quick_start_upload_data.md +5 -7
- package/bin/skills/lakehouse-doc-en/references/quick_start_user_management.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quick_start_workspace_user.md +8 -8
- package/bin/skills/lakehouse-doc-en/references/quickstart.md +69 -56
- package/bin/skills/lakehouse-doc-en/references/quickstart_datashare_between_companies.md +0 -5
- package/bin/skills/lakehouse-doc-en/references/quickstart_envirment_for_team.md +0 -24
- package/bin/skills/lakehouse-doc-en/references/realtime-pipeline-selection-guide.md +1 -2
- package/bin/skills/lakehouse-doc-en/references/realtime-sales-dashboard-with-dynamic-table.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/realtime_sync.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/release-note-2026-05-19.md +5 -3
- package/bin/skills/lakehouse-doc-en/references/revoke-privileges.md +3 -1
- package/bin/skills/lakehouse-doc-en/references/roles.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/row-filter.md +165 -0
- package/bin/skills/lakehouse-doc-en/references/row_level_permission.md +30 -19
- package/bin/skills/lakehouse-doc-en/references/scheduled_task.md +28 -21
- package/bin/skills/lakehouse-doc-en/references/security_overview.md +99 -21
- package/bin/skills/lakehouse-doc-en/references/set-command.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/setup.md +13 -15
- package/bin/skills/lakehouse-doc-en/references/show-grants.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/snowflake-dynamic-tables-to-lakehouse.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/spark-connector-summary.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sql_functions/context_functions/current_vcluster.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/sso-configuration.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/streaming_pipeline_with_dynamic_table.md +0 -1
- package/bin/skills/lakehouse-doc-en/references/studio-incremental-sync-practice.md +27 -23
- package/bin/skills/lakehouse-doc-en/references/studio-shell-task.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/supported-cloud-platforms.md +32 -0
- package/bin/skills/lakehouse-doc-en/references/table_rendering.md +18 -12
- package/bin/skills/lakehouse-doc-en/references/task-develop.md +89 -91
- package/bin/skills/lakehouse-doc-en/references/task_development.md +19 -17
- package/bin/skills/lakehouse-doc-en/references/task_group.md +16 -14
- package/bin/skills/lakehouse-doc-en/references/task_instance.md +21 -21
- package/bin/skills/lakehouse-doc-en/references/task_param.md +38 -35
- package/bin/skills/lakehouse-doc-en/references/task_param_reference.md +81 -79
- package/bin/skills/lakehouse-doc-en/references/task_scheduling_dependency.md +20 -21
- package/bin/skills/lakehouse-doc-en/references/tencentcloud_arn_and_externalid.md +1 -5
- package/bin/skills/lakehouse-doc-en/references/trial-account-quotas-and-limits.md +1 -3
- package/bin/skills/lakehouse-doc-en/references/tutorial_connect_to_lakehouse.md +69 -0
- package/bin/skills/lakehouse-doc-en/references/tutorials.md +4 -1
- package/bin/skills/lakehouse-doc-en/references/unique-key.md +167 -0
- package/bin/skills/lakehouse-doc-en/references/usageandbillingview.md +138 -0
- package/bin/skills/lakehouse-doc-en/references/use-dbt-dev.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-realtime-uploaddata.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/use-java-sdk-upload-data-local.md +3 -3
- package/bin/skills/lakehouse-doc-en/references/use-models.md +128 -0
- package/bin/skills/lakehouse-doc-en/references/use-mysql-client.md +81 -81
- package/bin/skills/lakehouse-doc-en/references/use-python-sdk-upload-data.md +10 -12
- package/bin/skills/lakehouse-doc-en/references/user-identification.md +2 -3
- package/bin/skills/lakehouse-doc-en/references/user_permission_grand_guide.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/using-udf-in-dynamic-table.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/vc_cache.md +18 -22
- package/bin/skills/lakehouse-doc-en/references/vcluster_size_description.md +33 -31
- package/bin/skills/lakehouse-doc-en/references/virtual-cluster.md +43 -45
- package/bin/skills/lakehouse-doc-en/references/web-job-history.md +94 -108
- package/bin/skills/lakehouse-doc-en/references/web_search.md +16 -7
- package/bin/skills/lakehouse-doc-en/references/zettapark-data-engineering-demo.md +1 -1
- package/bin/skills/lakehouse-doc-en/references/zettapark-dataframe-guide.md +144 -70
- package/bin/skills/lakehouse-doc-en/references/zettapark-dynamic-table-guide.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-etl-guide.md +73 -33
- package/bin/skills/lakehouse-doc-en/references/zettapark-feature-engineering.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-functions-guide.md +75 -46
- package/bin/skills/lakehouse-doc-en/references/zettapark-quick-start.md +2 -2
- package/bin/skills/lakehouse-doc-en/references/zettapark-stream-guide.md +4 -4
- package/bin/skills/lakehouse-doc-en/references/zettapark-volume-guide.md +93 -29
- package/package.json +1 -1
- package/bin/skills/lakehouse-doc-en/references/CLAUDE.md +0 -606
- package/bin/skills/lakehouse-doc-en/references/modelprice.md +0 -155
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
# Use Models
|
|
2
|
+
|
|
3
|
+
After selecting a model in the Model Marketplace, you can view the model's endpoint information and usage examples, and copy the code directly to start calling.
|
|
4
|
+
|
|
5
|
+
## Finding Model Endpoints
|
|
6
|
+
|
|
7
|
+
Go to the **Model Marketplace**, click on any model to enter its detail page, where you can see:
|
|
8
|
+
|
|
9
|
+
- **Endpoint Management**: Lists the available endpoint names, service providers, status, and billing information for that model
|
|
10
|
+
- **Model Usage Examples**: Provides curl and Python code examples that can be copied and used directly
|
|
11
|
+
|
|
12
|
+
The endpoint name format is `{provider}@{model_name}@{interface_type}`, for example:
|
|
13
|
+
|
|
14
|
+
- `aliyun_bailian_bj@qwen/qwen3.7-max@open_ai` — Alibaba Cloud Bailian-Beijing, OpenAI-compatible interface
|
|
15
|
+
- `aws_bedrock_us_east_1@anthropic/claude-opus-4.6@anthropic` — AWS Bedrock US-East-1, Anthropic interface
|
|
16
|
+
|
|
17
|
+
## Two Interface Formats
|
|
18
|
+
|
|
19
|
+
AI Gateway supports two calling interfaces, depending on the protocol type of the model:
|
|
20
|
+
|
|
21
|
+
| Interface Type | Path | Applicable Models |
|
|
22
|
+
|----------|------|----------|
|
|
23
|
+
| OpenAI-compatible | `/gateway/v1/chat/completions` | Qwen, DeepSeek, GLM, Kimi, Doubao, GPT, etc. |
|
|
24
|
+
| Anthropic-compatible | `/gateway/v1/messages` | Claude series |
|
|
25
|
+
|
|
26
|
+
## curl Calls
|
|
27
|
+
|
|
28
|
+
### OpenAI-Compatible Interface (Qwen, etc.)
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
curl -X POST https://cn-shanghai-alicloud-aimesh.api.clickzetta.com/gateway/v1/chat/completions \
|
|
32
|
+
-H "Authorization: Bearer $API_KEY" \
|
|
33
|
+
-H "Content-Type: application/json" \
|
|
34
|
+
-d '{
|
|
35
|
+
"model": "qwen/qwen3.7-max",
|
|
36
|
+
"messages": [{"role": "user", "content": "hello"}]
|
|
37
|
+
}'
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
### Anthropic-Compatible Interface (Claude)
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
curl -X POST https://cn-shanghai-alicloud-aimesh.api.clickzetta.com/gateway/v1/messages \
|
|
44
|
+
-H "x-api-key: $API_KEY" \
|
|
45
|
+
-H "anthropic-version: 2023-06-01" \
|
|
46
|
+
-H "Content-Type: application/json" \
|
|
47
|
+
-d '{
|
|
48
|
+
"model": "anthropic/claude-opus-4.6",
|
|
49
|
+
"messages": [{"role": "user", "content": "hello"}],
|
|
50
|
+
"max_tokens": 256
|
|
51
|
+
}'
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Python Calls
|
|
55
|
+
|
|
56
|
+
### OpenAI-Compatible Interface (Recommended)
|
|
57
|
+
|
|
58
|
+
Use the `openai` library, pointing `base_url` to the AI Gateway endpoint:
|
|
59
|
+
|
|
60
|
+
```python
|
|
61
|
+
from openai import OpenAI
|
|
62
|
+
|
|
63
|
+
client = OpenAI(
|
|
64
|
+
base_url="https://cn-shanghai-alicloud-aimesh.api.clickzetta.com/gateway/v1",
|
|
65
|
+
api_key="<your-api-key>",
|
|
66
|
+
)
|
|
67
|
+
|
|
68
|
+
response = client.chat.completions.create(
|
|
69
|
+
model="qwen/qwen3.7-max",
|
|
70
|
+
messages=[{"role": "user", "content": "hello"}],
|
|
71
|
+
)
|
|
72
|
+
print(response.choices[0].message.content)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
### Anthropic-Compatible Interface (Claude)
|
|
76
|
+
|
|
77
|
+
Use the `requests` library to call the Anthropic Messages interface directly:
|
|
78
|
+
|
|
79
|
+
```python
|
|
80
|
+
import requests
|
|
81
|
+
|
|
82
|
+
url = "https://cn-shanghai-alicloud-aimesh.api.clickzetta.com/gateway/v1/messages"
|
|
83
|
+
headers = {
|
|
84
|
+
"x-api-key": "${API_KEY}",
|
|
85
|
+
"anthropic-version": "2023-06-01",
|
|
86
|
+
"Content-Type": "application/json",
|
|
87
|
+
}
|
|
88
|
+
payload = {
|
|
89
|
+
"model": "anthropic/claude-opus-4.6",
|
|
90
|
+
"messages": [{"role": "user", "content": "hello"}],
|
|
91
|
+
"max_tokens": 256,
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
resp = requests.post(url, headers=headers, json=payload, timeout=60)
|
|
95
|
+
print(resp.status_code)
|
|
96
|
+
print(resp.text)
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
You can also use the official `anthropic` SDK, replacing `base_url` with the AI Gateway address:
|
|
100
|
+
|
|
101
|
+
```python
|
|
102
|
+
import anthropic
|
|
103
|
+
|
|
104
|
+
client = anthropic.Anthropic(
|
|
105
|
+
base_url="https://cn-shanghai-alicloud-aimesh.api.clickzetta.com/gateway/v1",
|
|
106
|
+
api_key="<your-api-key>",
|
|
107
|
+
)
|
|
108
|
+
|
|
109
|
+
message = client.messages.create(
|
|
110
|
+
model="anthropic/claude-opus-4.6",
|
|
111
|
+
max_tokens=256,
|
|
112
|
+
messages=[{"role": "user", "content": "hello"}],
|
|
113
|
+
)
|
|
114
|
+
print(message.content)
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
## Notes
|
|
118
|
+
|
|
119
|
+
- Endpoint addresses differ by deployment region; use the actual example code in the Model Marketplace as the reference
|
|
120
|
+
- OpenAI-compatible interface uses `Authorization: Bearer <key>`; Anthropic interface uses `x-api-key: <key>`
|
|
121
|
+
- Model names must exactly match the model identifier in the endpoint; copy from the Model Marketplace detail page
|
|
122
|
+
- API Keys must be created in advance in **API KEY Management** with calling permissions enabled for the corresponding model
|
|
123
|
+
|
|
124
|
+
## Related Documents
|
|
125
|
+
|
|
126
|
+
- [Quick Start](quickstart.md) — Create API Key and configure routing
|
|
127
|
+
- [Model Pricing](pricing-ai-gateway.md) — Billing standards for each model
|
|
128
|
+
- [Call LLMs in SQL](lakehouse-ai-sql-analysis.md) — Calling via AI_COMPLETE in Lakehouse SQL
|
|
@@ -1,6 +1,5 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Connect Using MySQL Protocol
|
|
2
2
|
|
|
3
|
-
\[**Preview Release] This feature is currently in public preview**.
|
|
4
3
|
|
|
5
4
|
| Cloud Provider | Region | Compatible MySQL Protocol Version |
|
|
6
5
|
| -------------- | --------- | --------------------------------- |
|
|
@@ -12,75 +11,75 @@
|
|
|
12
11
|
| Amazon | Beijing | MySQL8.x |
|
|
13
12
|
| | Singapore | MySQL8.x |
|
|
14
13
|
|
|
15
|
-
Lakehouse supports the MySQL client-server communication protocol, so you can use MySQL drivers to connect to Lakehouse. However, Lakehouse does not implement MySQL syntax and data types. When connecting to Lakehouse via a MySQL client,
|
|
14
|
+
Lakehouse supports the MySQL client-server communication protocol, so you can use MySQL drivers to connect to Lakehouse. However, Lakehouse does not implement MySQL syntax and data types. When connecting to Lakehouse via a MySQL client, SQL statements should use Lakehouse syntax, not MySQL syntax. For example, the `mysqldump` command is not available in Lakehouse.
|
|
16
15
|
|
|
17
|
-
To accommodate some BI reports, Lakehouse has implemented some commonly used MySQL functions, such as `str_to_date` and `date_format`.
|
|
18
|
-
|
|
16
|
+
To accommodate some BI reports, Lakehouse has implemented some commonly used MySQL functions, such as `str_to_date` and `date_format`.
|
|
17
|
+
|
|
18
|
+
When a MySQL 8.x client connects to the server, the default authentication method is `caching_sha2_password`. For more information on MySQL authentication methods, refer to the [official documentation](https://dev.mysql.com/doc/refman/8.0/en/authentication-plugins.html). Connection example for Alibaba Cloud Shanghai:
|
|
19
19
|
|
|
20
20
|
```
|
|
21
|
-
jdbc:mysql://
|
|
22
|
-
--Add port, optional
|
|
23
|
-
jdbc:mysql://
|
|
21
|
+
jdbc:mysql://cn-shanghai-alicloud-mysql.api.singdata.com/public
|
|
22
|
+
-- Add port, optional
|
|
23
|
+
jdbc:mysql://cn-shanghai-alicloud-mysql.api.singdata.com:3306/public
|
|
24
|
+
-- Use SSL, optional
|
|
25
|
+
jdbc:mysql://cn-shanghai-alicloud-mysql.api.singdata.com/public?useSSL=true
|
|
24
26
|
```
|
|
25
27
|
|
|
26
|
-
|
|
27
|
-
When the MySQL 5.x version client connects to the server, it uses the mysql_native_password authentication method by default. For more information on MySQL authentication methods, please refer to the [official documentation](https://dev.mysql.com/doc/refman/8.0/en/authentication-plugins.html). Connecting to Lakehouse does not require SSL to be enabled. The connection method is as follows, for example, connecting to Shanghai Alibaba Cloud.
|
|
28
|
+
When a MySQL 5.x client connects to the server, the default authentication method is `mysql_native_password`. For more information, refer to the [official documentation](https://dev.mysql.com/doc/refman/8.0/en/authentication-plugins.html). SSL is not required when connecting to Lakehouse. Connection example for Alibaba Cloud Shanghai:
|
|
28
29
|
|
|
29
|
-
* Currently, you need to reset the password
|
|
30
|
-
```
|
|
30
|
+
* Currently, you need to reset the password even for newly created accounts. This is because MySQL 5.x uses the `mysql_native_password` key, and Lakehouse needs to store MySQL's encryption algorithm. Lakehouse only stores MySQL key encryption algorithms when the password is changed. You can keep the same password when modifying it to avoid affecting other task connections.
|
|
31
31
|
|
|
32
32
|
```
|
|
33
|
-
jdbc:mysql://
|
|
34
|
-
--Add port, optional
|
|
35
|
-
jdbc:mysql://
|
|
33
|
+
jdbc:mysql://cn-shanghai-alicloud-mysql.api.singdata.com/public?useSSL=false
|
|
34
|
+
-- Add port, optional
|
|
35
|
+
jdbc:mysql://cn-shanghai-alicloud-mysql.api.singdata.com:3306/public?useSSL=false
|
|
36
36
|
```
|
|
37
37
|
|
|
38
|
-
Port: 3306
|
|
38
|
+
Port: 3306 (optional)
|
|
39
39
|
|
|
40
40
|
## Usage Scenarios
|
|
41
41
|
|
|
42
|
-
*
|
|
42
|
+
* Scenarios where the Lakehouse driver cannot be used — for example, when some BI report tools do not support custom JDBC drivers or the Lakehouse native JDBC driver. If possible, use the Lakehouse native driver.
|
|
43
43
|
|
|
44
44
|
## Usage Restrictions
|
|
45
45
|
|
|
46
|
-
*
|
|
46
|
+
* Full compatibility with MySQL functions and syntax is not currently available. Some MySQL-specific syntax and functions used in certain reports will produce errors when sent to Lakehouse (contact Lakehouse support).
|
|
47
47
|
* Data type restrictions
|
|
48
|
-
*
|
|
49
|
-
* Using the MySQL driver does not support complex types such as array, struct, map, json, binary, etc., which are unique to Lakehouse. This is because the MySQL client cannot handle Lakehouse-specific types.
|
|
48
|
+
* When using the Lakehouse MySQL protocol, if SQL statements contain MySQL-specific types (such as mediumint, numeric, bit, time, year, datetime, varbinary, text, blob, enum, set, or spatial data types), the system will report an error. For example, `CREATE TABLE mysql_table(col numeric)` or `SELECT CAST(xxxx AS text)` — any SQL containing these types will trigger an error. To avoid this, use Lakehouse types as replacements for MySQL data types.
|
|
50
49
|
* Syntax restrictions
|
|
51
|
-
*
|
|
50
|
+
* MySQL-specific syntax such as MySQL dump commands and LOAD commands are not supported.
|
|
52
51
|
* Function restrictions
|
|
53
|
-
* Lakehouse implements some commonly used MySQL functions
|
|
52
|
+
* Lakehouse implements some commonly used MySQL functions. Contact Lakehouse support if you need more.
|
|
54
53
|
* Data import restrictions
|
|
55
|
-
*
|
|
56
|
-
* Using the MySQL driver does not support Lakehouse local commands ([
|
|
57
|
-
* When MySQL lists
|
|
54
|
+
* MySQL bulk data import (e.g., MySQL LOAD command) is not supported.
|
|
55
|
+
* Using the MySQL driver does not support Lakehouse local commands ([PUT](put.md), [REMOVE](remove-volume.md), [GET](get.md), etc.), and does not support using the MySQL driver to call Lakehouse's Java SDK for bulk upload and real-time upload.
|
|
56
|
+
* When MySQL lists table structures, it needs to query the information schema. There is currently a 15-minute delay in Lakehouse for newly created tables appearing in the information schema, so newly created tables cannot be seen immediately.
|
|
58
57
|
|
|
59
58
|
## Connection Method
|
|
60
59
|
|
|
61
|
-
##
|
|
60
|
+
## Set Up the Virtual Cluster
|
|
62
61
|
|
|
63
|
-
Since
|
|
62
|
+
Since the MySQL protocol has no way to pass in a cluster setting, users can use SQL commands to add a default Virtual Cluster for the user. This cluster will be used when connecting via MySQL. Note that BI scenarios often have performance requirements for analysis, so it is recommended to choose an appropriate specification of analytical Virtual Cluster for the BI tool connecting user to provide the best query performance.
|
|
64
63
|
|
|
65
64
|
```sql
|
|
66
|
-
ALTER USER user_name SET DEFAULT_VCLUSTER =
|
|
65
|
+
ALTER USER user_name SET DEFAULT_VCLUSTER = DEFAULT;
|
|
67
66
|
|
|
68
|
-
-- Check if the cluster
|
|
67
|
+
-- Check if the cluster setting has taken effect
|
|
69
68
|
SHOW USERS;
|
|
70
69
|
```
|
|
71
70
|
|
|
72
|
-
##
|
|
71
|
+
## Set Username and Password
|
|
73
72
|
|
|
74
73
|
Since the MySQL protocol does not include workspace name and instance name, you need to concatenate the instance name and workspace name into the username.
|
|
75
74
|
|
|
76
|
-
The username format
|
|
75
|
+
The username format requirement is:
|
|
77
76
|
|
|
78
77
|
```
|
|
79
|
-
|
|
78
|
+
account_name@instance_name.workspace_name
|
|
80
79
|
```
|
|
81
80
|
|
|
82
|
-
* **
|
|
83
|
-
* **
|
|
81
|
+
* **instance_name**: Obtain from the JDBC connection string on the [Workspace](workspace-introduction.md) page. For example, in `jdbc:clickzetta://jnsxwfyr.api.singdata.com/quick_start?virtualCluster=DEFAULT`, `jnsxwfyr` is the instance_name.
|
|
82
|
+
* **workspace_name**: The name of the workspace.
|
|
84
83
|
|
|
85
84
|
Example:
|
|
86
85
|
|
|
@@ -88,97 +87,98 @@ Example:
|
|
|
88
87
|
user@jnsxwfyr.quick_start
|
|
89
88
|
```
|
|
90
89
|
|
|
91
|
-
##
|
|
90
|
+
## Set Connection Address
|
|
92
91
|
|
|
93
92
|
Port: 3306
|
|
94
93
|
|
|
95
|
-
| Cloud
|
|
96
|
-
|
|
|
97
|
-
|
|
|
98
|
-
|
|
|
94
|
+
| Cloud Provider | Region | Connection Address |
|
|
95
|
+
| -------------- | --------- | ---------------------------------------------------------- |
|
|
96
|
+
| Alibaba Cloud | Shanghai | cn-shanghai-alicloud-mysql.api.singdata.com |
|
|
97
|
+
| Tencent Cloud | Shanghai | ap-shanghai-tencentcloud-mysql.api.singdata.com |
|
|
98
|
+
| | Beijing | ap-beijing-tencentcloud-mysql.api.singdata.com |
|
|
99
|
+
| | Guangzhou | ap-guangzhou-tencentcloud-mysql.api.singdata.com |
|
|
100
|
+
| Amazon | Beijing | cn-north-1-aws-mysql.api.singdata.com |
|
|
99
101
|
|
|
100
|
-
##
|
|
102
|
+
## Other Parameters
|
|
101
103
|
|
|
102
|
-
* useSSL=true
|
|
103
|
-
* Database
|
|
104
|
-
* Port: 3306 optional
|
|
104
|
+
* `useSSL=true`: Lakehouse uses `caching_sha2_password` for MySQL 8.x clients, so when connecting with MySQL 8.x drivers, you must set `useSSL=true`. If using MySQL 5.x, set `useSSL=false`.
|
|
105
|
+
* Database name: The schema name in Lakehouse
|
|
106
|
+
* Port: 3306 (optional)
|
|
105
107
|
|
|
106
|
-
##
|
|
108
|
+
## BI Report Configuration Parameters
|
|
107
109
|
|
|
108
110
|
**Connection Tool**
|
|
109
111
|
|
|
110
|
-
|
|
112
|
+
MySQL Java 8.0 driver is recommended. However, some reports may initialize with MySQL-specific syntax — if you encounter errors, contact Lakehouse technical support.
|
|
111
113
|
|
|
112
114
|
**Set vcluster**:
|
|
113
|
-
You need to
|
|
115
|
+
You need to assign a default Virtual Cluster for the connecting user in Lakehouse. This cluster will be used when connecting via MySQL. Note that BI scenarios often have performance requirements for analysis, so it is recommended to choose an appropriate specification of analytical Virtual Cluster for the BI tool connecting user to provide the best query performance.
|
|
114
116
|
|
|
115
117
|
```SQL
|
|
116
|
-
ALTER USER user_name SET DEFAULT_VCLUSTER= default_ap;
|
|
117
|
-
--Check if the vcluster setting
|
|
118
|
-
|
|
118
|
+
ALTER USER user_name SET DEFAULT_VCLUSTER = default_ap;
|
|
119
|
+
-- Check if the vcluster setting has taken effect
|
|
120
|
+
SHOW USERS;
|
|
119
121
|
```
|
|
120
122
|
|
|
121
|
-
**Driver
|
|
123
|
+
**Driver requirements**: MySQL 8.0 or above is recommended
|
|
122
124
|
|
|
123
|
-
**Connection
|
|
125
|
+
**Connection address (required)**
|
|
124
126
|
|
|
125
|
-
**Username
|
|
127
|
+
**Username format (required)**
|
|
126
128
|
|
|
127
|
-
* Format:
|
|
129
|
+
* Format: username@instance_name.workspace_name, for example, test@jnsxwfyr.ql_ws
|
|
128
130
|
|
|
129
|
-
**Database
|
|
131
|
+
**Database name (required)**
|
|
130
132
|
|
|
131
133
|
* Format: schema name, such as public
|
|
132
134
|
|
|
133
|
-
**Port: 3306.
|
|
135
|
+
**Port: 3306. Required for some reports**
|
|
134
136
|
|
|
135
|
-
**Additional
|
|
137
|
+
**Additional parameters (required)**:
|
|
136
138
|
|
|
137
|
-
If using the 8.x driver, some reports can directly fill in the
|
|
139
|
+
If using the 8.x driver, some reports can directly fill in the following string in the configuration parameters:
|
|
138
140
|
|
|
139
141
|
* useSSL=true
|
|
140
142
|
|
|
141
|
-
If the report does not support filling in parameters
|
|
143
|
+
If the report does not support filling in parameters:
|
|
142
144
|
|
|
143
|
-
*
|
|
144
|
-
If using the 5.x
|
|
145
|
+
* In the report interface, there is usually an option to enable SSL — check it.
|
|
146
|
+
If using the 5.x driver, this does not need to be checked.
|
|
145
147
|
|
|
146
148
|
## Common Issues
|
|
147
149
|
|
|
148
150
|
* Unable to specify computing resources
|
|
149
151
|
|
|
150
|
-
*
|
|
151
|
-
* ```SQL
|
|
152
|
-
|
|
153
|
-
-- Check if the vcluster setting
|
|
154
|
-
|
|
155
|
-
|
|
152
|
+
* You need to assign a default Virtual Cluster for the connecting user in Lakehouse
|
|
153
|
+
* ```SQL
|
|
154
|
+
ALTER USER UAT_TEST SET DEFAULT_VCLUSTER = DEFAULT;
|
|
155
|
+
-- Check if the vcluster setting has taken effect
|
|
156
|
+
SHOW USERS;
|
|
157
|
+
```
|
|
156
158
|
|
|
157
159
|
## Troubleshooting
|
|
158
160
|
|
|
159
|
-
Troubleshoot issues through
|
|
161
|
+
Troubleshoot issues through the Lakehouse job history. After connecting via the MySQL protocol, MySQL connections to Lakehouse typically send some SQL, and job history records all submitted SQL. You can view the sent SQL through job history to see error information. Filter quickly using these three conditions:
|
|
160
162
|
|
|
161
|
-
*
|
|
162
|
-
* Submitter: The username connected in the BI report
|
|
163
|
+
* Virtual Cluster: The Virtual Cluster specified in the BI report
|
|
164
|
+
* Submitter: The username connected in the BI report — for example, if it is `test@jnsxwfyr.ql_ws`, filter by `test`
|
|
163
165
|
* Schema: The schema name specified in the BI report
|
|
164
|
-
*
|
|
166
|
+
* Click the refresh button in the upper right corner to monitor SQL sent by the BI report
|
|
165
167
|
|
|
166
|
-
|
|
168
|
+

|
|
167
169
|
|
|
168
|
-
##
|
|
170
|
+
## Examples
|
|
169
171
|
|
|
170
|
-
* Using
|
|
172
|
+
* Using mysql client to connect
|
|
171
173
|
|
|
172
|
-
|
|
174
|
+
mysql client connection to Lakehouse:
|
|
173
175
|
|
|
174
176
|
```
|
|
175
|
-
-- Add the -A option to avoid sending some
|
|
176
|
-
mysql -h cn-shanghai-alicloud-mysql.api.
|
|
177
|
+
-- Add the -A option to avoid sending some validation SQL when connecting to mysql
|
|
178
|
+
mysql -h cn-shanghai-alicloud-mysql.api.singdata.com -u user_name@instance_name.workspace_name -D public -p -A
|
|
177
179
|
|
|
178
|
-
-- When
|
|
179
|
-
mysql -h cn-shanghai-alicloud-mysql.api.
|
|
180
|
+
-- When connecting with the mysql client, if you get the error: ERROR 2059 (HY000): Authentication plugin 'mysql_clear_password' cannot be loaded: plugin not enabled, add parameters and use the following command to connect
|
|
181
|
+
mysql -h cn-shanghai-alicloud-mysql.api.singdata.com -u user_name@instance_name.workspace_name -Dpublic -p -A --enable-cleartext-plugin --default-auth=mysql_native_password
|
|
180
182
|
```
|
|
181
183
|
|
|
182
|
-
* Use [Power BI to connect to Lakehouse](
|
|
183
|
-
|
|
184
|
-
^
|
|
184
|
+
* Use [Power BI to connect to Lakehouse](PowerBI.md)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Batch Upload Data Using Python SDK
|
|
2
2
|
|
|
3
|
-
This document details how to use the BulkloadStream in the Python SDK to batch load data into Lakehouse. This method is suitable for importing large amounts of data at once, supports custom data sources, and provides flexibility for data import. This example uses a local CSV file
|
|
3
|
+
This document details how to use the BulkloadStream in the Python SDK to batch load data into Lakehouse. This method is suitable for importing large amounts of data at once, supports custom data sources, and provides flexibility for data import. This example uses a local CSV file. If the data source is in object storage or within the data integration scope supported by Lakehouse Studio, it is recommended to use the COPY command or the data integration feature instead.
|
|
4
4
|
|
|
5
5
|
# Reference Documentation
|
|
6
6
|
|
|
@@ -9,18 +9,18 @@ This document details how to use the BulkloadStream in the Python SDK to batch l
|
|
|
9
9
|
## Application Scenarios
|
|
10
10
|
|
|
11
11
|
* Suitable for business scenarios that require batch data uploads.
|
|
12
|
-
* Suitable for developers familiar with Python
|
|
12
|
+
* Suitable for developers familiar with Python who need to customize data import logic.
|
|
13
13
|
|
|
14
14
|
## Usage Restrictions
|
|
15
15
|
|
|
16
16
|
* BulkloadStream does not support writing to primary key (pk) tables.
|
|
17
17
|
* Not suitable for frequent data upload scenarios with intervals of less than five minutes.
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
# Use Case
|
|
20
20
|
|
|
21
21
|
This example uses the `olist_order_payments_dataset` from the [Brazilian E-commerce](https://www.kaggle.com/datasets/olistbr/brazilian-ecommerce?select=olist_order_items_dataset.csv) public dataset.
|
|
22
22
|
|
|
23
|
-
|
|
23
|
+
## Prerequisites
|
|
24
24
|
|
|
25
25
|
* Create the target table `bulk_order_payments`:
|
|
26
26
|
|
|
@@ -41,15 +41,15 @@ CREATE TABLE bulk_order_payments (
|
|
|
41
41
|
| ------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
42
42
|
| username | Y | Username |
|
|
43
43
|
| password | Y | Password |
|
|
44
|
-
| service | Y | Address to connect to the
|
|
45
|
-
| instance | Y | You can
|
|
44
|
+
| service | Y | Address to connect to the Lakehouse, region\_id.api.clickzetta.com. You can view the JDBC connection string in Lakehouse Studio under Administration -> Workspace  |
|
|
45
|
+
| instance | Y | You can view the JDBC connection string in Lakehouse Studio under Administration -> Workspace  |
|
|
46
46
|
| workspace | Y | Workspace in use |
|
|
47
|
-
| vcluster | Y |
|
|
47
|
+
| vcluster | Y | Virtual Cluster in use |
|
|
48
48
|
| schema | Y | Name of the schema to access |
|
|
49
49
|
|
|
50
50
|
## Develop with Python Code
|
|
51
51
|
|
|
52
|
-
Use pip to install the Python package dependencies for Lakehouse. Python version 3.
|
|
52
|
+
Use pip to install the Python package dependencies for Lakehouse. Python version 3.10 or above is required:
|
|
53
53
|
|
|
54
54
|
```SQL
|
|
55
55
|
pip install clickzetta-connector
|
|
@@ -65,11 +65,11 @@ def get_lakehouse_connect():
|
|
|
65
65
|
conn = connect(
|
|
66
66
|
username='',
|
|
67
67
|
password='',
|
|
68
|
-
service='api.
|
|
68
|
+
service='api.clickzetta.com',
|
|
69
69
|
instance='',
|
|
70
70
|
workspace='',
|
|
71
71
|
schema='public',
|
|
72
|
-
vcluster='
|
|
72
|
+
vcluster='DEFAULT_AP')
|
|
73
73
|
return conn
|
|
74
74
|
|
|
75
75
|
conn = get_lakehouse_connect()
|
|
@@ -99,5 +99,3 @@ Commit data import completion:
|
|
|
99
99
|
```SQL
|
|
100
100
|
bulkload_stream.commit()
|
|
101
101
|
```
|
|
102
|
-
|
|
103
|
-
^
|
|
@@ -13,7 +13,7 @@ This documentation covers the following topics in detail:
|
|
|
13
13
|
|
|
14
14
|
Users in Singdata can be divided into two levels based on their scope: **global account users** and **Lakehouse service instance users**. These two levels help enterprises manage all users under an account globally across multi-cloud and multi-region service instances, while also differentiating permission scopes for different users based on the service instance.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
|
|
17
17
|
|
|
18
18
|
^
|
|
19
19
|
|
|
@@ -21,7 +21,7 @@ Users in Singdata can be divided into two levels based on their scope: **global
|
|
|
21
21
|
|
|
22
22
|
Global users are users who perform global management and configuration on the Singdata platform. Each user has an independent identity in the system, with a unique username and password. Within an account, the username (user_name) must be unique to distinguish different user identities. Phone numbers, email addresses, and other information can be duplicated across different users.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
|
|
25
25
|
|
|
26
26
|
^
|
|
27
27
|
|
|
@@ -31,7 +31,6 @@ Service instance users are users within a specific service instance (instance) s
|
|
|
31
31
|
|
|
32
32
|
Service instance users are granted the "instance_user" role by default, which has no data or functional permissions. They need to be further granted permissions within the instance or workspace before they can perform operations.
|
|
33
33
|
|
|
34
|
-
:-: 
|
|
35
34
|
|
|
36
35
|
***
|
|
37
36
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
# Using UDF in Dynamic Tables
|
|
2
|
-
|
|
2
|
+
|
|
3
3
|
|
|
4
4
|
|
|
5
5
|
Dynamic tables extend support for user-defined functions. You can use custom functions created with External Function (including UDF, UDAF, and UDTF) in the SELECT clause of a dynamic table DDL. The system will automatically apply incremental computation optimization during refresh.
|
|
@@ -14,49 +14,47 @@ Among them:
|
|
|
14
14
|
|
|
15
15
|
Metadata Cache and Query Result Cache services belong to the service layer and can be shared within the workspace.
|
|
16
16
|
|
|
17
|
-
Compute Cluster Local Disk Cache is stored on the local nodes of the cluster and can only be used when using the specified
|
|
17
|
+
Compute Cluster Local Disk Cache is stored on the local nodes of the cluster and can only be used when using the specified Virtual Cluster.
|
|
18
18
|
|
|
19
19
|
In the storage-compute separation architecture of Lakehouse, data is stored in object storage. To address network request latency and improve response speed in analytical scenarios, we adopt caching strategies. Compute Cluster Cache stores frequently accessed data on local nodes, thereby accelerating queries.
|
|
20
20
|
|
|
21
21
|
Compute Cluster Cache is divided into two types:
|
|
22
22
|
|
|
23
|
-
1. **Active Cache**: Manually cache tables to the compute cluster through commands. Each time the compute cluster starts, these pre-cached tables
|
|
23
|
+
1. **Active Cache**: Manually cache tables to the compute cluster through commands. Each time the compute cluster starts, the latest data or partitions of these pre-cached tables are automatically loaded. Currently, only AP type clusters are supported. Suitable scenarios include BI report queries, which can significantly reduce query latency and improve data processing speed.
|
|
24
24
|
2. **Passive Cache**: During the first query, Lakehouse automatically caches the read files to the compute cluster. Subsequent queries involving the same table files will directly utilize the cache, speeding up the query process. Supports both GP and AP type clusters. For the second and subsequent queries, if they involve the initially cached tables, the cache will be directly hit.
|
|
25
25
|
|
|
26
26
|
## Usage
|
|
27
27
|
|
|
28
|
-
1. **
|
|
28
|
+
1. **Active Caching Method**:
|
|
29
29
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
30
|
+
```SQL
|
|
31
|
+
ALTER VCLUSTER DEFAULT SET PRELOAD_TABLES="schema1.table1,schema2.table2";
|
|
32
|
+
```
|
|
33
33
|
|
|
34
|
-
|
|
34
|
+
If you need to add a new table to the cache, you must include all previously configured tables, otherwise the original ones will be overwritten:
|
|
35
35
|
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
36
|
+
```SQL
|
|
37
|
+
ALTER VCLUSTER DEFAULT SET PRELOAD_TABLES="schema1.table1,schema2.table2,schema3.table3";
|
|
38
|
+
```
|
|
39
39
|
|
|
40
|
-
`PRELOAD_TABLES` supports wildcards. Use `schema_name.*` to cache all tables
|
|
40
|
+
`PRELOAD_TABLES` supports wildcards. Use `schema_name.*` to cache all tables under a Schema:
|
|
41
41
|
|
|
42
42
|
```SQL
|
|
43
|
-
-- Cache all tables in the sales Schema, plus dim_date
|
|
43
|
+
-- Cache all tables in the sales Schema, plus the dim_date table in the public Schema
|
|
44
44
|
ALTER VCLUSTER bi_vc SET PRELOAD_TABLES="sales.*,public.dim_date";
|
|
45
45
|
```
|
|
46
46
|
|
|
47
47
|
## **Viewing Cache Status**:
|
|
48
|
+
When tables are loaded into the compute cluster using the `ALTER..PRELOAD_TABLES` command, there may be a delay in the cache status update displayed by `SHOW PRELOAD`. However, the cached tables are actually already effective. Under normal circumstances, this delay is approximately 10 minutes.
|
|
48
49
|
|
|
49
|
-
When tables are loaded into the compute cluster using the `ALTER..PRELOAD_TABLES` command, there may be a delay in the update of the cache status displayed by `SHOW PRELOAD`. However, the cached tables are actually effective. Under normal circumstances, this delay is about 10 minutes.
|
|
50
|
-
|
|
51
|
-
1. **Active Cache Table Method**:
|
|
52
50
|
|
|
53
|
-
* Display the preload table/partition status of the current
|
|
51
|
+
* Display the preload table/partition status of the current Virtual Cluster:
|
|
54
52
|
|
|
55
53
|
```SQL
|
|
56
54
|
SHOW PRELOAD CACHED STATUS;
|
|
57
55
|
```
|
|
58
56
|
|
|
59
|
-
* Display the preload table/partition status of the specified
|
|
57
|
+
* Display the preload table/partition status of the specified Virtual Cluster:
|
|
60
58
|
|
|
61
59
|
```SQL
|
|
62
60
|
SHOW VCLUSTER preload_ap_vc_test PRELOAD CACHED STATUS;
|
|
@@ -68,18 +66,16 @@ SHOW VCLUSTER preload_ap_vc_test PRELOAD CACHED STATUS;
|
|
|
68
66
|
SHOW VCLUSTER preload_ap_vc_test PRELOAD CACHED STATUS WHERE table LIKE '%x_test';
|
|
69
67
|
```
|
|
70
68
|
|
|
71
|
-
* Display the preloaded cache summary information of the
|
|
69
|
+
* Display the preloaded cache summary information of the Virtual Cluster:
|
|
72
70
|
|
|
73
71
|
```SQL
|
|
74
72
|
SHOW EXTENDED PRELOAD CACHED STATUS;
|
|
75
73
|
```
|
|
76
74
|
|
|
77
|
-
##
|
|
75
|
+
## Notes
|
|
78
76
|
|
|
79
77
|
* Active caching (PRELOAD_TABLES) is only supported on **Analytics (AP)** clusters. General Purpose (GP) clusters do not support this feature.
|
|
80
78
|
* `PRELOAD_TABLES` is an **overwrite** operation. When adding a new table, you must include all previously configured tables, otherwise they will be removed from preloading.
|
|
81
|
-
* The cluster supports automatic start and stop. When the cluster stops, the local cache is automatically released.
|
|
79
|
+
* The cluster supports automatic start and stop. When the cluster stops, the local cache is automatically released. When an AP cluster restarts, only the most recently written data or partitions are cached.
|
|
82
80
|
* After executing the cache command, only newly written data will be cached.
|
|
83
81
|
* `SHOW PRELOAD` status updates may have approximately a 10-minute delay, but the cache is already effective.
|
|
84
|
-
|
|
85
|
-
^
|