@clickzetta/cz-cli-darwin-x64 0.3.94 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/JOIN.md

@@ -0,0 +1,265 @@
+# JOIN
+
+JOIN is used to merge data from two or more tables based on specified conditions. Lakehouse supports the following JOIN types:
+
+```Plain
+left_table_reference {
+    [ join_type ] JOIN right_table_reference join_criteria |
+    NATURAL JOIN right_table_reference |
+    CROSS JOIN right_table_reference
+}
+
+join_type ::=
+    { [ INNER ] | LEFT [ OUTER ] | [ LEFT ] SEMI | RIGHT [ OUTER ] | FULL [ OUTER ] | [ LEFT ] ANTI | CROSS }
+
+join_criteria ::=
+    { ON boolean_expression | USING ( column_name [, ...] ) }
+```
+
+`left_table_reference` and `right_table_reference` are the two tables (or subqueries) participating in the JOIN, `join_type` specifies the join type, and `join_criteria` specifies the join condition.
+
+## JOIN Types
+
+| Type | Description |
+|------|------|
+| **INNER JOIN** | Returns rows from both tables that satisfy the join condition (intersection); this is the default JOIN type |
+| **LEFT [OUTER] JOIN** | Returns all rows from the left table; fills with NULL when there is no match in the right table |
+| **RIGHT [OUTER] JOIN** | Returns all rows from the right table; fills with NULL when there is no match in the left table |
+| **FULL [OUTER] JOIN** | Returns all rows from both tables; fills with NULL on the side with no match |
+| **[LEFT] SEMI JOIN** | Returns rows from the left table that have a match in the right table; does not return right table columns |
+| **[LEFT] ANTI JOIN** | Returns rows from the left table that have no match in the right table; does not return right table columns |
+| **CROSS JOIN** | Returns the Cartesian product of both tables (all combinations of rows) |
+| **NATURAL JOIN** | Performs an implicit equi-join on all columns with the same name in both tables; no condition needs to be specified |
+
+## JOIN Conditions
+
+- `ON boolean_expression`: Specifies any boolean expression as the join condition. Subqueries are not supported in JOIN conditions.
+- `USING (column_name [, ...])`: Specifies one or more column names for an equi-join; these columns must exist in both tables.
+
+## Test Data Description
+
+The following examples use two tables in the `doc_test` schema:
+
+- `employees(id, name, dept, salary, hire_date, is_active)` — 5 employee records
+- `departments(dept_id, dept_name, manager)` — 3 departments
+
+`employees.dept` is joined with `departments.dept_name`. The `Dave` employee belongs to the `Marketing` department which exists in `departments`, but the `HR` department in `departments` has only 1 employee, and `departments` also has an extra `Finance` department (no employees belong to it in `employees`) — this clearly demonstrates how each JOIN type handles unmatched rows.
+
+## Usage Examples
+
+### INNER JOIN
+
+Returns rows where `dept = dept_name` matches in both `employees` and `departments`.
+
+```SQL
+SELECT e.id, e.name, e.dept, d.manager
+FROM doc_test.employees e
+INNER JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY e.id;
+```
+
+Result (only employees with matches in both tables):
+
+```
++----+-------+-------------+---------+
+| id | name  | dept        | manager |
++----+-------+-------------+---------+
+|  1 | Alice | Engineering | Charlie |
+|  2 | Bob   | Engineering | Charlie |
+|  3 | Carol | Marketing   | Diana   |
+|  4 | Dave  | Marketing   | Diana   |
+|  5 | Eve   | HR          | Frank   |
++----+-------+-------------+---------+
+```
+
+### LEFT [OUTER] JOIN
+
+Returns all rows from `employees`; fills with NULL when there is no match in `departments`.
+
+```SQL
+SELECT e.id, e.name, e.dept, d.dept_id, d.manager
+FROM doc_test.employees e
+LEFT JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY e.id;
+```
+
+Result (all employees are retained; `dept_id`/`manager` are NULL if `dept` has no corresponding record in `departments`):
+
+```
++----+-------+-------------+---------+---------+
+| id | name  | dept        | dept_id | manager |
++----+-------+-------------+---------+---------+
+|  1 | Alice | Engineering |       1 | Charlie |
+|  2 | Bob   | Engineering |       1 | Charlie |
+|  3 | Carol | Marketing   |       2 | Diana   |
+|  4 | Dave  | Marketing   |       2 | Diana   |
+|  5 | Eve   | HR          |       3 | Frank   |
++----+-------+-------------+---------+---------+
+```
+
+### RIGHT [OUTER] JOIN
+
+Returns all rows from `departments`; fills with NULL when there is no match in `employees`.
+
+```SQL
+SELECT e.id, e.name, d.dept_name, d.manager
+FROM doc_test.employees e
+RIGHT JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY d.dept_id, e.id;
+```
+
+Result (all departments are retained; `id`/`name` are NULL if a department has no employees):
+
+```
++------+-------+-------------+---------+
+| id   | name  | dept_name   | manager |
++------+-------+-------------+---------+
+|    1 | Alice | Engineering | Charlie |
+|    2 | Bob   | Engineering | Charlie |
+|    3 | Carol | Marketing   | Diana   |
+|    4 | Dave  | Marketing   | Diana   |
+|    5 | Eve   | HR          | Frank   |
+| NULL | NULL  | Finance     | Grace   |
++------+-------+-------------+---------+
+```
+
+### FULL [OUTER] JOIN
+
+Returns all rows from both tables; fills with NULL on the side with no match.
+
+```SQL
+SELECT e.id, e.name, e.dept, d.dept_name, d.manager
+FROM doc_test.employees e
+FULL JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY e.id, d.dept_id;
+```
+
+Result (both the employee side and department side are fully retained; corresponding columns are NULL when there is no match):
+
+```
++------+-------+-------------+-------------+---------+
+| id   | name  | dept        | dept_name   | manager |
++------+-------+-------------+-------------+---------+
+|    1 | Alice | Engineering | Engineering | Charlie |
+|    2 | Bob   | Engineering | Engineering | Charlie |
+|    3 | Carol | Marketing   | Marketing   | Diana   |
+|    4 | Dave  | Marketing   | Marketing   | Diana   |
+|    5 | Eve   | HR          | HR          | Frank   |
+| NULL | NULL  | NULL        | Finance     | Grace   |
++------+-------+-------------+-------------+---------+
+```
+
+### [LEFT] SEMI JOIN
+
+Returns rows from the left table that have a match in the right table; does not return right table columns.
+
+```SQL
+SELECT e.id, e.name, e.dept
+FROM doc_test.employees e
+SEMI JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY e.id;
+```
+
+Result (only employee information is returned, without department table columns):
+
+```
++----+-------+-------------+
+| id | name  | dept        |
++----+-------+-------------+
+|  1 | Alice | Engineering |
+|  2 | Bob   | Engineering |
+|  3 | Carol | Marketing   |
+|  4 | Dave  | Marketing   |
+|  5 | Eve   | HR          |
++----+-------+-------------+
+```
+
+### [LEFT] ANTI JOIN
+
+Returns rows from the left table that have **no** match in the right table; does not return right table columns.
+
+```SQL
+SELECT e.id, e.name, e.dept
+FROM doc_test.employees e
+ANTI JOIN doc_test.departments d
+ON e.dept = d.dept_name
+ORDER BY e.id;
+```
+
+Result (empty result when no employees have a `dept` that does not exist in `departments`):
+
+```
++----+------+------+
+| id | name | dept |
++----+------+------+
+(0 rows)
+```
+
+If there are employees in `employees` whose department is not in `departments`, those employees will appear in the result.
+
+### CROSS JOIN
+
+Returns the Cartesian product of both tables; every row is combined with every other row. Result rows = left table rows × right table rows.
+
+```SQL
+SELECT e.name, d.dept_name
+FROM doc_test.employees e
+CROSS JOIN doc_test.departments d
+ORDER BY e.id, d.dept_id
+LIMIT 9;
+```
+
+Result (5 employees × 3 departments = 15 rows; first 9 rows shown here):
+
+```
++-------+-------------+
+| name  | dept_name   |
++-------+-------------+
+| Alice | Engineering |
+| Alice | Marketing   |
+| Alice | HR          |
+| Bob   | Engineering |
+| Bob   | Marketing   |
+| Bob   | HR          |
+| Carol | Engineering |
+| Carol | Marketing   |
+| Carol | HR          |
++-------+-------------+
+```
+
+### NATURAL JOIN
+
+Performs an implicit equi-join based on all columns with the same name in both tables; no ON or USING condition needs to be specified.
+
+```SQL
+-- NATURAL JOIN: automatically joins on columns with the same name
+SELECT *
+FROM doc_test.employees e
+NATURAL JOIN doc_test.departments d;
+```
+
+> ⚠️ **Note**: `NATURAL JOIN` matches **all** columns with the same name in both tables. If both tables have multiple columns with the same name, all of them participate in the join condition, which can easily produce unexpected results. It is recommended to use this only when column names are unambiguous. For everyday development, it is recommended to use `ON` to explicitly specify join conditions.
+
+### USING Syntax
+
+When the join columns have the same name in both tables, you can use `USING` instead of `ON`; the column appears only once in the result set:
+
+```SQL
+-- Assuming both tables have a dept_name column, use USING to simplify
+SELECT e.name, e.salary, dept_name, d.manager
+FROM doc_test.employees e
+JOIN doc_test.departments d
+USING (dept_name);
+```
+
+## Notes
+
+- JOIN conditions (`ON` clause) do not support subqueries.
+- SEMI JOIN and ANTI JOIN only return left table columns; referencing right table columns in SELECT will cause an error.
+- FULL OUTER JOIN can have significant performance overhead in some distributed engines; pay attention to the execution plan when dealing with large data volumes.
+- CROSS JOIN does not require an ON condition; if you accidentally write `JOIN` without an `ON`, the optimizer will treat it as a CROSS JOIN.

package/bin/skills/lakehouse-doc-en/references/JSON.md

@@ -0,0 +1,164 @@
+# JSON Type
+
+JSON (JavaScript Object Notation) is the native semi-structured data type in Singdata Lakehouse. Compared to storing JSON text as STRING, using the native JSON type leverages **columnar storage optimization** and **predicate pushdown**, significantly improving query performance and reducing storage overhead.
+
+## Why Use the JSON Type?
+
+| Feature | STRING Type | JSON Type |
+|---|---|---|
+| **Storage** | Plain text, no structural compression | **Columnar storage**: high-frequency fields are automatically extracted into separate columns |
+| **Query Performance** | Requires full scan and text parsing | **Column pruning**: only reads the needed fields without parsing the entire row |
+| **Type Safety** | Requires manual CAST after extraction | Built-in functions like `json_extract_bigint` directly return SQL types |
+
+> **How it works**: When writing JSON data, Lakehouse automatically analyzes the schema. For high-frequency fields (such as `id`, `timestamp`), the system converts them to strongly-typed columns (such as `BIGINT`) for columnar storage at the underlying level.
+
+---
+
+## 1. Constructing JSON Data
+
+### 1.1 Parsing from a String (Recommended)
+Use the `parse_json` (or `json_parse`) function to convert a string into a JSON object. This is the most common way to handle externally imported data.
+
+```SQL
+SELECT parse_json('{"id": 1, "name": "Lakehouse"}') AS data;
+-- Result: {"id":1,"name":"Lakehouse"}
+```
+
+### 1.2 JSON Constants
+Define JSON data directly in SQL using the `JSON '...'` syntax.
+
+```SQL
+SELECT JSON '{"key": "value"}' AS const_json;
+```
+
+### 1.3 Using Constructor Functions
+Dynamically build JSON objects or arrays from SQL values.
+
+```SQL
+-- Build an object
+SELECT json_object('id', 1, 'name', 'Lakehouse');
+-- Result: {"id":1,"name":"Lakehouse"}
+
+-- Build an array
+SELECT json_array(1, 2, 3, 'test');
+-- Result: [1,2,3,"test"]
+```
+
+---
+
+## 2. Querying and Accessing JSON Data
+
+Lakehouse provides two ways to access data inside JSON: **shorthand syntax** (returns JSON type) and **extraction functions** (returns SQL types).
+
+### 2.1 Shorthand Syntax `['key']`
+Suitable for quick access; the result is still of **JSON type**. Supports nested access and array indexing.
+
+```SQL
+WITH t AS (
+    SELECT parse_json('{"user": {"id": 101, "tags": ["vip", "active"]}}') AS data
+)
+SELECT 
+    data['user']['id'] AS user_id_json,       -- Returns JSON type 101
+    data['user']['tags'][0] AS first_tag;     -- Returns JSON type "vip"
+```
+
+### 2.2 Type Extraction Functions (Recommended for Calculations)
+Use the `json_extract_*` family of functions to directly return **native SQL types** (such as BIGINT, STRING), making subsequent calculations and filtering easier.
+
+| Function | Return Type | Example |
+|---|---|---|
+| `json_extract_string` | STRING | `json_extract_string(data, '$.name')` |
+| `json_extract_bigint` | BIGINT | `json_extract_bigint(data, '$.id')` |
+| `json_extract_int` | INT | `json_extract_int(data, '$.age')` |
+| `json_extract_double` | DOUBLE | `json_extract_double(data, '$.price')` |
+| `json_extract_float` | FLOAT | `json_extract_float(data, '$.score')` |
+| `json_extract_boolean`| BOOLEAN| `json_extract_boolean(data, '$.is_active')` |
+| `json_extract_date`| DATE| `json_extract_date(data, '$.birth_date')` |
+| `json_extract_timestamp`| TIMESTAMP| `json_extract_timestamp(data, '$.created')` |
+
+**Example**:
+```SQL
+SELECT 
+    json_extract_bigint(data, '$.user.id') AS user_id,
+    json_extract_string(data, '$.user.tags[0]') AS first_tag
+FROM (SELECT parse_json('{"user": {"id": 101, "tags": ["vip"]}}') AS data) t;
+```
+
+> 💡 **Tip**: Using extraction functions (such as `json_extract_bigint`) in a `WHERE` clause can trigger **predicate pushdown**, leveraging the underlying columnar index to accelerate queries.
+
+---
+
+## 3. Type Conversion (Important Notes)
+
+When handling JSON strings, **`parse_json`** and **`::json` (CAST)** behave completely differently — be sure to pay attention:
+
+| Method | Semantics | Example Result |
+|---|---|---|
+| **`parse_json(str)`** | **Parses** the string content into a JSON structure | `parse_json('{"a":1}')` → JSON object `{"a":1}` |
+| **`str::json`** | Treats the entire string as a **JSON string value** | `'{"a":1}'::json` → JSON string `"{\"a\":1}"` |
+
+**Practical comparison**:
+
+```SQL
+-- Scenario 1: Correct parsing
+SELECT parse_json('{"a": 1}')['a'];  
+-- Result: 1 (value successfully retrieved)
+
+-- Scenario 2: Incorrect conversion (Cast)
+SELECT ('{"a": 1}'::json)['a'];     
+-- Result: NULL (because this is a string, there is no key 'a')
+```
+
+**Conclusion**: When your data source is a JSON-formatted string (such as logs or API responses), you **must use `parse_json`**. `::json` is typically used to convert a plain string (such as `"hello"`) to the JSON string type.
+
+---
+
+## 4. Performance Optimization in Practice
+
+### 4.1 Predicate Pushdown
+When filtering with strongly-typed functions like `json_extract_bigint`, Lakehouse can directly scan the underlying optimized columns, avoiding full-table parsing.
+
+```SQL
+-- Efficient query: leverages columnar index
+SELECT * FROM logs 
+WHERE json_extract_bigint(data, '$.status') = 200;
+
+-- Inefficient query: full table scan with parsing
+SELECT * FROM logs 
+WHERE data['status']::bigint = 200; -- Note: some versions may not support pushdown
+```
+
+### 4.2 Handling Invalid JSON
+`parse_json` is fault-tolerant. If the input is not valid JSON, it returns `NULL` instead of throwing an error, which is very useful when handling dirty data.
+
+```SQL
+SELECT parse_json('invalid json');  -- Result: NULL
+SELECT parse_json('{"a": 1}');      -- Result: {"a":1}
+```
+
+---
+
+## 5. Limitations and Notes
+
+*   **No sorting/grouping support**: JSON columns themselves do not support `ORDER BY`, `GROUP BY`, or `JOIN`. You must extract specific fields before performing these operations.
+*   **Size limit**: The default maximum JSON string length is 16MB. To adjust this, modify the table properties:
+    ```SQL
+    ALTER TABLE table_name SET PROPERTIES("cz.storage.write.max.json.bytes"="33554432");
+    ```
+*   **Key ordering**: The order of keys inside a JSON object may be rearranged during storage; output order being inconsistent with input is normal behavior.
+
+---
+
+## Related Documentation
+
+*   **JSON Function Reference**:
+    *   [JSON_EXTRACT Function Series](sql_functions/scalar_functions/json_functions/json_extract.md): Complete function list for extracting JSON data
+    *   [JSON_CONTAINS](sql_functions/scalar_functions/json_functions/json_contains.md): Check whether a JSON object contains specific content
+    *   [JSON_TYPE](sql_functions/scalar_functions/json_functions/json_type.md): Returns the type of a JSON value
+    *   [JSON_NORMALIZE](sql_functions/scalar_functions/json_functions/json_normalize.md): Normalize a JSON string (key sorting)
+    *   [JSON_MINIFY](sql_functions/scalar_functions/json_functions/json_minify.md): Compress a JSON string
+    *   [JSON_REMOVE](sql_functions/scalar_functions/json_functions/json_remove.md): Remove elements from a JSON object
+    *   [JSON_PARSE](sql_functions/scalar_functions/json_functions/json_parse.md): Parse a JSON string
+    *   [JSON_TUPLE](sql_functions/table_functions/json_tuple.md): Extract multiple key-value pairs from a JSON object
+*   [JSON Data Processing Guide](json_data_process_guide.md): Comprehensive guide for JSON data modeling, querying, and function usage
+*   [JSON Data Processing Guide for Complex Business Scenarios](json_guide_for_complex_biz_cases.md): JSON best practices for e-commerce, IoT, finance, and other scenarios

package/bin/skills/lakehouse-doc-en/references/JSON_DataType.md

@@ -0,0 +1,49 @@
+# JSON Data Processing
+
+## Create schema and table
+
+```
+CREATE SCHEMA IF NOT EXISTS clickzetta_demo_json_schema;
+use schema clickzetta_demo_json_schema;
+create table if not exists clickzetta_demo_json_schema.user_infor(info json);
+insert into table  clickzetta_demo_json_schema.user_infor values
+  (JSON '{"name": "Zhang San","age": 25,"gender": "Male","email": "zhangsan@example.com","phone": "13812xxx","address": "Chaoyang District, Beijing"}'),
+  (JSON '{"name": "Li Si","age": 23, "gender": "Female","email": "lisi@example.com","phone": "139876xxx1","address": "Pudong New Area, Shanghai"}'),
+  (JSON '{"name": "Wang Wu", "age": 27,"gender": "Male","email": "wangwu@example.com","phone": "1376xxxxx8","address": "Tianhe District, Guangzhou"}'),
+  (JSON '{"name": "Zhao Liu","age": 24,"gender": "Female","email": "zhaoliu@example.com","phone": "1369xxxx2","address": "Nanshan District, Shenzhen"}'),
+  (JSON '{"name": "Sun Qi","age": 26, "gender": "Male","email": "sunqi@example.com","phone": "1351xxxx679","address": "Xihu District, Hangzhou"}'),
+  (JSON '{"name": "Zhou Ba","age": 22,"gender": "Female","email": "zhouba@example.com","phone": "1347xxxx19","address": "Gulou District, Nanjing"}'),
+  (JSON '{"name": "Wu Jiu","age": 28,"gender": "Male","email": "wujiu@example.com","phone": "13345xxxx01","address": "Wuhou District, Chengdu"}'),
+  (JSON '{"name": "Zheng Shi","age": 21,"gender": "Female","email": "zhengshi@example.com","phone": "1326xxx123","address": "Yubei District, Chongqing"}'),
+  (JSON '{"name": "Chen Shiyi","age": 29,"gender": "Male","email": "chenshiyi@example.com","phone": "131xxxx789","address": "Beilin District, Xi'an"}'),
+  (JSON '{"name": "Lin Shier", "age": 20,"gender": "Female","email": "linshier@example.com","phone": "130xxxx5432","address": "Siming District, Xiamen"}');
+```
+
+## Using json\_extract to Parse Data
+
+```
+select json_extract_string(info,"$.address") as address,
+      json_extract_int(info,"$.age") as age,
+      json_extract_string(info,"$.email") as email
+from clickzetta_demo_json_schema.user_infor;
+```
+
+![](.topwrite/assets/image_1718703925062.png)
+
+## Cleanup
+
+```
+drop schema if exists clickzetta_demo_json_schema;
+```
+
+## Congratulations, it's done.
+
+Please enjoy and learn more!
+
+## Appendix
+
+### Download Zeppelin Notebook Source File
+
+The code in this document is also available in a version that runs on [Zeppelin](eco_integration/zeppelin.md). If you want to run the code directly, please follow the instructions in the documentation to install [Zeppelin](eco_integration/zeppelin.md).
+
+[03.JSON_Data_Processing.ipynb](https://github.com/yunqiqiliang/clickzetta_quickstart/blob/main/zeppelin_notebook/03.JSON%E6%95%B0%E6%8D%AE%E5%A4%84%E7%90%86.ipynb)

package/bin/skills/lakehouse-doc-en/references/KAFKA_Storage_connection.md

@@ -0,0 +1 @@
+# Kafka Storage Connection

package/bin/skills/lakehouse-doc-en/references/Kafka_connection.md

@@ -0,0 +1,36 @@
+## Feature Overview
+
+> **[Preview Release]** This feature is currently in public preview.
+
+This document describes how to create an external table in SQL connected to a Kafka message queue system. By defining an external table, you can conveniently read data streams from Kafka and query and analyze this data as a table.
+
+## Creating a Storage Connection
+
+First, you need to create a storage connection for connecting to the Kafka server. Connections requiring client certificates are not currently supported.
+
+### Syntax
+
+```SQL
+CREATE STORAGE CONNECTION connection_name
+    TYPE kafka
+    BOOTSTRAP_SERVERS = ['server1:port1', 'server2:port2', ...]
+    SECURITY_PROTOCOL = 'PLAINTEXT';
+```
+
+### Parameter Description
+
+* **connection_name**: The name of the connection, used for subsequent referencing.
+* **TYPE**: The connection type, which is `kafka` here.
+* **BOOTSTRAP_SERVERS**: The list of Kafka cluster addresses, in the format `['host1:port1', 'host2:port2', ...]`.
+* **SECURITY_PROTOCOL**: The security protocol, which can be `PLAINTEXT`, etc.
+
+### Example
+
+```SQL
+CREATE STORAGE CONNECTION test_kafka_conn
+    TYPE kafka
+    BOOTSTRAP_SERVERS = ['47.99.48.62:9092']
+    SECURITY_PROTOCOL = 'PLAINTEXT';
+```
+
+^

package/bin/skills/lakehouse-doc-en/references/Key_Concepts.md

@@ -0,0 +1,116 @@
+# Singdata Lakehouse Key Concepts
+
+Welcome to Singdata Lakehouse! This chapter will introduce you to the basic concepts of Singdata Lakehouse in detail, helping you better understand and use this product.
+
+## Account
+
+A Singdata account represents an organization or individual (usually a corporate account) that has established a business relationship with Singdata, covering actions such as account registration and service activation. The account is the basic entity for using Singdata product services, making payments, and obtaining service support. Upon registration, the system will automatically generate a globally unique account name, which serves as your unique identifier in Singdata products.
+
+* The account is responsible for recharging, billing, payment, and ordering/changing Singdata products;
+* All service instances and their resource objects activated on Singdata products by enterprises or individuals belong to a specific account.
+
+## Account URL
+
+Each account has a unique account URL for accessing the management center. You need to log in using a valid username and password under that account. The format of the account URL is: `<account_name>.accounts.singdata.com`. For example, in the following link, `0256c297` is the account name: `0267c297.accounts.singdata.com`.
+
+## User
+
+After the account is successfully registered, you can create multiple users to share the resources within the same account. Different users can allocate data and resources through permission control.
+
+## Lakehouse Instance
+
+A Lakehouse service instance is the carrier of Singdata Lakehouse product services. When activating the Lakehouse service, you need to create a service instance under your account based on the specified cloud service provider and region. An account can create one or more service instances (currently, only a single service instance creation is open by default).
+
+Within the service instance, data objects, computing resources, and job tasks are managed using unified metadata, and tools such as data integration, development scheduling, operation and maintenance monitoring, and data catalog are used for application building and data management under a unified permission system. The service instance has regional attributes, and its computing, data, and other service resources are all within the region of the cloud service provider. Different service instances are isolated from each other by default.
+
+:-: ![](.topwrite/assets/p1_1739877904983.png)
+
+## Workspace
+
+A workspace is a logical object used to organize Lakehouse resource objects (data objects, computing resources, users, etc.) and provide supporting data development capabilities (data integration, data development, data operation, and maintenance). You can create multiple workspaces under one service instance. Workspaces are isolated by default, and users need to join a workspace to use the objects within it. Objects within different workspaces under the same instance can be shared through cross-workspace authorization. For how to manage and use workspaces, refer to [Workspace Management](workspace-introduction.md).
+
+## Virtual Cluster
+
+Computing resources consist of multi-instance virtual computing clusters and computing services running in the clusters, providing a computing environment for your jobs, including CPU, memory, and temporary storage. For details, see [Compute Cluster](virtual-cluster.md).
+
+## Data Source
+
+A data source stores the connection string information for data storage, such as the database address, access username, password, etc. Data sources can be used to configure data synchronization tasks or reference and consume data during data analysis. For details, see [Data Source Management](config-datasource.md).
+
+## Schema
+
+Within a workspace, a schema is a namespace for a set of database objects, including tables, views, etc. You can include multiple schema objects with different names within a workspace.
+
+## Table
+
+A table is a formatted two-dimensional data table.
+
+## View
+
+A view is a virtual table that does not actually exist in the database. When used, the view is dynamically generated.
+
+## Materialized View
+
+A materialized view is a special type of view that, unlike a regular view, actually exists in the database and occupies storage resources. For details, see [Materialized View](materialized_ddl.md).
+
+## Lakehouse Studio
+
+Lakehouse Studio is a web-based integrated development and management toolset natively provided by Lakehouse, offering a graphical user interface designed to deliver comprehensive one-stop functionalities and an intuitive user experience. After logging into Lakehouse Studio via a browser, users can easily operate various product features on the interface, such as uploading/importing or synchronizing data, developing SQL tasks, scheduling jobs, configuring monitoring alerts, performing task operation, and managing users and permissions. Compared to accessing via JDBC protocol connections or CLI command-line tools, Lakehouse Studio provides richer native features and is more convenient and user-friendly. For details, see [Lakehouse Studio](studio_manual.md).
+
+## Task Development
+
+Task development is an integrated big data development environment where you can perform task development, debugging, configuration scheduling, and submission and release operations, completing the key processes of big data aggregation, processing, and analysis. In the left navigation menu, click "Development" to enter the main interface of the Web IDE. For details, see [Task Development Overview](task_development.md).
+
+## Task Group
+
+A task group is a virtual business management group used to manage a list of tasks within the development section. You can use task groups to organize and orchestrate a series of tasks, including adding/creating task nodes, creating dependencies between task nodes, configuring task group parameters, and performing unified batch submission operations. For details, see [Task Group](task_group.md).
+
+## Maintenance Center
+
+The maintenance center provides operational management operations for tasks and instances. The tasks managed by the maintenance center include manually triggered tasks and periodically scheduled tasks, and their corresponding instances, for centralized management. For details, see [Task and Instance Maintenance](task-instance-maintenance.md).
+
+## Data Catalog
+
+The data catalog is a bridge connecting data providers and data consumers. The data catalog builds the directory based on the metadata information of the tables and provides global search and filtering functions, making it easy to quickly find tables and view table details, assisting in the consumption and use of data. For more details, see[Data Asset Map](datacatalog.md).
+
+## Workflow
+
+A workflow is a data processing job flow object composed of one or more interdependent tasks. The workflow specifies the running environment and resources for the tasks. Jobs can be executed on-demand, periodically, or triggered by events.
+
+## Workflow Instance
+
+When a workflow task is triggered, a workflow instance is generated. The workflow instance includes a timestamp and task status, created by the scheduling system based on the trigger conditions.
+
+## Task
+
+A task is an atomic data development task, such as data synchronization tasks, SQL tasks, Python tasks, SHELL tasks, etc. Tasks are created, saved, and organized in a file format.
+
+### Lakehouse SQL Task
+
+Refers to SQL query tasks generated in the Lakehouse through web-based data development features or CLI, JDBC connections, etc. For how to develop and schedule a Lakehouse SQL task, refer to [Task Development and Scheduling](taskdevelop.md).
+
+### Synchronization Task
+
+Refers to data synchronization type tasks. Users generate synchronization tasks by configuring the synchronization data source and target data table. Synchronization tasks also consume computing resources and are metered. Synchronization tasks include two main categories: offline synchronization tasks and real-time synchronization tasks. For more details, see [Data Sync](data-integration.md).
+
+### Python Task
+
+Python tasks refer to tasks that can write and run Python code. For many data analysis and processing scenarios, especially for BI+AI analysis scenarios, combining Python and SQL can greatly improve the efficiency of data analysis and processing. In Singdata Lakehouse, Python code is run by providing a Python script task type. For more details, see [Python Task](python_task.md).
+
+### JDBC Task
+
+JDBC tasks are an important task node in data development, allowing you to write SQL code to connect to data sources that support the JDBC protocol, enabling operations such as data addition, deletion, modification, and query. For more details, see [JDBC Task](jdbc_task.md).
+
+^
+
+## Task Instance
+
+A specific execution instance generated when a data development task is executed manually or scheduled.
+
+^
+
+***
+
+More concepts and object model designs can be found in [Object Model](object_model_design.md).
+
+^