@clickzetta/cz-cli-darwin-x64 0.3.94 → 0.3.96

package/bin/skills/lakehouse-doc-en/references/object_identifier.md

@@ -0,0 +1,270 @@
+# Lakehouse Object Naming Rules
+
+In Lakehouse, a set of naming rules has been established to ensure consistency and readability in metadata naming (such as databases, tables, views, users, etc.) and to avoid potential issues. This document describes these rules in detail to help you better understand and use Lakehouse.
+
+## Identifier Types
+
+Lakehouse supports the following types of identifier naming:
+
+* **Regular identifiers**: No quotes required, but must comply with specific naming rules.
+* **Double-quoted identifiers**: Must be surrounded by double quotes (`"`), allowing keywords to be used as identifier names. Disabled by default; enable with `SET cz.sql.double.quoted.identifiers=true` (session level only). **Note**: Once enabled, double quotes no longer represent strings — `"hello"` is parsed as an identifier, not a string value.
+* **Backtick identifiers**: Must be surrounded by backticks (`` ` ``). For **column names**, backticks allow any characters (hyphens, spaces, Chinese characters, numeric prefixes, etc.); for **table names, schema names, and other object names**, backticks only bypass keyword restrictions and numeric-prefix restrictions — special characters such as hyphens, spaces, and Chinese characters are still not allowed. Backticks are used by default; switch to double quotes with `SET cz.sql.double.quoted.identifiers=true`.
+
+## Naming Rules
+
+### Regular Identifier Naming Conventions
+
+1. **Starting character**: Identifiers must start with a letter (uppercase A-Z or lowercase a-z) or an underscore (`_`). Starting with a digit or other special character is not allowed.
+2. **Character composition**: Identifiers may only contain letters, underscores, and decimal digits (0-9). Other special characters and keywords such as hyphens (`-`), spaces, or other non-alphanumeric characters are not allowed.
+3. **Case sensitivity**: Metadata undergoes case conversion during storage. Most objects (schema, table, view, column, etc.) are stored in **lowercase**; VCluster names are an exception and are stored in **uppercase**. References are case-insensitive — `MyTable` and `mytable` are treated as the same identifier. Backticks also cannot preserve case. Note: Column aliases in SELECT queries (`AS MyAlias`) preserve their original case and are not converted.
+4. **Language restriction**: Chinese characters are not supported.
+5. **Length limit**: 1–256 characters; WORKSPACE NAME is limited to 3–28 characters.
+
+Examples of valid identifiers:
+
+```SQL
+my_table
+table1
+table_2
+```
+
+### Backtick Identifiers
+
+The behavior of backticks differs by object type:
+
+**For column names**: Backticks bypass all naming restrictions, allowing hyphens, spaces, Chinese characters, numeric prefixes, and any other characters:
+
+```sql
+CREATE TABLE my_table(
+    `my-col`  INT,
+    `my col`  STRING,
+    `chinese_col`   DATE,
+    `123col`  BIGINT
+)
+```
+
+**For table names, schema names, and other object names**: Backticks only bypass keyword restrictions and numeric-prefix restrictions; special characters such as hyphens, spaces, and Chinese characters are still not allowed:
+
+```sql
+-- Numeric prefix allowed
+CREATE SCHEMA `123`;
+-- Keyword as name allowed
+CREATE SCHEMA `select`;
+-- Chinese column names (column names only)
+CREATE TABLE my_table(`chinese_col` STRING);
+```
+
+> ⚠️ **Note**: Backticks **cannot** preserve case. Whether for table names or column names, objects created with backticks are still stored in lowercase. For example, `` `MyTable` `` is stored as `mytable`.
+
+Use backticks only when necessary.
+
+### Double-Quoted Identifiers
+
+After enabling `SET cz.sql.double.quoted.identifiers=true`, double quotes act as identifier delimiters, compatible with SQL ANSI/ISO standard behavior. After enabling, note:
+
+- Double quotes no longer represent strings; `"hello"` is parsed as an identifier (column name/object name), not a string value
+- Objects created with keywords using double quotes must be referenced with double quotes
+
+```sql
+-- Enable double-quoted identifier mode
+SET cz.sql.double.quoted.identifiers=true;
+
+-- Use a keyword as a schema name
+CREATE SCHEMA "select";
+
+-- Must use double quotes when referencing
+SELECT * FROM "select".my_table;
+```
+
+## Naming Rules for Each Object Type
+
+## Instance (INSTANCE NAME)
+
+|      | Constraint | Condition |
+| ---- | -------- | --------------------------------------- |
+| Name rules | Naming rule | Generated by the system; starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|      | Length limit | 3–28 characters |
+|      | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|      | Chinese characters | Not supported |
+|      | Reserved words | None |
+|      | Duplicates | Unique identifier for the user's Lakehouse |
+
+## WORKSPACE
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | 3–28 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved word: `sys` |
+|         | Duplicates | Not allowed within the current instance |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | No more than 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed in the middle |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## SCHEMA
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved words: `information_schema`, `public` |
+|         | Duplicates | Not allowed within the current WORKSPACE; cannot duplicate a VOLUME name |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9) |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## TABLE
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | None |
+|         | Duplicates | Not allowed within the current schema |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## VIEW
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | None |
+|         | Duplicates | Not allowed within the current schema |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## VIRTUAL CLUSTER
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). **Lowercase letters are converted to uppercase** (opposite of other objects). |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Duplicates | Not allowed within the current workspace |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## COLUMN
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase; backticks support special characters, Chinese characters, and numeric prefixes. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Not supported for regular identifiers; backtick identifiers support hyphens, spaces, and other special characters |
+|         | Chinese characters | Supported (requires backticks, e.g., `` `chinese_col` ``) |
+|         | Reserved words | Reserved words exist |
+|         | Duplicates | Not allowed within the current table |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## VOLUME
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved words exist |
+|         | Duplicates | Not allowed within the current workspace; cannot duplicate a schema name |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## FUNCTION
+
+|         | Constraint | Condition |
+| ------- | -------- | ------------------------------------------ |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase; avoid duplicating built-in function names. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved words exist |
+|         | Duplicates | Duplicates allowed; distinguished by input parameters |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## ROLE
+
+|             | Constraint | Condition |
+| ----------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|             | Length limit | Maximum 1–256 characters |
+|             | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|             | Chinese characters | Not supported |
+|             | Reserved words | Reserved roles exist |
+|             | Duplicates | Not allowed within the current instance |
+| COMMENT     | Naming rule | Starts with a letter or Chinese character |
+|             | Length limit | 1024 characters |
+|             | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|             | Chinese characters | Supported |
+|             | Case sensitivity | Case-sensitive |
+| ROLE_ALIAS | Naming rule | Starts with a letter or Chinese character |
+|             | Length limit | 1024 characters |
+|             | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|             | Chinese characters | Supported |
+|             | Case sensitivity | Case-insensitive |
+
+## USER
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved words exist (e.g., `USER`) |
+|         | Duplicates | Not allowed within the current Account |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |
+
+## INDEX
+
+|         | Constraint | Condition |
+| ------- | -------- | -------------------------------- |
+| Name rules | Naming rule | Starts with a letter (A-Z, a-z) or underscore (`_`). Uppercase letters are converted to lowercase. |
+|         | Length limit | Maximum 1–256 characters |
+|         | Special characters | Only letters, underscores, and decimal digits (0-9); spaces not allowed |
+|         | Chinese characters | Not supported |
+|         | Reserved words | Reserved words exist |
+|         | Duplicates | Not allowed within the current schema; an index must be in the same schema as its table |
+| COMMENT | Naming rule | Starts with a letter or Chinese character |
+|         | Length limit | 1024 characters |
+|         | Special characters | Letters, Chinese characters, underscores, decimal digits (0-9); spaces allowed |
+|         | Chinese characters | Supported |
+|         | Case sensitivity | Case-sensitive |

package/bin/skills/lakehouse-doc-en/references/object_model_design.md

@@ -0,0 +1,244 @@
+# Object Model Overview
+
+The Singdata Lakehouse object model defines the types, hierarchy, and interactions of all manageable resources in the system. Understanding the object model helps you quickly locate the features you need, organize your data assets correctly, and design a sound data architecture.
+
+## Object Hierarchy
+
+Objects in Lakehouse are organized in the following hierarchy:
+
+![](/.topwrite/assets/05-object-hierarchy.png)
+
+**Hierarchy description**:
+
+| Level     | Description                                                                                                                                                                                               | Contained objects                                                                                                                                               |
+| --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| Instance  | The service instance is the top-level container for all Lakehouse resources, including compute, storage, and metadata                                                                                     | Network Policy, Instance Role, Share, Catalog (three types: Workspace / External Catalog / SHARED)                                                              |
+| Workspace | A Workspace is a MANAGED-type Catalog that provides both a data layer (Schemas and data objects) and a Studio layer (VCluster, users, job scheduling). Workspaces are isolated from each other by default | Connection, VCluster, Workspace Role, Workspace User, External Schema, Studio jobs (SQL / Python / Shell / sync tasks / workflows), and all subordinate Schemas |
+| Schema    | A Schema is the namespace for data objects within a Workspace, used for logical grouping and management of tables, views, and other objects                                                               | Table, Dynamic Table, View, Volume, Pipe, Table Stream, Index, Function, Synonym, Semantic View                                                                 |
+
+## Object Categories
+
+Objects in Lakehouse are grouped by purpose into the following categories:
+
+### Organizational Hierarchy
+
+Organizational hierarchy objects form the resource structure of Lakehouse:
+
+* [Workspace](workspace-introduction.md) — The native top-level namespace in Lakehouse, providing two layers of capability: a **data layer** (all Schemas and data objects, using three-part naming `workspace.schema.table`) and a **Studio layer** (independent user system, role-based permissions, VCluster, and job scheduling). Workspaces are fully isolated from each other by default. Belongs to the **Instance level**.
+
+* **Catalog** — The general top-level namespace concept introduced for federated queries. A Workspace is the MANAGED type (full capability). Two additional read-only types exist, neither of which has a Studio layer:
+  * [External Catalog](external-catalog-summary.md) (EXTERNAL) — Maps metadata from external data sources (Hive, Databricks, Iceberg, etc.), enabling direct queries on external data using three-part naming `catalog.schema.table` without data migration. Belongs to the **Instance level**.
+  * **SHARED** — System-built shared datasets (TPC-H / TPC-DS), read-only. Belongs to the **Instance level**.
+
+* [Schema](schema.md) — The logical namespace for data objects within a Workspace, used for layered management (e.g., ods / dwd / ads). Different Schemas within the same Workspace can reference each other. Belongs to the **Workspace level**.
+
+* [External Schema](external-schema.md) — Created from an External Catalog, maps a Schema from an external data system into the current Workspace so users can query external data with standard SQL without data migration. Belongs to the **Workspace level**.
+
+### Data Tables
+
+Data tables are the core objects for storing and processing data. All belong to the **Schema level**:
+
+* [Table](table.md) — A columnar-storage structured data table supporting INSERT / UPDATE / DELETE. The foundational storage unit for all layers of a data warehouse.
+* [Dynamic Table](dynamic-table.md) — Defines transformation logic in SQL; the system automatically refreshes results incrementally. Ideal for building ODS→DWD→ADS data pipelines without manual scheduling.
+* [View](view.md) — A virtual table that stores no data and is computed dynamically at query time. Useful for encapsulating complex SQL and enforcing column-level access control.
+* [Materialized View](materializedview.md) — Pre-computes and physically stores query results. Suitable for frequently executed fixed aggregation queries, trading storage for query speed.
+* [External Table](external-table-guide.md) — Data resides in an external system (Delta Lake, Hudi, Kafka, etc.); Lakehouse manages only the metadata. Suitable when you want to query data in place without migrating it.
+* [Semantic View](semantic-view-overview.md) — Encapsulates multi-table JOIN and aggregation logic as a business semantic layer. BI tools and AI agents access data through semantic views, hiding the complexity of the underlying table structure.
+
+### File Storage
+
+File storage objects belong to the **Schema level** and are used to manage unstructured data and object storage files:
+
+* [Volume](volume-introduction.md) — A file storage mount point. Pipes read files from a Volume and write them into tables; External Functions can read model files stored in a Volume.
+  * [Internal Volume](internal_volume.md) — User Volume (personal user space) and Table Volume (table-associated storage), created automatically with the instance.
+  * [External Volume](external_volume.md) — Mounts an existing object storage bucket (OSS / COS / S3). Data stays in place; Lakehouse accesses it via a Storage Connection.
+
+### Connection Objects
+
+Connection objects belong to the **Workspace level** and centrally store authentication credentials for third-party services, avoiding hard-coded secrets in SQL:
+
+* [Connection](connection-overview.md) — Securely stores authentication information for third-party services. Access is controlled by the Workspace administrator.
+  * [API Connection](create-api-connection.md) — Stores invocation credentials for cloud functions, used by External Functions to call services such as Alibaba Cloud FC and Tencent Cloud SCF.
+  * [Storage Connection](create-storage-connection.md) — Stores access keys for object storage, used by external Volumes and external tables (OSS, COS, S3).
+  * [Catalog Connection](create-catalog-connection.md) — Stores connection information for external metadata services, used by External Catalogs to connect to Hive Metastore and similar systems.
+
+### Data Pipelines and Change Capture
+
+Data pipeline objects belong to the **Schema level** and handle automatic data flow and change tracking:
+
+* [Pipe](pipe-introduction.md) — Continuously monitors a Volume or Kafka topic and automatically writes newly arrived files or messages into a target table. Replaces manual polling scripts for fully automated file ingestion.
+* [Table Stream](table-stream-title.md) — A cursor object that records incremental changes (INSERT / UPDATE / DELETE) on a table without storing the data itself. Downstream Dynamic Tables or jobs consume the Stream to implement CDC-driven incremental computation.
+
+### Indexes
+
+Indexes belong to the **Schema level** and build auxiliary data structures on tables to accelerate filter conditions without changing the physical storage layout:
+
+* [Bloomfilter Index](bloomfilter-summary.md) — Suited for equality queries (`=`, `IN`). Minimal storage overhead in exchange for significantly fewer unnecessary block reads.
+* [Inverted Index](inverted-index.md) — Suited for full-text search and keyword matching, with support for Chinese tokenization.
+* [Vector Index](vector-search.md) — Suited for semantic similarity search, supporting ANN (approximate nearest neighbor) acceleration for vector retrieval.
+
+### Partitions and Bucketing
+
+Partitions and bucketing belong to the **Schema level** and determine the physical organization of data. They are specified at table creation time and affect the data scan range at query time:
+
+* [Partition](partition_table_guide.md) — Physically groups data by time or business fields. Queries automatically skip irrelevant partitions, making this the primary optimization technique for large-table query performance.
+* [Bucketing](cluster-table-guide.md) — Hashes data into buckets by specified columns, co-locating rows with the same key in the same bucket. Significantly improves data locality for JOIN and aggregation workloads.
+
+### Functions
+
+Function objects belong to the **Schema level**:
+
+* [User-Defined Functions](user-external-function.md) — Encapsulate reusable computation logic in SQL or code, callable like built-in functions in any query.
+  * [SQL Function](create-sql-function.md) — Defined with SQL expressions and executed within the engine. Suitable for encapsulating business rules, calculation formulas, and other pure-SQL logic.
+  * [External Function](remotefunction-intro.md) — Registers an external HTTP service as a SQL function. Suitable for calling LLMs for text processing, vision services for image recognition, and other AI-augmented computation scenarios.
+
+### Synonyms
+
+Synonym objects belong to the **Schema level**:
+
+* [Synonym](synonym.md) — Creates a local alias for an object in another Schema. When the ADS layer references dimension tables from the DIM layer, synonyms avoid writing the full three-part path (`workspace.schema.table`) in every query.
+
+### Data Sharing
+
+Data sharing objects belong to the **Instance level**:
+
+* [Share](data-sharing.md) — A Provider instance grants access to specified tables or views to a Consumer instance within the same cloud and service region. The Consumer reads the Provider's original data directly — no data copying, no storage cost, no synchronization delay. Cross-cloud or cross-region sharing is not supported.
+
+### Studio Objects
+
+Studio objects belong to the **Workspace level** and form Lakehouse's built-in data development and scheduling environment. They share the same user system and permission controls as the SQL data objects in the same Workspace:
+
+* [SQL Job](task-develop.md) — Write and schedule SQL data processing logic in the Studio IDE, with support for dependency orchestration and time-based triggers.
+* [Python / Shell Job](python-task.md) — Run custom scripts to handle complex logic that SQL cannot cover.
+* [Data Sync Job](data-integration.md) — Visually configure real-time CDC sync or offline batch sync for 40+ data sources without writing code. Runs on an Integration VCluster under the hood.
+* [Workflow (Composite Job)](composite_task.md) — Orchestrates multiple jobs into a dependency-aware DAG for unified scheduling and monitoring.
+
+> Studio development objects are primarily managed through the graphical interface rather than SQL DDL. The [CZ-CLI](cz-cli.md) also supports Studio job development, deployment, and run inspection, making it suitable for command-line and AI agent workflows. See [Using Studio In Depth](studio_manual.md).
+
+### Compute Resources
+
+Compute resource objects belong to the **Workspace level**:
+
+* [VCluster (Compute Cluster)](virtual-cluster.md) — An elastic compute resource pool that starts and stops on demand with no charges when idle. Multiple VClusters can be created within the same Workspace to isolate different workloads.
+  * **General-purpose (GP VC**): Suitable for mixed ETL and query workloads.
+  * **Analytics (AP VC**): Optimized for large-scale OLAP queries; ideal for BI and ad-hoc analysis.
+  * **Integration (Integration VC**): Designed for real-time CDC sync tasks with low-latency writes.
+
+### Security Policies
+
+Security policy objects protect data and control access:
+
+* [Network Policy](network_policy.md) — IP-based access control (allowlist / blocklist) that blocks unauthorized sources at the instance entry point. Belongs to the **Instance level**.
+* [Dynamic Data Masking Policy](dynamic-mask.md) — Dynamically replaces sensitive values in specified columns based on the user's role (e.g., phone numbers displayed as `138****8888`). Query results are automatically masked; the underlying data is unchanged. Belongs to the **Schema level** (bound to table columns).
+
+### Identity and Permissions
+
+Users follow a **two-tier model**: created at the instance level, authorized at the workspace level.
+
+* [User](authority-management.md) — Created in the account console. Belongs to the **Instance level**. Newly created users have no data permissions by default; they must be added to a Workspace and granted a role before they can access its resources.
+* [Role](roles.md) — A collection of permissions. Roles simplify permission management through batch authorization.
+  * **Instance Role** — An instance-level role that applies across the entire service instance (e.g., `instance_admin`, `instance_user`). Belongs to the **Instance level**.
+  * **Workspace Role** — A workspace-level role that applies only within a specific Workspace (e.g., `workspace_admin`, `workspace_dev`). Belongs to the **Workspace level**.
+
+### Advanced Table Features
+
+The following are configurable features at the table level, not independent object types:
+
+* [Time Travel](timetravel-summary.md) — Access historical versions of a table to recover from accidental deletions or modifications. Use `TIMESTAMP AS OF` to query data at any historical point in time.
+* [Data Lifecycle Management](data-lifecycle.md) — Set expiration policies for tables or partitions to automatically reclaim expired data and control storage costs.
+
+## Typical Architecture Patterns
+
+### Multi-Cloud · Multi-Region · Multi-Instance
+
+Regardless of which cloud or region a deployment targets, every Lakehouse instance presents a fully consistent SQL syntax, object model, API, and permission system. Business teams can switch deployment environments without rewriting any code.
+
+![](/.topwrite/assets/06-multi-cloud_1779458399919.png)
+
+Each cloud provider's each region can independently host one or more Lakehouse instances. Instances are fully isolated from each other, each with its own compute, storage, metadata, and access controls.
+
+Currently supported cloud providers and regions:
+
+| Cloud Provider | Region                                                                |
+| -------------- | --------------------------------------------------------------------- |
+| Alibaba Cloud  | East China 2 (Shanghai), North China 2 (Beijing), Singapore           |
+| Tencent Cloud  | North China (Beijing), East China (Shanghai), South China (Guangzhou) |
+| AWS            | North China (Beijing), Singapore                                      |
+
+**Use cases**: Multi-region disaster recovery, independent instances for overseas operations, compliance requirements for data residency.
+
+***
+
+### Multiple Workspaces in One Instance — Business Line Isolation
+
+A single instance can host multiple Workspaces. Different business lines use independent Workspaces, achieving full isolation of users, permissions, compute clusters, and data objects. Workspaces are not accessible to each other by default; cross-Workspace access requires explicit authorization.
+
+![](/.topwrite/assets/07-multi-workspace.png)
+
+The core value of this pattern is **isolation**: ETL jobs from the data platform team do not affect BI team query performance; experimental operations by the algorithm team cannot accidentally modify production data; data permissions across business lines remain independent.
+
+**Typical division**:
+
+* **Data Platform Workspace**: Managed by data engineers, runs ETL and CDC sync jobs, holds write permissions.
+* **Business Analytics Workspace**: For analysts and BI teams, read-only access, connected to BI tools, uses a dedicated Analytics VCluster.
+* **AI / ML Workspace**: For algorithm engineers, runs vector search and LLM inference jobs, uses an AI-dedicated VCluster.
+
+***
+
+### Multiple Schemas in One Workspace — Data Warehouse Layering
+
+Within a single Workspace, Schemas implement data warehouse layering. Each layer's data objects are managed independently, and layers are connected through Dynamic Tables that automatically refresh incrementally. This is the recommended data warehouse architecture for Singdata Lakehouse: use Schemas to define layers, use Dynamic Tables to replace manual scheduling, and let the data pipeline run automatically.
+
+![](/.topwrite/assets/08-multi-schema.png)
+
+**Standard layers**:
+
+| Schema | Role                                        | Primary objects                                           |
+| ------ | ------------------------------------------- | --------------------------------------------------------- |
+| `ods`  | Raw data layer, source-aligned storage      | Table, Pipe, Table Stream, External Table                 |
+| `dwd`  | Detail data layer, cleansed and transformed | Dynamic Table, Partition, Dynamic Mask                    |
+| `dws`  | Summary data layer, aggregated metrics      | Dynamic Table, Materialized View, Bloomfilter Index       |
+| `ads`  | Application data layer, externally exposed  | Table, View, Semantic View, Synonym                       |
+| `dim`  | Dimension layer, reused across layers       | Table (slowly changing dimensions), Table Stream, Synonym |
+
+***
+
+### Zero-Copy Cross-Account Data Sharing Within the Same Cloud and Region
+
+Using the Share object, a Provider instance can share tables or views in real time with a Consumer instance in the same cloud and service region. The Consumer queries the Provider's original data directly — no copying, no storage cost, no synchronization delay.
+
+![](/.topwrite/assets/09-data-sharing.svg)
+
+This pattern is suitable for sharing data across subsidiaries within a corporate group, or for data service providers exposing datasets to customers. The Consumer has read-only access and cannot modify the Provider's data; the Provider can revoke authorization at any time.
+
+**Constraint**: Only cross-instance sharing within the same cloud provider and service region is supported. Cross-cloud or cross-region sharing is not supported.
+
+**Operation flow**:
+
+```sql
+-- Provider: create a Share object and grant access
+CREATE SHARE my_share;
+ALTER SHARE my_share ADD TABLE ads.order_summary;
+ALTER SHARE my_share ADD INSTANCE consumer_instance_id;
+
+-- Consumer: create a read-only Schema from the Share and query
+CREATE SCHEMA shared_data FROM SHARE provider_instance.my_share;
+SELECT * FROM shared_data.order_summary;
+```
+
+***
+
+## Object Relationship Quick Reference
+
+| Scenario                                                   | Objects involved                                                                                        |
+| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| Data ingestion (files)                                     | Volume → Pipe → Table                                                                                   |
+| Data ingestion (database CDC)                              | Connection → [Real-time sync job](realtime_sync.md) (Studio) → Table                                    |
+| Data processing (incremental)                              | Table → Dynamic Table → Table                                                                           |
+| Data processing (CDC consumption)                          | Table → Table Stream → Dynamic Table (or job) → Table                                                   |
+| Federated query (in-place acceleration, no data migration) | External Catalog / External Schema → Query — [Implementation guide](lakehouse-acceleration-guide.md)    |
+| Data sharing                                               | Share → Cross-instance access within the same cloud and region (cross-cloud/cross-region not supported) |
+| Query acceleration                                         | Materialized View / Index / Partition → Table                                                           |
+| AI-augmented analytics                                     | Vector Index + Inverted Index + Semantic View                                                           |
+
+^

package/bin/skills/lakehouse-doc-en/references/om-api-connection.md

@@ -0,0 +1,35 @@
+# API Connection
+
+API Connection is used to store authentication information for **third-party function compute services**, enabling External Function invocations.
+
+Currently supported services: Alibaba Cloud Function Compute (FC), Tencent Cloud Serverless Cloud Function (SCF).
+
+## Use Cases
+
+When you need to call custom logic written in Python/Java (External Function) within SQL, you must first create an API Connection to configure the access credentials for the function compute service.
+
+```
+SQL Query
+  └── External Function (invokes remote function)
+        └── API Connection (stores authentication info)
+              └── Alibaba Cloud FC / Tencent Cloud SCF
+```
+
+## Create Example
+
+```sql
+-- Alibaba Cloud Function Compute
+CREATE API CONNECTION my_fc_conn
+  TYPE cloud_function
+  PROVIDER = 'aliyun'
+  REGION = 'cn-hangzhou'
+  ROLE_ARN = 'acs:ram::138xxxxxxxx:role/czudfrole'
+  NAMESPACE = 'my-namespace'
+  CODE_BUCKET = 'my-bucket';
+```
+
+## Related Documents
+
+- [CREATE API CONNECTION](create-api-connection.md) — full syntax
+- [External Function](om-external-function.md) — how to use API Connection to create external functions
+- [Connection Overview](connection-guide.md)

package/bin/skills/lakehouse-doc-en/references/om-bloomfilter.md

@@ -0,0 +1,39 @@
+# Bloomfilter Index
+
+Bloomfilter Index is an **equality filter acceleration index** in Lakehouse. It records the presence of column values at the data file level, allowing the query engine to quickly skip files that do not contain the target value before scanning, reducing I/O.
+
+## How It Works
+
+Bloomfilter is a probabilistic data structure that quickly determines "whether a value may exist in a certain data block":
+- **Not present**: 100% accurate, the data block is skipped directly
+- **May be present**: needs actual read to verify (very low probability of false positives)
+
+## Applicable Scenarios
+
+| Scenario | Effect |
+|------|------|
+| Equality filtering on high-cardinality columns (e.g., `user_id = 12345`) | Significantly reduces the number of scanned files |
+| Point queries (look up single record by ID) | Greatly improves query speed |
+| Range queries (`BETWEEN`, `>`, `<`) | **Ineffective**, not suitable |
+| Low-cardinality columns (e.g., gender, status) | Limited benefit, not recommended |
+
+## Create Example
+
+```sql
+-- Create Bloomfilter Index at table creation
+CREATE TABLE orders (
+    order_id BIGINT,
+    user_id  BIGINT,
+    amount   DECIMAL(10,2),
+    INDEX idx_user_id (user_id) USING BLOOMFILTER
+);
+
+-- Add to an existing table
+ALTER TABLE orders ADD INDEX idx_order_id (order_id) USING BLOOMFILTER;
+```
+
+## Related Documents
+
+- [Bloomfilter Index Details](bloomfilter-summary.md)
+- [Index Usage Best Practices](lakehouse-index-best-practice.md)
+- [Inverted Index](om-inverted-index.md) — full-text search scenarios

package/bin/skills/lakehouse-doc-en/references/om-catalog-connection.md

@@ -0,0 +1,36 @@
+# Catalog Connection
+
+Catalog Connection is used to connect to **external metadata stores** (such as Hive Metastore), allowing Lakehouse to directly read data from external systems through federated queries without needing to migrate data in.
+
+## Use Cases
+
+When your data resides in a Hive data lake and you want to query it directly in Lakehouse without copying data, establish a connection via Catalog Connection, then create an External Catalog.
+
+```
+Lakehouse SQL Query
+  └── External Catalog
+        └── Catalog Connection (stores Hive Metastore connection info)
+              └── Hive Metastore → Data lake files
+```
+
+## Create Example
+
+```sql
+CREATE CATALOG CONNECTION my_hive_conn
+  TYPE HIVE
+  METASTORE_URI = 'thrift://hive-metastore-host:9083';
+```
+
+## Comparison with Other Connections
+
+| Connection Type | Connection Target | Used With |
+|----------------|---------|--------------|
+| API Connection | Function compute services | External Function |
+| Storage Connection | Object storage | External Volume, external tables |
+| **Catalog Connection** | Hive Metastore | External Catalog |
+
+## Related Documents
+
+- [CREATE CATALOG CONNECTION](create-catalog-connection.md) — full syntax
+- [External Catalog Federated Queries](external-catalog-concept.md)
+- [Connection Overview](connection-guide.md)