@thinkingai/ae-cli 6.0.18-1 → 6.0.20

package/skills/ae-dataops/references/dataops-integration.md

@@ -1,7 +1,7 @@
 ---
 name: dataops-integration
 version: 1.0.0
-description: "Datasource and data integration: create datasources, create sync solutions, configure field mapping, execute sync, monitor sync. Trigger keywords: datasource, sync, integration, sync, field mapping, data ingestion, datasource, MySQL, ClickHouse."
+description: "Datasource and data integration: create datasources, configure sync solutions and field mappings, execute and monitor runs. Trigger keywords: datasource, sync, integration, field mapping, data ingestion, MySQL, ClickHouse, DatabricksJdbc."
 metadata:
   requires:
     bins: ["ae-cli"]
@@ -16,6 +16,7 @@ Use the `dataops_integration` subcommand to manage datasources and sync solution
 **Core Rules:**
 - Configuration parameters differ significantly between preset repositories (te_etl) and non-preset repositories, must strictly follow templates
 - sourceConfig/sinkConfig/channelConfig/fieldsMapping are all JSON strings
+- Sync solution updates are not partial patches: call `+get_sync_detail --withParams true` first, then submit complete sourceConfig and sinkConfig JSON. Pass channelConfig and fieldsMapping when keeping or updating them
 - Test datasource connection before creating sync solution
 
 ---
@@ -23,20 +24,22 @@ Use the `dataops_integration` subcommand to manage datasources and sync solution
 ## Workflow A: Create Datasource
 
 ```bash
-# Step 1: View supported datasource component types
+# Step 1: View supported datasource component types. No spaceCode is required.
 ae-cli dataops_integration +list_datasource_components
 
-# Step 2: Test connection (temporary configuration, not saved)
-ae-cli dataops_integration +test_datasource_connect --spaceCode "${spaceCode}" \
-  --componentName "MySQL" \
-  --configValue '{"host":"localhost","port":3306,"username":"root","password":"xxx","database":"test"}'
+# Step 2: Get envJsonList template for one component
+ae-cli dataops_integration +get_datasource_component_template \
+  --componentName "MySQL"
 
 # Step 3: Create datasource
 ae-cli dataops_integration +add_datasource --spaceCode "${spaceCode}" \
   --componentName "MySQL" --dataSourceName "Production MySQL" \
   --sharedConfig true \
-  --envJsonList '[{"host":"localhost","port":3306,"username":"root","password":"xxx"}]' \
-  --confirmed true
+  --envJsonList '[{"host":"localhost","port":3306,"username":"root","password":"xxx"}]'
+
+# Step 4: Test the saved datasource connection by name.
+ae-cli dataops_integration +test_datasource_connect --spaceCode "${spaceCode}" \
+  --datasourceName "Production MySQL"
 ```
 
 ---
@@ -49,8 +52,12 @@ ae-cli dataops_integration +add_datasource --spaceCode "${spaceCode}" \
 # View all datasources in workspace
 ae-cli dataops_integration +list_space_datasources --spaceCode "${spaceCode}"
 
+# View one datasource detail when connection config or failure reason is needed
+ae-cli dataops_integration +get_datasource_detail --spaceCode "${spaceCode}" \
+  --datasourceName "${datasourceName}"
+
 # View datasources available for sync (categorized by source/sink)
-ae-cli dataops_integration +list_sync_datasources --spaceCode "${spaceCode}"
+ae-cli dataops_integration +list_sync_datasources --spaceCode "${spaceCode}" --env DEV
 ```
 
 ### Step 2: Browse Source Table Structure
@@ -63,10 +70,12 @@ ae-cli dataops_integration +list_datasource_databases --spaceCode "${spaceCode}"
 # List tables under database
 ae-cli dataops_integration +list_datasource_tables --spaceCode "${spaceCode}" \
   --datasourceId "${datasourceId}" --database "test"
+# For catalog-based sources such as Databricks, add --catalog "${catalog}".
 
-# Get table structure (column definitions, partitions)
+# Get table structure (columns and partitionColumns)
 ae-cli dataops_integration +get_table_structure --spaceCode "${spaceCode}" \
   --datasourceId "${datasourceId}" --database "test" --tablePath "users"
+# For catalog-based sources such as Databricks, add --catalog "${catalog}".
 ```
 
 ### Step 3: Create Sync Solution
@@ -77,12 +86,11 @@ ae-cli dataops_integration +get_table_structure --spaceCode "${spaceCode}" \
 ae-cli dataops_integration +add_sync_solution --spaceCode "${spaceCode}" \
   --syncName "MySQL to Preset Repository Sync" \
   --srcComponent "MySQL" --srcDatasourceId "${mysqlDatasourceId}" \
-  --sinkComponent "hive" --sinkDatasourceId "te_etl@TASK_ENGINE_TRINO" \
+  --sinkComponent "te_etl" --sinkDatasourceId "te_etl@TASK_ENGINE_TRINO" \
   --sourceConfig '{"component":"MySQL","datasourceId":"xxx","database":"test","tablePath":"users"}' \
-  --sinkConfig '{"component":"hive","datasourceId":"te_etl@TASK_ENGINE_TRINO","database":"","tablePath":"ods_users_mysql","tableType":"PHYSICAL_TABLE","bizClassify":"CURRENT","dbBizType":"TASK_ENV_DB","authedSpace":"","partitionKeys":[],"dataSaveMode":1,"batchSize":20000}' \
+  --sinkConfig '{"component":"te_etl","datasourceId":"te_etl@TASK_ENGINE_TRINO","database":"","tablePath":"ods_users_mysql","tableType":"PHYSICAL_TABLE","bizClassify":"CURRENT","dbBizType":"TASK_ENV_DB","authedSpace":"","partitionKeys":[],"dataSaveMode":1,"batchSize":20000}' \
   --channelConfig '{"limitType":"0","gatewayConfig":{"engineFlag":"TASK_ENGINE_TRINO","companyId":1,"appDefinition":"APP_GAIA","bizFlag":"BIZ_GAIA_TASK_RELEASE","repoCode":"te_etl","spaceCode":"default"}}' \
-  --fieldsMapping '{"mapping":[{"source":{"name":"id","type":"int","manual":false,"partitionKey":false,"primaryKey":false,"shardingKey":false,"sortingKey":false,"upsertKey":false},"target":{"name":"id","type":"int","manual":false,"partitionKey":false,"primaryKey":false,"shardingKey":false,"sortingKey":false,"upsertKey":false}}]}' \
-  --confirmed true
+  --fieldsMapping '{"mapping":[{"source":{"name":"id","type":"int","manual":false,"partitionKey":false,"primaryKey":false,"shardingKey":false,"sortingKey":false,"upsertKey":false},"target":{"name":"id","type":"int","manual":false,"partitionKey":false,"primaryKey":false,"shardingKey":false,"sortingKey":false,"upsertKey":false}}]}'
 ```
 
 ### Step 4: Execute Sync
@@ -90,30 +98,42 @@ ae-cli dataops_integration +add_sync_solution --spaceCode "${spaceCode}" \
 ```bash
 # Manually execute sync solution
 ae-cli dataops_integration +exec_sync_solution --spaceCode "${spaceCode}" \
-  --syncId "${syncId}" --confirmed true
+  --syncId "${syncId}" --baseDate "${baseDate}"
 
-# View execution history
-ae-cli dataops_integration +list_sync_exec_histories --spaceCode "${spaceCode}" \
+# List manual sync runs
+ae-cli dataops_integration +list_sync_runs --spaceCode "${spaceCode}" \
   --syncId "${syncId}" --limit 20
 
-# View execution details
-ae-cli dataops_integration +get_sync_exec_info --spaceCode "${spaceCode}" \
-  --syncId "${syncId}" --taskId "${taskId}"
-
 # Stop execution if necessary
 ae-cli dataops_integration +stop_sync_solution --spaceCode "${spaceCode}" \
-  --syncId "${syncId}" --taskId "${taskId}" --confirmed true
+  --syncId "${syncId}" --taskId "${taskId}"
 ```
 
 ---
 
 ## JSON Configuration Templates
 
+### Preset Repository as Source (sourceConfig)
+
+```json
+{
+  "component": "te_etl",
+  "datasourceId": "te_etl@TASK_ENGINE_TRINO",
+  "tablePath": "source_table",
+  "tableType": "PHYSICAL_TABLE",
+  "bizClassify": "CURRENT",
+  "dbBizType": "TASK_ENV_DB",
+  "authedSpace": "",
+  "partitionKeys": [],
+  "successOnEmpty": false
+}
+```
+
 ### Preset Repository as Target (Sink)
 
 ```json
 {
-  "component": "hive",
+  "component": "te_etl",
   "datasourceId": "te_etl@TASK_ENGINE_TRINO",
   "database": "",
   "tablePath": "ods_users_mysql",
@@ -191,28 +211,47 @@ ae-cli dataops_integration +stop_sync_solution --spaceCode "${spaceCode}" \
 
 | Command | Purpose | Key Flags |
 |---------|---------|-----------|
-| `+list_datasource_components` | List component types | None |
-| `+test_datasource_connect` | Test connection | `--spaceCode` `--componentName` `--configValue` |
-| `+add_datasource` | Create datasource | `--spaceCode` `--componentName` `--dataSourceName` `--sharedConfig` `--envJsonList` `--confirmed` |
-| `+modify_datasource` | Modify datasource | `--spaceCode` `--dataSourceName` `--dataSourceRemark` `--sharedConfig` `--envJsonList` `--confirmed` |
-| `+online_datasource` | Online datasource | `--spaceCode` `--dataSourceNames` `--confirmed` |
-| `+list_space_datasources` | List datasources | `--spaceCode` `--componentName` `--dataSourceName` |
-| `+list_sync_datasources` | Sync datasources | `--spaceCode` `--env` |
-| `+list_datasource_databases` | List databases | `--spaceCode` `--datasourceId` `--env` |
-| `+list_datasource_tables` | List tables | `--spaceCode` `--datasourceId` `--database` `--env` |
-| `+get_table_structure` | Table structure | `--spaceCode` `--datasourceId` `--database` `--tablePath` `--env` |
-| `+add_sync_solution` | Create sync solution | `--spaceCode` `--syncName` `--srcComponent` `--srcDatasourceId` `--sinkComponent` `--sinkDatasourceId` `--sourceConfig` `--sinkConfig` `--channelConfig` `--fieldsMapping` `--confirmed` |
-| `+save_sync_solution` | Update sync solution | `--spaceCode` `--syncId` `--syncName` `--sourceConfig` `--sinkConfig` `--channelConfig` `--fieldsMapping` `--confirmed` |
+| `+list_datasource_components` | List supported datasource components | None |
+| `+get_datasource_component_template` | Component envJsonList template | `--componentName` |
+| `+test_datasource_connect` | Test saved datasource connection | `--spaceCode` `--datasourceName` |
+| `+add_datasource` | Create datasource | `--spaceCode` `--componentName` `--dataSourceName` `--sharedConfig` `--envJsonList` `[--dataSourceRemark]` |
+| `+modify_datasource` | Modify datasource | `--spaceCode` `--dataSourceName` `[--dataSourceRemark]` `[--sharedConfig]` `[--envJsonList]` |
+| `+online_datasource` | Online datasource | `--spaceCode` `--dataSourceNames` |
+| `+list_space_datasources` | List datasources | `--spaceCode` `[--datasourceName]` `[--componentName]` |
+| `+get_datasource_detail` | Datasource detail | `--spaceCode` `--datasourceName` |
+| `+list_sync_datasources` | Sync datasources | `--spaceCode` `[--env]` |
+| `+list_datasource_databases` | List datasource databases | `--spaceCode` `--datasourceId` `[--catalog]` `[--env]` |
+| `+list_datasource_tables` | List datasource tables | `--spaceCode` `--datasourceId` `--database` `[--catalog]` `[--env]` |
+| `+get_table_structure` | Datasource table structure | `--spaceCode` `--datasourceId` `--database` `--tablePath` `[--catalog]` `[--env]` |
+| `+add_sync_solution` | Create sync solution | `--spaceCode` `--syncName` `--srcComponent` `--srcDatasourceId` `--sinkComponent` `--sinkDatasourceId` `--sourceConfig` `--sinkConfig` `[--channelConfig]` `[--fieldsMapping]` `[--remark]` |
+| `+save_sync_solution` | Update sync solution | `--spaceCode` `--syncId` `--sourceConfig` `--sinkConfig` `[--syncName]` `[--channelConfig]` `[--fieldsMapping]` `[--remark]` |
 | `+list_sync_solutions` | List sync solutions | `--spaceCode` |
-| `+get_sync_detail` | Sync solution details | `--spaceCode` `--syncId` `--withParams` |
-| `+exec_sync_solution` | Execute sync | `--spaceCode` `--syncId` `--execParams` `--confirmed` |
-| `+list_sync_exec_histories` | Execution history | `--spaceCode` `--syncId` `--execType` `--limit` |
-| `+get_sync_exec_info` | Execution details | `--spaceCode` `--syncId` `--taskId` |
-| `+stop_sync_solution` | Stop sync | `--spaceCode` `--syncId` `--taskId` `--confirmed` |
+| `+get_sync_detail` | Sync solution detail | `--spaceCode` `--syncId` `[--withParams]` |
+| `+exec_sync_solution` | Execute sync | `--spaceCode` `--syncId` `[--baseDate]` `[--comment]` |
+| `+list_sync_runs` | Manual sync runs | `--spaceCode` `--syncId` `[--limit]` |
+| `+stop_sync_solution` | Stop one running sync execution task | `--spaceCode` `--syncId` `--taskId` |
+
+- **Datasource components**: `+list_datasource_components` requires no arguments. It returns an array of `componentName`, `componentType`, and `description`.
+- **Datasource component template**: `+get_datasource_component_template` requires only `--componentName`. It returns component metadata, `requiredFields`, `optionalFields`, `envJsonExampleObject`, and `importantNotes`.
+- **Datasource list**: `+list_space_datasources` requires `--spaceCode`; `--datasourceName` and `--componentName` are optional filters. It returns datasource summary fields including `datasourceId`, `dataSourceComponentName`, `dataSourceName`, `dataSourceRemark`, `dataSourceStatus`, `connectStatus`, `syncTaskNum`, and `sharedConfig`.
+- **Datasource detail**: `+get_datasource_detail` requires `--spaceCode` and `--datasourceName`. It returns `datasourceId`, `dataSourceComponentName`, `dataSourceName`, `dataSourceRemark`, `dataSourceStatus`, `connectStatus`, `syncTaskNum`, `sharedConfig`, masked `connectConfig`, `connectFails`, and `lastConnectTime`.
+- **Datasource creation**: `+add_datasource` creates directly after the CLI write confirmation. `envJsonList` must be a JSON array string using keys from the component template `requiredFields`; `sharedConfig=true` uses one config for DEV/PROD, while `sharedConfig=false` requires two configs (DEV, PROD).
+- **Datasource connection test**: `+test_datasource_connect` requires `--spaceCode` and `--datasourceName`. It tests the saved datasource config and returns `datasourceName`, `connectStatus`, `connectFails`, `lastConnectTime`, and `nextAction`.
+- **Datasource modification**: `+modify_datasource` creates no preview; it updates only provided optional fields after the CLI write confirmation. Use `--envJsonList` with the same JSON array format as `+add_datasource`.
+- **Datasource online**: `+online_datasource` requires `--spaceCode` and `--dataSourceNames`. It executes after the CLI write confirmation and returns `failDataSources` and `successDataSourceNames`.
+- **Sync datasources**: `+list_sync_datasources` requires `--spaceCode`; `--env` is optional and defaults to `DEV`. It returns `sourceComponentSet` and `sinkComponentSet`, grouped by component, with `dataSourceList` and `supportableComponent`.
+- **Datasource databases**: `+list_datasource_databases` requires `--spaceCode` and `--datasourceId`; `--catalog` and `--env` are optional, and `--env` defaults to `DEV`. It returns an array of objects with `databaseName`.
+- **Datasource tables**: `+list_datasource_tables` requires `--spaceCode`, `--datasourceId`, and `--database`; `--catalog` and `--env` are optional, and `--env` defaults to `DEV`. It returns table metadata including `database`, `tableName`, `tableType`, `tableComment`, `engine`, `disabled`, `disabledReasons`, `sameVersion`, and `supportSharding`.
+- **Datasource table structure**: `+get_table_structure` requires `--spaceCode`, `--datasourceId`, `--database`, and `--tablePath`; `--catalog` and `--env` are optional, and `--env` defaults to `DEV`. It returns `columns` and `partitionColumns`.
+- **Sync solution list**: `+list_sync_solutions` requires only `--spaceCode`. It returns sync metadata including `syncId`, `syncName`, source/sink datasource and table fields, last execution/schedule status codes, owner, remark, and timestamps.
+- **Sync detail**: `+get_sync_detail` requires `--spaceCode` and `--syncId`; `--withParams` is optional and defaults to `false`. It returns source, sink, field mapping, last execution/schedule status, owner, and `nextAction`; `withParams=true` also returns `usedParams`.
+- **Sync update**: `+save_sync_solution` requires `--spaceCode`, `--syncId`, `--sourceConfig`, and `--sinkConfig`; `--syncName`, `--channelConfig`, `--fieldsMapping`, and `--remark` are optional. Passing `--syncName` renames the sync solution; omit it to keep the current name. `sourceConfig` and `sinkConfig` must be complete JSON strings from `+get_sync_detail --withParams true`; pass complete `channelConfig` and `fieldsMapping` when keeping or updating them.
+- **Sync runs**: `+list_sync_runs` requires `--spaceCode` and `--syncId`; `--limit` is optional and defaults to `20`. It returns `runs`, `returnedCount`, `limit`, and `nextAction`; each run includes `taskId`, `execType`, `status`, `execTime`, `channelMode`, and `submitter`.
+- **Stop sync run**: `+stop_sync_solution` requires `--spaceCode`, `--syncId`, and `--taskId`. Use `taskId` from `+list_sync_runs` for an active run. It returns `action`, `result` with `execStatus`, `syncId`, and `taskId`, and top-level `status`.
 
 ## Component Conditional Required Parameters
 
-Some components have conditional required parameters that vary based on deployment mode. The requiredFields from `+list_datasource_components` may not include these parameters (they are explained in optionalFields or importantNotes). When creating datasources, be sure to supplement corresponding parameters based on the user's selected mode.
+Some components have conditional required parameters that vary based on deployment mode. The requiredFields from `+get_datasource_component_template` may not include these parameters (they are explained in optionalFields or importantNotes). When creating datasources, be sure to supplement corresponding parameters based on the user's selected mode.
 
 ### MongoDB
 

package/skills/ae-dataops/references/dataops-query.md

@@ -15,6 +15,10 @@ Use the `dataops_ide` subcommand for data exploration and SQL queries.
 
 **Core Rule: IDE only allows query operations, prohibits creating/modifying/deleting data tables (use dataops_datatable for task table DDL).**
 
+**Table discovery rule:** use `dataops_datatable +dict_search_tables` first for the visible DataOps catalog. Use `dataops_ide +search_tables` only when raw engine metadata is needed, and `dataops_ide +ide_list_tables` only for known catalog/schema browsing.
+
+If `spaceCode` is unknown, run `ae-cli dataops_repo +list_spaces` first. Use the only returned `spaceCode` directly; when multiple spaces are returned, choose by user intent or ask the user instead of guessing.
+
 ---
 
 ## Workflow A: Browse Data Warehouse Metadata
@@ -22,67 +26,57 @@ Use the `dataops_ide` subcommand for data exploration and SQL queries.
 Explore data warehouse structure step by step: repository → catalog → schema → table → column.
 
 ```bash
-# Step 1: List available repositories
-ae-cli dataops_ide +list_repos --spaceCode "${spaceCode}"
+# Step 1: List available IDE warehouse repositories
+ae-cli dataops_ide +ide_list_repos --spaceCode "${spaceCode}"
+
+# Step 2: List accessible catalogs and schema names. Defaults: connType=SPACE, repoCode=te_etl.
+ae-cli dataops_ide +ide_list_catalogs --spaceCode "${spaceCode}"
 
-# Step 2: List all catalogs and their schemas in repository
-ae-cli dataops_ide +list_catalogs --spaceCode "${spaceCode}" --connType "SPACE" --repoCode "${repoCode}"
+# Step 3: List physical tables under one schema. Defaults: connType=SPACE, repoCode=te_etl, entityType=TABLE, pageNum=1, pageSize=20.
+# This command does not support keyword filtering; use +search_tables only for engine-side keyword search.
+ae-cli dataops_ide +ide_list_tables --spaceCode "${spaceCode}" \
+  --catalog "${catalog}" --schema "${schema}" --pageSize 20
 
-# Step 3: List tables/views under schema
-ae-cli dataops_ide +list_tables --spaceCode "${spaceCode}" --connType "SPACE" --repoCode "${repoCode}" \
-  --catalog "${catalog}" --schema "${schema}" --isView false --pageNum 1 --pageSize 100
+# To list views under the same schema
+ae-cli dataops_ide +ide_list_tables --spaceCode "${spaceCode}" \
+  --catalog "${catalog}" --schema "${schema}" --entityType VIEW --pageSize 20
 
-# Step 4: Get detailed table metadata (column definitions, DDL, partitions, row count)
-ae-cli dataops_ide +get_table_detail --spaceCode "${spaceCode}" --connType "SPACE" --repoCode "${repoCode}" \
-  --catalog "${catalog}" --schema "${schema}" --tableName "${tableName}" \
-  --engineType "TASK_ENGINE_TRINO" --isView false
+# Step 4: Get engine-side table/view detail. Defaults: connType=SPACE, repoCode=te_etl, engineType=TASK_ENGINE_TRINO; entityType is auto-detected; tableDdl is hidden unless --includeDdl true.
+ae-cli dataops_ide +ide_get_table_detail --spaceCode "${spaceCode}" \
+  --catalog "${catalog}" --schema "${schema}" --tableName "${tableName}"
 ```
 
 ---
 
 ## Workflow B: Search and View Tables
 
-When you don't know the exact table location, search by keyword.
+When you don't know the exact table location, search by keyword. Prefer `dataops_datatable +dict_search_tables`; use the IDE search below only when raw engine metadata is needed.
 
 ```bash
-# Fuzzy search tables across catalog/schema
-ae-cli dataops_ide +search_tables --spaceCode "${spaceCode}" --connType "SPACE" --repoCode "${repoCode}" \
-  --searchKey "user" --size 20
-
-# Fuzzy search column names across tables
-ae-cli dataops_ide +search_columns --spaceCode "${spaceCode}" --repoCode "${repoCode}" \
-  --searchKey "user_id" --tables '[{"catalog":"hive","isView":false,"schema":"ws_default_dev","tableName":"dwd_user"}]' \
-  --engineType "TASK_ENGINE_TRINO" --pageNum 1 --pageSize 100
-
-# Generate SELECT SQL based on table and column names
-ae-cli dataops_ide +generate_sql --spaceCode "${spaceCode}" --repoCode "${repoCode}" \
-  --catalog "hive" --schema "ws_default_dev" --tableName "dwd_user" \
-  --engineType "TASK_ENGINE_TRINO" --selectColumns '["user_id","user_name","age"]'
+# Search engine-side warehouse tables/views by keyword. Defaults: connType=SPACE, repoCode=te_etl, size=20.
+ae-cli dataops_ide +search_tables --spaceCode "${spaceCode}" --searchKey "user"
 ```
 
 ---
 
-## Workflow C: Execute SQL Query (Complete Async Flow)
+## Workflow C: Execute SQL Query (Download-Centered Async Flow)
 
-This is the most important workflow, containing 3 steps: execute → poll progress → get results.
+Use this flow for SQL query execution. Result rows are not returned through MCP/CLI; submit creates a Gaia download-center task directly.
 
 ```bash
-# Step 1: Execute SQL asynchronously (returns requestId)
-ae-cli dataops_ide +execute_sql --spaceCode "${spaceCode}" --repoCode "te_etl" \
+# Step 1: Submit SQL and create a download task. Defaults: repoCode=te_etl, engineType=TASK_ENGINE_TRINO.
+ae-cli dataops_ide +submit_sql_query --spaceCode "${spaceCode}" --repoCode "te_etl" \
   --sql "SELECT * FROM hive.ws_default_dev.dwd_user LIMIT 10" \
-  --engineType "TASK_ENGINE_TRINO" --confirmed true
+  --engineType "TASK_ENGINE_TRINO"
 
-# Step 2: Query execution progress (use requestId from Step 1, recommend 2-5 second polling)
-# state values: QUEUED / RUNNING / FINISHED / FAILED / CANCELLED
-ae-cli dataops_ide +get_query_progress --requestId "${requestId}"
+# Step 2: Poll the download task status by spaceCode/downloadTaskId. Rows are not returned through MCP/CLI.
+ae-cli dataops_ide +get_sql_query_status --spaceCode "${spaceCode}" --downloadTaskId ${downloadTaskId}
 
-# Step 3: After state=FINISHED, get query results
-# recordId obtained from query history or progress info
-ae-cli dataops_ide +get_query_result --spaceCode "${spaceCode}" --connType "SPACE" \
-  --repoCode "te_etl" --recordId "${recordId}"
+# Step 3: CLI-only local save after downloadStatus=SUCCESS. Use a .zip suffix.
+ae-cli dataops_ide +get_sql_query_status --spaceCode "${spaceCode}" --downloadTaskId ${downloadTaskId} --downloadTo "./result.zip"
 
-# (Optional) Cancel running query
-ae-cli dataops_ide +cancel_query --requestId "${requestId}" --confirmed true
+# (Optional) Cancel the download task.
+ae-cli dataops_ide +cancel_sql_query --spaceCode "${spaceCode}" --downloadTaskId ${downloadTaskId}
 ```
 
 ---
@@ -91,21 +85,28 @@ ae-cli dataops_ide +cancel_query --requestId "${requestId}" --confirmed true
 
 | Command | Purpose | Key Flags |
 |---------|---------|-----------|
-| `+list_repos` | List repositories | `--spaceCode` |
-| `+list_catalogs` | List catalog/schema | `--spaceCode` `--connType` `--repoCode` |
-| `+list_tables` | List tables | `--spaceCode` `--connType` `--repoCode` `--catalog` `--schema` `--isView` `--pageNum` `--pageSize` |
-| `+get_table_detail` | Table metadata | `--spaceCode` `--connType` `--repoCode` `--catalog` `--schema` `--tableName` `--engineType` `--isView` |
-| `+get_schema_info` | Schema statistics | `--spaceCode` `--connType` `--repoCode` `--catalog` `--schema` |
-| `+search_tables` | Fuzzy search tables | `--spaceCode` `--connType` `--repoCode` `--searchKey` `--size` |
-| `+search_columns` | Fuzzy search columns | `--spaceCode` `--repoCode` `--searchKey` `--tables` (JSON) `--engineType` `--pageNum` `--pageSize` |
-| `+generate_sql` | Generate SELECT | `--spaceCode` `--repoCode` `--catalog` `--schema` `--tableName` `--engineType` `--selectColumns` (JSON) |
-| `+execute_sql` | Execute SQL | `--spaceCode` `--repoCode` `--sql` `--engineType` `--confirmed` |
-| `+get_query_progress` | Query progress | `--requestId` |
-| `+get_query_result` | Get results | `--spaceCode` `--connType` `--repoCode` `--recordId` |
-| `+cancel_query` | Cancel query | `--requestId` `--confirmed` |
+| `+ide_list_repos` | List IDE repositories grouped by connType | `--spaceCode` |
+| `+ide_list_catalogs` | List IDE catalogs and schemas | `--spaceCode` `[--connType]` `[--repoCode]` |
+| `+ide_list_tables` | List tables or views in one catalog/schema | `--spaceCode` `--catalog` `--schema` `[--connType]` `[--repoCode]` `[--entityType]` `[--pageNum]` `[--pageSize]` |
+| `+ide_get_table_detail` | Get engine-side table/view detail; `tableDdl` only with `--includeDdl true` | `--spaceCode` `--catalog` `--schema` `--tableName` `[--connType]` `[--repoCode]` `[--engineType]` `[--entityType]` `[--includeDdl]` |
+| `+get_schema_info` | Get schema statistics only (`schema/tableNum/viewNum`) | `--spaceCode` `--catalog` `--schema` `[--connType]` `[--repoCode]` |
+| `+search_tables` | Search engine-side warehouse tables/views by keyword | `--spaceCode` `--searchKey` `[--connType]` `[--repoCode]` `[--size]` |
+| `+submit_sql_query` | Submit SQL and create a download task | `--spaceCode` `--sql` `[--repoCode]` `[--engineType]` |
+| `+get_sql_query_status` | Poll download task status; `--downloadTo` is CLI-only local save after `downloadStatus=SUCCESS` | `--spaceCode` `--downloadTaskId` `[--requestId]` `[--downloadTo]` |
+| `+cancel_sql_query` | Cancel download task; `--requestId` is trace-only | `--spaceCode` `--downloadTaskId` `[--requestId]` |
 
 ## Parameter Notes
 
 - **connType**: `SPACE` (data warehouse for daily queries, default) | `ETL` (ETL engine) | `APP` (app warehouse for external services)
-- **engineType**: `TASK_ENGINE_TRINO` (default, interactive queries) | `TASK_ENGINE_STARROCKS` (real-time analytics, high concurrency)
-- **isView**: `false` for physical tables | `true` for views
+- **Repository list**: `+ide_list_repos` requires `--spaceCode` and has no optional flags. It returns an array grouped by `connType`; each group has `repos` with `repoCode`, `repoDesc`, and `engineTypes`.
+- **Catalog list**: `+ide_list_catalogs` requires `--spaceCode`; `--connType` and `--repoCode` are optional and default to `SPACE` and `te_etl`. It returns an array of catalogs with `catalog`, `catalogBelongEnum`, `schemaNum`, and `schemas`.
+- **Table list**: `+ide_list_tables` requires `--spaceCode`, `--catalog`, and `--schema`. `--connType`, `--repoCode`, `--entityType`, `--pageNum`, and `--pageSize` are optional and default to `SPACE`, `te_etl`, `TABLE`, `1`, and `20`. It returns `items`, `pageNum`, `pageSize`, `returnedCount`, `hasMoreMaybe`, and `nextAction`.
+- **Table search**: `+search_tables` requires `--spaceCode` and `--searchKey`. `--connType`, `--repoCode`, and `--size` are optional and default to `SPACE`, `te_etl`, and `20`. It returns `items`, `searchKey`, `size`, `totalCount`, `tableCount`, `viewCount`, `returnedCount`, `hasMore`, and `nextAction`.
+- **Table detail**: `+ide_get_table_detail` requires `--spaceCode`, `--catalog`, `--schema`, and `--tableName`. `--connType`, `--repoCode`, `--engineType`, `--entityType`, and `--includeDdl` are optional and default to `SPACE`, `te_etl`, `TASK_ENGINE_TRINO`, auto-detect, and `false`. It returns identity, storage metadata, columns, partitions, partition keys, optional layout fields, and `tableDdl` only when requested.
+- **Schema info**: `+get_schema_info` requires `--spaceCode`, `--catalog`, and `--schema`. `--connType` and `--repoCode` are optional and default to `SPACE` and `te_etl`. It returns only `schema`, `tableNum`, and `viewNum`.
+- **SQL submit**: `+submit_sql_query` requires `--spaceCode` and `--sql`. `--repoCode` and `--engineType` are optional and default to `te_etl` and `TASK_ENGINE_TRINO`. On success it returns `requestId`, `spaceCode`, `repoCode`, `downloadTaskId`, `downloadStatus`, `downloadApi`, `downloadParams`, and `nextAction`; rows are never returned.
+- **SQL status**: `+get_sql_query_status` requires `--spaceCode` and `--downloadTaskId`. `--requestId` is optional trace-only. `--downloadTo` is CLI-only; after `downloadStatus=SUCCESS` it saves the result zip and adds `localFile`. It returns status/progress metadata, `nextAction`, `downloadApi`, and `downloadParams`; rows are never returned.
+- **SQL cancel**: `+cancel_sql_query` requires `--spaceCode` and `--downloadTaskId`. `--requestId` is optional trace-only. It returns cancellation request metadata such as `downloadCancelStatus`.
+- **engineType**: `TASK_ENGINE_TRINO` (default, interactive queries) | `TASK_ENGINE_STARROCKS` (high-concurrency analytics)
+- **entityType**: optional `TABLE` or `VIEW` hint. Omit it unless the target type must be forced.
+- **includeDdl**: defaults to false. Use `--includeDdl true` only when the raw DDL text is needed.

package/skills/ae-dataops/references/dataops-table.md

@@ -1,7 +1,7 @@
 ---
 name: dataops-table
 version: 1.0.0
-description: "Data table and view management: search tables, view table details, create physical tables/views, batch create linked views. Trigger keywords: create table, table creation, view, data dictionary, table details, linked view, linkview, datatable, DDL."
+description: "Data table and view management: search tables, view table details, create physical tables/views. Trigger keywords: create table, table creation, view, data dictionary, table details, datatable, DDL."
 metadata:
   requires:
     bins: ["ae-cli"]
@@ -15,29 +15,23 @@ Use the `dataops_datatable` subcommand to manage data tables.
 
 **Core Rules:**
 - Creating/modifying/deleting workspace task tables **must use dataops_datatable**, prohibited to use dataops_ide
+- Use `+dict_search_tables` as the default discovery command for the visible DataOps catalog
+- Use `dataops_ide +search_tables` only when raw engine-side metadata is needed
+- Use `dataops_ide +ide_list_tables` only after catalog/schema are known and you need schema browsing
 - Confirm no table with the same name exists before creation
 - When creating views or data tables, generate DDL according to **Trino DDL specifications**
-- Create task tables in DEV environment first, then PROD environment (can create simultaneously)
+- `+create_table` and `+create_view` create objects in the DEV environment only. Publish PROD separately with `+publish_entity`.
 
 ---
 
 ## Workflow A: Search and View Table Information
 
 ```bash
-# Search tables in data dictionary (fuzzy match by table name/description)
+# Default table discovery: search the DataOps table catalog. Prefer precise keywords; avoid broad scans.
 ae-cli dataops_datatable +dict_search_tables --spaceCode "${spaceCode}" --search "user" --maxResults 20
 
-# Get detailed table information (fields, DDL, data lineage)
+# Get compact detail. TASK_ENV without --env returns DEV and PRODUCT.
 ae-cli dataops_datatable +get_table_detail --spaceCode "${spaceCode}" --tableName "dwd_user_info"
-
-# Paginated query of registered tables in workspace
-ae-cli dataops_datatable +list_tables_by_page --spaceCode "${spaceCode}" --search "user" --pageNum 1 --pageSize 20
-
-# Get table hierarchy by repository dimension (repo→catalog→schema→tables)
-ae-cli dataops_datatable +struct_by_repo --spaceCode "${spaceCode}"
-
-# Search system tables (event tables, user tables, dimension tables)
-ae-cli dataops_datatable +list_system_tables --spaceCode "${spaceCode}" --projectName "my_project"
 ```
 
 ---
@@ -48,15 +42,13 @@ ae-cli dataops_datatable +list_system_tables --spaceCode "${spaceCode}" --projec
 # Step 1: Confirm no table with same name exists
 ae-cli dataops_datatable +dict_search_tables --spaceCode "${spaceCode}" --search "dwd_user_info"
 
-# Step 2: Create physical table in DEV environment (DDL must follow Trino specifications)
+# Step 2: Create physical table in DEV (DDL must follow Trino specifications)
 ae-cli dataops_datatable +create_table --spaceCode "${spaceCode}" \
-  --ddl "CREATE TABLE dwd_user_info (user_id VARCHAR, user_name VARCHAR, age INTEGER) WITH (format = 'ORC')" \
-  --repoCode "te_etl" --catalog "hive" --schema "ws_default_dev" --confirmed true
+  --ddl "CREATE TABLE dwd_user_info (user_id VARCHAR, user_name VARCHAR, age INTEGER) WITH (format = 'ORC')"
 
-# Step 3: Create physical table in PROD environment (same DDL, change schema to product database)
-ae-cli dataops_datatable +create_table --spaceCode "${spaceCode}" \
-  --ddl "CREATE TABLE dwd_user_info (user_id VARCHAR, user_name VARCHAR, age INTEGER) WITH (format = 'ORC')" \
-  --repoCode "te_etl" --catalog "hive" --schema "ws_default_product" --confirmed true
+# Step 3: Publish explicitly. name is enough when it resolves to one TASK_ENV entity.
+ae-cli dataops_datatable +publish_entity --spaceCode "${spaceCode}" \
+  --name "dwd_user_info"
 ```
 
 ---
@@ -67,34 +59,13 @@ ae-cli dataops_datatable +create_table --spaceCode "${spaceCode}" \
 # Step 1: Confirm no view with same name exists
 ae-cli dataops_datatable +dict_search_tables --spaceCode "${spaceCode}" --search "v_user_info"
 
-# Step 2: Create view (defaults to v_${source_table_name} if view name not specified)
+# Step 2: Create DataOps view in DEV. This example expands ${spaceCode} but keeps ${env} literal.
 ae-cli dataops_datatable +create_view --spaceCode "${spaceCode}" \
-  --ddl "CREATE VIEW v_user_info AS SELECT user_id, user_name FROM hive.ws_default_dev.dwd_user_info" \
-  --repoCode "te_etl" --catalog "hive" --schema "ws_default_dev" --confirmed true
-```
+  --ddl "CREATE VIEW v_user_info AS SELECT user_id, user_name FROM hive.ws_${spaceCode}_\${env}.dwd_user_info"
 
----
-
-## Workflow D: Batch Create Linked Views
-
-```bash
-# Step 1: List source tables available for creating linked views
-ae-cli dataops_datatable +linkview_list_source_tables --spaceCode "${spaceCode}" \
-  --srcRepoCode "te_etl" --srcCatalog "hive" --srcSchema "ws_default_hidden"
-
-# Step 2: Batch create linked views (async execution)
-ae-cli dataops_datatable +batch_create --spaceCode "${spaceCode}" \
-  --srcRepoCode "te_etl" --srcCatalog "hive" --srcSchema "ws_default_hidden" \
-  --processType 2 --batchCreationMode 1 --owner "${ownerOpenId}" \
-  --viewInfos '[{"viewName":"v_orders","srcTableName":"ta_event_123","srcTableType":"TABLE"}]' \
-  --confirmed true
-
-# Step 3: Poll batch creation status (recommend 3-5 second interval)
-# status: WAIT / RUNNING / SUCCESS / STOP
-ae-cli dataops_datatable +linkview_get_batch_status --spaceCode "${spaceCode}" --batchId "${batchId}"
-
-# View historical batch creation records
-ae-cli dataops_datatable +list_batches --spaceCode "${spaceCode}"
+# Step 3: Publish explicitly. name is enough when it resolves to one TASK_ENV entity.
+ae-cli dataops_datatable +publish_entity --spaceCode "${spaceCode}" \
+  --name "v_user_info"
 ```
 
 ---
@@ -103,20 +74,18 @@ ae-cli dataops_datatable +list_batches --spaceCode "${spaceCode}"
 
 | Command | Purpose | Key Flags |
 |---------|---------|-----------|
-| `+dict_search_tables` | Data dictionary search | `--spaceCode` `--search` `--maxResults` |
-| `+get_table_detail` | Table details | `--spaceCode` `--tableName` |
-| `+list_tables_by_page` | Paginated list | `--spaceCode` `--search` `--pageNum` `--pageSize` |
-| `+struct_by_repo` | By repository hierarchy | `--spaceCode` `--search` |
-| `+list_system_tables` | System table list | `--spaceCode` `--projectId`/`--projectName` `--pageNum` `--pageSize` |
-| `+create_table` | Create physical table | `--spaceCode` `--ddl` `--repoCode` `--catalog` `--schema` `--confirmed` |
-| `+create_view` | Create view | `--spaceCode` `--ddl` `--repoCode` `--catalog` `--schema` `--confirmed` |
-| `+batch_create` | Batch create views | `--spaceCode` `--srcRepoCode` `--srcCatalog` `--srcSchema` `--processType` `--batchCreationMode` `--owner` `--viewInfos` `--confirmed` |
-| `+linkview_get_batch_status` | Batch creation status | `--spaceCode` `--batchId` |
-| `+list_batches` | Historical batch records | `--spaceCode` `--status` |
+| `+dict_search_tables` | DataOps table catalog search, default 50 results | `--spaceCode` `--search` `--maxResults` |
+| `+get_table_detail` | DataOps catalog detail | `--spaceCode` `--tableName` `--manageMode` `--env` `--entityType TABLE|VIEW` |
+| `+create_table` | Create DataOps physical table in DEV | `--spaceCode` `--ddl` |
+| `+create_view` | Create DataOps view in DEV | `--spaceCode` `--ddl` |
+| `+publish_entity` | Publish table/view from DEV to PROD | `--spaceCode` `--name` `[--entityId]` `[--entityType]` |
 
 ## Parameter Notes
 
-- **schema naming**: DEV environment uses `ws_${spaceCode}_dev`, PROD environment uses `ws_${spaceCode}_product`
-- **processType**: `1`=DEV only | `2`=DEV+PROD
-- **batchCreationMode**: `1`=full sync | `2`=retry failures
+- **dict_search_tables**: Searches task, IDE, system, and authorized-space tables visible to `spaceCode`. Prefer precise `--search`; default `maxResults` is 50 and max is 200. Returns `tables`, `totalCount`, `returnedCount`, `hasMore`, and `hint` when truncated. `comment` and `remark` are returned only when present.
+- **get_table_detail**: Prefer exact `--tableName`. Add `--manageMode` or `--entityType TABLE|VIEW` when ambiguous. TASK_ENV without `--env` returns `environments.DEV/PRODUCT`; otherwise returns `detail`. Empty optional fields are omitted.
+- **create_table**: Requires `--spaceCode` and `--ddl`; no optional flags. Creates a DataOps physical table in DEV only. The backend parses Trino-compatible DDL and saves TASK_ENV metadata in the default workspace warehouse (`repo=te_etl`, `catalog=hive`). Publish by name with `+publish_entity --name <tableName>`.
+- **create_view**: Requires `--spaceCode` and `--ddl`; no optional command flags. Creates a DataOps view in DEV only. The backend saves TASK_ENV metadata through the DataView save flow in the default workspace warehouse (`repo=te_etl`, `catalog=hive`). Publish by name with `+publish_entity --name <viewName>`. Keep the literal `${env}` placeholder when referencing current-space task tables, for example `ws_${spaceCode}_${env}`.
+- **publish_entity**: Requires `--spaceCode` and `--name`. Publishes one existing TASK_ENV table/view from DEV to PROD. Optional `--entityId` disambiguates same-name matches; optional `--entityType TABLE|VIEW` validates the resolved type. Returns `action/result/status`; result includes published ids/names and `ONLINE` status, or `errorType`/`candidates`/`errors`.
+- **schema naming**: DEV environment uses `ws_${spaceCode}_dev`, PROD environment uses `ws_${spaceCode}_product`. Do not hardcode either schema in current-space view DDL; use `ws_${spaceCode}_${env}` with literal `${env}`.
 - **Table name rule**: `^[a-z][0-9a-z_]{0,127}$`