@thinkingai/ae-cli 6.0.18-1 → 6.0.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/README.md +7 -6
  2. package/README.zh.md +8 -7
  3. package/dist/index.js +11 -6
  4. package/dist/{te-agent-KZRIEN4N.js → te-agent-CYZUC4US.js} +543 -19
  5. package/dist/{te-analysis-VAR4KUOK.js → te-analysis-HJ22TUAA.js} +318 -40
  6. package/dist/te-audience-MYDE5WEH.js +590 -0
  7. package/dist/te-dataops-ZTS7XMXM.js +1842 -0
  8. package/dist/{te-kb-B62KFP2A.js → te-kb-OWM3RQRH.js} +46 -21
  9. package/dist/{te-meta-EPZGQN56.js → te-meta-JVO4QJBH.js} +28 -11
  10. package/package.json +1 -1
  11. package/skills/ae-agent/SKILL.md +103 -4
  12. package/skills/ae-analysis/SKILL.md +66 -19
  13. package/skills/ae-analysis/references/build_attribution_analysis_qp.md +60 -0
  14. package/skills/ae-analysis/references/build_cluster_definition.md +60 -0
  15. package/skills/ae-analysis/references/build_distribution_analysis_qp.md +65 -0
  16. package/skills/ae-analysis/references/build_heat_map_analysis_qp.md +52 -0
  17. package/skills/ae-analysis/references/build_interval_analysis_qp.md +58 -0
  18. package/skills/ae-analysis/references/build_path_analysis_qp.md +51 -0
  19. package/skills/ae-analysis/references/build_rank_list_analysis_qp.md +61 -0
  20. package/skills/ae-analysis/references/build_tag_definition.md +63 -0
  21. package/skills/ae-analysis/references/cancel_query.md +2 -3
  22. package/skills/ae-analysis/references/copy_dashboard.md +33 -0
  23. package/skills/ae-analysis/references/create_id_cluster.md +40 -0
  24. package/skills/ae-analysis/references/create_id_tag.md +39 -0
  25. package/skills/ae-analysis/references/delete_alert.md +27 -0
  26. package/skills/ae-analysis/references/delete_cluster.md +27 -0
  27. package/skills/ae-analysis/references/delete_dashboard.md +30 -0
  28. package/skills/ae-analysis/references/delete_metric.md +27 -0
  29. package/skills/ae-analysis/references/delete_report.md +27 -0
  30. package/skills/ae-analysis/references/delete_tag.md +27 -0
  31. package/skills/ae-analysis/references/drilldown_user_events.md +4 -4
  32. package/skills/ae-analysis/references/drilldown_users.md +3 -3
  33. package/skills/ae-analysis/references/freeze_dashboards.md +29 -0
  34. package/skills/ae-analysis/references/get_table_columns.md +11 -12
  35. package/skills/ae-analysis/references/list_cluster_members.md +4 -4
  36. package/skills/ae-analysis/references/list_clusters.md +3 -3
  37. package/skills/ae-analysis/references/list_dashboards.md +3 -3
  38. package/skills/ae-analysis/references/list_entities.md +1 -1
  39. package/skills/ae-analysis/references/list_events.md +3 -3
  40. package/skills/ae-analysis/references/list_metrics.md +2 -2
  41. package/skills/ae-analysis/references/list_properties.md +3 -3
  42. package/skills/ae-analysis/references/list_reports.md +3 -3
  43. package/skills/ae-analysis/references/list_spaces.md +27 -0
  44. package/skills/ae-analysis/references/list_tag_members.md +4 -4
  45. package/skills/ae-analysis/references/list_tags.md +2 -2
  46. package/skills/ae-analysis/references/move_dashboard.md +33 -0
  47. package/skills/ae-analysis/references/query_adhoc.md +2 -2
  48. package/skills/ae-analysis/references/query_bi_panel_data.md +2 -2
  49. package/skills/ae-analysis/references/query_dashboard_report_data.md +4 -4
  50. package/skills/ae-analysis/references/query_entity_details.md +4 -4
  51. package/skills/ae-analysis/references/query_event_details.md +3 -3
  52. package/skills/ae-analysis/references/query_report_data.md +37 -3
  53. package/skills/ae-analysis/references/update_id_cluster.md +36 -0
  54. package/skills/ae-analysis/references/update_id_tag.md +37 -0
  55. package/skills/ae-analysis/references/update_report.md +35 -0
  56. package/skills/ae-analysis-intent/SKILL.md +1 -1
  57. package/skills/ae-dataops/SKILL.md +54 -522
  58. package/skills/ae-dataops/references/dataops-flow-create.md +78 -75
  59. package/skills/ae-dataops/references/dataops-flow-monitor.md +58 -51
  60. package/skills/ae-dataops/references/dataops-integration.md +80 -41
  61. package/skills/ae-dataops/references/dataops-query.md +54 -53
  62. package/skills/ae-dataops/references/dataops-table.md +28 -59
  63. package/dist/te-audience-KMKYI2YJ.js +0 -400
  64. package/dist/te-dataops-GD3XBYCS.js +0 -1830
@@ -13,7 +13,7 @@ metadata:
13
13
 
14
14
  Use the `dataops_flow` subcommand to manage flow lifecycle.
15
15
 
16
- **Flow Lifecycle: Create → Configure Nodes → Configure Schedule → Test in DEV → Release to PROD → Online Schedule**
16
+ **Flow Lifecycle: Create → Configure Nodes → Configure Schedule → Preview Release → Release to PROD → Manual PROD Execution / Online Schedule**
17
17
 
18
18
  ---
19
19
 
@@ -25,68 +25,58 @@ Follow these steps in order to create a production-ready flow from scratch.
25
25
 
26
26
  ```bash
27
27
  ae-cli dataops_flow +create_flow --spaceCode "${spaceCode}" \
28
- --flowName "Daily ETL Process" --remark "Process user data" --confirmed true
28
+ --flowName "Daily ETL Process" --remark "Process user data"
29
29
  # Returns flowCode, required for subsequent steps
30
30
  ```
31
31
 
32
32
  ### Step 2: Create Task Nodes (can be called multiple times)
33
33
 
34
34
  ```bash
35
- # Create SQL task
36
- ae-cli dataops_flow +create_task --spaceCode "${spaceCode}" \
37
- --flowCode "${flowCode}" --taskName "Process User Data" --taskType "TRINO_SQL" --confirmed true
38
- # Returns taskCode
35
+ # Create SQL task and save SQL content
36
+ ae-cli dataops_flow +create_sql_task --spaceCode "${spaceCode}" \
37
+ --flowCode ${flowCode} --taskName "Process User Data" \
38
+ --sql "SELECT * FROM dwd_user"
39
+ # Returns result.taskCode
39
40
 
40
41
  # Create task with upstream dependency
41
- ae-cli dataops_flow +create_task --spaceCode "${spaceCode}" \
42
- --flowCode "${flowCode}" --taskName "Export Results" --taskType "TRINO_SQL" \
43
- --preTaskCode "${upstreamTaskCode}" --confirmed true
42
+ ae-cli dataops_flow +create_sql_task --spaceCode "${spaceCode}" \
43
+ --flowCode ${flowCode} --taskName "Export Results" \
44
+ --preTaskCode ${upstreamTaskCode} \
45
+ --sql "INSERT INTO ads_user SELECT * FROM dwd_user"
44
46
  ```
45
47
 
46
- **Task Node Types:**
48
+ `+create_sql_task` creates DEV Trino SQL task nodes and saves the SQL content.
47
49
 
48
- | Type | Description | Typical Use |
49
- |------|-------------|-------------|
50
- | TRINO_SQL | Trino SQL query | Data processing, ETL |
51
- | SHELL | Shell script | System commands, file operations |
52
- | FLOW_CHECK | Flow check | Check if upstream flow is complete |
53
- | TASK_CHECK | Task check | Check if specified task is complete |
54
- | OFFLINE_SYNC | Offline sync | Integration solution data sync (requires syncId) |
55
- | APP_SYNC | App sync | App table data sync (requires syncId) |
56
- | PLACE_HOLDER | Placeholder | Flow control, logical grouping |
57
-
58
- ### Step 3: Save Task Definition (configure SQL/Shell content)
50
+ ### Step 3: Create or Modify Integration Sync Task
59
51
 
60
52
  ```bash
61
- # SQL task
62
- ae-cli dataops_flow +save_task_definition --spaceCode "${spaceCode}" \
63
- --flowCode "${flowCode}" --taskCode "${taskCode}" \
64
- --taskContentJson '{"sql":"SELECT * FROM dwd_user","repoCode":"te_etl","catalog":"hive","schema":"ws_default_dev"}' \
65
- --failRetryTimes 3 --failRetryInterval 30 --timeout 60 --confirmed true
66
-
67
- # Shell task
68
- ae-cli dataops_flow +save_task_definition --spaceCode "${spaceCode}" \
69
- --flowCode "${flowCode}" --taskCode "${taskCode}" \
70
- --taskContentJson '{"script":"#!/bin/bash\necho hello"}' \
71
- --confirmed true
53
+ # Create integration sync task from an existing DataOps sync solution
54
+ ae-cli dataops_flow +create_integration_task --spaceCode "${spaceCode}" \
55
+ --flowCode ${flowCode} --taskName "Sync Employees" \
56
+ --syncId "${syncId}"
57
+
58
+ # Rebind an existing integration sync task to another sync solution
59
+ ae-cli dataops_flow +update_integration_task --spaceCode "${spaceCode}" \
60
+ --flowCode ${flowCode} --taskCode ${taskCode} \
61
+ --syncId "${syncId}"
72
62
  ```
73
63
 
74
- ### Step 4: Validate SQL (recommended, TRINO_SQL type only)
64
+ Use `dataops_integration +list_sync_solutions` or `+get_sync_detail` to find the `syncId` first. These commands expose workflow `OFFLINE_SYNC` tasks only; app sync tasks are not part of this flow.
65
+
66
+ ### Step 4: Modify SQL Task Content
75
67
 
76
68
  ```bash
77
- ae-cli dataops_flow +validate_task_sql --spaceCode "${spaceCode}" \
78
- --flowCode "${flowCode}" --taskCode "${taskCode}" \
79
- --sql "SELECT * FROM dwd_user" --repoCode "te_etl" \
80
- --catalog "hive" --schema "ws_default_dev"
81
- # Check valid=true, otherwise review message and errorPosition
69
+ # Update SQL task content without changing owner, remark, dependencies, retry, or timeout
70
+ ae-cli dataops_flow +update_sql_task --spaceCode "${spaceCode}" \
71
+ --flowCode ${flowCode} --taskCode ${taskCode} \
72
+ --sql "SELECT * FROM dwd_user"
82
73
  ```
83
74
 
84
75
  ### Step 5: Add Task Dependencies (DAG connections)
85
76
 
86
77
  ```bash
87
78
  ae-cli dataops_flow +add_task_relation --spaceCode "${spaceCode}" \
88
- --flowCode "${flowCode}" --preTaskCode "${upstreamTaskCode}" --taskCode "${downstreamTaskCode}" \
89
- --confirmed true
79
+ --flowCode ${flowCode} --preTaskCode ${upstreamTaskCode} --taskCode ${downstreamTaskCode}
90
80
  ```
91
81
 
92
82
  ### Step 6: Configure Schedule
@@ -94,8 +84,8 @@ ae-cli dataops_flow +add_task_relation --spaceCode "${spaceCode}" \
94
84
  ```bash
95
85
  # CRON expression for scheduled execution
96
86
  ae-cli dataops_flow +save_schedule_config --spaceCode "${spaceCode}" \
97
- --flowCode "${flowCode}" --scheduled 1 --scheduleType "CRON" \
98
- --cron "0 0 2 * * ?" --confirmed true
87
+ --flowCode ${flowCode} --enabled true \
88
+ --cron "0 0 2 * * ?"
99
89
  # Example: 0 0 2 * * ? = Daily at 2 AM
100
90
  ```
101
91
 
@@ -103,28 +93,34 @@ ae-cli dataops_flow +save_schedule_config --spaceCode "${spaceCode}" \
103
93
  - `0 0 */4 * * ?` — Every 4 hours
104
94
  - `0 30 8 * * 1-5` — Weekdays at 8:30
105
95
 
106
- ### Step 7: Test in DEV Environment
96
+ ### Step 7: Preview Release
107
97
 
108
98
  ```bash
109
- # Manually trigger execution
110
- ae-cli dataops_flow +execute_flow --spaceCode "${spaceCode}" \
111
- --flowCode "${flowCode}" --env "DEV" --confirmed true
112
-
113
- # Check execution status
114
- ae-cli dataops_flow +get_execute_dag --spaceCode "${spaceCode}" --executeId "${executeId}"
115
-
116
- # View task logs (get bsTaskInstanceId and taskCode from DAG)
117
- ae-cli dataops_flow +get_task_instance_log --bsTaskInstanceId "${instanceId}" --taskCode "${taskCode}"
118
-
119
- # Adjust configuration based on logs, then repeat testing
99
+ # Preview pending DEV-to-PROD release changes before publishing
100
+ ae-cli dataops_flow +preview_release_flow --spaceCode "${spaceCode}" \
101
+ --flowCode ${flowCode}
120
102
  ```
121
103
 
122
- ### Step 8: Release to Production
104
+ ### Step 8: Release to Production and Verify
123
105
 
124
106
  ```bash
125
107
  ae-cli dataops_flow +release_flow --spaceCode "${spaceCode}" \
126
- --flowCode "${flowCode}" --confirmed true
127
- # After release, PROD schedule takes effect immediately without interrupting running instances
108
+ --flowCode ${flowCode}
109
+
110
+ # Manually trigger PROD execution when verification is needed
111
+ ae-cli dataops_flow +execute_flow --spaceCode "${spaceCode}" \
112
+ --flowCode ${flowCode}
113
+
114
+ # Check execution status
115
+ ae-cli dataops_operations +search_flow_instances --spaceCode "${spaceCode}" \
116
+ --keyword "${flowCode}" --pageSize 20
117
+
118
+ # Inspect one instance and task logs when troubleshooting
119
+ ae-cli dataops_operations +get_flow_instance_detail --spaceCode "${spaceCode}" \
120
+ --flowCode ${flowCode} --flowInstanceId ${flowInstanceId}
121
+ ae-cli dataops_operations +get_task_instance_detail --spaceCode "${spaceCode}" \
122
+ --flowCode ${flowCode} --flowInstanceId ${flowInstanceId} \
123
+ --taskName "${taskName}" --includeLog true
128
124
  ```
129
125
 
130
126
  ---
@@ -133,31 +129,38 @@ ae-cli dataops_flow +release_flow --spaceCode "${spaceCode}" \
133
129
 
134
130
  | Command | Purpose | Key Flags |
135
131
  |---------|---------|-----------|
136
- | `+create_flow` | Create flow | `--spaceCode` `--flowName` `--remark` `--confirmed` |
137
- | `+create_task` | Create task node | `--spaceCode` `--flowCode` `--taskName` `--taskType` `--syncId` `--preTaskCode` `--confirmed` |
138
- | `+save_task_definition` | Save task definition | `--spaceCode` `--flowCode` `--taskCode` `--taskContentJson` `--failRetryTimes` `--failRetryInterval` `--timeout` `--confirmed` |
139
- | `+validate_task_sql` | Validate SQL | `--spaceCode` `--flowCode` `--taskCode` `--sql` `--repoCode` `--catalog` `--schema` |
140
- | `+add_task_relation` | Add dependency | `--spaceCode` `--flowCode` `--preTaskCode` `--taskCode` `--confirmed` |
141
- | `+save_schedule_config` | Configure schedule | `--spaceCode` `--flowCode` `--scheduled` `--scheduleType` `--cron` `--confirmed` |
142
- | `+get_flow_dag` | View DAG structure | `--spaceCode` `--flowCode` `--env` |
143
- | `+get_task_params` | View task parameters | `--spaceCode` `--flowCode` `--taskCode` `--env` |
144
- | `+execute_flow` | Manual execution | `--spaceCode` `--flowCode` `--env` `--confirmed` |
145
- | `+get_execute_dag` | View runtime DAG | `--spaceCode` `--executeId` |
146
- | `+get_task_instance_log` | View task logs | `--bsTaskInstanceId` `--taskCode` `--startOffset` `--limit` |
147
- | `+release_flow` | Release to production | `--spaceCode` `--flowCode` `--confirmed` |
132
+ | `+create_flow` | Create DEV flow | `--spaceCode` `--flowName` `[--remark]` |
133
+ | `+create_sql_task` | Create DEV Trino SQL task and save SQL | `--spaceCode` `--flowCode` `--taskName` `--sql` `[--preSql]` `[--postSql]` `[--preTaskCode]` `[--remark]` |
134
+ | `+update_sql_task` | Update DEV Trino SQL task content | `--spaceCode` `--flowCode` `--taskCode` `--sql` `[--preSql]` `[--postSql]` |
135
+ | `+create_integration_task` | Create DEV integration sync task from an existing sync solution | `--spaceCode` `--flowCode` `--taskName` `--syncId` `[--preTaskCode]` `[--remark]` |
136
+ | `+update_integration_task` | Rebind DEV integration sync task to a sync solution | `--spaceCode` `--flowCode` `--taskCode` `--syncId` |
137
+ | `+add_task_relation` | Add DEV dependency | `--spaceCode` `--flowCode` `--preTaskCode` `--taskCode` |
138
+ | `+save_schedule_config` | Save DEV schedule config | `--spaceCode` `--flowCode` `--enabled` `[--cron]` |
139
+ | `+get_task_params` | View DEV task parameter list | `--spaceCode` `--flowCode` `--taskCode` |
140
+ | `+execute_flow` | Manual PROD execution | `--spaceCode` `--flowCode` `[--baseDate]` |
141
+ | `dataops_operations +get_flow_instance_detail` | View operations instance DAG and task statuses | `--spaceCode` `--flowCode` `--flowInstanceId` |
142
+ | `dataops_operations +get_task_instance_detail` | View operations task detail and optional logs | `--spaceCode` `--flowCode` `--flowInstanceId` (`--taskCode` or `--taskName`) `[--includeLog]` |
143
+ | `+preview_release_flow` | Preview pending DEV-to-PROD release changes without publishing | `--spaceCode` `--flowCode` |
144
+ | `+release_flow` | Submit DEV-to-PROD release | `--spaceCode` `--flowCode` |
148
145
 
149
146
  ## Parameter Notes
150
147
 
151
148
  - **Parameter Reference**: Reference workspace parameters in tasks using `${paramKey}` (e.g., `${ws_run_date}`)
152
- - **Environment**: `DEV` (default) | `PROD`
153
- - **taskContentJson**: JSON string, format varies by taskType:
154
- - TRINO_SQL: `{"sql":"SELECT ...","repoCode":"xxx","catalog":"hive","schema":"db"}`
155
- - SHELL: `{"script":"#!/bin/bash\n..."}`
149
+ - **Execution**: `+execute_flow` requires `--spaceCode` and `--flowCode`; `--baseDate` is optional and maps to runtime parameter `bd`. It always runs PROD and returns `action/result/status`; `result` includes `flowCode`, `executeId`, `operationStatus`, `nextAction`, and optional `flowInstanceId`.
150
+ - **Schedule config**: `+save_schedule_config` requires `--spaceCode`, `--flowCode`, and `--enabled`. `--cron` is required only when `--enabled true`; omit it when disabling scheduling. It returns `action/result/status`; `result` includes `enabled`, `flow`, `message`, and `cron` only when enabled.
151
+ - **Release preview**: `+preview_release_flow` requires `--spaceCode` and `--flowCode`; it has no optional flags. It returns `flowCode`, `releaseStatus`, `message`, and `changes`. Each change may include `scheduleConfigChange` and `tasks`; task entries may include `changed`, `contentCompare`, and `targetTable`.
152
+ - **Release**: `+release_flow` requires `--spaceCode` and `--flowCode`; it has no optional flags. It returns `action/result/status`; `result` includes `flowCode`, `releaseStatus`, `message`, optional `packageCode`, and optional `changes`. Each change may include `scheduleConfigChange` and `tasks`; task entries include `changed`.
153
+ - **SQL task creation**: `+create_sql_task` requires `--spaceCode`, `--flowCode`, `--taskName`, and `--sql`; `--preSql`, `--postSql`, `--preTaskCode`, and `--remark` are optional. It returns `action/result/status`; `result` includes `flowCode`, `taskCode`, `taskName`, `taskType=TRINO_SQL`, and `sqlSaved=true`.
154
+ - **SQL task update**: `+update_sql_task` requires `--spaceCode`, `--flowCode`, `--taskCode`, and `--sql`; `--preSql` and `--postSql` are optional and keep existing values when omitted. It returns `action/result/status`; `result` includes `sqlSaved`, `flowCode`, `taskCode`, `taskType=TRINO_SQL`, and `task`.
155
+ - **Task dependency**: `+add_task_relation` requires `--spaceCode`, `--flowCode`, `--preTaskCode`, and `--taskCode`. `preTaskCode` is upstream and `taskCode` is downstream. It returns `action/result/status`; `result` includes `status`, `flowCode`, `preTaskCode`, `taskCode`, and `message`.
156
+ - **Task parameters**: `+get_task_params` requires `--spaceCode`, `--flowCode`, and `--taskCode`; it has no optional flags. It queries DEV and returns `data` as an array. Items include fields such as `paramKey`, `paramType`, `paramDataType`, `paramFrom`, and built-in flags like `isBd`.
157
+ - **Integration task creation**: `+create_integration_task` requires `--spaceCode`, `--flowCode`, `--taskName`, and `--syncId`; `--preTaskCode` and `--remark` are optional. It returns `action/result/status`; `result` includes `syncTaskSaved`, `flowCode`, `taskCode`, `taskName`, `taskType=OFFLINE_SYNC`, `syncId`, and `nextAction`.
158
+ - **SQL task dependencies**: Use optional `--preTaskCode` to create one upstream dependency with the task. Use `+add_task_relation` for additional DAG dependencies.
156
159
 
157
160
  ## Important Notes
158
161
 
159
162
  1. **Getting flowCode**: Use `+list_flows` (see `dataops-flow-monitor` Skill)
160
- 2. **Getting taskCode**: Returned by `+create_task` or query via `+get_flow_dag`
161
- 3. **Release Impact**: Syncs all DEV changes to PROD, PROD schedule takes effect immediately
163
+ 2. **Getting taskCode**: Returned by `+create_sql_task` / `+create_integration_task` or query via `+get_flow_overview`
164
+ 3. **Release Impact**: Submits current DEV changes to PROD; released schedule/config applies to future PROD runs
162
165
  4. **SQL Validation**: Recommended to validate before saving TRINO_SQL tasks
163
166
  5. **Cannot create circular dependencies**
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: dataops-flow-monitor
3
3
  version: 1.0.0
4
- description: "Flow execution and monitoring: view execution instances, runtime DAG, task logs, manually execute/stop flows. Trigger keywords: execute flow, running instance, monitor, logs, stop, DAG, troubleshoot, instance, flow instance."
4
+ description: "Flow execution and monitoring: view execution instances, inspect instance DAGs and task logs, manually execute/stop flows. Trigger keywords: execute flow, running instance, monitor, logs, stop, troubleshoot, instance, flow instance."
5
5
  metadata:
6
6
  requires:
7
7
  bins: ["ae-cli"]
@@ -11,11 +11,11 @@ metadata:
11
11
 
12
12
  > **Prerequisites:** Read [`ae-dataops/SKILL.md`](../SKILL.md) for general rules.
13
13
 
14
- Use the `dataops_flow` subcommand to execute and monitor flows.
14
+ Use `dataops_flow` to execute and control flows. Use `dataops_operations` for space-level operations instance search, instance DAG inspection, and task logs.
15
15
 
16
16
  **Core Concepts:**
17
- - **executeId** — Execution record ID used by flow_* tools (obtained via `+list_flow_instances`)
18
- - **flowInstanceId** — Operations instance ID used by operations_* tools (these two are not interchangeable)
17
+ - **executeId** — Execution ID returned immediately by `dataops_flow +execute_flow`; useful for stopping before the scheduler instance is available
18
+ - **flowInstanceId** — Operations workflow instance ID returned by `dataops_operations +search_flow_instances`; use it for operations detail, task logs, and stop
19
19
  - **Instance list queries PROD environment by default** (unlike other tools which default to DEV)
20
20
 
21
21
  ---
@@ -23,18 +23,24 @@ Use the `dataops_flow` subcommand to execute and monitor flows.
23
23
  ## Workflow A: Find and View Flow Execution Status
24
24
 
25
25
  ```bash
26
- # Step 1: Search flows (get flowCode)
27
- ae-cli dataops_flow +list_flows --spaceCode "${spaceCode}" --keyword "etl"
26
+ # Step 1: List/search current-space flows and get flowCode
27
+ ae-cli dataops_flow +list_flows --spaceCode "${spaceCode}" --keyword "etl" --pageSize 20
28
28
 
29
- # Step 2: View execution instance list (default PROD, sorted by time descending)
30
- ae-cli dataops_flow +list_flow_instances --spaceCode "${spaceCode}" \
31
- --flowCode "${flowCode}" --env "PROD" --size 30
29
+ # Optional: find frequently released flows in the last 30 days
30
+ ae-cli dataops_flow +list_high_frequency_release_flows --spaceCode "${spaceCode}" \
31
+ --days 30 --topN 10 --status SUCCESS
32
32
 
33
- # Step 3: View runtime DAG (with execution status for each node)
34
- ae-cli dataops_flow +get_execute_dag --spaceCode "${spaceCode}" --executeId "${executeId}"
33
+ # Optional: get one-environment overview by exact name or flowCode
34
+ ae-cli dataops_flow +get_flow_overview --spaceCode "${spaceCode}" \
35
+ --flowName "etl" --env "PROD"
35
36
 
36
- # Step 4: View execution record details
37
- ae-cli dataops_flow +get_execute_record --spaceCode "${spaceCode}" --executeId "${executeId}"
37
+ # Step 2: Search operation instances across the whole space, with statistics
38
+ ae-cli dataops_operations +search_flow_instances --spaceCode "${spaceCode}" \
39
+ --keyword "${flowKeyword}" --startDate "${startDate}" --endDate "${endDate}" --pageSize 20
40
+
41
+ # Step 3: Inspect one instance DAG and task statuses
42
+ ae-cli dataops_operations +get_flow_instance_detail --spaceCode "${spaceCode}" \
43
+ --flowCode ${flowCode} --flowInstanceId ${flowInstanceId}
38
44
  ```
39
45
 
40
46
  ---
@@ -42,21 +48,18 @@ ae-cli dataops_flow +get_execute_record --spaceCode "${spaceCode}" --executeId "
42
48
  ## Workflow B: Troubleshoot Task Failures
43
49
 
44
50
  ```bash
45
- # Step 1: Get runtime DAG, locate failed node
46
- ae-cli dataops_flow +get_execute_dag --spaceCode "${spaceCode}" --executeId "${executeId}"
47
- # Find node with status=FAILED, get its bsTaskInstanceId and taskCode
48
-
49
- # Step 2: View failed task logs
50
- ae-cli dataops_flow +get_task_instance_log --bsTaskInstanceId "${bsTaskInstanceId}" \
51
- --taskCode "${taskCode}" --limit 100
51
+ # Step 1: Inspect one workflow instance and locate the failed task
52
+ ae-cli dataops_operations +get_flow_instance_detail --spaceCode "${spaceCode}" \
53
+ --flowCode ${flowCode} --flowInstanceId ${flowInstanceId}
52
54
 
53
- # Step 3: View task definition (confirm SQL/configuration)
54
- ae-cli dataops_flow +get_task_params --spaceCode "${spaceCode}" \
55
- --flowCode "${flowCode}" --taskCode "${taskCode}"
55
+ # Step 2: View task detail and logs
56
+ ae-cli dataops_operations +get_task_instance_detail --spaceCode "${spaceCode}" \
57
+ --flowCode ${flowCode} --flowInstanceId ${flowInstanceId} \
58
+ --taskName "${taskName}" --includeLog true
56
59
 
57
- # Step 4: Re-execute after fixing
60
+ # Step 3: Re-execute after fixing
58
61
  ae-cli dataops_flow +execute_flow --spaceCode "${spaceCode}" \
59
- --flowCode "${flowCode}" --env "DEV" --confirmed true
62
+ --flowCode ${flowCode}
60
63
  ```
61
64
 
62
65
  ---
@@ -64,17 +67,18 @@ ae-cli dataops_flow +execute_flow --spaceCode "${spaceCode}" \
64
67
  ## Workflow C: Manual Execution and Stop
65
68
 
66
69
  ```bash
67
- # Manually trigger execution (default DEV, can specify PROD)
70
+ # Manually trigger PROD execution
68
71
  ae-cli dataops_flow +execute_flow --spaceCode "${spaceCode}" \
69
- --flowCode "${flowCode}" --env "DEV" --confirmed true
72
+ --flowCode ${flowCode}
70
73
  # Returns executeId
71
74
 
72
- # Monitor execution progress
73
- ae-cli dataops_flow +get_execute_dag --spaceCode "${spaceCode}" --executeId "${executeId}"
75
+ # Monitor recent executions
76
+ ae-cli dataops_operations +search_flow_instances --spaceCode "${spaceCode}" \
77
+ --keyword "${flowKeyword}" --startDate "${startDate}" --endDate "${endDate}" --pageSize 20
74
78
 
75
79
  # Stop execution if necessary (irreversible)
76
- ae-cli dataops_flow +stop_flow --spaceCode "${spaceCode}" \
77
- --flowCode "${flowCode}" --executeId "${executeId}" --confirmed true
80
+ ae-cli dataops_operations +stop_flow_instance --spaceCode "${spaceCode}" \
81
+ --flowCode ${flowCode} --executeId ${executeId}
78
82
  ```
79
83
 
80
84
  ---
@@ -82,14 +86,11 @@ ae-cli dataops_flow +stop_flow --spaceCode "${spaceCode}" \
82
86
  ## Workflow D: View Flow Structure
83
87
 
84
88
  ```bash
85
- # View flow definition DAG (static structure, dependencies)
86
- ae-cli dataops_flow +get_flow_dag --spaceCode "${spaceCode}" --flowCode "${flowCode}"
87
-
88
- # View flow basic information
89
- ae-cli dataops_flow +get_flow_detail --spaceCode "${spaceCode}" --flowCode "${flowCode}"
89
+ # View agent-friendly flow overview for one environment
90
+ ae-cli dataops_flow +get_flow_overview --spaceCode "${spaceCode}" \
91
+ --flowName "etl" --env "DEV"
90
92
 
91
- # View schedule configuration
92
- ae-cli dataops_flow +get_schedule_config --spaceCode "${spaceCode}" --flowCode "${flowCode}"
93
+ # The current schedule is included in +get_flow_overview output as "schedule".
93
94
  ```
94
95
 
95
96
  ---
@@ -98,21 +99,27 @@ ae-cli dataops_flow +get_schedule_config --spaceCode "${spaceCode}" --flowCode "
98
99
 
99
100
  | Command | Purpose | Key Flags |
100
101
  |---------|---------|-----------|
101
- | `+list_flows` | Search flows | `--spaceCode` `--keyword` |
102
- | `+get_flow_detail` | Flow information | `--spaceCode` `--flowCode` `--env` |
103
- | `+get_flow_dag` | Definition DAG | `--spaceCode` `--flowCode` `--env` |
104
- | `+get_schedule_config` | Schedule configuration | `--spaceCode` `--flowCode` `--env` |
105
- | `+execute_flow` | Manual execution | `--spaceCode` `--flowCode` `--env` `--confirmed` |
106
- | `+stop_flow` | Stop execution | `--spaceCode` `--flowCode` `--executeId` `--confirmed` |
107
- | `+list_flow_instances` | Instance list | `--spaceCode` `--flowCode` `--env` `--size` |
108
- | `+get_execute_record` | Execution record | `--spaceCode` `--executeId` |
109
- | `+get_execute_dag` | Runtime DAG | `--spaceCode` `--executeId` |
110
- | `+get_task_instance_log` | Task logs | `--bsTaskInstanceId` `--taskCode` `--startOffset` `--limit` |
111
- | `+get_task_params` | Task parameters | `--spaceCode` `--flowCode` `--taskCode` `--env` |
112
- | `+update_flow` | Update flow | `--spaceCode` `--flowCode` `--flowName` `--remark` `--confirmed` |
102
+ | `+list_flows` | List/search current-space flows | `--spaceCode` `[--keyword]` `[--pageNum]` `[--pageSize]` |
103
+ | `+list_high_frequency_release_flows` | List flows ranked by release count, not execution count | `--spaceCode` `[--days]` `[--topN]` `[--minCount]` `[--status]` |
104
+ | `+get_flow_overview` | DEV/PROD flow overview | `--spaceCode` (`--flowCode` or `--flowName`) `[--env]` |
105
+ | `+execute_flow` | Manual PROD execution | `--spaceCode` `--flowCode` `[--baseDate]` |
106
+ | `dataops_operations +search_flow_instances` | Search operations workflow instances | `--spaceCode` `[--keyword]` `[--startDate]` `[--endDate]` `[--status]` `[--pageNum]` `[--pageSize]` |
107
+ | `dataops_operations +get_flow_instance_detail` | Operations instance DAG and task statuses | `--spaceCode` `--flowCode` `--flowInstanceId` |
108
+ | `dataops_operations +get_task_instance_detail` | Operations task detail and optional logs | `--spaceCode` `--flowCode` `--flowInstanceId` (`--taskCode` or `--taskName`) `[--includeLog]` |
109
+ | `dataops_operations +stop_flow_instance` | Stop a running execution | `--spaceCode` `--flowCode` exactly one of `--executeId` or `--flowInstanceId` |
110
+ | `+get_task_params` | DEV task parameter list | `--spaceCode` `--flowCode` `--taskCode` |
111
+ | `+update_flow` | Update DEV workflow name and/or remark | `--spaceCode` `--flowCode` `[--flowName]` `[--remark]` |
113
112
 
114
113
  ## Parameter Notes
115
114
 
115
+ - **Execution**: `+execute_flow` requires `--spaceCode` and `--flowCode`; `--baseDate` is optional and maps to runtime parameter `bd`. It always runs PROD and returns `action/result/status`; `result` includes `flowCode`, `executeId`, `operationStatus`, `nextAction`, and optional `flowInstanceId`.
116
+ - **Stop flow instance**: `dataops_operations +stop_flow_instance` requires `--spaceCode`, `--flowCode`, and exactly one of `--executeId` or `--flowInstanceId`. `--executeId` comes from `dataops_flow +execute_flow`; `--flowInstanceId` comes from `dataops_operations +search_flow_instances`. It returns `success`, `selector`, and selector-specific stop result fields.
117
+ - **Flow overview**: `+get_flow_overview` requires `--spaceCode` and either `--flowCode` or exact `--flowName`; `--flowCode` takes precedence. `--env` defaults to `DEV`; use `PROD` to include latest production instance fields when available. It returns `success`, `env`, `resolvedBy`, `flow`, `schedule`, `dag`, and `summary`.
118
+ - **Flow update**: `+update_flow` requires `--spaceCode`, `--flowCode`, and at least one of `--flowName` or `--remark`. It returns `action/result/status`; `result` is an array with items containing `flowCode`, `operationStatus`, `nameChanged`, and optional `flowName`.
119
+ - **High-frequency release flows**: `+list_high_frequency_release_flows` requires `--spaceCode`; `--days`, `--topN`, `--minCount`, and `--status` are optional. Defaults are `days=30`, `topN=10`, and `status=SUCCESS`. It returns `period`, `filters`, `flows`, `returnedCount`, and `nextAction`; flow items include `rank`, `flowCode`, `flowName`, `releaseCount`, `lastReleaseTime`, and `avgIntervalHours`.
120
+ - **Operations instance search**: `dataops_operations +search_flow_instances` requires `--spaceCode`; `--keyword`, `--startDate`, `--endDate`, `--status`, `--pageNum`, and `--pageSize` are optional. `--keyword` fuzzy-matches `flowName` or `flowCode`. `pageNum` defaults to `1`; `pageSize` defaults to `20` and maxes at `100`. It returns `totalCount`, `returnedCount`, `pageNum`, `pageSize`, `hasMore`, `instances`, `statusCounts`, `triggerTypeCounts`, and `ownerCounts`.
121
+ - **Task instance detail**: `dataops_operations +get_task_instance_detail` requires `--spaceCode`, `--flowCode`, `--flowInstanceId`, and either `--taskCode` or exact `--taskName`. `--taskCode` takes precedence when both are passed. `--includeLog` is optional and defaults to `false`. It returns `success`, `flowInstanceId`, `taskCode`, `taskName`, `taskInstanceId`, `status`, `task`, `taskInstance`, `definition`, and `log` only when requested.
116
122
  - **env**: `DEV` (development) | `PROD` (production, instance list defaults to PROD)
123
+ - **flow list paging**: `+list_flows` requires `--spaceCode`; `--keyword`, `--pageNum`, and `--pageSize` are optional. It returns `flows`, `totalCount`, `returnedCount`, `pageNum`, `pageSize`, and `hasMore`; flow items include `latestProductionInstance` only when available. `pageSize` defaults to `20` and maxes at `100`.
124
+ - **Task parameters**: `+get_task_params` requires `--spaceCode`, `--flowCode`, and `--taskCode`; it has no optional flags. It queries DEV and returns `data` as an array. Items include fields such as `paramKey`, `paramType`, `paramDataType`, `paramFrom`, and built-in flags like `isBd`.
117
125
  - **Task Status**: `success` / `failure` / `running` / `waiting`
118
- - **DAG Difference**: `+get_flow_dag` returns static definition structure, `+get_execute_dag` returns runtime structure with execution status