@shulex/cxclawctl 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -22
- package/dist/cli.js +2202 -127
- package/dist/cli.js.map +1 -1
- package/package.json +19 -6
- package/capabilities.lock.json +0 -898
- package/requirements-cli.txt +0 -2
- package/skills/__lib__/__init__.py +0 -0
- package/skills/__lib__/skill-runtime/bootstrap.py +0 -134
- package/skills/__lib__/solvea-api/__init__.py +0 -0
- package/skills/__lib__/solvea-api/client.py +0 -1944
- package/skills/__lib__/solvea-api/dashboard.py +0 -100
- package/skills/__lib__/solvea-api/solvea_api.py +0 -1341
- package/skills/agent-analytics/SKILL.md +0 -74
- package/skills/agent-analytics/references/chat-report-fields.md +0 -36
- package/skills/agent-analytics/references/report-fields.md +0 -130
- package/skills/agent-analytics/references/transcript.md +0 -110
- package/skills/agent-builder/SKILL.md +0 -42
- package/skills/agent-builder/interfaces/fallback-chatflow.md +0 -47
- package/skills/agent-builder/references/chatflow/graph-spec.md +0 -259
- package/skills/agent-builder/references/chatflow/node-selection.md +0 -89
- package/skills/agent-builder/references/chatflow/nodes/agent.md +0 -350
- package/skills/agent-builder/references/chatflow/nodes/business-track.md +0 -43
- package/skills/agent-builder/references/chatflow/nodes/code.md +0 -157
- package/skills/agent-builder/references/chatflow/nodes/collect-order.md +0 -47
- package/skills/agent-builder/references/chatflow/nodes/finish.md +0 -34
- package/skills/agent-builder/references/chatflow/nodes/human-review.md +0 -60
- package/skills/agent-builder/references/chatflow/nodes/if-else.md +0 -100
- package/skills/agent-builder/references/chatflow/nodes/intent.md +0 -79
- package/skills/agent-builder/references/chatflow/nodes/knowledge-reply.md +0 -43
- package/skills/agent-builder/references/chatflow/nodes/llm.md +0 -117
- package/skills/agent-builder/references/chatflow/nodes/note.md +0 -38
- package/skills/agent-builder/references/chatflow/nodes/question.md +0 -97
- package/skills/agent-builder/references/chatflow/nodes/rag-reply.md +0 -68
- package/skills/agent-builder/references/chatflow/nodes/reply.md +0 -97
- package/skills/agent-builder/references/chatflow/nodes/start.md +0 -35
- package/skills/agent-builder/references/chatflow/nodes/sub-graph.md +0 -48
- package/skills/agent-builder/references/chatflow/nodes/tool.md +0 -138
- package/skills/agent-builder/references/chatflow/nodes/transfer-to-human.md +0 -63
- package/skills/agent-builder/references/chatflow/nodes/variable-assignment.md +0 -52
- package/skills/agent-builder/references/chatflow/nodes/variable-extract.md +0 -50
- package/skills/agent-builder/references/chatflow/nodes/variable-process.md +0 -70
- package/skills/agent-builder/references/chatflow/overview.md +0 -53
- package/skills/agent-builder/references/chatflow/visual-translation.md +0 -64
- package/skills/agent-builder/references/chatflow/workflow.md +0 -134
- package/skills/agent-builder/references/multi-agent/create-update.md +0 -185
- package/skills/agent-builder/references/multi-agent/graph-spec.md +0 -246
- package/skills/agent-builder/references/multi-agent/graph-structure.md +0 -12
- package/skills/agent-builder/references/multi-agent/migrate-to-chatflow.md +0 -13
- package/skills/agent-builder/references/multi-agent/migration/large.md +0 -554
- package/skills/agent-builder/references/multi-agent/migration/principles.md +0 -446
- package/skills/agent-builder/references/multi-agent/migration/small.md +0 -225
- package/skills/agent-builder/references/multi-agent/orchestration.md +0 -11
- package/skills/agent-builder/references/multi-agent/overview.md +0 -48
- package/skills/agent-builder/references/multi-agent/update.md +0 -128
- package/skills/agent-builder/references/multi-agent/workflow.md +0 -87
- package/skills/agent-builder/references/single-agent/overview.md +0 -48
- package/skills/agent-builder/references/single-agent/prompt.md +0 -16
- package/skills/agent-builder/references/single-agent/tools.md +0 -36
- package/skills/agent-builder/references/single-agent/voice.md +0 -46
- package/skills/agent-builder/references/tools/http-tool.md +0 -315
- package/skills/agent-builder/references/tools/mock-tool.md +0 -279
- package/skills/agent-builder/references/tools/overview.md +0 -46
- package/skills/agent-builder/references/tools/tool-config.md +0 -43
- package/skills/agent-builder/references/tools/x-token.md +0 -34
- package/skills/agent-builder/scripts/chatflow/chatflow_builder.py +0 -3458
- package/skills/agent-builder/scripts/chatflow/validate_graph.py +0 -1697
- package/skills/agent-builder/scripts/copy_marketplace_agent.py +0 -61
- package/skills/agent-builder/scripts/multi-agent/legacy_graph_analyzer.py +0 -5974
- package/skills/agent-builder/scripts/multi-agent/replay_compare.py +0 -679
- package/skills/agent-builder/scripts/patch_agent_tools.py +0 -135
- package/skills/agent-builder/scripts/tools/validate_tool_config.py +0 -384
- package/skills/agent-tester/SKILL.md +0 -35
- package/skills/agent-tester/batch-test/overview.md +0 -86
- package/skills/agent-tester/scripts/batch_test_sheet.py +0 -213
- package/skills/agent-tester/single-test/badcase-retest.md +0 -85
- package/skills/agent-tester/single-test/direct-test.md +0 -24
- package/skills/agent-tester/single-test/overview.md +0 -142
- package/skills/delivery/SKILL.md +0 -112
- package/skills/delivery/interfaces/classify.md +0 -38
- package/skills/delivery/interfaces/create-agents.md +0 -89
- package/skills/delivery/interfaces/delivery-test.md +0 -67
- package/skills/delivery/references/flow.md +0 -40
- package/skills/delivery/references/validation.md +0 -15
- package/skills/delivery/scripts/batch_test.py +0 -704
- package/skills/delivery/scripts/classify.py +0 -22
- package/skills/delivery/scripts/create_agents.py +0 -695
- package/skills/delivery/scripts/gen_delivery_report.py +0 -335
- package/skills/delivery/scripts/gen_excel.py +0 -234
- package/skills/delivery/scripts/gen_report.py +0 -208
- package/skills/delivery/scripts/plan.py +0 -163
- package/skills/delivery/scripts/render.py +0 -713
- package/skills/delivery/scripts/render_summary.py +0 -38
- package/skills/delivery/scripts/sample_history_cases.py +0 -109
- package/skills/kb-builder/SKILL.md +0 -80
- package/skills/kb-builder/interfaces/qa-upload.md +0 -55
- package/skills/kb-builder/interfaces/web-kb.md +0 -84
- package/skills/kb-builder/references/directory.md +0 -373
- package/skills/kb-builder/references/export.md +0 -282
- package/skills/kb-builder/references/isolation-keywords.md +0 -113
- package/skills/kb-builder/references/knowledge.md +0 -153
- package/skills/kb-builder/references/product.md +0 -52
- package/skills/kb-builder/references/qa-from-help-center.md +0 -58
- package/skills/kb-builder/references/qa-from-pdf.md +0 -94
- package/skills/kb-builder/references/qa-from-tickets.md +0 -168
- package/skills/kb-builder/references/qa-from-web.md +0 -98
- package/skills/kb-builder/references/qa-quality.md +0 -73
- package/skills/kb-builder/references/reply-template.md +0 -237
- package/skills/kb-builder/references/tag-isolation-migration.md +0 -61
- package/skills/kb-builder/references/tags.md +0 -45
- package/skills/kb-builder/references/upload.md +0 -130
- package/skills/kb-builder/scripts/batch_extract_ticket_qa.py +0 -333
- package/skills/kb-builder/scripts/build_directory_plan.py +0 -144
- package/skills/kb-builder/scripts/crawl_site.py +0 -433
- package/skills/kb-builder/scripts/create_dirs.py +0 -146
- package/skills/kb-builder/scripts/export_knowledge.py +0 -454
- package/skills/kb-builder/scripts/extract_ticket_qa.py +0 -441
- package/skills/kb-builder/scripts/extract_web_qa.py +0 -118
- package/skills/kb-builder/scripts/import_qa.py +0 -219
- package/skills/kb-builder/scripts/import_source_documents.py +0 -143
- package/skills/kb-builder/scripts/import_templates.py +0 -409
- package/skills/poc/SKILL.md +0 -125
- package/skills/poc/assets/default_manual.css +0 -279
- package/skills/poc/assets/notice_escalate.png +0 -0
- package/skills/poc/assets/notice_escalate2.png +0 -0
- package/skills/poc/assets/notice_new_chat.png +0 -0
- package/skills/poc/assets/poc_test_template.html +0 -645
- package/skills/poc/assets/poc_test_template.md +0 -104
- package/skills/poc/assets/step3_preview.png +0 -0
- package/skills/poc/assets/step4_order_note.png +0 -0
- package/skills/poc/interfaces/classify.md +0 -38
- package/skills/poc/interfaces/create-agents.md +0 -108
- package/skills/poc/interfaces/poc-test.md +0 -94
- package/skills/poc/references/flow.md +0 -47
- package/skills/poc/references/kb-directory.md +0 -22
- package/skills/poc/references/validation.md +0 -17
- package/skills/poc/scripts/classify.py +0 -22
- package/skills/poc/scripts/create_agents.py +0 -818
- package/skills/poc/scripts/gen_excel.py +0 -234
- package/skills/poc/scripts/gen_poc_manual.py +0 -249
- package/skills/poc/scripts/gen_report.py +0 -208
- package/skills/poc/scripts/inspect_chatflow.py +0 -209
- package/skills/poc/scripts/plan.py +0 -178
- package/skills/poc/scripts/poc_test.py +0 -495
- package/skills/poc/scripts/render.py +0 -713
- package/skills/poc/scripts/render_summary.py +0 -41
- package/skills/runtime-inspector/SKILL.md +0 -51
- package/skills/runtime-inspector/references/frontend-rendering.md +0 -126
- package/skills/runtime-inspector/references/model.md +0 -59
- package/skills/runtime-inspector/references/sandbox.md +0 -131
- package/skills/runtime-inspector/scripts/model_inspect.py +0 -68
- package/skills/runtime-inspector/scripts/prepare.py +0 -98
- package/skills/runtime-inspector/scripts/process.py +0 -84
- package/skills/runtime-inspector/scripts/read_state.py +0 -32
- package/skills/runtime-inspector/scripts/sandbox_inspect.py +0 -463
- package/skills/runtime-inspector/scripts/test_claude_proxy.py +0 -49
- package/skills/runtime-inspector/scripts/test_openai_proxy.py +0 -41
- package/skills/runtime-inspector/scripts/verify_persistence.py +0 -101
- package/skills/runtime-inspector/scripts/write_test.py +0 -41
- package/skills/setup/SKILL.md +0 -17
- package/skills/shulex-support/SKILL.md +0 -39
- package/skills/shulex-support/references/badcase-analysis/overview.md +0 -34
- package/skills/shulex-support/references/badcase-analysis/text/conversation-record.md +0 -105
- package/skills/shulex-support/references/badcase-analysis/text/core-flow.md +0 -89
- package/skills/shulex-support/references/badcase-analysis/text/log-query-rules.md +0 -36
- package/skills/shulex-support/references/badcase-analysis/text/m1-entry.md +0 -53
- package/skills/shulex-support/references/badcase-analysis/text/m10-chatflow-service.md +0 -259
- package/skills/shulex-support/references/badcase-analysis/text/m11-algorithm-service.md +0 -148
- package/skills/shulex-support/references/badcase-analysis/text/m12-quality-inspection.md +0 -57
- package/skills/shulex-support/references/badcase-analysis/text/m2-intent.md +0 -120
- package/skills/shulex-support/references/badcase-analysis/text/m3-strategy.md +0 -126
- package/skills/shulex-support/references/badcase-analysis/text/m4-aifield.md +0 -121
- package/skills/shulex-support/references/badcase-analysis/text/m5-agent.md +0 -208
- package/skills/shulex-support/references/badcase-analysis/text/m6-tool.md +0 -115
- package/skills/shulex-support/references/badcase-analysis/text/m7-reply.md +0 -72
- package/skills/shulex-support/references/badcase-analysis/text/m8-error.md +0 -92
- package/skills/shulex-support/references/badcase-analysis/text/m9-knowledge-search.md +0 -147
- package/skills/shulex-support/references/badcase-analysis/text/sls-query.md +0 -283
- package/skills/shulex-support/references/badcase-analysis/voice/core-flow.md +0 -77
- package/skills/shulex-support/references/badcase-analysis/voice/dependencies.md +0 -19
- package/skills/shulex-support/references/badcase-analysis/voice/platform-api.md +0 -23
- package/skills/shulex-support/references/badcase-analysis/voice/sls-playbook.md +0 -31
- package/skills/shulex-support/references/product-docs.md +0 -70
- package/skills/shulex-support/references/tickets.md +0 -184
- package/skills/shulex-support/scripts/support.py +0 -427
- package/skills/solvea-desk/SKILL.md +0 -39
- package/skills/solvea-desk/references/report/fields.md +0 -82
- package/skills/solvea-desk/references/report/report-design.md +0 -200
- package/skills/solvea-desk/references/report/report-template.html +0 -44
- package/skills/solvea-desk/references/report/report.css +0 -517
- package/skills/solvea-desk/references/report/workflow.md +0 -464
- package/skills/solvea-desk/references/tickets/query.md +0 -103
- package/skills/solvea-desk/references/tickets/tags.md +0 -83
- package/skills/solvea-desk/scripts/report/fetch_sources.py +0 -580
- package/skills/solvea-desk/scripts/report/render_report.py +0 -763
- package/skills/solvea-desk/scripts/tickets/desk_api.py +0 -73
- package/skills/solvea-desk/scripts/tickets/fetch_tickets.py +0 -403
- package/skills/solvea-desk/scripts/tickets/ticket_tags.py +0 -305
- package/skills/solvea-kit/SKILL.md +0 -84
- package/skills/solvea-kit/references/agent.md +0 -145
- package/skills/solvea-kit/references/chat.md +0 -244
- package/skills/solvea-kit/references/intent.md +0 -69
- package/skills/solvea-kit/references/plugin.md +0 -53
- package/skills/solvea-kit/references/quality-inspection.md +0 -75
- package/skills/solvea-kit/references/strategy.md +0 -62
- package/skills/solvea-kit/references/tagging.md +0 -231
- package/skills/solvea-kit/references/variables.md +0 -53
- package/skills/solvea-kit/scripts/generate_label_import_file.py +0 -151
- package/skills/solvea-kit/scripts/intent_classify.py +0 -843
|
@@ -1,244 +0,0 @@
|
|
|
1
|
-
# 对话测试
|
|
2
|
-
|
|
3
|
-
> 已过时:POC 测试走 `poc`,交付回归走 `delivery`,原始会话与运营数据走 `agent-analytics`,badcase 上下文排查走 `shulex-support` 的 Agent 会话诊断场景,CHAT_FLOW 测试注意事项走 `agent-builder`。本文件仅保留历史兼容,不作为新任务入口。
|
|
4
|
-
|
|
5
|
-
## 单轮测试
|
|
6
|
-
|
|
7
|
-
### API
|
|
8
|
-
|
|
9
|
-
| 操作 | 方法 | 路径 | 说明 |
|
|
10
|
-
|------|------|------|------|
|
|
11
|
-
| 对话测试 | POST | `/api_v2/gpt/bot/{botId}/chat` | 单轮或多轮(传 chatId) |
|
|
12
|
-
|
|
13
|
-
### CLI
|
|
14
|
-
|
|
15
|
-
```bash
|
|
16
|
-
npx --yes @shulex/cxclawctl@latest api chat -m "你好" --pretty
|
|
17
|
-
```
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## 多轮测试(Session)
|
|
22
|
-
|
|
23
|
-
多轮对话时,每轮请求必须传入上一轮返回的 `chatId`,否则每次开新会话、上下文断开。
|
|
24
|
-
|
|
25
|
-
### CLI
|
|
26
|
-
|
|
27
|
-
```bash
|
|
28
|
-
npx --yes @shulex/cxclawctl@latest api chat -m "你好" --pretty
|
|
29
|
-
# 从返回中取 chatId
|
|
30
|
-
npx --yes @shulex/cxclawctl@latest api chat -m "继续问" --chat-id <上轮chatId> --pretty
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
### Python 示例
|
|
34
|
-
|
|
35
|
-
```python
|
|
36
|
-
from client import SolveaClient
|
|
37
|
-
client = SolveaClient()
|
|
38
|
-
|
|
39
|
-
r1 = client.chat("你好")
|
|
40
|
-
chat_id = r1.get("chatId")
|
|
41
|
-
r2 = client.chat("继续问", chat_id=chat_id)
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
### 对话详情与 Badcase 溯源
|
|
45
|
-
|
|
46
|
-
| 操作 | 方法 | 路径 | 说明 |
|
|
47
|
-
|------|------|------|------|
|
|
48
|
-
| 收件箱列表 | POST | `/api_v2/gpt/project/{pid}/transcripts/filter` | 返回会话摘要,不含完整内容 |
|
|
49
|
-
| 对话详情(按 chatId) | GET | `/api_v2/gpt/project/{pid}/transcript/{chatId}?isCompare=true` | 获取单条完整对话 |
|
|
50
|
-
| Badcase 溯源(按轮次) | POST | `/api_v2/gpt/bot/{botId}/chat/context/compare` | 返回检索知识、tool call 详情、最终 answer |
|
|
51
|
-
|
|
52
|
-
```bash
|
|
53
|
-
npx --yes @shulex/cxclawctl@latest api list-transcripts --pretty
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
Badcase 分析标准流程:
|
|
57
|
-
1. `get-transcript --chat-id <chatId>` — 获取完整对话数据,定位问题轮次
|
|
58
|
-
2. `get_chat_context_compare(bot_id, turn_data)` — 传入该轮 context,返回检索知识、tool call 详情、最终 answer
|
|
59
|
-
|
|
60
|
-
---
|
|
61
|
-
|
|
62
|
-
## CHAT_FLOW Agent 测试注意事项
|
|
63
|
-
|
|
64
|
-
CHAT_FLOW Agent 不能用 `metadata.specified_agent_id`(或老版 `specified_agent_data`)这类强制 agent 路由直接测试,必须通过策略路由。
|
|
65
|
-
|
|
66
|
-
完整测试流程:
|
|
67
|
-
1. 创建意图(`create-intent`)
|
|
68
|
-
2. 创建策略(`create-strategy`),将意图路由到目标 CHAT_FLOW Agent
|
|
69
|
-
3. 启用策略(`toggle-strategy --enable`)
|
|
70
|
-
4. 编写测试用例 JSON
|
|
71
|
-
5. 运行当前业务 skill 指定的测试 runner
|
|
72
|
-
6. 根据失败项调整 Agent → 重跑测试直到通过率达标
|
|
73
|
-
|
|
74
|
-
---
|
|
75
|
-
|
|
76
|
-
## 批量测试
|
|
77
|
-
|
|
78
|
-
> POC 测试业务流程归属 `poc` skill,交付回归业务流程归属 `delivery` skill。
|
|
79
|
-
> 测试 runner 已归属到对应业务 skill。
|
|
80
|
-
|
|
81
|
-
### 用法
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
# POC 测试:先读 /skills/poc/interfaces/poc-test.md
|
|
85
|
-
# Delivery 回归:先读 /skills/delivery/interfaces/delivery-test.md
|
|
86
|
-
|
|
87
|
-
# 校验 chatflow 图结构(chatflow 能力在 agent-builder skill 下)
|
|
88
|
-
npx --yes @shulex/cxclawctl@latest script skills/agent-builder/scripts/chatflow/validate_graph.py --data-file graph.json
|
|
89
|
-
```
|
|
90
|
-
|
|
91
|
-
### 测试用例格式(JSON Schema)
|
|
92
|
-
|
|
93
|
-
```json
|
|
94
|
-
[
|
|
95
|
-
{
|
|
96
|
-
"name": "测试名称",
|
|
97
|
-
"category": "分类",
|
|
98
|
-
"messages": ["用户消息1", "用户消息2(多轮)"],
|
|
99
|
-
"expected": {
|
|
100
|
-
"intent": "期望意图名(模糊匹配)",
|
|
101
|
-
"content_contains": ["回复须包含的关键词"],
|
|
102
|
-
"content_not_contains": ["回复不应包含的关键词"],
|
|
103
|
-
"handoff": false,
|
|
104
|
-
"flow_status": "success",
|
|
105
|
-
"nodes_executed": ["REPLY", "AGENT"]
|
|
106
|
-
}
|
|
107
|
-
}
|
|
108
|
-
]
|
|
109
|
-
```
|
|
110
|
-
|
|
111
|
-
### expected 字段说明
|
|
112
|
-
|
|
113
|
-
| 字段 | 类型 | 说明 |
|
|
114
|
-
|------|------|------|
|
|
115
|
-
| `intent` | string | 期望意图名,模糊匹配 |
|
|
116
|
-
| `content_contains` | string[] | 回复内容必须包含的关键词列表 |
|
|
117
|
-
| `content_not_contains` | string[] | 回复内容不应包含的关键词列表 |
|
|
118
|
-
| `handoff` | boolean | 是否期望转人工 |
|
|
119
|
-
| `flow_status` | string | 期望流程状态,如 `"success"` |
|
|
120
|
-
| `nodes_executed` | string[] | 期望执行的节点类型列表 |
|
|
121
|
-
|
|
122
|
-
### 含节点执行检查的测试用例示例
|
|
123
|
-
|
|
124
|
-
```json
|
|
125
|
-
{
|
|
126
|
-
"name": "修改地址-完整流程",
|
|
127
|
-
"messages": ["I want to change my address for order 12345"],
|
|
128
|
-
"expected": {
|
|
129
|
-
"intent": "Modify Order",
|
|
130
|
-
"flow_status": "success",
|
|
131
|
-
"nodes_executed": ["REPLY", "AGENT"],
|
|
132
|
-
"content_contains": ["address"]
|
|
133
|
-
}
|
|
134
|
-
}
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
### 自动判定为失败的条件
|
|
138
|
-
|
|
139
|
-
以下情况无论回复内容如何,测试工具自动判定为失败:
|
|
140
|
-
- 回复内容为 `INTERNAL_ERROR`
|
|
141
|
-
- `response.type` 为 `STOPPED` 或 `CONFUSED`
|
|
142
|
-
|
|
143
|
-
## 自动化测试闭环
|
|
144
|
-
|
|
145
|
-
用真实对话数据验证 Agent 配置效果,替代人工编写测试用例。
|
|
146
|
-
|
|
147
|
-
```
|
|
148
|
-
Step 1: 拉取真实对话数据
|
|
149
|
-
client.export_transcripts() → 轮询 → 下载 xlsx
|
|
150
|
-
(可选)client.list_intents/list_agents/list_plugins
|
|
151
|
-
|
|
152
|
-
Step 2: 转化为测试用例
|
|
153
|
-
从 inbox 提取每条对话的用户消息序列(按提问顺序)
|
|
154
|
-
单轮对话 → messages: [首条消息]
|
|
155
|
-
多轮对话 → messages: [消息1, 消息2, ...](保持原始顺序)
|
|
156
|
-
|
|
157
|
-
Step 3: 批量测试
|
|
158
|
-
POC 按 /skills/poc/interfaces/poc-test.md
|
|
159
|
-
Delivery 按 /skills/delivery/interfaces/delivery-test.md
|
|
160
|
-
检查:意图识别、节点执行链路、INTERNAL_ERROR、回复质量
|
|
161
|
-
|
|
162
|
-
Step 4: 分析结果 → 生成优化方案
|
|
163
|
-
- 意图误识别 → 调整意图 description
|
|
164
|
-
- 节点执行失败 → 修复 CHAT_FLOW 图
|
|
165
|
-
- 回复不准确 → 调整 Agent prompt
|
|
166
|
-
- 策略未命中 → 修复策略条件
|
|
167
|
-
列出待修改清单,和用户确认后自动执行
|
|
168
|
-
|
|
169
|
-
Step 5: 自动执行修改
|
|
170
|
-
用户确认后,调 API 更新意图/策略/Agent
|
|
171
|
-
|
|
172
|
-
Step 6: 重新测试
|
|
173
|
-
再次运行 batch test,对比前后通过率
|
|
174
|
-
循环 Step 4-6 直到达标
|
|
175
|
-
```
|
|
176
|
-
|
|
177
|
-
### 从线上导出 inbox 数据
|
|
178
|
-
|
|
179
|
-
```python
|
|
180
|
-
from client import SolveaClient
|
|
181
|
-
import time
|
|
182
|
-
|
|
183
|
-
client = SolveaClient()
|
|
184
|
-
|
|
185
|
-
# 1. 提交导出任务(默认近 1 个月)
|
|
186
|
-
result = client.export_transcripts(page_size=1000, time_offset=8)
|
|
187
|
-
task_id = result.get("taskId")
|
|
188
|
-
|
|
189
|
-
# 2. 轮询任务状态
|
|
190
|
-
while True:
|
|
191
|
-
tasks = client.get_export_tasks()
|
|
192
|
-
task = next((t for t in tasks if str(t["id"]) == str(task_id)), None)
|
|
193
|
-
if task and task["status"] == "SUCCESS":
|
|
194
|
-
file_url = task["file"]
|
|
195
|
-
break
|
|
196
|
-
time.sleep(5)
|
|
197
|
-
|
|
198
|
-
# 3. 下载 xlsx 并读取
|
|
199
|
-
# curl -o /tmp/inbox_export.xlsx '<file_url>'
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
| 操作 | 方法 | 路径 | 说明 |
|
|
203
|
-
|------|------|------|------|
|
|
204
|
-
| 提交导出任务 | POST | `/api_v2/gpt/project/{pid}/transcripts/export/async` | |
|
|
205
|
-
| 轮询导出状态 | POST | `/api_v2/gpt/label/project/{pid}/getLabelTask` | status=SUCCESS 时取 file URL |
|
|
206
|
-
|
|
207
|
-
```bash
|
|
208
|
-
npx --yes @shulex/cxclawctl@latest api export-transcripts --pretty
|
|
209
|
-
npx --yes @shulex/cxclawctl@latest api get-export-tasks --pretty
|
|
210
|
-
```
|
|
211
|
-
|
|
212
|
-
### 转化真实数据为测试用例
|
|
213
|
-
|
|
214
|
-
```python
|
|
215
|
-
test_cases = []
|
|
216
|
-
for conversation in inbox_data:
|
|
217
|
-
user_messages = [msg["content"] for msg in conversation["messages"]
|
|
218
|
-
if msg["role"] == "user"]
|
|
219
|
-
if user_messages:
|
|
220
|
-
test_cases.append({
|
|
221
|
-
"name": f"real_{conversation['id']}",
|
|
222
|
-
"category": "production_data",
|
|
223
|
-
"messages": user_messages,
|
|
224
|
-
"expected": {}
|
|
225
|
-
})
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
### 优化调整规则
|
|
229
|
-
|
|
230
|
-
1. 意图误识别率 > 10% → 修改意图 description,扩展或收窄匹配范围
|
|
231
|
-
2. 策略未命中 → 检查条件规则,修复 apiName/relation/values
|
|
232
|
-
3. INTERNAL_ERROR → 检查 CHAT_FLOW 图结构(validate_graph)
|
|
233
|
-
4. 回复不准确 → 调整 prompt,增加业务规则说明
|
|
234
|
-
5. 多轮断开 → 检查 maxRounds 配置
|
|
235
|
-
|
|
236
|
-
---
|
|
237
|
-
|
|
238
|
-
## 原则
|
|
239
|
-
|
|
240
|
-
- 多轮对话必须传 chatId,否则每次开新会话、上下文断开
|
|
241
|
-
- 测试驱动配置:配置 Agent 后必须通过批量测试验证,不能只配置不测试
|
|
242
|
-
- 测试用例必须覆盖 CHAT_FLOW 的每一个流程节点和分支,包括负面用例(不应匹配的意图)
|
|
243
|
-
- 仅检查回复内容不够,CHAT_FLOW 测试必须同时检查 `nodes_executed` 和 `flow_status`
|
|
244
|
-
- 每次调整后必须重跑测试,对比通过率变化
|
|
@@ -1,69 +0,0 @@
|
|
|
1
|
-
# 意图管理
|
|
2
|
-
|
|
3
|
-
## 能力概览
|
|
4
|
-
|
|
5
|
-
创建、查询、更新、启用/禁用 Shulex 策略路由使用的系统意图。
|
|
6
|
-
|
|
7
|
-
意图(Intent)代表用户的问题类型,如"退款"、"查询订单"。策略路由依赖意图 id 将对话分派给正确的 Agent。
|
|
8
|
-
|
|
9
|
-
CHAT_FLOW 内的 INTENT 节点直接在节点配置里定义意图(填 name + desc),不需要调用系统意图接口创建。策略路由的意图才需要通过 API 创建。
|
|
10
|
-
|
|
11
|
-
## API
|
|
12
|
-
|
|
13
|
-
| 操作 | 方法 | 路径 | 说明 |
|
|
14
|
-
|------|------|------|------|
|
|
15
|
-
| 意图列表 | GET | `/api_v2/gpt/project/{pid}/intents` | |
|
|
16
|
-
| 创建意图 | POST | `/api_v2/gpt/project/{pid}/intent` | |
|
|
17
|
-
| 意图详情 | GET | `/api_v2/gpt/project/{pid}/intent/{id}` | |
|
|
18
|
-
| 更新意图 | PUT | `/api_v2/gpt/project/{pid}/intent/{id}` | |
|
|
19
|
-
| 启用/禁用意图 | PUT | `/api_v2/gpt/project/{pid}/intent/{id}/status` | |
|
|
20
|
-
|
|
21
|
-
## CLI
|
|
22
|
-
|
|
23
|
-
```bash
|
|
24
|
-
npx --yes @shulex/cxclawctl@latest api list-intents --pretty
|
|
25
|
-
|
|
26
|
-
npx --yes @shulex/cxclawctl@latest api get-intent --id <intent_id> --pretty
|
|
27
|
-
|
|
28
|
-
npx --yes @shulex/cxclawctl@latest api create-intent \
|
|
29
|
-
--data '{"name":"退款","description":"用户要求退款","type":"Other","status":"ENABLED","category":"E_COMMERCE"}'
|
|
30
|
-
|
|
31
|
-
# 更新意图描述(PUT body 字段是 description,name 必传)
|
|
32
|
-
npx --yes @shulex/cxclawctl@latest api update-intent --id <intent_id> \
|
|
33
|
-
--data-file /workspace/state/solvea-kit/intent_update.json
|
|
34
|
-
|
|
35
|
-
# 短描述可直接使用 --data;长描述先写入 JSON 文件,再通过 --data-file 提交
|
|
36
|
-
|
|
37
|
-
npx --yes @shulex/cxclawctl@latest api toggle-intent --id <intent_id> --disable
|
|
38
|
-
npx --yes @shulex/cxclawctl@latest api toggle-intent --id <intent_id> --enable
|
|
39
|
-
```
|
|
40
|
-
|
|
41
|
-
## update-intent 字段说明(GET / PUT 字段命名不一致,必看)
|
|
42
|
-
|
|
43
|
-
API 在 GET 和 PUT 之间字段名不对齐,照搬 GET 输出回写会失败:
|
|
44
|
-
|
|
45
|
-
| GET 返回 | PUT body | 用途 |
|
|
46
|
-
|---------|----------|------|
|
|
47
|
-
| `name` | `name` | 意图名(必传) |
|
|
48
|
-
| **`example`** | **`description`** | 意图描述。GET 返回的 `example` 字段对应 PUT 的 `description`,**不要照搬 GET 字段名** |
|
|
49
|
-
| `enableCollect` | `enableCollect` | 一致 |
|
|
50
|
-
| `collects` | `collects` | 一致 |
|
|
51
|
-
| 其它字段(`callCount` / `createdAt` / `source` 等只读) | 不传 | 服务端维护 |
|
|
52
|
-
|
|
53
|
-
仅更新描述时,body 最小必填:
|
|
54
|
-
|
|
55
|
-
```json
|
|
56
|
-
{
|
|
57
|
-
"name": "Package issues",
|
|
58
|
-
"description": "...",
|
|
59
|
-
"enableCollect": false,
|
|
60
|
-
"collects": []
|
|
61
|
-
}
|
|
62
|
-
```
|
|
63
|
-
|
|
64
|
-
## 原则
|
|
65
|
-
|
|
66
|
-
- 创建意图前先查 `list-intents`,避免重复创建同名意图
|
|
67
|
-
- 更新意图:先 `get-intent` 拿当前 `name`(必传),用 `description` 字段(不是 `example`)写新描述
|
|
68
|
-
- CHAT_FLOW 的 INTENT 节点内直接定义意图(`config.intents[]` 的 name + desc),`intents[].intentId` 留空,不要调用 `create-intent` 接口
|
|
69
|
-
- 意图 name 建议纯 ASCII,与 INTENT 节点 name 字段约定一致
|
|
@@ -1,53 +0,0 @@
|
|
|
1
|
-
# 工具管理(Legacy 兼容入口)
|
|
2
|
-
|
|
3
|
-
> 本文件不再维护工具 schema。工具查看、创建、更新、调试、MockData 和 X-Token 配置统一由 `agent-builder` 负责。
|
|
4
|
-
|
|
5
|
-
权威文档:
|
|
6
|
-
|
|
7
|
-
- 总入口:`../../agent-builder/references/tools/overview.md`
|
|
8
|
-
- HTTP 参数、状态映射、relation/value 契约:`../../agent-builder/references/tools/http-tool.md`
|
|
9
|
-
- Mock / Excel / CSV:`../../agent-builder/references/tools/mock-tool.md`
|
|
10
|
-
- X-Token:`../../agent-builder/references/tools/x-token.md`
|
|
11
|
-
|
|
12
|
-
底层兼容命名仍是 `plugin` / `pluginId` / `queryPlugin`,常见工具类型仍是 `EXTERNAL_HTTP_CALL`。旧任务进入本页后,应立即转到上述 agent-builder 文档,不在这里复制或推导另一套 payload。
|
|
13
|
-
|
|
14
|
-
## 核心命令入口
|
|
15
|
-
|
|
16
|
-
统一使用 `/workspace/state/agent-builder/` 保存跨轮配置:
|
|
17
|
-
|
|
18
|
-
```bash
|
|
19
|
-
npx --yes @shulex/cxclawctl@latest api list-plugins --pretty
|
|
20
|
-
|
|
21
|
-
npx --yes @shulex/cxclawctl@latest api get-plugin --id <tool_id> --pretty
|
|
22
|
-
|
|
23
|
-
npx --yes @shulex/cxclawctl@latest script skills/agent-builder/scripts/tools/validate_tool_config.py \
|
|
24
|
-
--data-file /workspace/state/agent-builder/tool_<name>.json --pretty
|
|
25
|
-
|
|
26
|
-
npx --yes @shulex/cxclawctl@latest api create-plugin \
|
|
27
|
-
--data-file /workspace/state/agent-builder/tool_<name>.json --pretty
|
|
28
|
-
|
|
29
|
-
npx --yes @shulex/cxclawctl@latest api update-plugin \
|
|
30
|
-
--data-file /workspace/state/agent-builder/tool_<name>.json --pretty
|
|
31
|
-
|
|
32
|
-
npx --yes @shulex/cxclawctl@latest api replace-mock-data \
|
|
33
|
-
--name <mock_type> \
|
|
34
|
-
--data-file /workspace/state/agent-builder/mock_records.json --pretty
|
|
35
|
-
|
|
36
|
-
npx --yes @shulex/cxclawctl@latest api query-mock-data \
|
|
37
|
-
--name <mock_type> --pretty
|
|
38
|
-
|
|
39
|
-
npx --yes @shulex/cxclawctl@latest api run-plugin \
|
|
40
|
-
--data-file /workspace/state/agent-builder/run_tool_config.json --pretty
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
`run_tool_config.json` 必须使用 `externalHttpCallConfig` + `externalHttpCallRequestBody` envelope;具体结构只看 agent-builder 的 `http-tool.md`。
|
|
44
|
-
|
|
45
|
-
## 兼容与安全边界
|
|
46
|
-
|
|
47
|
-
- 创建前先查重并运行静态 validator;写入前确认环境和影响范围;创建后使用 `get-plugin` 回查完整定义。
|
|
48
|
-
- Authorization、X-Token 等鉴权 Header 按普通 `headerParams` 配置。
|
|
49
|
-
- `run-plugin` 会真实调用目标接口,只在用户明确要求或授权后执行。目标为写接口时,执行前还要额外确认请求体和影响范围;它不是创建或上线的默认步骤。
|
|
50
|
-
- MockData 的 `replace-mock-data` 是全量替换写操作,执行前必须确认;`--name` 在 MockData 命令中表示 `mock_type`。
|
|
51
|
-
- BOOLEAN 状态规则兼容 JSON boolean `true` / `false` 和前端保存的字符串 `"true"` / `"false"`;手写 JSON 推荐 boolean。完整类型与 relation 约束以 agent-builder 的 `http-tool.md` 为准。
|
|
52
|
-
- `actionStatuses` 默认分支兼容 `"condition": {}` 和 `"condition": {"or": []}`;新建配置推荐 canonical `{}`,但 `or: []` 同样合法。
|
|
53
|
-
- 工具绑定格式和节点边界按具体 Agent 形态的 agent-builder 文档处理,不沿用本 legacy 文件的历史示例。
|
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
# AI 质检管理
|
|
2
|
-
|
|
3
|
-
本模块管理系统的 AI 质检开关、系统规则、自定义规则和预警。质检结果运营分析仍由 `agent-analytics` 负责;这里仅在用户要调整配置或查看规则命中时使用。
|
|
4
|
-
|
|
5
|
-
## 查询与总开关
|
|
6
|
-
|
|
7
|
-
先读取现状。开启或关闭会影响项目后续 Agent 回复的质检行为,必须说明影响并在用户确认后写入:
|
|
8
|
-
|
|
9
|
-
```bash
|
|
10
|
-
npx --yes @shulex/cxclawctl@latest api get-quality-status --pretty
|
|
11
|
-
npx --yes @shulex/cxclawctl@latest api list-quality-rules \
|
|
12
|
-
--begin-ts <epoch_ms> --end-ts <epoch_ms> --pretty
|
|
13
|
-
npx --yes @shulex/cxclawctl@latest api set-quality-status --enable --pretty
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
关闭时使用 `--disable`。不要修改 GPT 的 Nacos `quality-inspection-gate`;它是平台灰度与消息过滤配置,不是租户可管理的质检开关。
|
|
17
|
-
|
|
18
|
-
## 系统规则
|
|
19
|
-
|
|
20
|
-
先读取规则清单与详情,再展示将要修改的范围、识别/拦截动作、启停状态和适用 Agent。系统规则只能更新,不能删除或更改为自定义规则:
|
|
21
|
-
|
|
22
|
-
```bash
|
|
23
|
-
npx --yes @shulex/cxclawctl@latest api get-quality-rule \
|
|
24
|
-
--inspection-key <rule_key> --pretty
|
|
25
|
-
npx --yes @shulex/cxclawctl@latest api update-quality-rule \
|
|
26
|
-
--data-file /workspace/state/solvea-kit/quality_rule_update.json --pretty
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
## 自定义规则
|
|
30
|
-
|
|
31
|
-
自定义规则可包含以下配置:
|
|
32
|
-
|
|
33
|
-
- `name`:规则名称
|
|
34
|
-
- `customPrompt`:自然语言质检要求
|
|
35
|
-
- `words`:敏感词数组;可与 `customPrompt` 二选一或同时使用
|
|
36
|
-
- `scope`:`0` 客服回复、`1` 客户来信、`2` 全部消息
|
|
37
|
-
- `type`:`1` 仅识别、`2` 拦截;`scope=2` 时只能使用仅识别
|
|
38
|
-
- `checkAllAgent`、`agentIds`、`autoAddAgent`:适用 Agent 范围
|
|
39
|
-
- `filterAmazon`:是否过滤 Amazon 场景
|
|
40
|
-
|
|
41
|
-
先生成 JSON 并向用户展示,确认后再创建、更新或删除:
|
|
42
|
-
|
|
43
|
-
```bash
|
|
44
|
-
npx --yes @shulex/cxclawctl@latest api create-custom-quality-rule \
|
|
45
|
-
--data-file /workspace/state/solvea-kit/custom_quality_rule_create.json --pretty
|
|
46
|
-
npx --yes @shulex/cxclawctl@latest api get-custom-quality-rule \
|
|
47
|
-
--inspection-key <rule_key> --pretty
|
|
48
|
-
npx --yes @shulex/cxclawctl@latest api update-custom-quality-rule \
|
|
49
|
-
--data-file /workspace/state/solvea-kit/custom_quality_rule_update.json --pretty
|
|
50
|
-
npx --yes @shulex/cxclawctl@latest api delete-custom-quality-rule \
|
|
51
|
-
--inspection-key <rule_key> --pretty
|
|
52
|
-
```
|
|
53
|
-
|
|
54
|
-
不要自行把规则设为拦截;用户没有明确要求时默认建议“仅识别”。删除规则前必须说明该规则将停止对后续消息生效。
|
|
55
|
-
|
|
56
|
-
## 预警
|
|
57
|
-
|
|
58
|
-
预警规则关联一个或多个质检规则,并可配置即时或每日汇总、邮箱或机器人 Webhook 通知。Webhook 和邮箱地址属于敏感配置:展示时脱敏,更新前必须确认。
|
|
59
|
-
|
|
60
|
-
```bash
|
|
61
|
-
npx --yes @shulex/cxclawctl@latest api list-quality-alert-rules --pretty
|
|
62
|
-
npx --yes @shulex/cxclawctl@latest api get-quality-alert-rule --id <alert_id> --pretty
|
|
63
|
-
npx --yes @shulex/cxclawctl@latest api create-quality-alert-rule \
|
|
64
|
-
--data-file /workspace/state/solvea-kit/quality_alert_create.json --pretty
|
|
65
|
-
npx --yes @shulex/cxclawctl@latest api update-quality-alert-rule \
|
|
66
|
-
--data-file /workspace/state/solvea-kit/quality_alert_update.json --pretty
|
|
67
|
-
npx --yes @shulex/cxclawctl@latest api delete-quality-alert-rule --id <alert_id> --pretty
|
|
68
|
-
npx --yes @shulex/cxclawctl@latest api list-quality-alert-messages \
|
|
69
|
-
--data-file /workspace/state/solvea-kit/quality_alert_messages_query.json --pretty
|
|
70
|
-
npx --yes @shulex/cxclawctl@latest api get-quality-alert-message --id <message_id> --pretty
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
## 结果排查边界
|
|
74
|
-
|
|
75
|
-
规则命中趋势和汇总可通过 `report-inspection` / `report-inspection-history` 查询;要分析具体会话或质检判断原因,转 `agent-analytics`。Agent 回复后的运行质检由 GPT 调 Intelli 执行,字段配置由 Tars 管理,不要跨这两个边界猜测数据来源。
|
|
@@ -1,62 +0,0 @@
|
|
|
1
|
-
# 策略管理
|
|
2
|
-
|
|
3
|
-
Strategy(策略)定义了:当某个条件(意图、触点、实体等)满足时,执行什么动作,例如路由到 Agent、回复模板、AI 回复、转人工或不处理。
|
|
4
|
-
|
|
5
|
-
## 条件构建(Condition)
|
|
6
|
-
|
|
7
|
-
先调 `list-entities` 获取可用条件实体(apiName、apiType、source、metadata),然后组装 condition:
|
|
8
|
-
|
|
9
|
-
```json
|
|
10
|
-
{
|
|
11
|
-
"and": [
|
|
12
|
-
{"rules": [{"apiName": "__touch_point", "apiType": "TEXT", "source": "SYSTEM", "relation": "IN", "values": ["LIVE_CHAT"]}]},
|
|
13
|
-
{"rules": [{"apiName": "__intent_ids", "apiType": "TEXT", "source": "SYSTEM", "relation": "IN", "values": [9750]}]}
|
|
14
|
-
]
|
|
15
|
-
}
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
`__intent_ids` 的 values 字段使用 `int(intent_id)`,不要用字符串。
|
|
19
|
-
|
|
20
|
-
## 动作类型(Action)
|
|
21
|
-
|
|
22
|
-
| 类型 | 说明 | 必需字段 |
|
|
23
|
-
|------|------|---------|
|
|
24
|
-
| `AGENT` | 路由到指定 Agent | agentId |
|
|
25
|
-
| `TEMPLATE_REPLY` | 回复固定模板 | templateId |
|
|
26
|
-
| `TEMPLATE_REPLY_AND_TRANSFER_TO_AGENT` | 模板回复后转人工 | templateId |
|
|
27
|
-
| `AI_REPLY` | AI 生成回复 | - |
|
|
28
|
-
| `AI_REPLY_AND_TRANSFER_TO_AGENT` | AI 回复后转人工 | - |
|
|
29
|
-
| `DO_NOTHING` | 不做任何操作 | - |
|
|
30
|
-
|
|
31
|
-
## API
|
|
32
|
-
|
|
33
|
-
| 操作 | 方法 | 路径 | 说明 |
|
|
34
|
-
|------|------|------|------|
|
|
35
|
-
| 可用实体列表 | GET | `/api_v2/gpt/project/{pid}/entities` | 获取 apiName、apiType、source |
|
|
36
|
-
| 创建策略 | POST | `/api_v2/gpt/project/{pid}/strategy?v4=true` | 必须带 v4=true |
|
|
37
|
-
| 更新策略 | PUT | `/api_v2/gpt/project/{pid}/strategy/{id}?v4=true` | 必须带 v4=true |
|
|
38
|
-
| 启用/禁用策略 | - | - | 通过 toggle-strategy CLI |
|
|
39
|
-
|
|
40
|
-
## CLI
|
|
41
|
-
|
|
42
|
-
```bash
|
|
43
|
-
npx --yes @shulex/cxclawctl@latest api list-entities --pretty
|
|
44
|
-
npx --yes @shulex/cxclawctl@latest api list-strategies --pretty
|
|
45
|
-
|
|
46
|
-
npx --yes @shulex/cxclawctl@latest api create-strategy \
|
|
47
|
-
--data '{"name":"退款路由","trigger":"MESSAGE","condition":{...},"action":{"type":"AGENT","agentId":2270}}'
|
|
48
|
-
|
|
49
|
-
npx --yes @shulex/cxclawctl@latest api update-strategy --id <strategy_id> \
|
|
50
|
-
--data-file /workspace/state/solvea-kit/update_strategy.json
|
|
51
|
-
|
|
52
|
-
npx --yes @shulex/cxclawctl@latest api toggle-strategy --id <strategy_id> --enable
|
|
53
|
-
npx --yes @shulex/cxclawctl@latest api toggle-strategy --id <strategy_id> --disable
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
## 原则
|
|
57
|
-
|
|
58
|
-
- 创建/更新策略接口参数由 SolveaClient 自动处理,无需额外传参。
|
|
59
|
-
- `__intent_ids` 的 values 字段使用 int,不要使用字符串形式。
|
|
60
|
-
- 配置策略后必须执行 `toggle-strategy --enable` 启用,否则不生效。
|
|
61
|
-
- 测试 CHAT_FLOW 时,必须先创建策略将意图路由到 Agent,再通过 chat 接口发送匹配消息测试;不能用 `metadata.specified_agent_id` / `metadata.specified_agent_data` 这类强制 agent 路由直接测试 CHAT_FLOW Agent。
|
|
62
|
-
- Agent 构建、工具配置和 CHAT_FLOW 图不在本文档维护,走 `agent-builder`。
|