@shulex/cxclawctl 0.1.2 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -22
- package/capabilities.lock.json +535 -131
- package/dist/cli.js +2203 -127
- package/dist/cli.js.map +1 -1
- package/package.json +20 -6
- package/requirements-cli.txt +0 -2
- package/skills/__lib__/__init__.py +0 -0
- package/skills/__lib__/skill-runtime/bootstrap.py +0 -134
- package/skills/__lib__/solvea-api/__init__.py +0 -0
- package/skills/__lib__/solvea-api/client.py +0 -1944
- package/skills/__lib__/solvea-api/dashboard.py +0 -100
- package/skills/__lib__/solvea-api/solvea_api.py +0 -1341
- package/skills/agent-analytics/SKILL.md +0 -74
- package/skills/agent-analytics/references/chat-report-fields.md +0 -36
- package/skills/agent-analytics/references/report-fields.md +0 -130
- package/skills/agent-analytics/references/transcript.md +0 -110
- package/skills/agent-builder/SKILL.md +0 -42
- package/skills/agent-builder/interfaces/fallback-chatflow.md +0 -47
- package/skills/agent-builder/references/chatflow/graph-spec.md +0 -259
- package/skills/agent-builder/references/chatflow/node-selection.md +0 -89
- package/skills/agent-builder/references/chatflow/nodes/agent.md +0 -350
- package/skills/agent-builder/references/chatflow/nodes/business-track.md +0 -43
- package/skills/agent-builder/references/chatflow/nodes/code.md +0 -157
- package/skills/agent-builder/references/chatflow/nodes/collect-order.md +0 -47
- package/skills/agent-builder/references/chatflow/nodes/finish.md +0 -34
- package/skills/agent-builder/references/chatflow/nodes/human-review.md +0 -60
- package/skills/agent-builder/references/chatflow/nodes/if-else.md +0 -100
- package/skills/agent-builder/references/chatflow/nodes/intent.md +0 -79
- package/skills/agent-builder/references/chatflow/nodes/knowledge-reply.md +0 -43
- package/skills/agent-builder/references/chatflow/nodes/llm.md +0 -117
- package/skills/agent-builder/references/chatflow/nodes/note.md +0 -38
- package/skills/agent-builder/references/chatflow/nodes/question.md +0 -97
- package/skills/agent-builder/references/chatflow/nodes/rag-reply.md +0 -68
- package/skills/agent-builder/references/chatflow/nodes/reply.md +0 -97
- package/skills/agent-builder/references/chatflow/nodes/start.md +0 -35
- package/skills/agent-builder/references/chatflow/nodes/sub-graph.md +0 -48
- package/skills/agent-builder/references/chatflow/nodes/tool.md +0 -138
- package/skills/agent-builder/references/chatflow/nodes/transfer-to-human.md +0 -63
- package/skills/agent-builder/references/chatflow/nodes/variable-assignment.md +0 -52
- package/skills/agent-builder/references/chatflow/nodes/variable-extract.md +0 -50
- package/skills/agent-builder/references/chatflow/nodes/variable-process.md +0 -70
- package/skills/agent-builder/references/chatflow/overview.md +0 -53
- package/skills/agent-builder/references/chatflow/visual-translation.md +0 -64
- package/skills/agent-builder/references/chatflow/workflow.md +0 -134
- package/skills/agent-builder/references/multi-agent/create-update.md +0 -185
- package/skills/agent-builder/references/multi-agent/graph-spec.md +0 -246
- package/skills/agent-builder/references/multi-agent/graph-structure.md +0 -12
- package/skills/agent-builder/references/multi-agent/migrate-to-chatflow.md +0 -13
- package/skills/agent-builder/references/multi-agent/migration/large.md +0 -554
- package/skills/agent-builder/references/multi-agent/migration/principles.md +0 -446
- package/skills/agent-builder/references/multi-agent/migration/small.md +0 -225
- package/skills/agent-builder/references/multi-agent/orchestration.md +0 -11
- package/skills/agent-builder/references/multi-agent/overview.md +0 -48
- package/skills/agent-builder/references/multi-agent/update.md +0 -128
- package/skills/agent-builder/references/multi-agent/workflow.md +0 -87
- package/skills/agent-builder/references/single-agent/overview.md +0 -48
- package/skills/agent-builder/references/single-agent/prompt.md +0 -16
- package/skills/agent-builder/references/single-agent/tools.md +0 -36
- package/skills/agent-builder/references/single-agent/voice.md +0 -46
- package/skills/agent-builder/references/tools/http-tool.md +0 -315
- package/skills/agent-builder/references/tools/mock-tool.md +0 -279
- package/skills/agent-builder/references/tools/overview.md +0 -46
- package/skills/agent-builder/references/tools/tool-config.md +0 -43
- package/skills/agent-builder/references/tools/x-token.md +0 -34
- package/skills/agent-builder/scripts/chatflow/chatflow_builder.py +0 -3458
- package/skills/agent-builder/scripts/chatflow/validate_graph.py +0 -1697
- package/skills/agent-builder/scripts/copy_marketplace_agent.py +0 -61
- package/skills/agent-builder/scripts/multi-agent/legacy_graph_analyzer.py +0 -5974
- package/skills/agent-builder/scripts/multi-agent/replay_compare.py +0 -679
- package/skills/agent-builder/scripts/patch_agent_tools.py +0 -135
- package/skills/agent-builder/scripts/tools/validate_tool_config.py +0 -384
- package/skills/agent-tester/SKILL.md +0 -35
- package/skills/agent-tester/batch-test/overview.md +0 -86
- package/skills/agent-tester/scripts/batch_test_sheet.py +0 -213
- package/skills/agent-tester/single-test/badcase-retest.md +0 -85
- package/skills/agent-tester/single-test/direct-test.md +0 -24
- package/skills/agent-tester/single-test/overview.md +0 -142
- package/skills/delivery/SKILL.md +0 -112
- package/skills/delivery/interfaces/classify.md +0 -38
- package/skills/delivery/interfaces/create-agents.md +0 -89
- package/skills/delivery/interfaces/delivery-test.md +0 -67
- package/skills/delivery/references/flow.md +0 -40
- package/skills/delivery/references/validation.md +0 -15
- package/skills/delivery/scripts/batch_test.py +0 -704
- package/skills/delivery/scripts/classify.py +0 -22
- package/skills/delivery/scripts/create_agents.py +0 -695
- package/skills/delivery/scripts/gen_delivery_report.py +0 -335
- package/skills/delivery/scripts/gen_excel.py +0 -234
- package/skills/delivery/scripts/gen_report.py +0 -208
- package/skills/delivery/scripts/plan.py +0 -163
- package/skills/delivery/scripts/render.py +0 -713
- package/skills/delivery/scripts/render_summary.py +0 -38
- package/skills/delivery/scripts/sample_history_cases.py +0 -109
- package/skills/kb-builder/SKILL.md +0 -80
- package/skills/kb-builder/interfaces/qa-upload.md +0 -55
- package/skills/kb-builder/interfaces/web-kb.md +0 -84
- package/skills/kb-builder/references/directory.md +0 -373
- package/skills/kb-builder/references/export.md +0 -282
- package/skills/kb-builder/references/isolation-keywords.md +0 -113
- package/skills/kb-builder/references/knowledge.md +0 -153
- package/skills/kb-builder/references/product.md +0 -52
- package/skills/kb-builder/references/qa-from-help-center.md +0 -58
- package/skills/kb-builder/references/qa-from-pdf.md +0 -94
- package/skills/kb-builder/references/qa-from-tickets.md +0 -168
- package/skills/kb-builder/references/qa-from-web.md +0 -98
- package/skills/kb-builder/references/qa-quality.md +0 -73
- package/skills/kb-builder/references/reply-template.md +0 -237
- package/skills/kb-builder/references/tag-isolation-migration.md +0 -61
- package/skills/kb-builder/references/tags.md +0 -45
- package/skills/kb-builder/references/upload.md +0 -130
- package/skills/kb-builder/scripts/batch_extract_ticket_qa.py +0 -333
- package/skills/kb-builder/scripts/build_directory_plan.py +0 -144
- package/skills/kb-builder/scripts/crawl_site.py +0 -433
- package/skills/kb-builder/scripts/create_dirs.py +0 -146
- package/skills/kb-builder/scripts/export_knowledge.py +0 -454
- package/skills/kb-builder/scripts/extract_ticket_qa.py +0 -441
- package/skills/kb-builder/scripts/extract_web_qa.py +0 -118
- package/skills/kb-builder/scripts/import_qa.py +0 -219
- package/skills/kb-builder/scripts/import_source_documents.py +0 -143
- package/skills/kb-builder/scripts/import_templates.py +0 -409
- package/skills/poc/SKILL.md +0 -125
- package/skills/poc/assets/default_manual.css +0 -279
- package/skills/poc/assets/notice_escalate.png +0 -0
- package/skills/poc/assets/notice_escalate2.png +0 -0
- package/skills/poc/assets/notice_new_chat.png +0 -0
- package/skills/poc/assets/poc_test_template.html +0 -645
- package/skills/poc/assets/poc_test_template.md +0 -104
- package/skills/poc/assets/step3_preview.png +0 -0
- package/skills/poc/assets/step4_order_note.png +0 -0
- package/skills/poc/interfaces/classify.md +0 -38
- package/skills/poc/interfaces/create-agents.md +0 -108
- package/skills/poc/interfaces/poc-test.md +0 -94
- package/skills/poc/references/flow.md +0 -47
- package/skills/poc/references/kb-directory.md +0 -22
- package/skills/poc/references/validation.md +0 -17
- package/skills/poc/scripts/classify.py +0 -22
- package/skills/poc/scripts/create_agents.py +0 -818
- package/skills/poc/scripts/gen_excel.py +0 -234
- package/skills/poc/scripts/gen_poc_manual.py +0 -249
- package/skills/poc/scripts/gen_report.py +0 -208
- package/skills/poc/scripts/inspect_chatflow.py +0 -209
- package/skills/poc/scripts/plan.py +0 -178
- package/skills/poc/scripts/poc_test.py +0 -495
- package/skills/poc/scripts/render.py +0 -713
- package/skills/poc/scripts/render_summary.py +0 -41
- package/skills/runtime-inspector/SKILL.md +0 -51
- package/skills/runtime-inspector/references/frontend-rendering.md +0 -126
- package/skills/runtime-inspector/references/model.md +0 -59
- package/skills/runtime-inspector/references/sandbox.md +0 -131
- package/skills/runtime-inspector/scripts/model_inspect.py +0 -68
- package/skills/runtime-inspector/scripts/prepare.py +0 -98
- package/skills/runtime-inspector/scripts/process.py +0 -84
- package/skills/runtime-inspector/scripts/read_state.py +0 -32
- package/skills/runtime-inspector/scripts/sandbox_inspect.py +0 -463
- package/skills/runtime-inspector/scripts/test_claude_proxy.py +0 -49
- package/skills/runtime-inspector/scripts/test_openai_proxy.py +0 -41
- package/skills/runtime-inspector/scripts/verify_persistence.py +0 -101
- package/skills/runtime-inspector/scripts/write_test.py +0 -41
- package/skills/setup/SKILL.md +0 -17
- package/skills/shulex-support/SKILL.md +0 -39
- package/skills/shulex-support/references/badcase-analysis/overview.md +0 -34
- package/skills/shulex-support/references/badcase-analysis/text/conversation-record.md +0 -105
- package/skills/shulex-support/references/badcase-analysis/text/core-flow.md +0 -89
- package/skills/shulex-support/references/badcase-analysis/text/log-query-rules.md +0 -36
- package/skills/shulex-support/references/badcase-analysis/text/m1-entry.md +0 -53
- package/skills/shulex-support/references/badcase-analysis/text/m10-chatflow-service.md +0 -259
- package/skills/shulex-support/references/badcase-analysis/text/m11-algorithm-service.md +0 -148
- package/skills/shulex-support/references/badcase-analysis/text/m12-quality-inspection.md +0 -57
- package/skills/shulex-support/references/badcase-analysis/text/m2-intent.md +0 -120
- package/skills/shulex-support/references/badcase-analysis/text/m3-strategy.md +0 -126
- package/skills/shulex-support/references/badcase-analysis/text/m4-aifield.md +0 -121
- package/skills/shulex-support/references/badcase-analysis/text/m5-agent.md +0 -208
- package/skills/shulex-support/references/badcase-analysis/text/m6-tool.md +0 -115
- package/skills/shulex-support/references/badcase-analysis/text/m7-reply.md +0 -72
- package/skills/shulex-support/references/badcase-analysis/text/m8-error.md +0 -92
- package/skills/shulex-support/references/badcase-analysis/text/m9-knowledge-search.md +0 -147
- package/skills/shulex-support/references/badcase-analysis/text/sls-query.md +0 -283
- package/skills/shulex-support/references/badcase-analysis/voice/core-flow.md +0 -77
- package/skills/shulex-support/references/badcase-analysis/voice/dependencies.md +0 -19
- package/skills/shulex-support/references/badcase-analysis/voice/platform-api.md +0 -23
- package/skills/shulex-support/references/badcase-analysis/voice/sls-playbook.md +0 -31
- package/skills/shulex-support/references/product-docs.md +0 -70
- package/skills/shulex-support/references/tickets.md +0 -184
- package/skills/shulex-support/scripts/support.py +0 -427
- package/skills/solvea-desk/SKILL.md +0 -39
- package/skills/solvea-desk/references/report/fields.md +0 -82
- package/skills/solvea-desk/references/report/report-design.md +0 -200
- package/skills/solvea-desk/references/report/report-template.html +0 -44
- package/skills/solvea-desk/references/report/report.css +0 -517
- package/skills/solvea-desk/references/report/workflow.md +0 -464
- package/skills/solvea-desk/references/tickets/query.md +0 -103
- package/skills/solvea-desk/references/tickets/tags.md +0 -83
- package/skills/solvea-desk/scripts/report/fetch_sources.py +0 -580
- package/skills/solvea-desk/scripts/report/render_report.py +0 -763
- package/skills/solvea-desk/scripts/tickets/desk_api.py +0 -73
- package/skills/solvea-desk/scripts/tickets/fetch_tickets.py +0 -403
- package/skills/solvea-desk/scripts/tickets/ticket_tags.py +0 -305
- package/skills/solvea-kit/SKILL.md +0 -84
- package/skills/solvea-kit/references/agent.md +0 -145
- package/skills/solvea-kit/references/chat.md +0 -244
- package/skills/solvea-kit/references/intent.md +0 -69
- package/skills/solvea-kit/references/plugin.md +0 -53
- package/skills/solvea-kit/references/quality-inspection.md +0 -75
- package/skills/solvea-kit/references/strategy.md +0 -62
- package/skills/solvea-kit/references/tagging.md +0 -231
- package/skills/solvea-kit/references/variables.md +0 -53
- package/skills/solvea-kit/scripts/generate_label_import_file.py +0 -151
- package/skills/solvea-kit/scripts/intent_classify.py +0 -843
|
@@ -1,259 +0,0 @@
|
|
|
1
|
-
# M10:ChatFlow 服务排查
|
|
2
|
-
|
|
3
|
-
M10 是 CHAT_FLOW 运行时排查索引。负责从会话记录、graph 配置和日志里确认图是否按预期执行;节点字段、图规范和节点选型不要在这里重复维护,按需读取 agent-builder 的 chatflow 文档。
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## 边界
|
|
8
|
-
|
|
9
|
-
M10 只处理已经进入 CHAT_FLOW 之后的问题:
|
|
10
|
-
|
|
11
|
-
- graph 定义加载、解析、校验失败。
|
|
12
|
-
- 节点没有进入、执行异常、输出错误。
|
|
13
|
-
- 边路由、`sourceHandle`、`output_handle` 不一致。
|
|
14
|
-
- 条件、变量系统字段、上游节点输出读取异常。
|
|
15
|
-
- RAG / KU / tool / 子图 / 人审节点在 chatflow 内部的传参、路由和结果消费异常。
|
|
16
|
-
- graph 没有正常结束,或停在等待用户输入、人审、子图等状态。
|
|
17
|
-
|
|
18
|
-
---
|
|
19
|
-
|
|
20
|
-
## 排查顺序
|
|
21
|
-
|
|
22
|
-
### 1. 读取当前生效 graph / 配置
|
|
23
|
-
|
|
24
|
-
从会话记录的 `strategyAction`、`context.ext` 或已知测试结果确认 `agentId`,然后执行:
|
|
25
|
-
|
|
26
|
-
```bash
|
|
27
|
-
npx --yes @shulex/cxclawctl@latest script skills/agent-builder/scripts/chatflow/chatflow_builder.py inspect --agent-id <agentId> --pretty
|
|
28
|
-
```
|
|
29
|
-
|
|
30
|
-
只要本轮确认进入 CHAT_FLOW,报告前必须读取当前生效的 ChatFlow 图配置。读取图配置的目的不是只做节点/边校验,而是看到全局业务流程,理解失败节点的上游来源、下游去向、变量和工具结果如何传递,必要时给出准确可行的配置建议。
|
|
31
|
-
|
|
32
|
-
`chatflow_run_info` 只能说明“实际跑了哪些节点”,不能说明“图上原本应该怎么连、节点怎么配、上下游业务和数据流是什么、用户后台当前看到的是否就是这张图”。
|
|
33
|
-
|
|
34
|
-
`get-agent` 只返回 Agent 元信息,不等于已经读取 ChatFlow 图。inspect 成功前不得输出最终报告,也不得只凭会话记录给出修改节点配置的强建议。
|
|
35
|
-
|
|
36
|
-
需要补充 Agent 类型、版本等元信息时,可执行:
|
|
37
|
-
|
|
38
|
-
```bash
|
|
39
|
-
npx --yes @shulex/cxclawctl@latest api get-agent --id <agentId> --pretty
|
|
40
|
-
```
|
|
41
|
-
|
|
42
|
-
关注点:
|
|
43
|
-
|
|
44
|
-
- 当前线上生效图是否就是用户以为的那张图。
|
|
45
|
-
- 对比运行记录中的 `flow_version`、节点 id 和当前 graph。当前图与历史运行版本无法对应时,只能说明当前状态,不能据此断言历史配置根因。
|
|
46
|
-
- 节点 title / id / type 是否能和 `chatflow_run_info`、日志对上。
|
|
47
|
-
- 按 `node_id` 在 graph 中定位失败节点,不要只按 title 猜节点。
|
|
48
|
-
- CODE 节点报错时,读取 CODE 节点的实际 `source_code`、输入变量选择器和输出字段,确认报错语句确实存在于当前生效配置。
|
|
49
|
-
- 核对失败节点的入边和出边,说明它为什么被执行、失败后原本会走向哪里。
|
|
50
|
-
- 上一节点 `output_handle` 是否能在 graph 边里找到对应 `sourceHandle`。
|
|
51
|
-
- 失败节点的上游业务意图、变量来源、工具/知识结果来源是否合理。
|
|
52
|
-
- 失败节点的下游回复、转人工、继续追问或结束路径是否符合业务目标。
|
|
53
|
-
- 条件、变量、工具入参、模板、知识目录、子图版本是否和业务预期一致。
|
|
54
|
-
- 完整 graph 里的 `NOTE` / `CUSTOM_NOTE` 是画布注释,不参与运行,不要把未连线 NOTE 当作 badcase 根因。
|
|
55
|
-
|
|
56
|
-
如果 agentId / graphId 无法从会话记录的 `strategyAction` 或 `context.ext` 确认,再查询相关 Agent 配置。拿不到图配置时,报告里必须写明“图配置未能确认”,不要只凭执行日志断定配置根因,也不要给出依赖全局流程判断的强建议。
|
|
57
|
-
|
|
58
|
-
### 2. 对齐会话执行记录
|
|
59
|
-
|
|
60
|
-
优先读取:
|
|
61
|
-
|
|
62
|
-
| 字段 | 含义 |
|
|
63
|
-
|------|------|
|
|
64
|
-
| `messages[].context.ext.chatflow_run_info` | 节点执行序列、节点状态、部分节点输出 |
|
|
65
|
-
| `messages[].context.ext.graph_finished` | graph 是否正常结束 |
|
|
66
|
-
| `messages[].type/reason/reasonDesc` | 最终回复类型和失败原因 |
|
|
67
|
-
| `messages[].context.metadata.ai_fields` | 条件、变量和工具节点可能依赖的变量系统字段 |
|
|
68
|
-
| `messages[].context.timing.__stage_time_total` | 本轮总耗时,辅助判断是否有慢节点 |
|
|
69
|
-
|
|
70
|
-
常见信号:
|
|
71
|
-
|
|
72
|
-
| 会话记录信号 | 下一步 |
|
|
73
|
-
|----------------|-------|
|
|
74
|
-
| `chatflow_run_info` 为空 | 回 M3 / M5,先确认是否真的进入 CHAT_FLOW |
|
|
75
|
-
| `graph_finished=false` | 找最后一个执行节点,确认是异常、等待用户输入、人审 interrupt,还是子图未完成 |
|
|
76
|
-
| 某个预期节点未出现 | 查上一节点的 `output_handle`、`goto`、routing map 和 graph 边 |
|
|
77
|
-
| 条件节点无匹配分支 | 查变量来源、条件配置、`FALSE` / default 兜底边 |
|
|
78
|
-
| RAG / KU 节点有执行但内容为空或不准 | M10 只确认 query、知识目录、结果消费;召回内部质量跳 M9 |
|
|
79
|
-
| TOOL 节点执行后分支不对 | 查工具入参、原始返回、action status / result routing、下游字段消费 |
|
|
80
|
-
| 停在 `USER_INPUT`,或上一节点 `output_handle=WAIT` | 先判断是否正常等待下一轮用户输入,不要直接判定 graph 异常 |
|
|
81
|
-
| 停在 `HUMAN_REVIEW` | 先判断是否正常等待人审 resume |
|
|
82
|
-
| 停在 `SUB_GRAPH` | 同时查外层 `SUB_GRAPH` 节点和内层 namespace / 子图执行日志 |
|
|
83
|
-
|
|
84
|
-
### 3. 定位到节点后读 chatflow 文档
|
|
85
|
-
|
|
86
|
-
报告前必须读取对应 chatflow 文档,避免只按报错字面推断。
|
|
87
|
-
|
|
88
|
-
| 问题类型 | 必读说明 |
|
|
89
|
-
|----------|----------|
|
|
90
|
-
| 图结构、边、source JSON、全局校验 | `/skills/agent-builder/references/chatflow/graph-spec.md` |
|
|
91
|
-
| 节点选型、是否该用模板/问题/工具/Agent/变量节点 | `/skills/agent-builder/references/chatflow/node-selection.md` |
|
|
92
|
-
| 具体节点字段、约束、常见误用 | `/skills/agent-builder/references/chatflow/nodes/<node>.md` |
|
|
93
|
-
|
|
94
|
-
节点文档映射规则:
|
|
95
|
-
|
|
96
|
-
- 节点类型转小写并用短横线连接,例如 `IF_ELSE` -> `nodes/if-else.md`,`RAG_REPLY` -> `nodes/rag-reply.md`。
|
|
97
|
-
- 特殊节点按文件名读取:`TRANSFER_TO_HUMAN` -> `transfer-to-human.md`,`VARIABLE_PROCESS` -> `variable-process.md`,`SUB_GRAPH` -> `sub-graph.md`。
|
|
98
|
-
- `USER_INPUT` 和 `AGENT_TOOL` 是运行时注入节点,优先读其归属业务节点文档;例如 `QUESTION` 等待用户输入时读 `question.md`,`AGENT_TOOL` 工具调用时读 `agent.md` 和 `tool.md`。
|
|
99
|
-
- `WAIT` 不是 source graph 里的业务节点;它是运行时 handle,表示节点选择进入注入的 `USER_INPUT`。
|
|
100
|
-
|
|
101
|
-
### 4. 最后查聚焦日志
|
|
102
|
-
|
|
103
|
-
日志用于确认会话记录和 graph 无法完全解释的运行时事实。先按 `/skills/shulex-support/references/badcase-analysis/text/sls-query.md` 定位 GPT 侧 trace;进入 chatflow 后读取“跨服务 trace 拼接”,优先用 `armsTrace`。
|
|
104
|
-
|
|
105
|
-
日志没查到时,报告中写清“日志未能确认”的原因,不要把推断写成日志事实。
|
|
106
|
-
|
|
107
|
-
---
|
|
108
|
-
|
|
109
|
-
## 运行模型
|
|
110
|
-
|
|
111
|
-
chatflow 服务把前端配置的 JSON graph 转成 LangGraph 后执行:
|
|
112
|
-
|
|
113
|
-
```text
|
|
114
|
-
JSON graph
|
|
115
|
-
-> get_or_load_graph_def()
|
|
116
|
-
-> GraphJSONLoader.load_from_dict()
|
|
117
|
-
-> GraphJSONLoader.inject_builtin_nodes()
|
|
118
|
-
-> GraphBuilder.build_graph()
|
|
119
|
-
-> GraphExecutor.execute_stream()
|
|
120
|
-
-> FlowService.execute_flow_stream()
|
|
121
|
-
```
|
|
122
|
-
|
|
123
|
-
排查时重点对齐这些运行时对象:
|
|
124
|
-
|
|
125
|
-
| 对象 | 排查价值 |
|
|
126
|
-
|------|----------|
|
|
127
|
-
| `node_results` | 和会话记录的 `chatflow_run_info` 对齐,确认失败或异常输出来自哪个节点 |
|
|
128
|
-
| `output.output_handle` | 当前节点实际选择的出边 handle |
|
|
129
|
-
| `output.goto` / `Command.goto` | 当前节点实际跳转的下一节点 |
|
|
130
|
-
| `execution_path` | 判断节点是否漏走、重复走、或卡在等待/人审/子图 |
|
|
131
|
-
| `current_node_name` | `graph_finished=false` 时定位最后卡点 |
|
|
132
|
-
| `extra_info.node_extra_info.enter_count` | 判断是否超过 `maxRounds` 或多轮重复等待 |
|
|
133
|
-
| `aifield_overrides` | 变量系统字段被 chatflow 内部抽取/赋值覆盖时必须看 |
|
|
134
|
-
| `sub_graph_reply_turns` / `sub_graph_path` | SUB_GRAPH badcase 要同时看外层图和内层图 |
|
|
135
|
-
|
|
136
|
-
注意:
|
|
137
|
-
|
|
138
|
-
- `USER_INPUT`、`AGENT_TOOL` 是运行时注入节点,可能不在前端原始 graph 中。
|
|
139
|
-
- `WAIT` 是 `sourceHandle` / `output_handle`,不是业务节点类型。看到 `output_handle=WAIT` 通常表示当前业务节点进入等待用户下一轮输入。
|
|
140
|
-
- 节点名字可能来自 `data.func_name` 或 `data.title`。查日志时同时用 node title、node id、node name 搜索。
|
|
141
|
-
- 如果某节点成功执行但没进预期下一节点,优先对齐当前节点的 `output_handle` 和 graph 边的 `sourceHandle`。
|
|
142
|
-
- 如果上游已经通过 tars 提前抽取变量系统字段,chatflow 条件和变量节点可能直接消费 `metadata.ai_fields`,不一定会看到 GPT 发起的抽取日志。
|
|
143
|
-
|
|
144
|
-
---
|
|
145
|
-
|
|
146
|
-
## 按现象排查
|
|
147
|
-
|
|
148
|
-
| 现象 | 先看 | 关键判断 |
|
|
149
|
-
|------|------|----------|
|
|
150
|
-
| 图加载或解析失败 | `get_or_load_graph_def`、graph 配置、`graph-spec.md` | 是 graph 数据缺失、JSON 不合法、节点字段不符合约束,还是后端 parser 规则拦截 |
|
|
151
|
-
| 节点没进入 | 上一节点 `output_handle/goto`、graph 边 `sourceHandle` | 上一节点是否走了预期 handle;graph 是否缺对应边 |
|
|
152
|
-
| 条件分支不对 | 变量来源、条件配置、`if-else.md` | 变量是否为空;比较符和变量类型是否匹配;是否有 `FALSE` 兜底 |
|
|
153
|
-
| 问题节点卡住 | `QUESTION` 节点结果、`question.md`、`USER_INPUT` | 是正常等待用户输入、选项未命中、必填变量缺失,还是轮次达到上限 |
|
|
154
|
-
| 回复内容不对 | `REPLY` 节点配置、模板/Prompt、上游节点输出 | 固定话术是否应使用模板;是否禁用了上游结果;模板或变量是否缺失 |
|
|
155
|
-
| RAG / KU 没答好 | query、知识目录、召回结果、结果消费 | M10 判断 chatflow 侧 query 和消费;召回内部质量跳 M9 |
|
|
156
|
-
| TOOL 入参或路由不对 | 工具配置、入参映射、原始返回、action status | 入参来源是否正确;工具返回是否包含路由字段;下游是否消费了正确字段 |
|
|
157
|
-
| AGENT 工具调用异常 | `agent.md`、`tool.md`、绑定工具、next_steps | 工具是否绑定;Agent 是否走到工具调用;next_steps 是否和出边匹配 |
|
|
158
|
-
| CODE 节点失败 | `code.md`、resolved inputs、source code、outputs | 先看输入是否解析成功,再看代码运行异常、超时、输出字段/类型 |
|
|
159
|
-
| 变量节点不生效 | `variable-process.md`、变量来源、LLM response、assignments | 抽取和赋值分开判断;确认目标变量系统字段是否被覆盖 |
|
|
160
|
-
| SUB_GRAPH 失败 | 外层 SUB_GRAPH、子图版本、内层执行路径、`sub-graph.md` | 只看外层节点不够,真正失败点可能在内层图 |
|
|
161
|
-
| HUMAN_REVIEW 未继续 | interrupt / resume 日志、`human-review.md` | 是正常等待人审,还是 resume 数据缺失/异常 |
|
|
162
|
-
| graph 未结束 | 最后一个节点、`graph_execution_error`、ERROR | 区分正常等待、配置缺边、节点异常、外部服务异常 |
|
|
163
|
-
|
|
164
|
-
---
|
|
165
|
-
|
|
166
|
-
## Log 层
|
|
167
|
-
|
|
168
|
-
### graph 入口、加载、执行
|
|
169
|
-
|
|
170
|
-
```bash
|
|
171
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
172
|
-
--from <begin> \
|
|
173
|
-
--to <end> \
|
|
174
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"get_or_load_graph_def" OR rest:"graph_execution_start" OR rest:"graph_execution_finished" OR rest:"graph_execution_error")' \
|
|
175
|
-
--line 10
|
|
176
|
-
```
|
|
177
|
-
|
|
178
|
-
### 通用节点更新、节点输入、节点错误
|
|
179
|
-
|
|
180
|
-
```bash
|
|
181
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
182
|
-
--from <begin> \
|
|
183
|
-
--to <end> \
|
|
184
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"node_update" OR rest:"node_input_build_failed" OR rest:"node error" OR rest:"Unhandled node error" OR rest:"node_cancelled")' \
|
|
185
|
-
--line 20
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
### 节点进入、条件、意图
|
|
189
|
-
|
|
190
|
-
```bash
|
|
191
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
192
|
-
--from <begin> \
|
|
193
|
-
--to <end> \
|
|
194
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"node_enter" OR rest:"evaluate_condition" OR rest:"case_matched" OR rest:"no_case_matched" OR rest:"intent classification" OR rest:"load_intent_list")' \
|
|
195
|
-
--line 20
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
### RAG / KU 节点
|
|
199
|
-
|
|
200
|
-
```bash
|
|
201
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
202
|
-
--from <begin> \
|
|
203
|
-
--to <end> \
|
|
204
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"rag_reply query_from_selector" OR rest:"rag_summary" OR rest:"rag_reply retrieve" OR rest:"retrieve_query" OR rest:"kb_reply retrieve")' \
|
|
205
|
-
--line 20
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
### 工具、Agent 工具、变量
|
|
209
|
-
|
|
210
|
-
```bash
|
|
211
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
212
|
-
--from <begin> \
|
|
213
|
-
--to <end> \
|
|
214
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"tool_node args" OR rest:"tool_node result" OR rest:"tool_node routing" OR rest:"agent_tool_node" OR rest:"tool_executed" OR rest:"selected variable not found" OR rest:"variable extract" OR rest:"variable_assignment" OR rest:"variable_process")' \
|
|
215
|
-
--line 20
|
|
216
|
-
```
|
|
217
|
-
|
|
218
|
-
### 代码、子图、人审、等待
|
|
219
|
-
|
|
220
|
-
```bash
|
|
221
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
222
|
-
--from <begin> \
|
|
223
|
-
--to <end> \
|
|
224
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"code_node" OR rest:"sub_graph" OR rest:"human_review" OR rest:"_user_input" OR rest:"output_handle" OR rest:"enter finish node")' \
|
|
225
|
-
--line 20
|
|
226
|
-
```
|
|
227
|
-
|
|
228
|
-
### fallback / ERROR
|
|
229
|
-
|
|
230
|
-
```bash
|
|
231
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
232
|
-
--from <begin> \
|
|
233
|
-
--to <end> \
|
|
234
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"llm_call_with_fallbacks" OR severity:"ERROR")' \
|
|
235
|
-
--line 20
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
如果 `armsTrace` 不可用,用 `chatId`、graphId / runId、接口路径、节点名和时间窗组合查询:
|
|
239
|
-
|
|
240
|
-
```bash
|
|
241
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
242
|
-
--from <begin> \
|
|
243
|
-
--to <end> \
|
|
244
|
-
--query 'rest:"<chatId>" AND (rest:"graph_execution" OR rest:"node_update" OR rest:"node_enter" OR rest:"tool_node")' \
|
|
245
|
-
--line 20
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
---
|
|
249
|
-
|
|
250
|
-
## 报告口径
|
|
251
|
-
|
|
252
|
-
最终报告不要堆源码名。用业务语言先说发生了什么,再说明为什么,并给出最小证据:
|
|
253
|
-
|
|
254
|
-
- 必须写清楚节点 title / id / type,以及它前后走了哪条边。
|
|
255
|
-
- 配置问题标 `[可自行调整]`,说明用户可在后台检查哪个节点、模板、变量、工具或知识配置。
|
|
256
|
-
- 系统异常、parser 错误、服务 ERROR、外部接口异常标 `[需联系研发]`,给出 trace、armsTrace、节点名和关键日志。
|
|
257
|
-
- 如果只是停在 `USER_INPUT` 或人审等待,先说明这是等待态;只有超时、resume 失败、用户输入没有恢复执行时才判异常。
|
|
258
|
-
- 如果 NOTE 未连接、未从 START 可达,不要作为运行时根因;NOTE 是画布注释。
|
|
259
|
-
- 如果日志没查到,写“日志未能确认:<原因>”,不要伪造日志结论。
|
|
@@ -1,148 +0,0 @@
|
|
|
1
|
-
# M11:算法服务通用排查
|
|
2
|
-
|
|
3
|
-
**职责代码**:ShulexService 的 intent、NER、classification、query rewrite、AI reply、input check、response check 等算法接口。
|
|
4
|
-
|
|
5
|
-
**服务路径**:`/Users/jianghao/PycharmProjects/ShulexService`
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## 职责与边界
|
|
10
|
-
|
|
11
|
-
M11 是算法服务公共兜底手册,用于确认 ShulexService 是否收到请求、入参上下文是否正确、是否发生 fallback / 超时 / ERROR。
|
|
12
|
-
|
|
13
|
-
优先级:
|
|
14
|
-
- 意图识别问题先看 M2,只有确认落到 ShulexService 算法接口时才进入 M11。
|
|
15
|
-
- 变量系统 / NER / 分类问题先看 M4,算法侧通用日志再用 M11。
|
|
16
|
-
- 知识召回内部逻辑先看 M9;只有 embedding / rerank / LLM fallback 明确落到算法服务时才进入 M11。
|
|
17
|
-
- ChatFlow 内部节点问题先看 M10,节点调用算法接口失败时再进入 M11。
|
|
18
|
-
|
|
19
|
-
不要把所有下游异常都归到 M11;M11 只回答“算法服务这一层是否正常”。
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## 关键入口
|
|
24
|
-
|
|
25
|
-
关键 router:
|
|
26
|
-
|
|
27
|
-
- `app/router/intent_identify.py`
|
|
28
|
-
- `app/router/ner.py`
|
|
29
|
-
- `app/router/query.py`
|
|
30
|
-
- `app/router/reply.py`
|
|
31
|
-
- `app/router/check.py`
|
|
32
|
-
- `app/router/agent.py`
|
|
33
|
-
|
|
34
|
-
常见接口 / 能力:
|
|
35
|
-
|
|
36
|
-
| 能力 | 典型接口或路径 | 常见上游 |
|
|
37
|
-
|------|----------------|---------|
|
|
38
|
-
| 意图识别 | `/v2/intent_identify`、`/intent_pure` | qa-bot / GPT |
|
|
39
|
-
| NER / 实体抽取 | `ner_api/general_ner`、`/ner/text_extract_x` | GPT / tars / chatflow |
|
|
40
|
-
| 分类 | `classification_api/batch_classify` | GPT / 变量系统 |
|
|
41
|
-
| query rewrite | `/query_rewrite` | GPT / qa-bot / chatflow |
|
|
42
|
-
| AI reply | `/ai_reply` | GPT / chatflow |
|
|
43
|
-
| 输入检查 | check router | GPT / chatflow |
|
|
44
|
-
| 输出检查 | check router | GPT / chatflow |
|
|
45
|
-
|
|
46
|
-
---
|
|
47
|
-
|
|
48
|
-
## Trace / 上下文
|
|
49
|
-
|
|
50
|
-
ShulexService 的上下文由中间件从 body/header 提取:
|
|
51
|
-
|
|
52
|
-
- `app/middleware/tenant_context.py`
|
|
53
|
-
- `tenant_id`
|
|
54
|
-
- `request_id`
|
|
55
|
-
- `chat_id`
|
|
56
|
-
- `armtrack`
|
|
57
|
-
- `app/infra/trace_middleware.py`
|
|
58
|
-
- `traceparent`
|
|
59
|
-
- B3 multi header
|
|
60
|
-
- B3 single header
|
|
61
|
-
- `app/infra/http_logging.py`
|
|
62
|
-
- `METHOD path || http_cost_time || ...`
|
|
63
|
-
|
|
64
|
-
排查时要确认:
|
|
65
|
-
- `tenant_id` 是否和 GPT / qa-bot 一致。
|
|
66
|
-
- `chat_id` 是否是当前 badcase 的 chatId。
|
|
67
|
-
- `request_id` 是否能串起一次算法调用。
|
|
68
|
-
- `armtrack` / `armsTrace` 是否能和 GPT 主链路关联。
|
|
69
|
-
|
|
70
|
-
---
|
|
71
|
-
|
|
72
|
-
## Log 层
|
|
73
|
-
|
|
74
|
-
通用日志:
|
|
75
|
-
|
|
76
|
-
```text
|
|
77
|
-
{chat_id} || {request_id} || {api} || request_start
|
|
78
|
-
{chat_id} || {request_id} || {api} || request_end
|
|
79
|
-
{chat_id} || {request_id} || {api} || ERROR
|
|
80
|
-
METHOD path || http_cost_time || ...
|
|
81
|
-
llm_call_with_fallbacks
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
查询模板:
|
|
85
|
-
|
|
86
|
-
```bash
|
|
87
|
-
# 按 armsTrace 查算法服务目标接口
|
|
88
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
89
|
-
--from <begin> \
|
|
90
|
-
--to <end> \
|
|
91
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"request_start" OR rest:"request_end" OR rest:"ERROR")' \
|
|
92
|
-
--line 20
|
|
93
|
-
|
|
94
|
-
# 按 chatId / request_id 兜底
|
|
95
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
96
|
-
--from <begin> \
|
|
97
|
-
--to <end> \
|
|
98
|
-
--query '(rest:"<chatId>" OR rest:"<request_id>") AND (rest:"request_start" OR rest:"request_end" OR rest:"ERROR")' \
|
|
99
|
-
--line 20
|
|
100
|
-
|
|
101
|
-
# HTTP 耗时
|
|
102
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
103
|
-
--from <begin> \
|
|
104
|
-
--to <end> \
|
|
105
|
-
--query 'armsTrace:"<armsTrace>" AND rest:"http_cost_time"' \
|
|
106
|
-
--line 20
|
|
107
|
-
|
|
108
|
-
# LLM fallback
|
|
109
|
-
npx --yes @shulex/cxclawctl@latest sls query \
|
|
110
|
-
--from <begin> \
|
|
111
|
-
--to <end> \
|
|
112
|
-
--query 'armsTrace:"<armsTrace>" AND rest:"llm_call_with_fallbacks"' \
|
|
113
|
-
--line 20
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
按能力缩小查询:
|
|
117
|
-
|
|
118
|
-
```bash
|
|
119
|
-
# 意图识别
|
|
120
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"intent_identify" OR rest:"intent_pure")'
|
|
121
|
-
|
|
122
|
-
# NER / 实体
|
|
123
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"general_ner" OR rest:"text_extract_x" OR rest:"ner")'
|
|
124
|
-
|
|
125
|
-
# 分类
|
|
126
|
-
--query 'armsTrace:"<armsTrace>" AND (rest:"batch_classify" OR rest:"classification")'
|
|
127
|
-
|
|
128
|
-
# query rewrite
|
|
129
|
-
--query 'armsTrace:"<armsTrace>" AND rest:"query_rewrite"'
|
|
130
|
-
|
|
131
|
-
# AI reply
|
|
132
|
-
--query 'armsTrace:"<armsTrace>" AND rest:"ai_reply"'
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
---
|
|
136
|
-
|
|
137
|
-
## 常见根因与建议
|
|
138
|
-
|
|
139
|
-
| 根因 | 建议 |
|
|
140
|
-
|------|------|
|
|
141
|
-
| 没有 `request_start` | `[需联系研发]` 上游可能未发起调用、trace 未串上,回对应模块确认 GPT / qa-bot / chatflow 调用点 |
|
|
142
|
-
| 有 `request_start` 无 `request_end` | `[需联系研发]` 算法接口超时或中途异常,提供 `request_id` 给算法侧 |
|
|
143
|
-
| 有 `ERROR` | `[需联系研发]` 提供 ERROR 日志、接口名、request_id、chat_id |
|
|
144
|
-
| 入参 tenant / chatId 不一致 | `[需联系研发]` 检查上游透传和 ShulexService tenant_context |
|
|
145
|
-
| `llm_call_with_fallbacks` 频繁 fallback | `[需联系研发]` 检查模型供应商、限流、超时和 fallback 配置 |
|
|
146
|
-
| `http_cost_time` 明显过高 | `[需联系研发]` 判断算法服务慢、下游模型慢,或上游等待超时 |
|
|
147
|
-
| 算法返回空但无 ERROR | `[需联系研发]` 提供入参与输出,交算法侧判断模型效果或阈值 |
|
|
148
|
-
| 算法结果正确但 GPT 侧未使用 | 回到对应业务模块:意图回 M2,变量系统回 M4,回复回 M7,ChatFlow 回 M10 |
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
# M12:回复后质检排查
|
|
2
|
-
|
|
3
|
-
M12 处理 Agent 生成回复之后的实时/后置质检:是否送检、命中什么标签、标签是否影响最终 AgentAction,以及质检服务是否调用失败。它不分析 ChatFlow 图或节点业务逻辑;图内执行异常先走 M10。
|
|
4
|
-
|
|
5
|
-
## 边界
|
|
6
|
-
|
|
7
|
-
- 质检可能记录标签,也可能根据命中结果改变回复、拦截回复或转人工;必须先确认实际影响,不能把“命中标签”等同于“回复被拦截”。
|
|
8
|
-
- `quality-inspection-gate` 是服务端过滤/灰度控制,不是租户的质检规则配置。badcase 排查只取运行证据,不修改它。
|
|
9
|
-
- 当前系统规则、自定义规则和开关只用于和历史结果对照,不是目标轮次的配置快照,也不能单独证明历史命中原因。
|
|
10
|
-
|
|
11
|
-
## 排查顺序
|
|
12
|
-
|
|
13
|
-
### 1. 对齐会话记录中的目标事实
|
|
14
|
-
|
|
15
|
-
先读取目标回复及其 `messages[].context`,检查:
|
|
16
|
-
|
|
17
|
-
| 证据 | 说明 |
|
|
18
|
-
|---|---|
|
|
19
|
-
| `context.metadata.real_time_inspection_tags` | 实时质检返回的标签,是优先证据 |
|
|
20
|
-
| `messages[].reason` / `reasonDesc` / `reasonList` | AgentAction 的原因及多原因列表 |
|
|
21
|
-
| `messages[].type` / `handoff` | 最终是否变为 `TO_AGENT`、`STOPPED` 或转人工 |
|
|
22
|
-
| `context.aiContent`、`filteredContent` 与最终 `content` | 候选回复是否被后处理改变或清空 |
|
|
23
|
-
|
|
24
|
-
先写清“生成了什么”和“最终展示了什么”。标签存在但最终 content、type 和 handoff 未受影响时,只能结论为“已记录命中,未证明拦截”。
|
|
25
|
-
|
|
26
|
-
### 2. 判断是否进入 M12
|
|
27
|
-
|
|
28
|
-
- 有 `real_time_inspection_tags`,或 reason 明确指向合规/质检:进入 M12。
|
|
29
|
-
- 回复被拦截、转人工或清空,但没有标签:先在 M12 确认是否送检;没有送检事实再回 M7/M8/M11。
|
|
30
|
-
- ChatFlow 节点本身失败:回 M10;不要把节点失败归为质检。
|
|
31
|
-
- 回复内容本身不符合业务预期、且最终未被质检改变:回 M7。
|
|
32
|
-
|
|
33
|
-
### 3. 对照当前质检配置
|
|
34
|
-
|
|
35
|
-
需要说明规则语义或复现建议时,可读取 solvea-kit 的质检管理说明和当前配置,核对总开关、系统规则、自定义规则及启用状态。
|
|
36
|
-
|
|
37
|
-
当前配置只能回答“现在什么规则会生效”。若会话记录没有保存当时规则版本或具体命中详情,报告必须写为“当前规则对照”,不能写成历史根因。
|
|
38
|
-
|
|
39
|
-
### 4. 最后才查日志
|
|
40
|
-
|
|
41
|
-
会话记录不能说明送检、标签来源、拦截原因或调用异常时,按 `sls-query.md` 的 M12 模板查日志:先查 GPT 的 `QualityCheckPostCheck`、`realTimeInspection`、`replyInspection`、`inspectionDataResponse`、豁免及 gate 线索;GPT 已确认外调失败时,再通过 `armsTrace` 查 Intelli。
|
|
42
|
-
|
|
43
|
-
日志查询次数和止损规则遵守 `log-query-rules.md`。没有日志证据时,不要从“没有标签”断言该轮一定未送检。
|
|
44
|
-
|
|
45
|
-
## 报告口径
|
|
46
|
-
|
|
47
|
-
只使用证据能支持的结论:
|
|
48
|
-
|
|
49
|
-
| 结论 | 最低证据 |
|
|
50
|
-
|---|---|
|
|
51
|
-
| 命中质检且影响 AgentAction | 标签/质检日志 + 最终 reason、type 或 handoff 的对应变化 |
|
|
52
|
-
| 命中质检但只记录、未拦截 | 有标签,且最终回复和动作未见质检影响 |
|
|
53
|
-
| 未送检 | 明确的豁免、过滤或未调用日志;不能只凭标签为空 |
|
|
54
|
-
| 质检调用失败 | GPT 或 Intelli 的实际异常日志 |
|
|
55
|
-
| 无法确认历史命中原因 | 仅能读取当前规则,缺少当时标签、规则快照或日志 |
|
|
56
|
-
|
|
57
|
-
不要把“质检命中”表述为原始 ChatFlow、意图、知识或 Agent 生成错误的根因;它是回复后的独立处理结果。建议修改规则时应保留原业务口径,并用原 badcase 回归验证。
|
|
@@ -1,120 +0,0 @@
|
|
|
1
|
-
# M2:意图识别排查
|
|
2
|
-
|
|
3
|
-
M2 负责确认用户表达经过查询整理后,被识别成了什么意图,以及这个结果是否符合当前意图配置。先还原目标轮次的事实,再读取相关意图;不要从最终命中的策略或回复倒推意图识别一定正确。
|
|
4
|
-
|
|
5
|
-
---
|
|
6
|
-
|
|
7
|
-
## 边界
|
|
8
|
-
|
|
9
|
-
M2 只处理进入策略匹配之前的问题:
|
|
10
|
-
|
|
11
|
-
- 用户原话经过整理或翻译后语义发生偏移。
|
|
12
|
-
- 原始识别意图与用户核心诉求不一致。
|
|
13
|
-
- 原始识别结果映射到项目意图后出现偏差。
|
|
14
|
-
- 正确意图不存在、未启用,或描述边界与相邻意图重叠。
|
|
15
|
-
- 多个候选意图的选择不符合业务预期。
|
|
16
|
-
|
|
17
|
-
如果 `intents[]` 已包含预期意图,但策略为空、命中错误或动作错误,进入 M3;不要继续把策略问题归到 M2。
|
|
18
|
-
|
|
19
|
-
---
|
|
20
|
-
|
|
21
|
-
## 排查顺序
|
|
22
|
-
|
|
23
|
-
### 1. 对齐会话记录中的目标事实
|
|
24
|
-
|
|
25
|
-
只读取目标用户消息、对应回复和必要的前置轮次:
|
|
26
|
-
|
|
27
|
-
| 字段 | 要回答的问题 |
|
|
28
|
-
|------|--------------|
|
|
29
|
-
| 用户消息 `content`,以及邮件的 `subject/snippet` | 用户真正的主诉是什么;订单号、产品名等背景词是否掩盖了核心动作 |
|
|
30
|
-
| `context.mixedQuery.userQuery/userEnQuery` | 整理或翻译后的问题是否仍保持原意 |
|
|
31
|
-
| `context.originIntent` | 当轮保存的原始识别名称是什么 |
|
|
32
|
-
| `context.intents[]` | 实际落到项目配置中的候选意图有哪些,顺序如何 |
|
|
33
|
-
| `context.intent` | 兼容字段中的最终意图;通常与 `intents[0].name` 对齐 |
|
|
34
|
-
| 回复 `type` | 是正常进入后续处理,还是 `CONFUSED` 等未完成路由状态 |
|
|
35
|
-
| `contextComparison.contextModifiedList` | 相关意图是否在会话发生后被修改或删除 |
|
|
36
|
-
|
|
37
|
-
按下面的顺序形成事实链:
|
|
38
|
-
|
|
39
|
-
1. 用一句话概括用户核心诉求,不直接复用系统生成的摘要。
|
|
40
|
-
2. 对比原话与 `mixedQuery`,确认偏差是否在识别前已经出现。
|
|
41
|
-
3. 对比 `originIntent`、`intents[]` 和 `intent`,指出偏差发生在原始识别还是配置意图映射。
|
|
42
|
-
4. 再看策略字段;只有意图结果已正确时才转 M3。
|
|
43
|
-
|
|
44
|
-
`originIntent` 与 `intent` 不同,不代表策略把意图过滤或改写了。它们分别是原始识别名称和项目配置中的结果,差异本身只是一条待解释的映射信号。
|
|
45
|
-
|
|
46
|
-
### 2. 读取当前配置
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
npx --yes @shulex/cxclawctl@latest api list-intents --pretty
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
先按 ID / 名称找到当轮实际意图和用户预期意图;需要确认单个意图完整信息时再执行:
|
|
53
|
-
|
|
54
|
-
```bash
|
|
55
|
-
npx --yes @shulex/cxclawctl@latest api get-intent --id <intent_id> --pretty
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
逐项比较:
|
|
59
|
-
|
|
60
|
-
- `status`:实际意图和预期意图是否启用。
|
|
61
|
-
- `description`:是否覆盖用户这类表达,并明确适用范围与排除边界;`get-intent` 详情中的对应字段名是 `example`。
|
|
62
|
-
- 相邻意图:名称或描述是否同时覆盖“对象相同、动作不同”的混合表达。
|
|
63
|
-
- 重复配置:是否存在同名、近义或新旧两套意图,状态和用途不同。
|
|
64
|
-
|
|
65
|
-
不要只看名称判断语义,也不要批量扫完所有意图后笼统说“模型区分能力不足”。报告必须引用与本条问题直接相关的实际意图、预期意图及其描述差异。
|
|
66
|
-
|
|
67
|
-
### 3. 结合字段值判断
|
|
68
|
-
|
|
69
|
-
| 会话事实 | 判断 |
|
|
70
|
-
|----------|------|
|
|
71
|
-
| `mixedQuery` 已改变用户核心诉求 | 偏差发生在识别前的问题整理;不能仅靠修改意图描述解释 |
|
|
72
|
-
| `originIntent` 已经与用户诉求不符 | 原始识别错误;再用当前相关意图描述判断是覆盖不足、边界重叠,还是仍缺历史证据 |
|
|
73
|
-
| `originIntent` 合理,但 `intents[]/intent` 落到另一配置意图 | 关注名称映射、可用意图集合或覆盖结果;不要归因到策略条件 |
|
|
74
|
-
| `intents[]` 含预期意图,但第一项不是预期结果 | 记录候选顺序;当前配置只能解释语义边界,不能证明当时的排序原因 |
|
|
75
|
-
| `intents[]` 正确,但 `strategyName` 为空或错误 | M2 已完成,转 M3 |
|
|
76
|
-
| `originIntent` 为空、`intents[]` 非空 | 只能确认保存结果不完整或存在兜底/覆盖,不能直接断言“没有执行识别” |
|
|
77
|
-
| `type=CONFUSED` 且 `intents[]` 非空 | 意图已有结果但后续未完成路由,优先转 M3 |
|
|
78
|
-
| 当前预期意图为 `DISABLED` | 说明当前不可命中;只有历史变更证据能证明当时也是该状态 |
|
|
79
|
-
|
|
80
|
-
### 4. 最后才查日志
|
|
81
|
-
|
|
82
|
-
第一版诊断默认停在会话事实和当前配置。只有以下问题仍无法回答时,才读取 `sls-query.md` 做聚焦查询:
|
|
83
|
-
|
|
84
|
-
- 无法确认当轮使用的是正常识别、上轮复用、指定意图还是兜底结果。
|
|
85
|
-
- 需要证明当轮实际送入识别的候选意图和描述。
|
|
86
|
-
- 需要确认原始识别结果、候选顺序或下游返回是否为空。
|
|
87
|
-
- 会话记录与当前配置明显矛盾,且没有历史变更信息。
|
|
88
|
-
|
|
89
|
-
需要继续判断算法请求和返回时转 M11。没有日志证据时,把这些内容写成“待确认”,不要用当前配置补成历史事实。
|
|
90
|
-
|
|
91
|
-
---
|
|
92
|
-
|
|
93
|
-
## 历史与当前状态边界
|
|
94
|
-
|
|
95
|
-
- `messages[].context` 是当轮保存的结果,优先作为历史事实。
|
|
96
|
-
- `list-intents/get-intent` 读取的是当前配置,只能解释现在的状态和语义边界。
|
|
97
|
-
- `contextComparison` 是当前资源与历史记录的比较,不是重新执行意图识别。
|
|
98
|
-
- 当前描述更合理,不代表当轮模型一定见过同一版本;当前描述不合理,也不能单独证明它就是历史误判原因。
|
|
99
|
-
|
|
100
|
-
---
|
|
101
|
-
|
|
102
|
-
## 常见根因与建议
|
|
103
|
-
|
|
104
|
-
| 根因 | 建议 |
|
|
105
|
-
|------|------|
|
|
106
|
-
| 用户核心诉求在 `mixedQuery` 中被改写偏 | `[需联系研发]` 提供原话与改写结果,继续定位问题整理环节 |
|
|
107
|
-
| 正确意图当前未启用 | `[可自行调整]` 确认业务范围后启用,并补齐对应策略 |
|
|
108
|
-
| 意图描述覆盖不足 | `[可自行调整]` 增加真实表达和适用边界,不只堆关键词 |
|
|
109
|
-
| 相邻意图描述重叠 | `[可自行调整]` 同时补充正向范围和排除条件,区分对象与用户动作 |
|
|
110
|
-
| 原始识别正确但配置意图结果异常 | `[需联系研发]` 提供 `originIntent`、`intents[]` 和相关意图 ID,继续确认映射过程 |
|
|
111
|
-
| 意图结果正确但没有进入预期策略 | 转 M3,不修改意图来掩盖策略配置问题 |
|
|
112
|
-
|
|
113
|
-
---
|
|
114
|
-
|
|
115
|
-
## 报告口径
|
|
116
|
-
|
|
117
|
-
- 先说用户主诉、`mixedQuery`、`originIntent`、`intents[]` 分别是什么,再说偏差发生在哪一步。
|
|
118
|
-
- 引用实际意图和预期意图的当前状态与描述;描述读取失败时明确写“配置未确认”。
|
|
119
|
-
- 不要把 `originIntent != intent` 写成策略阶段修改了意图。
|
|
120
|
-
- 没有历史版本或日志时,不要断言当前意图描述就是当轮误判根因。
|