@shulex/cxclawctl 0.1.2 → 0.1.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (208) hide show
  1. package/README.md +25 -22
  2. package/capabilities.lock.json +535 -131
  3. package/dist/cli.js +2203 -127
  4. package/dist/cli.js.map +1 -1
  5. package/package.json +20 -6
  6. package/requirements-cli.txt +0 -2
  7. package/skills/__lib__/__init__.py +0 -0
  8. package/skills/__lib__/skill-runtime/bootstrap.py +0 -134
  9. package/skills/__lib__/solvea-api/__init__.py +0 -0
  10. package/skills/__lib__/solvea-api/client.py +0 -1944
  11. package/skills/__lib__/solvea-api/dashboard.py +0 -100
  12. package/skills/__lib__/solvea-api/solvea_api.py +0 -1341
  13. package/skills/agent-analytics/SKILL.md +0 -74
  14. package/skills/agent-analytics/references/chat-report-fields.md +0 -36
  15. package/skills/agent-analytics/references/report-fields.md +0 -130
  16. package/skills/agent-analytics/references/transcript.md +0 -110
  17. package/skills/agent-builder/SKILL.md +0 -42
  18. package/skills/agent-builder/interfaces/fallback-chatflow.md +0 -47
  19. package/skills/agent-builder/references/chatflow/graph-spec.md +0 -259
  20. package/skills/agent-builder/references/chatflow/node-selection.md +0 -89
  21. package/skills/agent-builder/references/chatflow/nodes/agent.md +0 -350
  22. package/skills/agent-builder/references/chatflow/nodes/business-track.md +0 -43
  23. package/skills/agent-builder/references/chatflow/nodes/code.md +0 -157
  24. package/skills/agent-builder/references/chatflow/nodes/collect-order.md +0 -47
  25. package/skills/agent-builder/references/chatflow/nodes/finish.md +0 -34
  26. package/skills/agent-builder/references/chatflow/nodes/human-review.md +0 -60
  27. package/skills/agent-builder/references/chatflow/nodes/if-else.md +0 -100
  28. package/skills/agent-builder/references/chatflow/nodes/intent.md +0 -79
  29. package/skills/agent-builder/references/chatflow/nodes/knowledge-reply.md +0 -43
  30. package/skills/agent-builder/references/chatflow/nodes/llm.md +0 -117
  31. package/skills/agent-builder/references/chatflow/nodes/note.md +0 -38
  32. package/skills/agent-builder/references/chatflow/nodes/question.md +0 -97
  33. package/skills/agent-builder/references/chatflow/nodes/rag-reply.md +0 -68
  34. package/skills/agent-builder/references/chatflow/nodes/reply.md +0 -97
  35. package/skills/agent-builder/references/chatflow/nodes/start.md +0 -35
  36. package/skills/agent-builder/references/chatflow/nodes/sub-graph.md +0 -48
  37. package/skills/agent-builder/references/chatflow/nodes/tool.md +0 -138
  38. package/skills/agent-builder/references/chatflow/nodes/transfer-to-human.md +0 -63
  39. package/skills/agent-builder/references/chatflow/nodes/variable-assignment.md +0 -52
  40. package/skills/agent-builder/references/chatflow/nodes/variable-extract.md +0 -50
  41. package/skills/agent-builder/references/chatflow/nodes/variable-process.md +0 -70
  42. package/skills/agent-builder/references/chatflow/overview.md +0 -53
  43. package/skills/agent-builder/references/chatflow/visual-translation.md +0 -64
  44. package/skills/agent-builder/references/chatflow/workflow.md +0 -134
  45. package/skills/agent-builder/references/multi-agent/create-update.md +0 -185
  46. package/skills/agent-builder/references/multi-agent/graph-spec.md +0 -246
  47. package/skills/agent-builder/references/multi-agent/graph-structure.md +0 -12
  48. package/skills/agent-builder/references/multi-agent/migrate-to-chatflow.md +0 -13
  49. package/skills/agent-builder/references/multi-agent/migration/large.md +0 -554
  50. package/skills/agent-builder/references/multi-agent/migration/principles.md +0 -446
  51. package/skills/agent-builder/references/multi-agent/migration/small.md +0 -225
  52. package/skills/agent-builder/references/multi-agent/orchestration.md +0 -11
  53. package/skills/agent-builder/references/multi-agent/overview.md +0 -48
  54. package/skills/agent-builder/references/multi-agent/update.md +0 -128
  55. package/skills/agent-builder/references/multi-agent/workflow.md +0 -87
  56. package/skills/agent-builder/references/single-agent/overview.md +0 -48
  57. package/skills/agent-builder/references/single-agent/prompt.md +0 -16
  58. package/skills/agent-builder/references/single-agent/tools.md +0 -36
  59. package/skills/agent-builder/references/single-agent/voice.md +0 -46
  60. package/skills/agent-builder/references/tools/http-tool.md +0 -315
  61. package/skills/agent-builder/references/tools/mock-tool.md +0 -279
  62. package/skills/agent-builder/references/tools/overview.md +0 -46
  63. package/skills/agent-builder/references/tools/tool-config.md +0 -43
  64. package/skills/agent-builder/references/tools/x-token.md +0 -34
  65. package/skills/agent-builder/scripts/chatflow/chatflow_builder.py +0 -3458
  66. package/skills/agent-builder/scripts/chatflow/validate_graph.py +0 -1697
  67. package/skills/agent-builder/scripts/copy_marketplace_agent.py +0 -61
  68. package/skills/agent-builder/scripts/multi-agent/legacy_graph_analyzer.py +0 -5974
  69. package/skills/agent-builder/scripts/multi-agent/replay_compare.py +0 -679
  70. package/skills/agent-builder/scripts/patch_agent_tools.py +0 -135
  71. package/skills/agent-builder/scripts/tools/validate_tool_config.py +0 -384
  72. package/skills/agent-tester/SKILL.md +0 -35
  73. package/skills/agent-tester/batch-test/overview.md +0 -86
  74. package/skills/agent-tester/scripts/batch_test_sheet.py +0 -213
  75. package/skills/agent-tester/single-test/badcase-retest.md +0 -85
  76. package/skills/agent-tester/single-test/direct-test.md +0 -24
  77. package/skills/agent-tester/single-test/overview.md +0 -142
  78. package/skills/delivery/SKILL.md +0 -112
  79. package/skills/delivery/interfaces/classify.md +0 -38
  80. package/skills/delivery/interfaces/create-agents.md +0 -89
  81. package/skills/delivery/interfaces/delivery-test.md +0 -67
  82. package/skills/delivery/references/flow.md +0 -40
  83. package/skills/delivery/references/validation.md +0 -15
  84. package/skills/delivery/scripts/batch_test.py +0 -704
  85. package/skills/delivery/scripts/classify.py +0 -22
  86. package/skills/delivery/scripts/create_agents.py +0 -695
  87. package/skills/delivery/scripts/gen_delivery_report.py +0 -335
  88. package/skills/delivery/scripts/gen_excel.py +0 -234
  89. package/skills/delivery/scripts/gen_report.py +0 -208
  90. package/skills/delivery/scripts/plan.py +0 -163
  91. package/skills/delivery/scripts/render.py +0 -713
  92. package/skills/delivery/scripts/render_summary.py +0 -38
  93. package/skills/delivery/scripts/sample_history_cases.py +0 -109
  94. package/skills/kb-builder/SKILL.md +0 -80
  95. package/skills/kb-builder/interfaces/qa-upload.md +0 -55
  96. package/skills/kb-builder/interfaces/web-kb.md +0 -84
  97. package/skills/kb-builder/references/directory.md +0 -373
  98. package/skills/kb-builder/references/export.md +0 -282
  99. package/skills/kb-builder/references/isolation-keywords.md +0 -113
  100. package/skills/kb-builder/references/knowledge.md +0 -153
  101. package/skills/kb-builder/references/product.md +0 -52
  102. package/skills/kb-builder/references/qa-from-help-center.md +0 -58
  103. package/skills/kb-builder/references/qa-from-pdf.md +0 -94
  104. package/skills/kb-builder/references/qa-from-tickets.md +0 -168
  105. package/skills/kb-builder/references/qa-from-web.md +0 -98
  106. package/skills/kb-builder/references/qa-quality.md +0 -73
  107. package/skills/kb-builder/references/reply-template.md +0 -237
  108. package/skills/kb-builder/references/tag-isolation-migration.md +0 -61
  109. package/skills/kb-builder/references/tags.md +0 -45
  110. package/skills/kb-builder/references/upload.md +0 -130
  111. package/skills/kb-builder/scripts/batch_extract_ticket_qa.py +0 -333
  112. package/skills/kb-builder/scripts/build_directory_plan.py +0 -144
  113. package/skills/kb-builder/scripts/crawl_site.py +0 -433
  114. package/skills/kb-builder/scripts/create_dirs.py +0 -146
  115. package/skills/kb-builder/scripts/export_knowledge.py +0 -454
  116. package/skills/kb-builder/scripts/extract_ticket_qa.py +0 -441
  117. package/skills/kb-builder/scripts/extract_web_qa.py +0 -118
  118. package/skills/kb-builder/scripts/import_qa.py +0 -219
  119. package/skills/kb-builder/scripts/import_source_documents.py +0 -143
  120. package/skills/kb-builder/scripts/import_templates.py +0 -409
  121. package/skills/poc/SKILL.md +0 -125
  122. package/skills/poc/assets/default_manual.css +0 -279
  123. package/skills/poc/assets/notice_escalate.png +0 -0
  124. package/skills/poc/assets/notice_escalate2.png +0 -0
  125. package/skills/poc/assets/notice_new_chat.png +0 -0
  126. package/skills/poc/assets/poc_test_template.html +0 -645
  127. package/skills/poc/assets/poc_test_template.md +0 -104
  128. package/skills/poc/assets/step3_preview.png +0 -0
  129. package/skills/poc/assets/step4_order_note.png +0 -0
  130. package/skills/poc/interfaces/classify.md +0 -38
  131. package/skills/poc/interfaces/create-agents.md +0 -108
  132. package/skills/poc/interfaces/poc-test.md +0 -94
  133. package/skills/poc/references/flow.md +0 -47
  134. package/skills/poc/references/kb-directory.md +0 -22
  135. package/skills/poc/references/validation.md +0 -17
  136. package/skills/poc/scripts/classify.py +0 -22
  137. package/skills/poc/scripts/create_agents.py +0 -818
  138. package/skills/poc/scripts/gen_excel.py +0 -234
  139. package/skills/poc/scripts/gen_poc_manual.py +0 -249
  140. package/skills/poc/scripts/gen_report.py +0 -208
  141. package/skills/poc/scripts/inspect_chatflow.py +0 -209
  142. package/skills/poc/scripts/plan.py +0 -178
  143. package/skills/poc/scripts/poc_test.py +0 -495
  144. package/skills/poc/scripts/render.py +0 -713
  145. package/skills/poc/scripts/render_summary.py +0 -41
  146. package/skills/runtime-inspector/SKILL.md +0 -51
  147. package/skills/runtime-inspector/references/frontend-rendering.md +0 -126
  148. package/skills/runtime-inspector/references/model.md +0 -59
  149. package/skills/runtime-inspector/references/sandbox.md +0 -131
  150. package/skills/runtime-inspector/scripts/model_inspect.py +0 -68
  151. package/skills/runtime-inspector/scripts/prepare.py +0 -98
  152. package/skills/runtime-inspector/scripts/process.py +0 -84
  153. package/skills/runtime-inspector/scripts/read_state.py +0 -32
  154. package/skills/runtime-inspector/scripts/sandbox_inspect.py +0 -463
  155. package/skills/runtime-inspector/scripts/test_claude_proxy.py +0 -49
  156. package/skills/runtime-inspector/scripts/test_openai_proxy.py +0 -41
  157. package/skills/runtime-inspector/scripts/verify_persistence.py +0 -101
  158. package/skills/runtime-inspector/scripts/write_test.py +0 -41
  159. package/skills/setup/SKILL.md +0 -17
  160. package/skills/shulex-support/SKILL.md +0 -39
  161. package/skills/shulex-support/references/badcase-analysis/overview.md +0 -34
  162. package/skills/shulex-support/references/badcase-analysis/text/conversation-record.md +0 -105
  163. package/skills/shulex-support/references/badcase-analysis/text/core-flow.md +0 -89
  164. package/skills/shulex-support/references/badcase-analysis/text/log-query-rules.md +0 -36
  165. package/skills/shulex-support/references/badcase-analysis/text/m1-entry.md +0 -53
  166. package/skills/shulex-support/references/badcase-analysis/text/m10-chatflow-service.md +0 -259
  167. package/skills/shulex-support/references/badcase-analysis/text/m11-algorithm-service.md +0 -148
  168. package/skills/shulex-support/references/badcase-analysis/text/m12-quality-inspection.md +0 -57
  169. package/skills/shulex-support/references/badcase-analysis/text/m2-intent.md +0 -120
  170. package/skills/shulex-support/references/badcase-analysis/text/m3-strategy.md +0 -126
  171. package/skills/shulex-support/references/badcase-analysis/text/m4-aifield.md +0 -121
  172. package/skills/shulex-support/references/badcase-analysis/text/m5-agent.md +0 -208
  173. package/skills/shulex-support/references/badcase-analysis/text/m6-tool.md +0 -115
  174. package/skills/shulex-support/references/badcase-analysis/text/m7-reply.md +0 -72
  175. package/skills/shulex-support/references/badcase-analysis/text/m8-error.md +0 -92
  176. package/skills/shulex-support/references/badcase-analysis/text/m9-knowledge-search.md +0 -147
  177. package/skills/shulex-support/references/badcase-analysis/text/sls-query.md +0 -283
  178. package/skills/shulex-support/references/badcase-analysis/voice/core-flow.md +0 -77
  179. package/skills/shulex-support/references/badcase-analysis/voice/dependencies.md +0 -19
  180. package/skills/shulex-support/references/badcase-analysis/voice/platform-api.md +0 -23
  181. package/skills/shulex-support/references/badcase-analysis/voice/sls-playbook.md +0 -31
  182. package/skills/shulex-support/references/product-docs.md +0 -70
  183. package/skills/shulex-support/references/tickets.md +0 -184
  184. package/skills/shulex-support/scripts/support.py +0 -427
  185. package/skills/solvea-desk/SKILL.md +0 -39
  186. package/skills/solvea-desk/references/report/fields.md +0 -82
  187. package/skills/solvea-desk/references/report/report-design.md +0 -200
  188. package/skills/solvea-desk/references/report/report-template.html +0 -44
  189. package/skills/solvea-desk/references/report/report.css +0 -517
  190. package/skills/solvea-desk/references/report/workflow.md +0 -464
  191. package/skills/solvea-desk/references/tickets/query.md +0 -103
  192. package/skills/solvea-desk/references/tickets/tags.md +0 -83
  193. package/skills/solvea-desk/scripts/report/fetch_sources.py +0 -580
  194. package/skills/solvea-desk/scripts/report/render_report.py +0 -763
  195. package/skills/solvea-desk/scripts/tickets/desk_api.py +0 -73
  196. package/skills/solvea-desk/scripts/tickets/fetch_tickets.py +0 -403
  197. package/skills/solvea-desk/scripts/tickets/ticket_tags.py +0 -305
  198. package/skills/solvea-kit/SKILL.md +0 -84
  199. package/skills/solvea-kit/references/agent.md +0 -145
  200. package/skills/solvea-kit/references/chat.md +0 -244
  201. package/skills/solvea-kit/references/intent.md +0 -69
  202. package/skills/solvea-kit/references/plugin.md +0 -53
  203. package/skills/solvea-kit/references/quality-inspection.md +0 -75
  204. package/skills/solvea-kit/references/strategy.md +0 -62
  205. package/skills/solvea-kit/references/tagging.md +0 -231
  206. package/skills/solvea-kit/references/variables.md +0 -53
  207. package/skills/solvea-kit/scripts/generate_label_import_file.py +0 -151
  208. package/skills/solvea-kit/scripts/intent_classify.py +0 -843
@@ -1,213 +0,0 @@
1
- #!/usr/bin/env python3
2
- """Create and submit system-compatible Agent Tester batch-test workbooks.
3
-
4
- Input JSON is a list of cases. Chatflow cases accept ``turns`` with ``question``
5
- and optional ``metadata``; ordinary Agent cases accept one ``question``.
6
- """
7
-
8
- import os
9
- import sys
10
- sys.path.insert(0, os.path.join(os.environ.get("SOLVEA_LIB_DIR", os.path.join(os.path.dirname(__file__), "..", "..", "__lib__")), "skill-runtime"))
11
- from bootstrap import * # noqa: F401,F403
12
-
13
- import argparse
14
- import json
15
- from pathlib import Path
16
-
17
- from openpyxl import Workbook
18
- from openpyxl.styles import Alignment, Font, PatternFill
19
- from openpyxl.utils import get_column_letter
20
-
21
- from client import SolveaClient
22
-
23
-
24
- CHATFLOW_HEADERS = ["case_name", "chat_id"]
25
- REPLY_HEADERS = [
26
- "question", "origin_answer", "ai_answer", "similarity", "judgement",
27
- "reason", "chat_id", "meta_data", "fail_reason",
28
- ]
29
-
30
- REPLY_INSTRUCTION = (
31
- "Do not delete this row. Fill question and optional meta_data; all result columns are generated by the system. "
32
- "REPLY meta_data must use plain text key: value pairs separated by semicolons, for example "
33
- "touchPoint: LIVE_CHAT; order_number: A-1001."
34
- )
35
-
36
-
37
- def _metadata_text(value):
38
- if value in (None, ""):
39
- return ""
40
- if isinstance(value, str):
41
- return value
42
- if not isinstance(value, dict):
43
- raise ValueError("turn metadata must be an object or a string")
44
- return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
45
-
46
-
47
- def _reply_metadata_text(value):
48
- """Render metadata in the key/value syntax consumed by the REPLY backend."""
49
- if value in (None, ""):
50
- return ""
51
- if isinstance(value, dict):
52
- return "; ".join(f"{key}: {val}" for key, val in value.items())
53
- if isinstance(value, str):
54
- stripped = value.strip()
55
- if stripped.startswith("{") and stripped.endswith("}"):
56
- try:
57
- parsed = json.loads(stripped)
58
- except json.JSONDecodeError:
59
- parsed = None
60
- if isinstance(parsed, dict):
61
- return _reply_metadata_text(parsed)
62
- return value
63
- raise ValueError("REPLY metadata must be an object or a string")
64
-
65
-
66
- def _load_cases(path):
67
- cases = json.loads(Path(path).read_text(encoding="utf-8"))
68
- if not isinstance(cases, list) or not cases:
69
- raise ValueError("cases JSON must be a non-empty array")
70
- return cases
71
-
72
-
73
- def build_workbook(cases, mode):
74
- if mode not in {"chatflow", "agent"}:
75
- raise ValueError("mode must be chatflow or agent")
76
- if len(cases) > (600 if mode == "chatflow" else 200):
77
- raise ValueError(f"{mode} batch testing supports at most {600 if mode == 'chatflow' else 200} cases")
78
-
79
- workbook = Workbook()
80
- sheet = workbook.active
81
- sheet.title = "Batch Test"
82
- instruction = sheet.cell(row=1, column=1)
83
- instruction.alignment = Alignment(wrap_text=True, vertical="top")
84
- instruction.font = Font(italic=True, color="666666")
85
-
86
- if mode == "chatflow":
87
- normalized = []
88
- max_turns = 0
89
- for index, case in enumerate(cases, start=1):
90
- turns = case.get("turns") if isinstance(case, dict) else None
91
- if not isinstance(turns, list) or not turns:
92
- raise ValueError(f"case {index} requires a non-empty turns array")
93
- normalized_turns = []
94
- for turn in turns:
95
- question = turn.get("question", turn.get("message", "")) if isinstance(turn, dict) else ""
96
- if not isinstance(question, str) or not question.strip():
97
- raise ValueError(f"case {index} contains an empty question")
98
- normalized_turns.append((turn.get("metadata") if isinstance(turn, dict) else None, question.strip()))
99
- max_turns = max(max_turns, len(normalized_turns))
100
- normalized.append((str(case.get("case_name") or f"case_{index}"), normalized_turns))
101
-
102
- headers = list(CHATFLOW_HEADERS)
103
- for turn_number in range(1, max_turns + 1):
104
- headers.extend([f"metadata_{turn_number}", f"question_{turn_number}", f"ai_answer_{turn_number}"])
105
- instruction.value = (
106
- "Do not delete this row. One row is one multi-turn case. Fill case_name, metadata_N and question_N; "
107
- "chat_id and ai_answer_N are generated by the system."
108
- )
109
- sheet.append(headers)
110
- for case_name, turns in normalized:
111
- row = [case_name, ""]
112
- for metadata, question in turns:
113
- row.extend([_metadata_text(metadata), question, ""])
114
- row.extend([""] * (len(headers) - len(row)))
115
- sheet.append(row)
116
- else:
117
- instruction.value = REPLY_INSTRUCTION
118
- sheet.append(REPLY_HEADERS)
119
- for index, case in enumerate(cases, start=1):
120
- if not isinstance(case, dict):
121
- raise ValueError(f"case {index} must be an object")
122
- question = case.get("question", case.get("message", ""))
123
- if not isinstance(question, str) or not question.strip():
124
- raise ValueError(f"case {index} requires question")
125
- sheet.append([
126
- question.strip(), "", "", "", "", "", "",
127
- _reply_metadata_text(case.get("metadata")), "",
128
- ])
129
-
130
- for cell in sheet[2]:
131
- cell.font = Font(bold=True, color="FFFFFF")
132
- cell.fill = PatternFill("solid", fgColor="1E3A5F")
133
- cell.alignment = Alignment(horizontal="center", vertical="center", wrap_text=True)
134
- sheet.freeze_panes = "A3"
135
- sheet.row_dimensions[1].height = 48
136
- for column in range(1, sheet.max_column + 1):
137
- sheet.column_dimensions[get_column_letter(column)].width = 22
138
- return workbook
139
-
140
-
141
- def _write_state(data):
142
- state_path = STATE_DIR / "latest_batch_test.json"
143
- state_path.parent.mkdir(parents=True, exist_ok=True)
144
- state_path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
145
-
146
-
147
- def create_sheet(args):
148
- cases = _load_cases(args.cases_file)
149
- output = Path(args.output) if args.output else OUTPUTS_DIR / f"{args.mode}_batch_test_{timestamp()}.xlsx"
150
- output.parent.mkdir(parents=True, exist_ok=True)
151
- build_workbook(cases, args.mode).save(output)
152
- result = {"mode": args.mode, "case_count": len(cases), "file": str(output)}
153
- _write_state({"stage": "sheet_created", **result})
154
- return result
155
-
156
-
157
- def submit(args):
158
- if args.flow_version is not None and args.agent_id is None:
159
- raise ValueError("--flow-version requires --agent-id")
160
- if args.agent_id is not None and args.flow_version is None:
161
- raise ValueError("--agent-id requires --flow-version")
162
- client = SolveaClient(env=args.env, project_id=args.project_id)
163
- file_key = client.upload_batch_test_file(args.file)
164
- task_type = "MULTI_TURN_REPLY" if args.mode == "chatflow" else "REPLY"
165
- metadata = None
166
- if args.agent_id is not None:
167
- metadata = {"specified_agent_id": args.agent_id, "specified_flow_version": args.flow_version}
168
- result = client.submit_batch_test(file_key, task_type, bot_id=args.bot_id or client.bot_id, metadata=metadata)
169
- state = {
170
- "stage": "submitted", "mode": args.mode, "file": str(Path(args.file).resolve()),
171
- "file_key": file_key, "task_type": task_type, "agent_id": args.agent_id,
172
- "flow_version": args.flow_version, "submission_result": result,
173
- }
174
- _write_state(state)
175
- return state
176
-
177
-
178
- def get_tasks(args):
179
- client = SolveaClient(env=args.env, project_id=args.project_id)
180
- task_type = "MULTI_TURN_REPLY" if args.mode == "chatflow" else "REPLY"
181
- return client.get_label_tasks(page=args.page, page_size=args.page_size, task_type=task_type)
182
-
183
-
184
- def main():
185
- parser = argparse.ArgumentParser(description=__doc__)
186
- parser.add_argument("command", choices=["create-sheet", "submit", "get-tasks"])
187
- parser.add_argument("--mode", choices=["chatflow", "agent"], required=True)
188
- parser.add_argument("--cases-file")
189
- parser.add_argument("--output")
190
- parser.add_argument("--file")
191
- parser.add_argument("--agent-id", type=int)
192
- parser.add_argument("--flow-version", type=int)
193
- parser.add_argument("--bot-id", type=int)
194
- parser.add_argument("--project-id")
195
- parser.add_argument("--env", choices=["staging", "prod"], default=SOLVEA_ENV)
196
- parser.add_argument("--page", type=int, default=1)
197
- parser.add_argument("--page-size", type=int, default=20)
198
- args = parser.parse_args()
199
- if args.command == "create-sheet":
200
- if not args.cases_file:
201
- parser.error("create-sheet requires --cases-file")
202
- result = create_sheet(args)
203
- elif args.command == "submit":
204
- if not args.file:
205
- parser.error("submit requires --file")
206
- result = submit(args)
207
- else:
208
- result = get_tasks(args)
209
- print(json.dumps(result, ensure_ascii=False, indent=2))
210
-
211
-
212
- if __name__ == "__main__":
213
- main()
@@ -1,85 +0,0 @@
1
- # Badcase 复测
2
-
3
- 本场景用于根据历史 ChatId 重放客户问题,验证修复,并比较原会话与新测试的 AgentAction。不要生成预期字段,也不要让模型自拟预期后再据此判定。
4
-
5
- ## 读取原会话
6
-
7
- 执行:
8
-
9
- ```bash
10
- npx --yes @shulex/cxclawctl@latest api get-transcript \
11
- --chat-id <historicalChatId> \
12
- --pretty
13
- ```
14
-
15
- 从原会话中:
16
-
17
- - 只提取客户 `USER` 消息,保持原顺序。
18
- - 不重放原 AI、人工客服、系统或流程消息。
19
- - 为每轮 USER 消息保留紧随其后的原 AgentAction,作为本地 baseline,仅用于测试后比较,不传入 `live_test`。
20
- - 原会话发生转人工,或 AgentAction 为 `TO_AGENT`、`STOPPED`、`HUMAN_REVIEW` 后停止提取后续轮次。
21
- - 记录原回复、Agent、意图、策略、工具调用、节点路径、图状态和 handoff 信息。
22
-
23
- ## 确定请求参数
24
-
25
- 按以下优先级确定:
26
-
27
- - `agentId`:用户显式指定的目标 Agent优先,否则使用原 AgentAction 实际调用的 Agent。
28
- - `botId`:当前 CXClaw 请求携带的 Bot ID 优先;没有时使用原会话 Bot ID;仍无法确定才询问。
29
- - `flowVersion`:用户显式版本优先,否则工具入参省略;`live_test` 会通过 `chatflow/queryGraph/latest` 取得当前用户草稿或最新发布版本,再将实际版本明确写入测试请求。复测修复时不要默认使用历史旧版本。
30
- - `touchPoint`:用户显式值优先,其次使用原会话 touchPoint,仍缺失时默认 `LIVE_CHAT`。
31
- - `metadata`:经用户确认后,只复制完成测试确实需要的业务字段。
32
-
33
- 不要复制旧 ChatId、运行状态、内部路由字段或工具固定注入的 metadata。
34
-
35
- 开始复测前,若用户未明确 metadata 策略,必须提醒并确认:是否固定复用原会话 metadata(例如订单号、用户邮箱、商品或渠道),还是不带 metadata 验证默认路由。只复制用户确认需要的字段,不能默认带入或自行补值。
36
-
37
- ## 调用 live_test
38
-
39
- 将提取出的全部客户问题组成一个固定 `case`,一次调用 `live_test`:
40
-
41
- ```json
42
- {
43
- "agentId": 2584,
44
- "touchPoint": "LIVE_CHAT",
45
- "case": {
46
- "caseId": "historical-chat-id",
47
- "turns": [
48
- {"message": "第一轮客户问题"},
49
- {"message": "第二轮客户问题"}
50
- ]
51
- },
52
- "metadata": {
53
- "order_number": "A-1001"
54
- }
55
- }
56
- ```
57
-
58
- 不要把 baseline、原 AI 回复、原 ChatId 或预期字段放入工具参数。多轮问题必须在同一个 `case.turns` 中提交,由工具内部维护新 ChatId。
59
-
60
- ## 读取新会话
61
-
62
- 工具完成后,获取结果中的新 ChatId,再执行一次:
63
-
64
- ```bash
65
- npx --yes @shulex/cxclawctl@latest api get-transcript \
66
- --chat-id <newTestChatId> \
67
- --pretty
68
- ```
69
-
70
- 按 USER 消息顺序将新 AgentAction 与原 baseline 对齐。不要仅根据 `live_test.messages` 的回复文本判断是否修复,因为完整路由、工具和节点信息在会话记录中。
71
-
72
- ## 对比结论
73
-
74
- 逐轮比较:
75
-
76
- 1. AgentAction 类型和 handoff 是否变化。
77
- 2. 实际 Agent、意图、策略是否变化。
78
- 3. 工具是否新增、减少、报错或得到不同关键结果。
79
- 4. Chatflow 节点路径和图状态是否变化。
80
- 5. AI 回复的业务结论、关键事实和下一步建议是否发生显著变化。
81
- 6. 原问题是否仍复现、已修复、出现新问题,或证据不足需要人工判断。
82
-
83
- 回复文本不同不自动代表失败;但回答结论、订单状态、承诺、路由、工具行为或节点路径发生明显变化时必须指出。
84
-
85
- 如果新测试出现系统错误、工具错误、失败节点或没有最终 AgentAction,先给出测试结论和新 ChatId,再主动引导进入 `shulex-support` 的 Agent 会话诊断场景。用户已经要求定位或修复时,直接使用新 ChatId 衔接排查。
@@ -1,24 +0,0 @@
1
- # 直接测试
2
-
3
- 用户只给 Agent 名称时,先查询并精确匹配:
4
-
5
- ```bash
6
- npx --yes @shulex/cxclawctl@latest api list-agents --pretty
7
- ```
8
-
9
- 默认使用定位的 `agentId`、当前 Bot ID、`LIVE_CHAT` 和当前用户草稿。用户说“随便测一下”但未给问题时,使用“你好,请介绍一下你可以帮助我处理哪些问题?”。
10
-
11
- 无论问题看起来是否简单,只要用户没有明确 metadata 策略,开始测试前都要询问一次:是否需要固定 metadata(例如订单号、用户邮箱、商品或渠道)?用户明确“不需要”才省略;不得默认空 metadata 或自行编造业务值。
12
-
13
- 明确的问题使用固定 `case`;一轮也使用 `case.turns`,多个已确定的问题放进同一次 `live_test` 调用:
14
-
15
- ```json
16
- {
17
- "agentId": 2584,
18
- "case": {"turns": [{"message": "我的订单什么时候发货?"}]}
19
- }
20
- ```
21
-
22
- 只有下一问必须根据前一轮 AI 回复决定时,使用探索式 `message`;继续同一对话时回传上次结果的 `chatId`,新的独立测试不要复用旧 ChatId。metadata 仅补经用户确认的真实业务字段,例如 `order_number` 或 `CustomerEmail`。
23
-
24
- 完成后按 `overview.md` 获取完整会话并分析 AgentAction。
@@ -1,142 +0,0 @@
1
- # 单 case 测试
2
-
3
- 这里的“单测”指一次测试一个 Agent。执行测试必须调用 `live_test`,不要通过脚本、临时 JSON 或本地状态文件自行调用 Chat 接口。
4
-
5
- ## 场景路由
6
-
7
- | 场景 | 读取文档 |
8
- |---|---|
9
- | 用户指定 Agent、直接给问题、说“随便测一下”或要求继续探索 | `direct-test.md` |
10
- | 用户给出历史 ChatId,要求复测 badcase、验证修复或比较变化 | `badcase-retest.md` |
11
-
12
- 先读本文件,再只读命中的场景文档。
13
-
14
- ## 两种调用方式
15
-
16
- `case` 和 `message` 必须且只能提供一个:
17
-
18
- - **固定用例**:传 `case.turns`,一次工具调用自动按顺序执行全部用户问题,并在内部复用同一个测试 ChatId。直接测试和 badcase 复测优先使用这种方式。
19
- - **灵活探索**:传 `message` 执行一轮;继续同一对话时,把上次结果中的 `chatId` 原样传回。适用于 Agent 根据上一轮结果临时生成下一问。
20
-
21
- ## metadata 确认(单测必做)
22
-
23
- 用户没有明确说明 metadata 策略时,调用 `live_test` 前必须先提醒并确认:本次是否需要固定 metadata(例如订单号、用户邮箱、商品或渠道),以保证路由和工具上下文符合预期。用户确认不需要后,才省略 metadata;不要默认为空,也不要自行编造值。
24
-
25
- ## 参数契约
26
-
27
- | 参数 | 必填 | 规则 |
28
- |---|---|---|
29
- | `agentId` | 是 | 目标 Agent ID。不要传名称。 |
30
- | `botId` | 否 | 默认使用当前 CXClaw 请求携带的 Bot ID;只有当前请求没有 Bot ID 时才向用户确认。 |
31
- | `flowVersion` | 否 | 工具入参默认省略;`live_test` 会调用 `chatflow/queryGraph/latest`,取得当前用户草稿或最新发布版本,再将实际版本明确写入测试请求。 |
32
- | `touchPoint` | 否 | 默认 `LIVE_CHAT`,传入时统一使用大写值。 |
33
- | `case` | 二选一 | `{ "caseId"?: string, "turns": [{ "message": string }] }`。至少一轮。 |
34
- | `message` | 二选一 | 灵活探索模式的一条用户问题。 |
35
- | `chatId` | 否 | 只在 `message` 模式继续上一轮测试对话时传入。固定 `case` 模式由工具内部维护。 |
36
- | `metadata` | 否 | 经用户确认后,仅追加完成测试所需的业务上下文。 |
37
- | `option` | 否 | 仅追加接口支持的其它选项。 |
38
-
39
- 调用示例:
40
-
41
- ```json
42
- {
43
- "agentId": 2584,
44
- "botId": 80,
45
- "flowVersion": 17,
46
- "touchPoint": "LIVE_CHAT",
47
- "case": {
48
- "caseId": "optional-case-name",
49
- "turns": [
50
- {"message": "我的订单什么时候发货?"},
51
- {"message": "请帮我确认一下这个订单。"}
52
- ]
53
- },
54
- "metadata": {"order_number": "A-1001"},
55
- "option": {}
56
- }
57
- ```
58
-
59
- ## 工具固定字段
60
-
61
- 不要在外部传入或覆盖以下字段:
62
-
63
- - `option.debug`
64
- - `option.verbose`
65
- - `metadata.specified_agent_id`
66
- - `metadata.specified_flow_version`
67
- - `metadata.show_chat_flow`
68
-
69
- `live_test` 会固定注入 `debug=true`、`verbose=true` 和三个路由 metadata。外部同名字段会被忽略,不能用它们改变测试目标。
70
-
71
- CHAT_FLOW 草稿按当前用户隔离,版本号为 `-userId`。保存草稿不会递增 `meta.chatFlowVersion`;只有发布才生成新的正版本并更新该字段。因此 Agent 刚修改并保存草稿后,调用 `live_test` 时不用自行计算 `flowVersion`:工具会查询当前登录用户的草稿版本,并把该负数版本明确写入测试请求。若已发布且草稿已删除,工具会改用最新发布版本。只有明确回放某个历史或已发布版本时才在工具入参中显式传正版本号。
72
-
73
- ## 接口与进度
74
-
75
- - `touchPoint=LIVE_CHAT` 时,工具内部只调用 `/api_v2/gpt/bot/{botId}/chat/stream`。
76
- - 其它 touchPoint 只调用同步 `/api_v2/gpt/bot/{botId}/chat`。
77
- - 两条路径不允许 fallback。流式路径只取最终 AgentAction,不把完整 SSE events 放入工具结果。
78
- - 工具通过 `tool_progress_delta` 展示用户问题和 AI 流式回复,通过持久化的 `turn_completed` 记录每轮成功、失败、耗时和总进度。
79
- - 不要因为工具已经展示实时进度而额外创建过程文件。
80
-
81
- ## 执行顺序与阶段反馈
82
-
83
- 1. 参数确定后,先用一句话说明目标 Agent、测试轮数和即将开始测试,再调用 `live_test`。
84
- 2. `live_test` 内部的逐轮问题、回复和状态已经由 Live Test 组件展示,不要在工具执行期间重复输出相同进度。
85
- 3. 必须等待 `live_test` 完整结束并取得新测试 `chatId`。完成后先输出一句阶段进展,说明完成轮数、成功或失败状态以及新 ChatId。
86
- 4. 阶段进展之后,再执行一次 `get-transcript` 获取完整会话并分析 AgentAction。不要把读取会话命令放到 Live Test 之前,也不要与 Live Test 并行执行。
87
- 5. `get-transcript` 完成后再输出逐轮结论;不要把读取会话动作描述成新的测试轮次。
88
-
89
- `live_test` 入参会提供给用户查看,因此只传本次测试真正需要的 Agent、case/message、渠道和业务 metadata。不要把 baseline、原会话全文、分析草稿或内部固定字段塞入工具参数。
90
-
91
- ## 工具结果
92
-
93
- `live_test` 的 `content` 是可供当前 Agent 继续分析的 JSON 字符串,结构化 `payload` 同时提供给前端:
94
-
95
- ```json
96
- {
97
- "type": "live_test",
98
- "version": 1,
99
- "status": "completed",
100
- "mode": "case",
101
- "chatId": "new-test-chat-id",
102
- "botId": 80,
103
- "agentId": 2584,
104
- "flowVersion": -11302,
105
- "totalTurns": 2,
106
- "completedTurns": 2,
107
- "messages": [
108
- {"turn": 1, "role": "USER", "content": "..."},
109
- {"turn": 1, "role": "ASSISTANT", "id": "...", "content": "..."}
110
- ]
111
- }
112
- ```
113
-
114
- 失败时 `status=failed`,并包含 `error`;工具整体错误状态以 `is_error` 为准。结果过大时可能出现 `truncated=true`,但 `chatId`、轮次和消息 ID 必须保留。
115
-
116
- ## 获取 AgentAction
117
-
118
- 工具完成后,使用返回的新测试 `chatId` 获取一次完整会话记录:
119
-
120
- ```bash
121
- npx --yes @shulex/cxclawctl@latest api get-transcript \
122
- --chat-id <newTestChatId> \
123
- --pretty
124
- ```
125
-
126
- staging 环境按公共 `solvea-api` 规则追加 `--env staging`。不要为每一轮分别请求;一次完整会话已经包含所有消息及其执行过程。
127
-
128
- ## 输出结论
129
-
130
- 按轮次输出:
131
-
132
- 1. 用户问题和 AI 回复摘要。
133
- 2. AgentAction 类型、handoff、原因。
134
- 3. 实际 Agent、意图和策略。
135
- 4. 工具调用、工具错误和关键结果。
136
- 5. Chatflow 节点路径、图状态和耗时。
137
- 6. 与原 badcase 有基线时,指出路由、工具、节点和回复业务语义的显著变化。
138
- 7. 当前轮是成功、失败还是需要人工判断。
139
-
140
- 显著变化不自动等于失败,但必须明确指出。不要生成单测报告、过程文件或内部状态文件,直接在聊天回复中给出结构化结论。
141
-
142
- 当测试出现 `SYSTEM_ERROR`、图状态失败、工具报错或没有最终 AgentAction 时,展示当前测试 ChatId,并引导用户使用该 ChatId 进入 `shulex-support` 的 Agent 会话诊断场景。用户已经要求排查根因时可直接衔接,不要再次询问 ChatId。
@@ -1,112 +0,0 @@
1
- ---
2
- name: delivery
3
- description: 正式交付业务编排。只负责阶段推进、用户确认和交付物汇总;具体意图、KB、批量回归能力通过其他 skill 的 interfaces 调用。
4
- requires:
5
- skills: [kb-builder]
6
- ---
7
-
8
- # Delivery
9
-
10
- 签约后正式交付编排 skill。职责是串联意图创建、知识库搭建和回归测试,不复制其他 skill 的内部职责。
11
-
12
- ## 边界原则
13
-
14
- - 只调用其他 skill 暴露的 `interfaces/` 能力。
15
- - 不在本 skill 维护 intent 创建、X-Token patch、KB crawl、QA 上传、批测 runner 的内部实现。
16
- - Delivery 顶层状态只保存跨阶段引用。
17
- - 每个阶段完成后先向用户确认,再进入下一阶段。
18
-
19
- ## 必读文件
20
-
21
- | 阶段 | Interface |
22
- |---|---|
23
- | 意图分类 | `/skills/delivery/interfaces/classify.md` |
24
- | Delivery agent / 策略创建 | `/skills/delivery/interfaces/create-agents.md` |
25
- | 知识库,必做或确认已有 | `/skills/kb-builder/interfaces/web-kb.md` / `/skills/kb-builder/interfaces/qa-upload.md` |
26
- | 回归测试 | `/skills/delivery/interfaces/delivery-test.md` |
27
-
28
- ## 顶层状态
29
-
30
- 状态文件:`/workspace/state/delivery/progress.json`。
31
-
32
- 每次开始处理 Delivery 前,先运行:
33
-
34
- ```bash
35
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/plan.py --init
36
- ```
37
-
38
- 阶段完成后用 `plan.py` 更新顶层状态,例如:
39
-
40
- ```bash
41
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/plan.py \
42
- --status agents_created \
43
- --set intent.created_log_path=/workspace/outputs/delivery/created_<ts>.json
44
- ```
45
-
46
- 需要汇总当前交付物时运行:
47
-
48
- ```bash
49
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/render_summary.py
50
- ```
51
-
52
- ```json
53
- {
54
- "status": "empty | classified | agents_created | kb_done | regression_done | done",
55
- "tenant": "",
56
- "brand_name": "",
57
- "intent": {
58
- "progress_path": "/workspace/state/delivery/progress.json",
59
- "classification_report_path": "",
60
- "classification_excel_path": "",
61
- "created_log_path": ""
62
- },
63
- "kb": {
64
- "required": true,
65
- "source_url": "",
66
- "directory_plan_path": "",
67
- "qa_plan_path": "",
68
- "upload_result_path": ""
69
- },
70
- "regression": {
71
- "input_path": "",
72
- "touch_point": "LIVE_CHAT",
73
- "results_path": "",
74
- "report_html": "",
75
- "report_md": ""
76
- },
77
- "updated_at": ""
78
- }
79
- ```
80
-
81
- ## 起点判断
82
-
83
- | 用户消息含 | 起点 |
84
- |---|---|
85
- | 「补充知识 / 提取 QA / 加进 KB」+ 知识源附件 | 知识库 |
86
- | 「建知识库目录 / 搭知识库」+ 网站/知识来源 URL | 知识库 |
87
- | 「上传 qa_plan.json」 | 知识库上传 |
88
- | 「批量回归 / 跑测试」+ 测试用例 xlsx 或意图列表 | 回归测试 |
89
- | 给了 xlsx 路径 | 意图分类 |
90
- | 「接着上次」 | 读 `/workspace/state/delivery/progress.json` |
91
-
92
- 如果只输入 `/delivery` 或信息不足,询问:
93
-
94
- ```text
95
- 您想从哪个阶段开始?
96
-
97
- 1. 阶段一:意图分类与意图创建
98
- 2. 阶段二:知识库搭建
99
- 3. 阶段三:批量回归测试
100
-
101
- 请告诉我从哪一步开始以及已有素材路径。
102
- ```
103
-
104
- ## 流程
105
-
106
- 详见 `/skills/delivery/references/flow.md`。
107
-
108
- ## 可恢复缺口
109
-
110
- Delivery 的固定广场 agent 只是默认来源,不是硬阻塞。执行创建脚本后如果 state 或 created log 出现
111
- `preset_agent_missing` / `preset_agent_errors`,必须向用户说明缺失项,并询问是从零新建 CHAT_FLOW
112
- agent,还是使用其他现有 agent 替代;不要直接报告“预置 Agent 不存在,必须开发修复后才能继续”。
@@ -1,38 +0,0 @@
1
- # Delivery Intent Classification
2
-
3
- Delivery 专属分类入口。入口和状态归属 Delivery skill;通用意图分类实现调用 `solvea-kit/scripts/intent_classify.py`,状态写入 `/workspace/state/delivery/progress.json`。
4
-
5
- ## Commands
6
-
7
- ```bash
8
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/classify.py --xlsx <xlsx> --messages-col <col> [--title-col <col>]
9
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/classify.py --no-excel
10
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/gen_report.py --brand <brand>
11
- npx --yes @shulex/cxclawctl@latest script skills/delivery/scripts/gen_excel.py --brand <brand>
12
- ```
13
-
14
- ## Inputs
15
-
16
- - `xlsx`:用户上传的客诉表,可选。
17
- - `--messages-col`:xlsx 模式必填,由 Delivery 先读样例推断。
18
- - `--title-col`:可选。
19
- - `--no-excel`:使用固定 taxonomy。
20
-
21
- ## Guarantees
22
-
23
- - xlsx 模式不会自动降级到 `--no-excel`。
24
- - 模型选择由 Solvea Kit 的 service-managed LLM 通道决定,Delivery 不维护具体模型名。
25
- - 分类结果通过 `results_path` 落盘。
26
- - 报告和 Excel 路径通过 Delivery state 暴露。
27
-
28
- ## State Fields
29
-
30
- - `status`
31
- - `tenant`
32
- - `run_id`
33
- - `results_path`
34
- - `items_path`
35
- - `report_path`
36
- - `excel_path`
37
- - `mode`
38
- - `updated_at`