kc-beta 0.7.3 → 0.7.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (88) hide show
  1. package/README.md +10 -4
  2. package/bin/kc-beta.js +20 -6
  3. package/package.json +1 -1
  4. package/src/agent/engine.js +131 -60
  5. package/src/agent/pipelines/_milestone-derive.js +140 -4
  6. package/src/agent/pipelines/initializer.js +4 -1
  7. package/src/agent/skill-loader.js +433 -111
  8. package/src/agent/tools/consult-skill.js +112 -0
  9. package/src/agent/tools/copy-to-workspace.js +4 -3
  10. package/src/agent/tools/release.js +128 -1
  11. package/src/agent/tools/workspace-file.js +7 -7
  12. package/src/config.js +1 -1
  13. package/template/AGENT.md +182 -7
  14. package/template/skills/en/{meta-meta/auto-model-selection → auto-model-selection}/SKILL.md +1 -0
  15. package/template/skills/en/{meta-meta/bootstrap-workspace → bootstrap-workspace}/SKILL.md +1 -0
  16. package/template/skills/{zh/meta → en}/compliance-judgment/SKILL.md +1 -0
  17. package/template/skills/en/{meta/confidence-system → confidence-system}/SKILL.md +1 -0
  18. package/template/skills/en/{meta/corner-case-management → corner-case-management}/SKILL.md +1 -0
  19. package/template/skills/en/{meta/cross-document-verification → cross-document-verification}/SKILL.md +1 -0
  20. package/template/skills/en/{meta-meta/dashboard-reporting → dashboard-reporting}/SKILL.md +1 -0
  21. package/template/skills/en/{meta/data-sensibility → data-sensibility}/SKILL.md +1 -0
  22. package/template/skills/{zh/meta → en}/document-chunking/SKILL.md +1 -0
  23. package/template/skills/en/{meta/document-parsing → document-parsing}/SKILL.md +1 -0
  24. package/template/skills/{zh/meta → en}/entity-extraction/SKILL.md +1 -0
  25. package/template/skills/en/{meta-meta/evolution-loop → evolution-loop}/SKILL.md +1 -0
  26. package/template/skills/en/{meta-meta/pdf-review-dashboard → pdf-review-dashboard}/SKILL.md +1 -0
  27. package/template/skills/en/{meta-meta/quality-control → quality-control}/SKILL.md +1 -0
  28. package/template/skills/en/{meta-meta/rule-extraction → rule-extraction}/SKILL.md +1 -0
  29. package/template/skills/en/{meta-meta/rule-graph → rule-graph}/SKILL.md +1 -0
  30. package/template/skills/en/{meta-meta/skill-authoring → skill-authoring}/SKILL.md +1 -0
  31. package/template/skills/en/skill-creator/SKILL.md +2 -1
  32. package/template/skills/en/{meta-meta/skill-to-workflow → skill-to-workflow}/SKILL.md +5 -4
  33. package/template/skills/en/{meta-meta/task-decomposition → task-decomposition}/SKILL.md +1 -0
  34. package/template/skills/en/{meta/tree-processing → tree-processing}/SKILL.md +1 -0
  35. package/template/skills/en/{meta-meta/version-control → version-control}/SKILL.md +1 -0
  36. package/template/skills/en/{meta-meta/work-decomposition → work-decomposition}/SKILL.md +17 -6
  37. package/template/skills/phase_skills.yaml +107 -0
  38. package/template/skills/zh/{meta-meta/auto-model-selection → auto-model-selection}/SKILL.md +1 -0
  39. package/template/skills/zh/{meta-meta/bootstrap-workspace → bootstrap-workspace}/SKILL.md +1 -0
  40. package/template/skills/{en/meta → zh}/compliance-judgment/SKILL.md +1 -0
  41. package/template/skills/zh/{meta/confidence-system → confidence-system}/SKILL.md +1 -0
  42. package/template/skills/zh/{meta/corner-case-management → corner-case-management}/SKILL.md +1 -0
  43. package/template/skills/zh/{meta/cross-document-verification → cross-document-verification}/SKILL.md +1 -0
  44. package/template/skills/zh/{meta-meta/dashboard-reporting → dashboard-reporting}/SKILL.md +1 -0
  45. package/template/skills/zh/{meta/data-sensibility → data-sensibility}/SKILL.md +1 -0
  46. package/template/skills/{en/meta → zh}/document-chunking/SKILL.md +1 -0
  47. package/template/skills/zh/{meta/document-parsing → document-parsing}/SKILL.md +1 -0
  48. package/template/skills/{en/meta → zh}/entity-extraction/SKILL.md +1 -0
  49. package/template/skills/zh/{meta-meta/evolution-loop → evolution-loop}/SKILL.md +1 -0
  50. package/template/skills/zh/{meta-meta/pdf-review-dashboard → pdf-review-dashboard}/SKILL.md +1 -0
  51. package/template/skills/zh/{meta-meta/quality-control → quality-control}/SKILL.md +1 -0
  52. package/template/skills/zh/{meta-meta/rule-extraction → rule-extraction}/SKILL.md +1 -0
  53. package/template/skills/zh/{meta-meta/rule-graph → rule-graph}/SKILL.md +1 -0
  54. package/template/skills/zh/{meta-meta/skill-authoring → skill-authoring}/SKILL.md +1 -0
  55. package/template/skills/zh/skill-creator/SKILL.md +2 -1
  56. package/template/skills/zh/skill-to-workflow/SKILL.md +190 -0
  57. package/template/skills/zh/{meta-meta/task-decomposition → task-decomposition}/SKILL.md +1 -0
  58. package/template/skills/zh/{meta/tree-processing → tree-processing}/SKILL.md +1 -0
  59. package/template/skills/zh/{meta-meta/version-control → version-control}/SKILL.md +1 -0
  60. package/template/skills/zh/{meta-meta/work-decomposition → work-decomposition}/SKILL.md +15 -4
  61. package/template/CLAUDE.md +0 -150
  62. package/template/skills/zh/meta-meta/skill-to-workflow/SKILL.md +0 -188
  63. /package/template/skills/en/{meta/compliance-judgment → compliance-judgment}/references/output-format.md +0 -0
  64. /package/template/skills/en/{meta/cross-document-verification → cross-document-verification}/references/contradiction-taxonomy.md +0 -0
  65. /package/template/skills/en/{meta-meta/dashboard-reporting → dashboard-reporting}/scripts/generate_dashboard.py +0 -0
  66. /package/template/skills/en/{meta/document-parsing → document-parsing}/references/parser-catalog.md +0 -0
  67. /package/template/skills/en/{meta-meta/evolution-loop → evolution-loop}/references/convergence-guide.md +0 -0
  68. /package/template/skills/en/{meta-meta/pdf-review-dashboard → pdf-review-dashboard}/scripts/generate_review.js +0 -0
  69. /package/template/skills/en/{meta-meta/quality-control → quality-control}/references/qa-layers.md +0 -0
  70. /package/template/skills/en/{meta-meta/quality-control → quality-control}/references/sampling-strategies.md +0 -0
  71. /package/template/skills/en/{meta-meta/rule-extraction → rule-extraction}/references/chunking-strategies.md +0 -0
  72. /package/template/skills/en/{meta-meta/skill-authoring → skill-authoring}/references/skill-format-spec.md +0 -0
  73. /package/template/skills/en/{meta-meta/skill-to-workflow → skill-to-workflow}/references/worker-llm-catalog.md +0 -0
  74. /package/template/skills/en/{meta-meta/task-decomposition → task-decomposition}/references/decision-matrix.md +0 -0
  75. /package/template/skills/en/{meta-meta/version-control → version-control}/references/trace-id-spec.md +0 -0
  76. /package/template/skills/zh/{meta/compliance-judgment → compliance-judgment}/references/output-format.md +0 -0
  77. /package/template/skills/zh/{meta/cross-document-verification → cross-document-verification}/references/contradiction-taxonomy.md +0 -0
  78. /package/template/skills/zh/{meta-meta/dashboard-reporting → dashboard-reporting}/scripts/generate_dashboard.py +0 -0
  79. /package/template/skills/zh/{meta/document-parsing → document-parsing}/references/parser-catalog.md +0 -0
  80. /package/template/skills/zh/{meta-meta/evolution-loop → evolution-loop}/references/convergence-guide.md +0 -0
  81. /package/template/skills/zh/{meta-meta/pdf-review-dashboard → pdf-review-dashboard}/scripts/generate_review.js +0 -0
  82. /package/template/skills/zh/{meta-meta/quality-control → quality-control}/references/qa-layers.md +0 -0
  83. /package/template/skills/zh/{meta-meta/quality-control → quality-control}/references/sampling-strategies.md +0 -0
  84. /package/template/skills/zh/{meta-meta/rule-extraction → rule-extraction}/references/chunking-strategies.md +0 -0
  85. /package/template/skills/zh/{meta-meta/skill-authoring → skill-authoring}/references/skill-format-spec.md +0 -0
  86. /package/template/skills/zh/{meta-meta/skill-to-workflow → skill-to-workflow}/references/worker-llm-catalog.md +0 -0
  87. /package/template/skills/zh/{meta-meta/task-decomposition → task-decomposition}/references/decision-matrix.md +0 -0
  88. /package/template/skills/zh/{meta-meta/version-control → version-control}/references/trace-id-spec.md +0 -0
@@ -1,5 +1,6 @@
1
1
  ---
2
2
  name: work-decomposition
3
+ tier: meta-meta
3
4
  description: 在 rule_extraction → skill_authoring 过渡阶段决定如何把规则集拆分为 TaskBoard 任务。涵盖排序方法(难度优先 / Shannon–Huffman、广度优先、深度优先、二分切分)、分组策略(多条规则合并到一个任务 vs. 各自独立的判断标准)、三轴难度评估、以及如何写一份贯穿全流程都能用得上的 PATTERNS.md 项目记忆。当进入 rule_extraction、进入 skill_authoring 或感觉 TaskBoard 走偏想要重新拆分时使用。
4
5
  ---
5
6
 
@@ -7,7 +8,7 @@ description: 在 rule_extraction → skill_authoring 过渡阶段决定如何把
7
8
 
8
9
  KC 的 main agent 是指挥者。指挥者决定下一步做什么——而这个决定凌驾于后续所有选择之上。错误的拆分会让整个会话变得昂贵:规则顺序错了,agent 会把同一种结构重新设计三遍;不相关的规则被合并到一个 skill 里,最终 check.py 就会变成 E2E #4 那种"统一执行器"反模式;本应合并的相关规则被分散到不同 skill,agent 会把同样的 chunker 逻辑重新推导 17 次。
9
10
 
10
- 这份 skill 是指挥者做这类决定的操作手册。它放在 `meta-meta/` 下,因为工作拆分是系统级的纪律,不是某条规则的具体技巧。互补的 `task-decomposition`(同样在 `meta-meta/` 下)覆盖单条规则**内部**的结构——locate → extract → normalize → judge → comment。本 skill 覆盖的是规则**集合**该如何切分成 TaskBoard 任务。
11
+ 这份 skill 是指挥者做这类决定的操作手册。它的层级标记是 `tier: meta-meta`,因为工作拆分是系统级的纪律,不是某条规则的具体技巧。互补的 `task-decomposition`(同样 `tier: meta-meta`)覆盖单条规则**内部**的结构——locate → extract → normalize → judge → comment。本 skill 覆盖的是规则**集合**该如何切分成 TaskBoard 任务。
11
12
 
12
13
  ## 何时使用本 skill
13
14
 
@@ -339,13 +340,23 @@ PATTERNS.md 全文控制在约 5 KB 之内。超过时,剪掉最不可执行
339
340
 
340
341
  ### 调用 TaskCreate / TaskUpdate / TaskComplete
341
342
 
342
- 引擎注册了三个任务面板工具(v0.7.3+):
343
+ 引擎注册了三个任务面板工具(v0.7.4):
343
344
 
344
- - `TaskCreate({id, title, phase, ruleId?})` —— 在 `tasks.json` 中新增一条任务。`id` 在本会话内必须唯一;per-rule 任务建议用 `<rule_id>-<phase>` 这种稳定形状,分组 / 非规则任务用 `<group-name>-<phase>`。`phase` 是该任务所属的阶段(当前阶段或你预先排好的未来阶段)。`ruleId` 可选 —— 设上之后,引擎在里程碑推导时能把这个 rule_id 计入覆盖。
345
+ - `TaskCreate({id, title, phase, ruleId?})` —— 在 `tasks.json` 中新增一条任务。`id` 在本会话内必须唯一;per-rule 任务建议用 `<rule_id>-<phase>` 这种稳定形状,分组 / 非规则任务用 `<group-name>-<phase>`。`phase` 是该任务所属的当前阶段。`ruleId` 可选 —— 设上之后引擎在里程碑推导时能把这个 rule_id 计入覆盖。
345
346
  - `TaskUpdate({id, status?, summary?})` —— 把任务状态改为 `pending` / `in_progress` / `completed` / `failed`,可选附一行简要 summary。
346
347
  - `TaskComplete({id, summary?})` —— `TaskUpdate({id, status:"completed", summary})` 的语法糖。完成一个工作单元后走这条最常用的路径。
347
348
 
348
- 调用 `TaskCreate` 把你的拆分写进面板、本回合结束之后,Ralph 循环会取下一条 pending 任务执行。完成工作、调 `TaskComplete`,循环再前进。如果一条任务无法完成(不可恢复的错误),调 `TaskUpdate({id, status:"failed", summary:"原因"})`,让队列继续推进而不是被堵在那里。
349
+ ### Ralph 循环范围 —— 仅限当前阶段
350
+
351
+ 重要契约(v0.7.4 在团队反馈后调整):
352
+
353
+ - **循环范围 = 仅当前阶段**。TaskCreate 只能为当前阶段建任务,Ralph 循环在阶段内逐条处理。
354
+ - **阶段边界 = 循环退出**。当前阶段任务全部完成、或阶段推进(你调 `phase_advance`、或任何其他地方改了 `currentPhase`)时,循环干净退出,控制权回到用户。
355
+ - **引擎不再自动推进阶段**。即使所有任务完成 + 退出条件满足,引擎也不会自动跳到下一阶段。阶段推进是你**显式调** `phase_advance`,或用户重新 prompt 的事。
356
+ - **不要为未来阶段预先建任务**。它们会被忽略 —— 循环在阶段边界先退出,根本不处理它们。只为你**当前所在**的阶段建任务。
357
+ - **阶段边界 = 用户检查点**。这是有意为之。团队需要在自然断点上看进度。完成你这一批任务、调 `phase_advance` 之后,循环退出,你在最后一条消息里向用户汇报进度,用户再 prompt 你开始下一阶段。
358
+
359
+ "从 bootstrap 一路跑到 finalization 不停"这种端到端无人值守,**不是引擎该做的事** —— 这个能力以后会以外部 driver 的形式实现(`/loop` 风格命令),由它跨阶段地反复调用 agent。在一次调用内,你就把当前阶段做完、推进、回到用户。
349
360
 
350
361
  示例:
351
362
 
@@ -1,150 +0,0 @@
1
- # KC Reborn — Document Verification Workspace
2
-
3
- ## What This Workspace Is
4
-
5
- You are a coding agent tasked with building a document verification app for the developer user's specific business scenario. The meta skills in `skills/` encode the methodology of experienced verification system architects and business analysts. You bring the intelligence and judgment to apply this methodology to the specific case at hand.
6
-
7
- Your goal: build a verification system that starts with you doing the work, then gradually distills your capability into cheap, fast workflows powered by worker LLMs. You are the ground truth. The workflows you create are the deliverables.
8
-
9
- ## Roles
10
-
11
- - **Developer user**: The human you serve. They are a domain expert (e.g., tech lead at a bank's loan department). They provide the rules, the documents, and the business context. Discuss decisions with them.
12
- - **You (the coding agent)**: You are both the Builder (creating skills and workflows) and the Observer (judging quality). You do the verification first, prove it works, then teach smaller models to replicate your results.
13
- - **Worker LLMs**: The performers. Models configured in `.env` (TIER1 through TIER4) that will execute the workflows you build. Your job is to find the smallest model that works for each task.
14
-
15
- ## Workspace Layout
16
-
17
- ```
18
- Rules/ — Regulation documents, compliance notes from the developer user
19
- Samples/ — Sample documents for testing (your training set)
20
- Input/ — Production document batches awaiting verification
21
- Output/ — Verification results
22
- skills/ — Meta skills encoding verification methodology
23
- .env — Configuration: API keys, model tiers, thresholds, language
24
- ```
25
-
26
- Note: KC's session workspace under `~/.kc_agent/workspaces/<sessionId>/`
27
- uses lowercase counterparts (`rules/`, `samples/`, `input/`, `output/`,
28
- `logs/`, `workflows/`, `rule_skills/`) — these are runtime-internal and
29
- separate from this project's user-facing folders above. The asymmetry
30
- is intentional: title-case for human-facing project dirs, lowercase for
31
- KC's working state.
32
-
33
- ## Your Mission
34
-
35
- Follow this lifecycle. Each step references the skill(s) to consult:
36
-
37
- 1. **Bootstrap** → Read `bootstrap-workspace`. Understand the business scenario, read Rules/, scan Samples/, configure .env with the developer user.
38
- 2. **Extract Rules** → Read `rule-extraction`. Decompose regulation documents into atomic, testable verification rules.
39
- 3. **Decompose Tasks** → Read `task-decomposition`. For each rule, break the verification into sub-tasks and assign the optimal method (rule, code, LLM, or manual) to each.
40
- 4. **Map Rule Relationships** → Read `rule-graph`. Identify shared entities, dependencies, and conflicts between rules. Each rule stays independently executable.
41
- 5. **Write Rule Skills** → Read `skill-authoring`. Write each rule into a skill folder. Before writing extraction logic for a new document type, consult `data-sensibility` to observe the data first.
42
- 6. **Test Skills** → Apply each skill to Samples/. Use `evolution-loop` to diagnose failures and iterate. Continue until accuracy meets SKILL_ACCURACY threshold in .env.
43
- 7. **Distill to Workflows** → Read `skill-to-workflow`. Convert proven skills into Python code + worker LLM prompts. Test workflows against your own results as ground truth. Iterate until WORKFLOW_ACCURACY is met.
44
- 8. **Production QC** → Read `quality-control` and `confidence-system`. Run workflows on Input/. Sample and review results based on confidence scores. For multi-document cases, read `cross-document-verification`. Use `evolution-loop` when quality drops.
45
- 9. **Stabilize** → Gradually reduce monitoring as workflows prove reliable. Only intervene when rules change or quality drops.
46
- 10. **Report** → Read `dashboard-reporting`. Generate HTML dashboards so the developer user can see results, progress, and issues. Ensure dashboards include feedback collection mechanisms for users.
47
-
48
- Throughout: use `version-control` to track all changes. Use `corner-case-management` to handle edge cases without polluting workflows. Use `task-decomposition` and `rule-graph` to inform optimization decisions.
49
-
50
- ## Core Principles
51
-
52
- - **Minimum viable model**: Always use the smallest, cheapest, fastest model that meets the accuracy threshold. Start simple, escalate only when necessary.
53
- - **JIT structure**: Do not design schemas or formats prematurely. Define them when needed, keep them consistent once defined.
54
- - **OTF evolution**: The system you build today may look completely different tomorrow. Embrace change.
55
- - **Skills before workflows**: Prove each rule works as a skill (you executing it) before distilling into code + worker LLM prompts.
56
- - **Log everything**: Every test iteration, every evolution decision, every version change. Both JSON (machine-readable) and plain text (human-readable).
57
-
58
- ## How to Read Skills
59
-
60
- Skills use progressive disclosure:
61
- 1. **Frontmatter** (name + description) — always visible, ~100 words. Tells you WHEN to use the skill.
62
- 2. **SKILL.md body** — read when the skill is relevant. Under 500 lines. Conveys methodology, not recipes.
63
- 3. **references/** — read on demand for detailed technical reference.
64
- 4. **scripts/** — executable code you can run or adapt.
65
- 5. **assets/** — data files, templates, examples.
66
-
67
- Skills convey philosophy and decision frameworks. Adapt them to the specific business case. Do not follow them rigidly.
68
-
69
- ## Communication with Developer User
70
-
71
- - **Proactively discuss**: rule granularity, accuracy thresholds, model selection, edge cases.
72
- - **Report progress**: after each testing round, share results and next steps.
73
- - **Escalate**: when you cannot resolve an issue after iterating, surface it with evidence.
74
- - **Ask**: the developer user is a domain expert. When in doubt about a rule's intent, ask.
75
-
76
- ---
77
-
78
- # KC Reborn — 文档核查工作区
79
-
80
- ## 这是什么
81
-
82
- 你是一个编程智能体,负责为开发者用户的具体业务场景构建文档核查应用。`skills/` 中的元技能编码了资深核查系统架构师和业务分析师的方法论。你负责运用智慧和判断力,将这些方法论应用到具体场景中。
83
-
84
- 你的目标:构建一个核查系统,先由你亲自执行核查工作,然后逐步将你的能力蒸馏为由 Worker LLM(执行模型)驱动的低成本、高速度的工作流。你是基准真值。你创建的工作流是最终交付物。
85
-
86
- ## 角色定义
87
-
88
- - **开发者用户**:你服务的人。他们是领域专家(如银行信贷部门的技术负责人)。他们提供规则、文档和业务背景。与他们讨论决策。
89
- - **你(编程智能体)**:你既是构建者(创建技能和工作流),也是观察者(评判质量)。你先执行核查,证明方法可行,再教小模型复现你的结果。
90
- - **Worker LLM**:执行者。在 `.env` 中配置的模型(TIER1到TIER4),将执行你构建的工作流。你的任务是为每项工作找到能胜任的最小模型。
91
-
92
- ## 工作区结构
93
-
94
- ```
95
- Rules/ — 法规文件、开发者用户的合规注释
96
- Samples/ — 用于测试的样本文件(你的训练集)
97
- Input/ — 等待核查的生产批次文件
98
- Output/ — 核查结果
99
- skills/ — 编码核查方法论的元技能
100
- .env — 配置:API密钥、模型层级、阈值、语言
101
- ```
102
-
103
- 注:KC 在 `~/.kc_agent/workspaces/<sessionId>/` 下的会话工作区使用
104
- 小写对应目录(`rules/`、`samples/`、`input/`、`output/`、`logs/`、
105
- `workflows/`、`rule_skills/`)—— 这些是运行时内部目录,与本项目上面
106
- 那些用户可见的目录是分开的。这种大小写不对称是有意的:项目里给人看
107
- 的目录用首字母大写;KC 自己的工作状态用小写。
108
-
109
- ## 你的使命
110
-
111
- 遵循以下生命周期。每一步标注了需要参考的技能:
112
-
113
- 1. **初始化** → 阅读 `bootstrap-workspace`。理解业务场景,阅读 Rules/,浏览 Samples/,与开发者用户配置 .env。
114
- 2. **提取规则** → 阅读 `rule-extraction`。将法规文件分解为原子级、可测试的核查规则。
115
- 3. **任务分解** → 阅读 `task-decomposition`。对每条规则,将核查过程拆解为子任务,为每个子任务分配最优方法(规则、代码、LLM 或人工)。
116
- 4. **构建规则图谱** → 阅读 `rule-graph`。识别规则间的共享实体、依赖关系和潜在冲突。每条规则保持独立可执行。
117
- 5. **编写规则技能** → 阅读 `skill-authoring`。将每条规则写入技能文件夹。编写新文档类型的提取逻辑前,先阅读 `data-sensibility` 观察数据。
118
- 6. **测试技能** → 在 Samples/ 上应用每个技能。使用 `evolution-loop` 诊断失败并迭代。直到准确率达到 .env 中的 SKILL_ACCURACY 阈值。
119
- 7. **蒸馏为工作流** → 阅读 `skill-to-workflow`。将验证过的技能转化为 Python 代码 + Worker LLM 提示词。用你自己的结果作为基准测试工作流。迭代直到达到 WORKFLOW_ACCURACY。
120
- 8. **生产质控** → 阅读 `quality-control` 和 `confidence-system`。在 Input/ 上运行工作流。根据置信度分数抽样审查结果。涉及多文档案件时,阅读 `cross-document-verification`。质量下降时使用 `evolution-loop`。
121
- 9. **稳定运行** → 随着工作流稳定,逐步降低监控频率。仅在规则变更或质量下降时介入。
122
- 10. **报告** → 阅读 `dashboard-reporting`。生成 HTML 仪表板,让开发者用户直观地看到结果、进度和问题。确保仪表盘内置用户反馈收集机制。
123
-
124
- 全程使用 `version-control` 跟踪所有变更。使用 `corner-case-management` 处理边缘案例,不要污染主工作流。使用 `task-decomposition` 和 `rule-graph` 指导优化决策。
125
-
126
- ## 核心原则
127
-
128
- - **最小可用模型**:始终使用能达到准确率阈值的最小、最便宜、最快的模型。从简单开始,必要时才升级。
129
- - **即时结构(JIT)**:不要过早设计数据结构或格式。需要时定义,定义后保持一致。
130
- - **即时演进(OTF)**:你今天构建的系统明天可能面目全非。拥抱变化。
131
- - **先技能后工作流**:先证明每条规则作为技能(你执行)可行,再蒸馏为代码 + Worker LLM 提示词。
132
- - **记录一切**:每次测试迭代、每个演进决策、每次版本变更。同时保存 JSON(机器可读)和纯文本(人类可读)。
133
-
134
- ## 如何阅读技能
135
-
136
- 技能采用渐进式披露:
137
- 1. **前置元数据**(名称 + 描述)— 始终可见,约100字。告诉你何时使用该技能。
138
- 2. **SKILL.md 正文** — 技能相关时阅读。500行以内。传达方法论,而非配方。
139
- 3. **references/** — 按需阅读,获取详细技术参考。
140
- 4. **scripts/** — 可执行代码,可直接运行或修改。
141
- 5. **assets/** — 数据文件、模板、示例。
142
-
143
- 技能传达的是理念和决策框架。请根据具体业务场景灵活运用,不要机械照搬。
144
-
145
- ## 与开发者用户的沟通
146
-
147
- - **主动讨论**:规则粒度、准确率阈值、模型选择、边缘案例。
148
- - **汇报进度**:每轮测试后,分享结果和下一步计划。
149
- - **升级问题**:迭代后仍无法解决的问题,附带证据提交给开发者用户。
150
- - **多问**:开发者用户是领域专家。对规则意图有疑问时,问他们。
@@ -1,188 +0,0 @@
1
- ---
2
- name: skill-to-workflow
3
- description: Distill a proven verification skill into a Python workflow with worker LLM prompts. Use when a rule skill has been tested and reaches the SKILL_ACCURACY threshold defined in .env. Covers the decision of what to implement as code vs LLM calls, prompt engineering for small context windows, model tier selection and progressive downgrade, and testing workflows against the coding agent's own results as ground truth. Also use when optimizing existing workflows for cost or speed.
4
- ---
5
-
6
- # Skill to Workflow
7
-
8
- The skill is the ground truth. The workflow is a cheaper, faster approximation. Your job is to make the approximation as good as the original while being as cheap as possible.
9
-
10
- ## Engineering Goal
11
-
12
- Optimize the full chain: **shortest workflow** (fewest nodes) → **smallest model per node** (cheapest tier that meets accuracy) → **shortest prompt per model** (minimum tokens). This is the engineering objective — not prompt template sophistication or framework compliance.
13
-
14
- ## When to Start
15
-
16
- A skill is ready for workflow distillation when:
17
- - It has been tested on all documents in Samples/.
18
- - Its accuracy meets or exceeds the SKILL_ACCURACY threshold in `.env`.
19
- - Edge cases are documented in the skill's `assets/corner_cases.json`.
20
- - You understand the rule well enough to explain exactly how you verify it.
21
-
22
- If any of these are not true, go back and iterate on the skill first.
23
-
24
- ## The Distillation Decision
25
-
26
- For each step in your skill-based verification process, ask:
27
-
28
- ### Can this be done with regex or Python? (Cost: zero)
29
- - Date extraction with known formats → regex
30
- - Numeric comparison against threshold → Python arithmetic
31
- - Chinese numeral conversion → Python lookup table
32
- - Format validation (ID numbers, codes) → regex
33
- - Table cell extraction from structured markdown → string manipulation
34
-
35
- If yes, write it as code. These are free, fast, and deterministic.
36
-
37
- ### Does this require language understanding? (Cost: worker LLM call)
38
- - Finding the relevant section in a document → LLM
39
- - Extracting an entity described in natural language → LLM
40
- - Judging semantic adequacy ("adequate risk disclosure") → LLM
41
- - Resolving ambiguous references → LLM
42
-
43
- If yes, design a worker LLM prompt. Use the smallest model tier that maintains accuracy.
44
-
45
- ### The hybrid approach (most common)
46
- Most rules are a mix: regex extracts the number, Python compares it to the threshold, LLM handles the exceptional cases. Design the workflow as a pipeline where cheap steps run first and expensive steps run only when needed.
47
-
48
- ### When regex alone isn't enough — decision rubric
49
-
50
- Before declaring distillation complete, audit each rule's `verification_type` / `metric` / `evidence_type` (or equivalent fields in your catalog). For rules where the required verification is one of:
51
-
52
- - **Semantic** ("is this a positive guarantee or a disclaimer?")
53
- - **Contextual** ("interpret this in light of the document's product type")
54
- - **Counterfactual** ("what should this value be, given the other fields?")
55
- - **Cross-field arithmetic** ("does 期初 + 收益 - 分配 = 期末?")
56
-
57
- regex alone rarely suffices. Three acceptable forms:
58
-
59
- 1. **Pure regex with documented limits** — write the regex check, include a comment explaining the fragility (e.g., "matches syntactic pattern only; cannot detect semantic guarantees")
60
- 2. **Hybrid regex + LLM** — regex baseline catches obvious cases, `worker_llm_call` (tier1-2) handles ambiguous ones. The hybrid workflow declares which rule_ids escalate.
61
- 3. **Pure LLM via `worker_llm_call`** — for fully semantic rules where no regex baseline is meaningful.
62
-
63
- Don't ship pure regex for a rule whose `verification_type` is `judgment` / `semantic` without the documented-limits note. Future-you or a colleague will assume the regex is sufficient and that bug will hide for months.
64
-
65
- ### Worker LLM cost-aware tier choice
66
-
67
- If you do escalate to LLM:
68
- - **tier1** (most capable, ~¥0.001-0.002/doc): cross-field reasoning, ambiguity resolution, rules that benefit from chain-of-thought
69
- - **tier2-3**: bulk extraction with simple semantic checks
70
- - **tier4** (cheapest): high-volume keyword-spotting that regex can't handle. Note: tier4 models on SiliconFlow are Qwen3.5 thinking-mode — `content` can return empty if `reasoning_content` consumes max_tokens. Test with realistic prompts before relying. If you see empty responses, either bump max_tokens to ≥8192, shorten your prompt, or fall back to tier1-2.
71
-
72
- Both v0.7.1 audit conductors (DS and GLM) defaulted to all-regex distillation and only added LLM escalation when the human user explicitly asked for "V2 with worker LLM". If your rule catalog has any rules where the verification is genuinely semantic, you should reach for `worker_llm_call` yourself — don't wait to be asked.
73
-
74
- ## Workflow Structure
75
-
76
- A workflow is a Python file (or small set of files) in `workflows/`:
77
-
78
- ```
79
- workflows/
80
- rule_001_capital_adequacy/
81
- workflow_v1.py # The main workflow script
82
- prompts/
83
- extract.txt # Worker LLM prompt for extraction
84
- judge.txt # Worker LLM prompt for judgment (if needed)
85
- config.json # Model assignments, thresholds
86
- ```
87
-
88
- The workflow file should have a clear entry point:
89
-
90
- ```python
91
- def verify(document_text: str, config: dict) -> dict:
92
- """
93
- Returns:
94
- {
95
- "rule_id": "R001",
96
- "result": "pass" | "fail" | "missing" | "error",
97
- "extracted_value": ...,
98
- "confidence": 0.0-1.0,
99
- "comment": "..." (only when fail),
100
- "model_used": "...",
101
- "llm_calls": int,
102
- "llm_tokens": int
103
- }
104
- """
105
- ```
106
-
107
- This is a reference, not a rigid contract. Adapt the structure to the specific rule. The important thing is that every workflow produces a result that can be compared against the skill-based ground truth.
108
-
109
- ## Prompt Engineering for Worker LLMs
110
-
111
- Worker LLMs have smaller context windows (typically 16K-32K tokens). Design prompts that:
112
-
113
- 1. **Are self-contained.** Include everything the model needs in the prompt. Do not assume the model has context from previous calls.
114
- 2. **Specify the output format.** "Return a JSON object with fields: value, confidence, reasoning." Structured output reduces parsing errors.
115
- 3. **Include the narrowed context.** Do not send the entire document. Use the tree-processing pipeline (full document → relevant chapter → relevant section) to narrow the context before calling the worker LLM.
116
- 4. **Are written in the document's language.** Chinese documents get Chinese prompts. English documents get English prompts. Do not mix languages in a single prompt.
117
- 5. **Provide examples sparingly.** One or two examples help. Ten examples waste context window and risk overfitting.
118
-
119
- ## Model Tier Selection
120
-
121
- Start with the highest tier (TIER1) for each step. Measure accuracy. Then try lower tiers:
122
-
123
- 1. Run the workflow with TIER1 on all Samples/. Record accuracy per step.
124
- 2. For each step, try TIER2. If accuracy stays above WORKFLOW_ACCURACY, keep TIER2.
125
- 3. Continue downgrading per step until accuracy drops below threshold.
126
- 4. Record the optimal tier per step in `config.json`.
127
-
128
- Different steps within the same workflow can use different model tiers. Extraction might need TIER2 while judgment might work fine with TIER3.
129
-
130
- ### Formal Downgrade Protocol
131
-
132
- The basic approach above works, but a more rigorous protocol prevents premature tier commitments:
133
-
134
- **Direction**: Start top-down (TIER1 → TIER4) to establish the accuracy ceiling first. You need to know the best possible accuracy before trading it for cost savings.
135
-
136
- **Minimum test runs**: Run at least a meaningful number of documents (e.g., min(10, total_samples)) at each candidate tier before making a tier decision. Small samples are unreliable — a 3-document test could be misleading.
137
-
138
- **Accuracy delta trigger**: If a lower tier's accuracy is significantly below the higher tier (e.g., >5 percentage points), stay at the higher tier for that step. If the delta is within tolerance, use the cheaper tier.
139
-
140
- **Per-step independence**: Each workflow step is assessed separately. Record the optimal tier per step in `config.json`. Do not assume the whole workflow must use one tier.
141
-
142
- **Re-assessment trigger**: If production quality control shows a step's accuracy degrading (e.g., due to new document formats), re-run the tier assessment for that step.
143
-
144
- **Model-task recommendation list**: Maintain a per-project mapping of (task_type → recommended_tier) based on your testing experience. Over time, these lists can be collected across projects to build generalized tier recommendations.
145
-
146
- All numbers here (10 documents, 5 percentage points, etc.) are recommended starting points. The coding agent and developer user should calibrate these — or replace them entirely with a different assessment approach — based on their specific volume, accuracy requirements, and cost constraints. The pattern matters: **test at each tier → compare accuracy → commit when within tolerance → re-assess on degradation**.
147
-
148
- This follows the same tier-transition framework as parser escalation in `document-parsing`: a quality/accuracy score drives the decision to stay, escalate, or skip.
149
-
150
- ## Testing Against Ground Truth
151
-
152
- The coding agent's skill-based results are the ground truth. For each document in Samples/:
153
-
154
- 1. Run the workflow.
155
- 2. Compare the workflow's result against the skill-based result.
156
- 3. Log discrepancies: which step failed, what was expected vs actual.
157
- 4. Compute accuracy: `(matching results) / (total documents)`.
158
- 5. If accuracy < WORKFLOW_ACCURACY, diagnose and fix. Use `evolution-loop` methodology.
159
-
160
- ## Versioning
161
-
162
- Each iteration of a workflow is a new version file: `workflow_v1.py`, `workflow_v2.py`, etc. Track which version is active in `config.json`. See `version-control` skill for the full methodology.
163
-
164
- ## Releasing Workflows
165
-
166
- Once workflows hit accuracy threshold, they can be packaged for end users via the `release` tool. Each release is a self-contained directory under `output/releases/<slug>/` with the pinned workflows, a Python runner, a confidence scorer, an HTML dashboard generator, and a `serve.sh` helper. The bundle has no kc-beta dependency — anyone with Python and a worker LLM API key can run `python run.py <doc>` and produce verification results.
167
-
168
- What to include is your call: all rules in catalog, or a curated subset via the `include` parameter; bundling 1-3 representative samples as `fixtures/` if you want the recipient to be able to dry-run without their own data.
169
-
170
- The `release` tool snapshots the workspace first (git tag `snap/release-<slug>`), so the bundle is regenerable from git even if `output/releases/` is later cleaned. Decide when to release — there's no automation, no forced cadence. Typical triggers: workflows reach SKILL/WORKFLOW_ACCURACY thresholds, a stakeholder needs a hand-off, a production cron should run pinned versions instead of latest. Discuss with the developer user.
171
-
172
- ## Cost Tracking
173
-
174
- Track the cost of each workflow run:
175
- - Number of LLM calls per document.
176
- - Total tokens consumed per document.
177
- - Model tier used per call.
178
-
179
- This data helps the developer user understand the production cost and informs further optimization.
180
-
181
- ## Worker LLM API
182
-
183
- Worker LLMs are accessed via SiliconFlow API. Connection details are in `.env`:
184
- - `SILICONFLOW_API_KEY` for authentication
185
- - `SILICONFLOW_BASE_URL` for the API endpoint
186
- - Model names in `TIER1` through `TIER4`
187
-
188
- See `references/worker-llm-catalog.md` for current model capabilities and context window sizes.