loopforge 1.0.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/autonomous.d.ts +84 -0
- package/dist/autonomous.d.ts.map +1 -0
- package/dist/autonomous.js +250 -0
- package/dist/autonomous.js.map +1 -0
- package/dist/backends/ai.d.ts +37 -0
- package/dist/backends/ai.d.ts.map +1 -0
- package/dist/backends/ai.js +98 -0
- package/dist/backends/ai.js.map +1 -0
- package/dist/cli.d.ts +1 -1
- package/dist/cli.js +68 -6
- package/dist/cli.js.map +1 -1
- package/dist/engine.d.ts +15 -1
- package/dist/engine.d.ts.map +1 -1
- package/dist/engine.js +102 -1
- package/dist/engine.js.map +1 -1
- package/dist/generate-schema.js +56 -4
- package/dist/generate-schema.js.map +1 -1
- package/dist/index.d.ts +23 -5
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +22 -4
- package/dist/index.js.map +1 -1
- package/dist/loop-compiler.d.ts +4 -0
- package/dist/loop-compiler.d.ts.map +1 -1
- package/dist/loop-compiler.js +43 -0
- package/dist/loop-compiler.js.map +1 -1
- package/dist/mcp/server.d.ts +14 -0
- package/dist/mcp/server.d.ts.map +1 -0
- package/dist/mcp/server.js +93 -0
- package/dist/mcp/server.js.map +1 -0
- package/dist/mcp/session.d.ts +57 -0
- package/dist/mcp/session.d.ts.map +1 -0
- package/dist/mcp/session.js +177 -0
- package/dist/mcp/session.js.map +1 -0
- package/dist/mcp/tools.d.ts +108 -0
- package/dist/mcp/tools.d.ts.map +1 -0
- package/dist/mcp/tools.js +191 -0
- package/dist/mcp/tools.js.map +1 -0
- package/dist/mcp-server.d.ts +12 -0
- package/dist/mcp-server.d.ts.map +1 -0
- package/dist/mcp-server.js +14 -0
- package/dist/mcp-server.js.map +1 -0
- package/dist/policy.d.ts +8 -0
- package/dist/policy.d.ts.map +1 -1
- package/dist/policy.js +7 -0
- package/dist/policy.js.map +1 -1
- package/dist/protocol.d.ts +89 -1
- package/dist/protocol.d.ts.map +1 -1
- package/dist/protocol.js +20 -1
- package/dist/protocol.js.map +1 -1
- package/dist/runtime.d.ts +57 -0
- package/dist/runtime.d.ts.map +1 -0
- package/dist/runtime.js +414 -0
- package/dist/runtime.js.map +1 -0
- package/dist/tests/_helpers.d.ts +25 -0
- package/dist/tests/_helpers.d.ts.map +1 -0
- package/dist/tests/_helpers.js +47 -0
- package/dist/tests/_helpers.js.map +1 -0
- package/dist/tests/autonomous.test.d.ts +7 -0
- package/dist/tests/autonomous.test.d.ts.map +1 -0
- package/dist/tests/autonomous.test.js +294 -0
- package/dist/tests/autonomous.test.js.map +1 -0
- package/dist/tests/generate-schema.test.js +2 -2
- package/dist/tests/generate-schema.test.js.map +1 -1
- package/dist/tests/mcp.test.d.ts +7 -0
- package/dist/tests/mcp.test.d.ts.map +1 -0
- package/dist/tests/mcp.test.js +212 -0
- package/dist/tests/mcp.test.js.map +1 -0
- package/dist/tests/replay.test.js +69 -41
- package/dist/tests/replay.test.js.map +1 -1
- package/dist/tests/runtime.test.d.ts +7 -0
- package/dist/tests/runtime.test.d.ts.map +1 -0
- package/dist/tests/runtime.test.js +323 -0
- package/dist/tests/runtime.test.js.map +1 -0
- package/loop_policy.json +7 -0
- package/package.json +9 -3
- package/skills/perception/SKILL.md +186 -0
- package/skills/prompt-techniques/SKILL.md +52 -0
- package/skills/prompt-techniques/references/chain-of-thought.md +207 -0
- package/skills/prompt-techniques/references/few-shot.md +174 -0
- package/skills/prompt-techniques/references/least-to-most.md +182 -0
- package/skills/prompt-techniques/references/step-back.md +160 -0
- package/skills/prompt-techniques/references/tree-of-thought.md +191 -0
- package/skills/prompt-techniques/references/zero-shot.md +97 -0
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: perception
|
|
3
|
+
description: Multi-round autonomous loop driver powered by LoopForge MCP. Compiles context-aware prompts per iteration, tracks quality, inherits constraints, and stops when work is done — not when rounds run out.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Perception — Loop-Time Intelligence Skill
|
|
7
|
+
|
|
8
|
+
**Perception** turns any AI agent into a self-aware multi-round executor. It
|
|
9
|
+
drives the LoopForge MCP server (`loopforge-mcp`) to compile per-iteration
|
|
10
|
+
prompts from running quality trajectory, constraint inheritance, and drift
|
|
11
|
+
detection — maintaining cognitive stability across long-horizon loops.
|
|
12
|
+
|
|
13
|
+
Works with Claude Code, Codex, and any MCP-capable host. Zero configuration
|
|
14
|
+
beyond having `loopforge-mcp` registered as an MCP server.
|
|
15
|
+
|
|
16
|
+
## When to Activate
|
|
17
|
+
|
|
18
|
+
Activate when the user says any of:
|
|
19
|
+
- `/loop <task>` — start an autonomous multi-round loop
|
|
20
|
+
- "do this in a loop", "keep going until done", "iterate on this"
|
|
21
|
+
- Any task that clearly needs 3+ rounds (audit, refactor, migrate, review-and-fix cycles)
|
|
22
|
+
|
|
23
|
+
Also activate implicitly when the user's request is too large for a single
|
|
24
|
+
round (e.g. "audit the entire codebase and fix every issue").
|
|
25
|
+
|
|
26
|
+
## Core Workflow
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
┌─────────────────────────────────────────────────────┐
|
|
30
|
+
│ 1. loopforge_start(task, constraints, maxRounds) │
|
|
31
|
+
│ → sessionId + Round 1 prompt │
|
|
32
|
+
│ │
|
|
33
|
+
│ 2. Execute the prompt (read, write, test, fix) │
|
|
34
|
+
│ MUST include ---loopforge-eval block in output │
|
|
35
|
+
│ │
|
|
36
|
+
│ 3. loopforge_next(sessionId, output) │
|
|
37
|
+
│ → if prompt: go to step 2 │
|
|
38
|
+
│ → if prompt=null: loop ended, read stopReason │
|
|
39
|
+
└─────────────────────────────────────────────────────┘
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
### Step-by-step
|
|
43
|
+
|
|
44
|
+
**Step 1 — Start the loop:**
|
|
45
|
+
```
|
|
46
|
+
Call loopforge_start with:
|
|
47
|
+
task: required — the full task description
|
|
48
|
+
maxRounds: optional — default 20 from policy
|
|
49
|
+
constraints: optional — hard constraints as string[]
|
|
50
|
+
domain: optional — domain hint (solidity, react, rust…)
|
|
51
|
+
|
|
52
|
+
Keep the returned sessionId — you will need it for every subsequent call.
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Step 2 — Execute the round:**
|
|
56
|
+
Execute the compiled prompt exactly as you would any user task. Read files,
|
|
57
|
+
write code, run tests, fix bugs. The prompt already contains the task scope,
|
|
58
|
+
constraints, and quality expectations for this specific round.
|
|
59
|
+
|
|
60
|
+
**CRITICAL: Every round's output MUST end with a self-evaluation block:**
|
|
61
|
+
```
|
|
62
|
+
---loopforge-eval
|
|
63
|
+
{
|
|
64
|
+
"success": true,
|
|
65
|
+
"output_summary": "Fixed 3 reentrancy bugs in withdraw() and deposit(). All 12 tests pass.",
|
|
66
|
+
"constraint_violations": [],
|
|
67
|
+
"should_continue": true
|
|
68
|
+
}
|
|
69
|
+
---end-loopforge-eval
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
**Field rules:**
|
|
73
|
+
|
|
74
|
+
| Field | Type | Rule |
|
|
75
|
+
|-------|------|------|
|
|
76
|
+
| `success` | boolean | `true` ONLY if all hard constraints met AND the task goal achieved |
|
|
77
|
+
| `output_summary` | string | Specific, actionable. What was DONE this round — not what was attempted |
|
|
78
|
+
| `constraint_violations` | string[] | Constraints the agent actually violated this round. Be honest — hiding violations corrupts the quality signal and defeats the circuit breaker |
|
|
79
|
+
| `should_continue` | boolean | `false` ONLY when the ENTIRE task is complete. Partial progress = `true` |
|
|
80
|
+
|
|
81
|
+
**Why this matters:** Without a valid `---loopforge-eval` block, extraction
|
|
82
|
+
fails and the loop stops with `stopReason: stalled`. The circuit breaker
|
|
83
|
+
uses `success` + `violations` to compute quality scores — lying here defeats
|
|
84
|
+
the only mechanism that prevents infinite loops.
|
|
85
|
+
|
|
86
|
+
**Step 3 — Advance to next round:**
|
|
87
|
+
```
|
|
88
|
+
Call loopforge_next with:
|
|
89
|
+
sessionId: from loopforge_start
|
|
90
|
+
output: your FULL output from this round (including the eval block)
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
If the result has `prompt: null`, the loop has ended. Read `stopReason`:
|
|
94
|
+
- `task_complete` — you reported should_continue=false. Work is done.
|
|
95
|
+
- `circuit_breaker` — quality flat or declining for 3+ rounds. Review your approach.
|
|
96
|
+
- `max_rounds` — hit the round limit. Decide whether to extend or accept.
|
|
97
|
+
- `stalled` — no valid self-eval block found. Check your output format.
|
|
98
|
+
|
|
99
|
+
If the result has a non-null `prompt`, execute it and repeat from Step 2.
|
|
100
|
+
|
|
101
|
+
## Tool Reference
|
|
102
|
+
|
|
103
|
+
| Tool | When to call | Key input | Key output |
|
|
104
|
+
|------|-------------|-----------|------------|
|
|
105
|
+
| `loopforge_start` | Beginning of every loop | `task`, `maxRounds?`, `constraints?` | `sessionId`, round 1 `prompt` |
|
|
106
|
+
| `loopforge_next` | After executing every round | `sessionId`, `output` (with eval block) | next `prompt` or `null` + `stopReason` |
|
|
107
|
+
| `loopforge_status` | Mid-loop checkpoint, user asks "how's it going" | `sessionId` | `round`, `qualityTrajectory`, `status` |
|
|
108
|
+
| `loopforge_stop` | User wants to abort, fatal error | `sessionId` | `roundsCompleted`, final trajectory |
|
|
109
|
+
| `loopforge_list` | User asks "what loops are running" | — | `sessions[]` |
|
|
110
|
+
| `loopforge_replay` | After loop ends, user asks "show me what happened" | `sessionId` | `timeline[]` with all rounds |
|
|
111
|
+
|
|
112
|
+
## Quality Signals (Read the Room)
|
|
113
|
+
|
|
114
|
+
The prompt changes between rounds based on your trajectory. Pay attention to:
|
|
115
|
+
|
|
116
|
+
- **Compile level** — `l0` (cached, fast path), `l1` (patched, mild drift),
|
|
117
|
+
`l2` (full recompile, significant change). L2 means the compiler detected a
|
|
118
|
+
major shift — read the prompt carefully.
|
|
119
|
+
- **Warnings** — constraint violations accumulating, task drift, quality
|
|
120
|
+
decline. These are the compiler begging you to course-correct.
|
|
121
|
+
- **Technique rotation** — the compiler may switch from `zero-shot-cot` to
|
|
122
|
+
`tree-of-thought` if quality drops. Follow the new technique.
|
|
123
|
+
|
|
124
|
+
## Stop Condition Matrix
|
|
125
|
+
|
|
126
|
+
| Condition | stopReason | What to do |
|
|
127
|
+
|-----------|-----------|------------|
|
|
128
|
+
| You set `should_continue: false` | `task_complete` | Report final results to user |
|
|
129
|
+
| 3+ rounds of flat/declining quality | `circuit_breaker` | Tell user the approach is stuck, suggest a different strategy |
|
|
130
|
+
| Round count reached `maxRounds` | `max_rounds` | Summarize progress, ask user whether to extend |
|
|
131
|
+
| No `---loopforge-eval` block found | `stalled` | Fix your output format and restart |
|
|
132
|
+
| User interrupts or fatal error | (call `loopforge_stop`) | Clean stop with trajectory preserved |
|
|
133
|
+
|
|
134
|
+
## Rules
|
|
135
|
+
|
|
136
|
+
1. **Never skip the eval block.** No exceptions. If you forget, the loop stalls.
|
|
137
|
+
2. **Be honest in self-evaluation.** Lying about success or hiding violations
|
|
138
|
+
produces a false quality signal. The circuit breaker exists to stop bad
|
|
139
|
+
loops — if you fake quality=5 every round, it fires anyway.
|
|
140
|
+
3. **One loop per task.** Don't reuse a sessionId across different tasks.
|
|
141
|
+
4. **Execute the prompt you're given.** The compiler may change technique or
|
|
142
|
+
narrow scope between rounds. Trust it — it sees the trajectory.
|
|
143
|
+
5. **Call loopforge_next within the same turn.** Don't leave a round hanging
|
|
144
|
+
across conversation turns. The MCP server is in-memory.
|
|
145
|
+
|
|
146
|
+
## Example
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
User: /loop "Audit the ERC20 token in contracts/Token.sol for security issues"
|
|
150
|
+
|
|
151
|
+
Agent:
|
|
152
|
+
→ loopforge_start({ task: "Audit ERC20 token in contracts/Token.sol for security issues",
|
|
153
|
+
domain: "solidity",
|
|
154
|
+
constraints: ["must not break existing tests", "follow check-effects-interactions"] })
|
|
155
|
+
← { sessionId: "abc-123", round: 1, prompt: "## LoopForge Loop Compile — Round 1\n...", level: "l2" }
|
|
156
|
+
|
|
157
|
+
[Executes prompt: reads Token.sol, identifies 4 issues, writes audit notes]
|
|
158
|
+
|
|
159
|
+
Output ends with:
|
|
160
|
+
---loopforge-eval
|
|
161
|
+
{"success":false,"output_summary":"Found 4 potential issues: 2 reentrancy, 1 overflow, 1 access control. Not yet fixed.","constraint_violations":[],"should_continue":true}
|
|
162
|
+
---end-loopforge-eval
|
|
163
|
+
|
|
164
|
+
→ loopforge_next({ sessionId: "abc-123", output: <full output> })
|
|
165
|
+
← { sessionId: "abc-123", round: 2, prompt: "## LoopForge Loop Compile — Round 2\n...", level: "l1" }
|
|
166
|
+
|
|
167
|
+
[Executes prompt: fixes 3 issues, 1 needs more investigation]
|
|
168
|
+
|
|
169
|
+
---loopforge-eval
|
|
170
|
+
{"success":false,"output_summary":"Fixed reentrancy in withdraw() and overflow in calcReward(). Access control issue in setOwner() needs design discussion.","constraint_violations":[],"should_continue":true}
|
|
171
|
+
---end-loopforge-eval
|
|
172
|
+
|
|
173
|
+
→ loopforge_next({ sessionId: "abc-123", output: <full output> })
|
|
174
|
+
← { sessionId: "abc-123", round: 3, prompt: "## LoopForge Loop Compile — Round 3\n...", level: "l0" }
|
|
175
|
+
|
|
176
|
+
[Executes prompt: documents access control issue for user decision, all fixes pass tests]
|
|
177
|
+
|
|
178
|
+
---loopforge-eval
|
|
179
|
+
{"success":true,"output_summary":"All fixable issues resolved. 1 access control design question flagged for user. 24/24 tests pass.","constraint_violations":[],"should_continue":false}
|
|
180
|
+
---end-loopforge-eval
|
|
181
|
+
|
|
182
|
+
→ loopforge_next({ sessionId: "abc-123", output: <full output> })
|
|
183
|
+
← { sessionId: "abc-123", round: 3, prompt: null, stopReason: "task_complete", quality: 5 }
|
|
184
|
+
|
|
185
|
+
Reports to user: "Audit complete in 3 rounds. 3 issues fixed, 1 design question flagged."
|
|
186
|
+
```
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: prompt-techniques
|
|
3
|
+
description: >
|
|
4
|
+
Reference catalog of 7 prompt engineering techniques. This skill should be
|
|
5
|
+
used when the AI needs to recall the purpose, method steps, and use cases
|
|
6
|
+
of techniques like zero-shot, few-shot, chain-of-thought, step-back,
|
|
7
|
+
least-to-most, or tree-of-thought. It is a pure reference skill — no
|
|
8
|
+
workflow instructions, only technique documentation loaded on demand.
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Prompt Techniques Reference
|
|
12
|
+
|
|
13
|
+
This skill catalogs seven prompt engineering techniques. Use `references/` files
|
|
14
|
+
to load detailed technique documentation on demand. Each reference includes:
|
|
15
|
+
purpose, input requirements, recommended JSON input templates, method steps,
|
|
16
|
+
design rules, when-to-use guidance, and example use cases.
|
|
17
|
+
|
|
18
|
+
## Technique Overview
|
|
19
|
+
|
|
20
|
+
| Technique | Cognitive Load | Independence | When to Use |
|
|
21
|
+
|-----------|---------------|-------------|-------------|
|
|
22
|
+
| zero-shot | Low | Either | Simple code explanation, formatting, renaming. |
|
|
23
|
+
| few-shot | Medium | Continuous | Standard CRUD modules, unit tests with fixed patterns. |
|
|
24
|
+
| chain-of-thought | Medium-High | Either | Multi-step reasoning; use `zero-shot-cot` without examples, `few-shot-cot` with examples. |
|
|
25
|
+
| step-back | High | Independent | Vague errors, messy legacy refactoring — abstract principles first. |
|
|
26
|
+
| least-to-most | High | Continuous | Large multi-step modules — decompose into ordered sub-tasks. |
|
|
27
|
+
| tree-of-thought | High | Independent | Core algorithms, crypto/signature verification, Assembly ops — explore multiple candidate paths. |
|
|
28
|
+
|
|
29
|
+
## How to Use
|
|
30
|
+
|
|
31
|
+
1. Read `references/<technique>.md` for the selected technique. Each reference contains:
|
|
32
|
+
- **Purpose** — what problem the technique solves.
|
|
33
|
+
- **Input Requirements** — required and optional parameters.
|
|
34
|
+
- **Recommended JSON Input** — a complete input template with example values.
|
|
35
|
+
- **Method Steps** — step-by-step construction guide.
|
|
36
|
+
- **Design Rules** — specific rules for high-quality prompt construction.
|
|
37
|
+
- **Case Generation Rules** — how to generate examples, decompositions, branches, or abstractions (where applicable).
|
|
38
|
+
- **Single-Pass vs Two-Stage** — when to use each execution mode (step-back, least-to-most).
|
|
39
|
+
- **Search Strategy Selection** — beam/dfs/expert-panel guidance (tree-of-thought).
|
|
40
|
+
- **Prompt Output Template** — a complete 8-section output skeleton with section-level format specifications. Defines the exact structure the generated prompt must follow. The technique-specific section (section 5) format varies per technique: Few-Shot uses input→output pairs, Few-Shot-CoT uses reasoning triples, Step-Back uses abstraction framework boxes, Least-to-Most uses ordered subproblems, Tree-of-Thought uses state tables + evaluation criteria. Zero-Shot omits section 5 entirely for a lightweight 7-section prompt.
|
|
41
|
+
- **When to Use** — concrete scenarios and example use cases.
|
|
42
|
+
2. `references/chain-of-thought.md` covers both `zero-shot-cot` and `few-shot-cot`, including paper citations for Few-Shot-CoT.
|
|
43
|
+
3. Do not load all references at once — read only the one selected by the router in Step 1.
|
|
44
|
+
|
|
45
|
+
## Reference Files
|
|
46
|
+
|
|
47
|
+
- `references/zero-shot.md`
|
|
48
|
+
- `references/few-shot.md`
|
|
49
|
+
- `references/chain-of-thought.md` (covers zero-shot-cot and few-shot-cot)
|
|
50
|
+
- `references/step-back.md`
|
|
51
|
+
- `references/least-to-most.md`
|
|
52
|
+
- `references/tree-of-thought.md`
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# Chain-of-Thought (CoT) Prompting
|
|
2
|
+
|
|
3
|
+
Covers both Zero-Shot-CoT and Few-Shot-CoT.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Zero-Shot-CoT
|
|
8
|
+
|
|
9
|
+
### Purpose
|
|
10
|
+
|
|
11
|
+
适合需要分析、判断或多步推理,但用户未提供推理示例的任务。在 Prompt 中要求模型"一步一步思考 (Let's think step by step)",让模型自行展开推理过程。核心技巧是在 Instruction 末尾加入:**"Let's work this out in a step by step way to be sure we have the right answer."**
|
|
12
|
+
|
|
13
|
+
### Input Requirements
|
|
14
|
+
|
|
15
|
+
- **用户任务**:说明希望模型完成什么任务。
|
|
16
|
+
- **输出格式**:说明最终答案的结构,通常为"先推理过程,再最终答案"。
|
|
17
|
+
- **可选信息**:角色设定、约束条件、是否隐藏详细推理。
|
|
18
|
+
|
|
19
|
+
零样本思维链不插入任务级推理示例;如果用户能提供推理示例,应改用少样本思维链。
|
|
20
|
+
|
|
21
|
+
### Recommended JSON Input
|
|
22
|
+
|
|
23
|
+
```json
|
|
24
|
+
{
|
|
25
|
+
"task": "解决这道多步骤数学题",
|
|
26
|
+
"role": "你是一名严谨的数学推理助手",
|
|
27
|
+
"target_input": "{用户输入的数学题}",
|
|
28
|
+
"output_format": "先输出简明推理过程,再输出最终答案。",
|
|
29
|
+
"constraints": ["检查算术", "最终答案单独成行"]
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### Method Steps
|
|
34
|
+
|
|
35
|
+
1. 在 Prompt 中加入触发语句(如 "请逐步推理"、"Let's think step by step")。
|
|
36
|
+
2. 要求模型先给出推理过程,再给出最终答案。
|
|
37
|
+
3. 推理过程应简洁、可验证,避免冗长发散。
|
|
38
|
+
4. 区分推理和答案的输出位置,确保最终答案清晰可识别。
|
|
39
|
+
|
|
40
|
+
### Design Rules
|
|
41
|
+
|
|
42
|
+
- 保持角色、任务指令、上下文和输出格式明确。
|
|
43
|
+
- 可以提供"示例输出格式"或格式骨架,用于约束结构,但不要把它写成 few-shot-cot。
|
|
44
|
+
- 输出格式应可验证,如字段名、顺序、句数、列表数量、Markdown 样式或 JSON 结构。
|
|
45
|
+
- 使用 `###`、三引号 `"""` 或代码块清晰分隔指令和待处理内容。
|
|
46
|
+
- `{用户输入的文本}` 或 `{text input here}` 必须替换为实际用户文本,或作为待用户后续填入的占位符保留。
|
|
47
|
+
|
|
48
|
+
### When to Use
|
|
49
|
+
|
|
50
|
+
- 用户未提供完整推理示例。
|
|
51
|
+
- 任务需要推理但难度中等(不需要多分支探索)。
|
|
52
|
+
- 推理步骤是可预期的线性过程。
|
|
53
|
+
|
|
54
|
+
### Prompt Output Template (Zero-Shot-CoT)
|
|
55
|
+
|
|
56
|
+
当你用 Zero-Shot-CoT 生成提示词时,按以下 7 节骨架输出。第 5 节为推理骨架(非示例)。
|
|
57
|
+
|
|
58
|
+
| # | 章节 | 必需 | 说明 |
|
|
59
|
+
|---|------|------|------|
|
|
60
|
+
| 1 | 角色 | ✓ | 推理型专家角色 |
|
|
61
|
+
| 2 | 任务 | ✓ | 一句话 |
|
|
62
|
+
| 3 | 输入 | ✓ | 目标问题/数据 |
|
|
63
|
+
| 4 | 输出格式 | ✓ | "先输出推理过程,再输出最终答案" |
|
|
64
|
+
| 5 | **推理骨架(Zero-Shot-CoT)** | ✓ | 不是示例,而是格式提示:`请按以下步骤推理:步骤1: [做什么] → 步骤2: [做什么] → ... → 最终答案:` |
|
|
65
|
+
| 6 | 具体实现要求 | ✓ | 按输出格式展开 |
|
|
66
|
+
| 7 | 硬约束 | ✓ | 含触发语如"请逐步推理"、"Let's think step by step" |
|
|
67
|
+
| 8 | 生成要求 | ✓ | 验收标准 |
|
|
68
|
+
|
|
69
|
+
**质量自检:**
|
|
70
|
+
- [ ] 不含任务级输入/输出示例(那是 Few-Shot-CoT 的事)
|
|
71
|
+
- [ ] 推理骨架只给步骤名,不给具体推理内容
|
|
72
|
+
- [ ] 明确区分推理过程和最终答案的输出位置
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## Few-Shot-CoT
|
|
77
|
+
|
|
78
|
+
### Purpose
|
|
79
|
+
|
|
80
|
+
适合用户提供完整推理示例(问题→推理→答案三元组)的任务。让模型模仿示例中的分步推理方式解决新问题。适用于前后功能依赖但任务极难的场景(接力推理)。
|
|
81
|
+
|
|
82
|
+
Few-Shot-CoT 的关键不是只给 `输入 -> 输出`,而是给出 `问题/输入 -> 推理过程 -> 最终答案` 的示例三元组,让模型模仿示例中的分步推理方式,再解决新的目标问题。若用户没有提供完整推理案例,不追问,自动退回零样本思维链。
|
|
83
|
+
|
|
84
|
+
### Input Requirements
|
|
85
|
+
|
|
86
|
+
- **用户任务**:说明希望模型解决哪类问题,如数学应用题、常识推理、符号推理、日期计算、代码逻辑分析等。
|
|
87
|
+
- **目标问题**:用户希望最终处理的真实问题,或 `{用户输入的问题}` 占位符。
|
|
88
|
+
- **可选推理示例**:用户可以在第一次请求中直接提供 1~5 组与目标任务同类型的示例。
|
|
89
|
+
- **少样本思维链案例推荐包含**:问题/输入、推理过程、最终答案。
|
|
90
|
+
- **输出格式**:说明最终 Prompt 应要求模型输出哪些部分,如"推理过程 + 最终答案"、JSON、选项字母、代码块等。
|
|
91
|
+
|
|
92
|
+
若用户已附上完整推理案例,直接吸收案例中的分步推理方式和答案格式;若未检测到完整推理案例,不要追问或中断流程,按零样本思维链生成。
|
|
93
|
+
|
|
94
|
+
### Recommended JSON Input
|
|
95
|
+
|
|
96
|
+
```json
|
|
97
|
+
{
|
|
98
|
+
"task": "解决小学数学应用题",
|
|
99
|
+
"role": "你是一名严谨的数学解题助手",
|
|
100
|
+
"examples": [
|
|
101
|
+
{
|
|
102
|
+
"input": "小明有 5 支铅笔,又买了 2 盒,每盒 3 支。他现在有几支铅笔?",
|
|
103
|
+
"reasoning": "小明原来有 5 支铅笔。2 盒铅笔每盒 3 支,所以新增 2 × 3 = 6 支。总数是 5 + 6 = 11 支。",
|
|
104
|
+
"output": "11"
|
|
105
|
+
}
|
|
106
|
+
],
|
|
107
|
+
"target_input": "{用户输入的数学应用题}",
|
|
108
|
+
"output_format": "先输出推理过程,再输出最终答案。",
|
|
109
|
+
"constraints": ["推理步骤要简洁", "最终答案单独成行"]
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Case Rules
|
|
114
|
+
|
|
115
|
+
1. `examples` 可省略;省略时按零样本思维链生成,并在示例区说明原因。
|
|
116
|
+
2. 如果提供 `examples`,校验数量为 1~5,且每组都有非空 `input`、`reasoning` 和 `output`。
|
|
117
|
+
3. 将完整示例格式化为 `示例 N / 问题 / 推理过程 / 最终答案`,插入生成的 Prompt。
|
|
118
|
+
4. 推理示例不要只写"因为显然如此",要写出关键中间步骤。
|
|
119
|
+
|
|
120
|
+
### Paper Citations
|
|
121
|
+
|
|
122
|
+
根据《Chain-of-Thought Prompting Elicits Reasoning in Large Language Models》的方法,少样本思维链通过在提示词中提供带有自然语言中间推理步骤的示例,帮助模型处理多步推理任务。核心启发:
|
|
123
|
+
|
|
124
|
+
- 示例应是三元组:输入、思维链推理过程、最终输出。
|
|
125
|
+
- 推理过程应位于最终答案之前,让模型先分解问题再回答。
|
|
126
|
+
- 自然语言推理步骤比只给公式或只增加 token 更有效。
|
|
127
|
+
- 该方法更适合需要多步推理的任务,如算术推理、常识推理、符号操作和状态追踪。
|
|
128
|
+
- 简单单步任务不一定需要少样本思维链,可使用零样本提示或少样本提示。
|
|
129
|
+
|
|
130
|
+
### Method Steps
|
|
131
|
+
|
|
132
|
+
1. 将用户示例格式化为:示例N / 问题 / 推理过程 / 最终答案。
|
|
133
|
+
2. 示例应展示完整的中间推理步骤,不只是"显然如此"。
|
|
134
|
+
3. 在生成的 Prompt 末尾放置目标问题,要求模型模仿示例的推理结构。
|
|
135
|
+
4. 如果示例中推理不完整,退回 zero-shot-cot。
|
|
136
|
+
|
|
137
|
+
### When to Use
|
|
138
|
+
|
|
139
|
+
- 用户提供了 1-5 组高质量推理示例。
|
|
140
|
+
- 前后任务高度依赖,需要推理接力。
|
|
141
|
+
- 常见场景:连续的安全审计阶段、多步逻辑模块开发。
|
|
142
|
+
|
|
143
|
+
### Prompt Output Template (Few-Shot-CoT)
|
|
144
|
+
|
|
145
|
+
当你用 Few-Shot-CoT 生成提示词时,按以下 8 节骨架输出。第 5 节为推理模式参考。
|
|
146
|
+
|
|
147
|
+
| # | 章节 | 必需 | 说明 |
|
|
148
|
+
|---|------|------|------|
|
|
149
|
+
| 1 | 角色 | ✓ | 推理型专家 |
|
|
150
|
+
| 2 | 任务 | ✓ | 一句话 |
|
|
151
|
+
| 3 | 输入 | ✓ | 目标问题/模块 |
|
|
152
|
+
| 4 | 输出格式 | ✓ | "1. DDL, 2. API, 3. 业务代码..." |
|
|
153
|
+
| 5 | **推理模式参考(Few-Shot Examples)** | ✓ | 见下方格式规范 |
|
|
154
|
+
| 6 | 具体实现要求 | ✓ | 每个 API 端点附推理链 |
|
|
155
|
+
| 7 | 硬约束 | ✓ | 编号列表 |
|
|
156
|
+
| 8 | 生成要求 | ✓ | 验收标准 |
|
|
157
|
+
|
|
158
|
+
**第 5 节格式规范:** 必须包含两部分:
|
|
159
|
+
|
|
160
|
+
**A. 2 组输入→推理→输出三元组**,每组格式:
|
|
161
|
+
|
|
162
|
+
```
|
|
163
|
+
### 示例 N:[场景]
|
|
164
|
+
|
|
165
|
+
| 维度 | 内容 |
|
|
166
|
+
|------|------|
|
|
167
|
+
| 输入 | [具体输入] |
|
|
168
|
+
| 输出 | [期望输出] |
|
|
169
|
+
| 推理 | [一句话概括推理方向] |
|
|
170
|
+
|
|
171
|
+
**推理模式拆解:**
|
|
172
|
+
步骤1: [做什么]
|
|
173
|
+
步骤2: [做什么]
|
|
174
|
+
...
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
**B. 推理模式迁移规则框**:一个 ASCII 框图,从示例中提取通用推理模式并分层描述(如:参数校验层→业务校验层→业务处理层→数据持久化层→响应组装层)。
|
|
178
|
+
|
|
179
|
+
格式范本(来自 MCP 黄金标准 `prompt_01_user_system`):
|
|
180
|
+
|
|
181
|
+
```
|
|
182
|
+
┌─────────────────────────────────────────────────────────────┐
|
|
183
|
+
│ 通用推理模式 │
|
|
184
|
+
├─────────────────────────────────────────────────────────────┤
|
|
185
|
+
│ 1. 参数校验层 — 检查必填参数、格式、业务约束 │
|
|
186
|
+
│ 2. 业务校验层 — 检查冲突、合法性、权限 │
|
|
187
|
+
│ 3. 业务处理层 — 执行核心逻辑 │
|
|
188
|
+
│ 4. 数据持久化层 — 写入 DB/缓存,处理事务 │
|
|
189
|
+
│ 5. 响应组装层 — 统一格式,敏感字段脱敏 │
|
|
190
|
+
└─────────────────────────────────────────────────────────────┘
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
**质量自检:**
|
|
194
|
+
- [ ] 推理过程写出关键中间步骤,不是"因为显然如此"
|
|
195
|
+
- [ ] 迁移规则框是分层框架,不是单个步骤的复述
|
|
196
|
+
- [ ] 三元组的推理模式与目标任务同类型
|
|
197
|
+
|
|
198
|
+
---
|
|
199
|
+
|
|
200
|
+
## Common Design Rules (Both CoT Variants)
|
|
201
|
+
|
|
202
|
+
- 推理过程应优先 2-3 个示例。
|
|
203
|
+
- 对选择题,最终答案应明确选项字母和选项内容。
|
|
204
|
+
- 对数学、日期、状态追踪等任务,推理中要保留关键计算或状态变化。
|
|
205
|
+
- 使用链式推理(CoT)时,要求模型公开简洁可验证的理由,而不是隐藏思维链。
|
|
206
|
+
- 如果用户不希望暴露完整推理,可在生成的 Prompt 中要求输出"简要依据 + 最终答案",但示例仍应体现推理结构。
|
|
207
|
+
- 输出格式要区分推理过程和最终答案,避免模型把答案埋在推理中。
|
|
@@ -0,0 +1,174 @@
|
|
|
1
|
+
# Few-Shot Prompting
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
适合已有示例、需要复用格式、标签或表达风格的任务。通过提供 1-5 组输入/输出示例让模型理解任务模式和输出风格。用于标准 CRUD 模块、常规单元测试等有固定范式的任务。
|
|
6
|
+
|
|
7
|
+
## Input Requirements
|
|
8
|
+
|
|
9
|
+
- **用户任务**:说明希望模型完成什么任务。
|
|
10
|
+
- **输出格式**:说明最终回答的结构、字段、长度、语言或代码格式。
|
|
11
|
+
- **可选少样本示例**:用户可以在第一次请求中直接提供 1~5 组 `输入 -> 期望输出`,或 `输入 -> 推理过程 -> 期望输出`,用于展示任务模式、判断依据和输出风格。
|
|
12
|
+
- **可选信息**:角色设定、待处理文本、约束条件、目标用户或使用场景。
|
|
13
|
+
|
|
14
|
+
示例不是必需项。若用户已经附上案例,直接吸收案例中的输入输出映射和输出风格;若未检测到案例,不要追问或中断流程,按零样本提示生成,并在生成结果中简短说明"未检测到用户案例,因此按零样本提示生成"。不要替用户编造隐藏思维链;只保存用户明确提供或可公开展示的推理摘要。
|
|
15
|
+
|
|
16
|
+
## Recommended JSON Input
|
|
17
|
+
|
|
18
|
+
```json
|
|
19
|
+
{
|
|
20
|
+
"task": "从客服对话中抽取客户诉求和情绪",
|
|
21
|
+
"role": "你是一名严谨的信息抽取提示词设计师",
|
|
22
|
+
"examples": [
|
|
23
|
+
{
|
|
24
|
+
"input": "客户:我等了三天还没收到退款。",
|
|
25
|
+
"reasoning": "用户关注退款进度,表达等待过久带来的不满。",
|
|
26
|
+
"output": "{\"诉求\":\"查询退款进度\",\"情绪\":\"不满\"}"
|
|
27
|
+
}
|
|
28
|
+
],
|
|
29
|
+
"target_input": "{用户输入的客服对话}",
|
|
30
|
+
"output_format": "仅输出 JSON,字段为 诉求、情绪。",
|
|
31
|
+
"constraints": ["不要输出解释", "字段缺失时填 null"],
|
|
32
|
+
"persist_sample": true
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Case Injection Pipeline
|
|
37
|
+
|
|
38
|
+
### Step 1: 检测(Detect)
|
|
39
|
+
|
|
40
|
+
检查用户请求中是否包含 `examples` 字段或自然语言描述的输入→输出样例。识别示例中的输入输出映射和输出风格。
|
|
41
|
+
|
|
42
|
+
### Step 2: 校验(Validate)
|
|
43
|
+
|
|
44
|
+
- 如果提供 `examples`,校验数量为 1~5。
|
|
45
|
+
- 每组必须有非空 `input` 和 `output`;`reasoning` 可选。
|
|
46
|
+
- 示例应与用户任务同类型,避免混入无关领域案例。
|
|
47
|
+
- 没有示例时不要臆造示例,按零样本生成即可。
|
|
48
|
+
|
|
49
|
+
### Step 3: 格式化(Format)
|
|
50
|
+
|
|
51
|
+
将示例格式化为清晰的结构,插入生成 Prompt 的示例区:
|
|
52
|
+
|
|
53
|
+
```
|
|
54
|
+
示例 1:
|
|
55
|
+
输入: {input}
|
|
56
|
+
推理过程: {reasoning} (可选,无则省略此块)
|
|
57
|
+
输出: {output}
|
|
58
|
+
|
|
59
|
+
示例 2:
|
|
60
|
+
输入: ...
|
|
61
|
+
输出: ...
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### Step 4: 注入(Inject)
|
|
65
|
+
|
|
66
|
+
将格式化后的示例插入到最终的 Prompt 中,位于任务指令之后、目标输入之前。在 Prompt 末尾放置目标输入,要求模型按示例风格输出。
|
|
67
|
+
|
|
68
|
+
## Method Steps
|
|
69
|
+
|
|
70
|
+
1. 从用户提供的示例中提取输入到输出的映射规则。
|
|
71
|
+
2. 在最终 Prompt 中包含 1-5 组格式化的示例(推荐 2-3 个)。
|
|
72
|
+
3. 示例应展示:输入 → 期望输出 的完整对应关系。
|
|
73
|
+
4. 在 Prompt 末尾放置目标输入,要求模型按示例风格输出。
|
|
74
|
+
5. 如果用户未提供示例,退回 zero-shot 并简短说明。
|
|
75
|
+
|
|
76
|
+
## Dynamic Sample Persistence
|
|
77
|
+
|
|
78
|
+
> **Status: Future feature — not yet implemented.** The `persist_sample: true` flag
|
|
79
|
+
> and sample library described below are a planned enhancement. Currently, samples
|
|
80
|
+
> live only within the session — they are not persisted across sessions by any
|
|
81
|
+
> script. The checkpoint.py + hydrate.py vault stores prompts and summaries, not
|
|
82
|
+
> raw input→output examples.
|
|
83
|
+
|
|
84
|
+
当用户明确要求"把这个例子保存下来""加入样例库"或 payload 设置 `persist_sample: true` 时,将本轮示例追加到样本库。追加条目应包含:
|
|
85
|
+
|
|
86
|
+
- 任务、角色、输出格式、待处理内容占位和约束。
|
|
87
|
+
- 1~5 组用户示例(输入、可选推理过程、输出)。
|
|
88
|
+
- 本轮生成的完整少样本 Prompt。
|
|
89
|
+
|
|
90
|
+
若示例里包含隐私、密钥、真实身份信息或用户没有明确要求持久化,先脱敏或仅保存在 session 临时区。追加而非覆盖已有样本。
|
|
91
|
+
|
|
92
|
+
## When to Use
|
|
93
|
+
|
|
94
|
+
- 任务有固定的输入输出模式(如代码转换、格式化、分类)。
|
|
95
|
+
- 用户能提供高质量示例。
|
|
96
|
+
- 输出风格或格式需要严格遵循约定。
|
|
97
|
+
- 常见场景:标准 CRUD 模块、单元测试编写、代码风格转换、文本分类。
|
|
98
|
+
|
|
99
|
+
## Example Use Cases
|
|
100
|
+
|
|
101
|
+
- "根据这些示例,把后面的函数也写成同样的风格"
|
|
102
|
+
- "参考前两个单元测试,为这个新函数写测试"
|
|
103
|
+
- "模仿这两个 API 端点的写法,实现第三个端点"
|
|
104
|
+
|
|
105
|
+
## Prompt Output Template
|
|
106
|
+
|
|
107
|
+
当你用 Few-Shot 技术生最终提示词时,**必须**按以下 8 节骨架输出。技术特定的第 5 节格式要求见下方详细说明。
|
|
108
|
+
|
|
109
|
+
### 章节骨架
|
|
110
|
+
|
|
111
|
+
| # | 章节 | 必需 | 说明 |
|
|
112
|
+
|---|------|------|------|
|
|
113
|
+
| 1 | 角色 | ✓ | 领域专家 + 技术栈,如"你是一名资深 Go 后端工程师,精通 Gin + MySQL" |
|
|
114
|
+
| 2 | 任务 | ✓ | 一句话说清要做什么 |
|
|
115
|
+
| 3 | 输入 | ✓ | 目标数据/代码/模块的描述 |
|
|
116
|
+
| 4 | 输出格式 | ✓ | 编号列表,每项对应下方具体实现的子节。如"1. DDL, 2. API接口, 3. 校验逻辑, 4. 业务代码, 5. 单元测试" |
|
|
117
|
+
| 5 | **格式参考示例(Few-Shot)** | ✓ | 见下方详细格式规范 |
|
|
118
|
+
| 6 | 具体实现要求 | ✓ | 按输出格式的编号逐节展开,含代码示例、DDL、API 示例等 |
|
|
119
|
+
| 7 | 硬约束 | ✓ | 编号列表,含技术栈、范围值、架构限制等 |
|
|
120
|
+
| 8 | 生成要求 | ✓ | 编号列表,最终验收标准 |
|
|
121
|
+
|
|
122
|
+
### 第 5 节格式规范:格式参考示例(Few-Shot)
|
|
123
|
+
|
|
124
|
+
这是本技术最关键的章节。必须包含:
|
|
125
|
+
|
|
126
|
+
**A. 2-3 组输入→输出对**,每组格式:
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
### 示例 N:[场景描述]
|
|
130
|
+
|
|
131
|
+
**输入:**
|
|
132
|
+
[具体的输入数据,如 API 请求体、代码片段、数据结构]
|
|
133
|
+
|
|
134
|
+
**输出:**
|
|
135
|
+
[对应的期望输出,如 API 响应、代码结果、处理结果]
|
|
136
|
+
|
|
137
|
+
**映射规则:** [一句话解释输入如何变成输出]
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
**B. 输入→输出映射规则总结框**:一个 ASCII 框图,汇总所有示例中提取的通用规则。
|
|
141
|
+
|
|
142
|
+
格式范本(来自 MCP 黄金标准 `prompt_02_vital_signs`):
|
|
143
|
+
|
|
144
|
+
```
|
|
145
|
+
┌─────────────────────────────────────────────────────────────────┐
|
|
146
|
+
│ 数值校验映射规则 │
|
|
147
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
148
|
+
│ │
|
|
149
|
+
│ 输入值 → 判断1: 是否在有效范围内? │
|
|
150
|
+
│ ├─ 否 → 返回错误码 + 拒绝录入 │
|
|
151
|
+
│ └─ 是 → 判断2: 是否在正常范围内? │
|
|
152
|
+
│ ├─ 是 → status: "normal" │
|
|
153
|
+
│ └─ 否 → status: "abnormal" + warning │
|
|
154
|
+
│ │
|
|
155
|
+
└─────────────────────────────────────────────────────────────────┘
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
### 质量自检
|
|
159
|
+
|
|
160
|
+
- [ ] 示例是**任务领域的真实输入→输出对**,不是"提示词设计的元示例"
|
|
161
|
+
- [ ] 每组示例有明确的映射规则说明
|
|
162
|
+
- [ ] 映射规则总结框放在所有示例之后
|
|
163
|
+
- [ ] 输出格式的编号与具体实现要求的子节编号一一对应
|
|
164
|
+
- [ ] 硬约束≥8 条,含技术栈、数值范围、架构限制
|
|
165
|
+
|
|
166
|
+
## Notes
|
|
167
|
+
|
|
168
|
+
- 少样本示例如果存在,必须体现"输入如何变成输出",不要只写输出格式骨架。
|
|
169
|
+
- "推理过程"应是可公开的简要判断依据,用来帮助样例复用;不要保存隐藏思维链、敏感信息或不可公开的内部分析。
|
|
170
|
+
- 示例应与用户任务同类型,避免混入无关领域案例;没有示例时不要臆造示例,按零样本生成即可。
|
|
171
|
+
- 示例数量优先 2~3 个;任务简单可用 1 个,格式复杂可用 4~5 个。
|
|
172
|
+
- 输出格式要量化,如字段名、顺序、句数、Markdown 样式或 JSON Schema。
|
|
173
|
+
- 生成的 Prompt 中保留 `{用户输入的文本}` 等占位符,除非用户已经提供真实待处理内容。
|
|
174
|
+
- 案例仅供参考,不可直接复制为用户任务的生成内容。
|