@ckelsoe/prompt-architect 3.4.0 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -5,14 +5,14 @@
5
5
  "email": "charles@kelsoe.com"
6
6
  },
7
7
  "metadata": {
8
- "description": "Prompt engineering skill using 29 frameworks across 7 intent categories"
8
+ "description": "Prompt engineering skill using 31 frameworks across 7 intent categories"
9
9
  },
10
10
  "plugins": [
11
11
  {
12
12
  "name": "prompt-architect",
13
13
  "source": "./",
14
- "description": "Analyzes and improves prompts using 29 frameworks. Intent-based selection routes to the right framework — from ultra-minimal (APE) to agentic (ReAct).",
15
- "version": "3.4.0",
14
+ "description": "Analyzes and improves prompts using 31 frameworks. Intent-based selection routes to the right framework — from ultra-minimal (APE) to agentic (ReAct).",
15
+ "version": "3.5.0",
16
16
  "keywords": ["prompt-engineering", "prompt-frameworks", "agent-skills"],
17
17
  "category": "productivity"
18
18
  }
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "prompt-architect",
3
- "description": "29-framework prompt engineering skill for analyzing and improving prompts across 7 intent categories",
4
- "version": "3.4.0",
3
+ "description": "31-framework prompt engineering skill for analyzing and improving prompts across 7 intent categories",
4
+ "version": "3.5.0",
5
5
  "author": {
6
6
  "name": "Charles Kelsoe",
7
7
  "email": "charles@kelsoe.com"
package/CHANGELOG.md CHANGED
@@ -5,6 +5,21 @@ All notable changes to the Prompt Architect Claude Code skill will be documented
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [3.5.0] - 2026-07-22
9
+
10
+ Adds two research-backed frameworks, one composable technique, and a guardrail against unfalsifiable framework-selection rationale. Framework count is now 31.
11
+
12
+ ### Added
13
+ - **Self-Consistency** (REASON) — sample the same reasoning prompt N times at non-zero temperature and take a plain majority vote over the final answers, aggregated *outside* the model. Cited to Wang et al., "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (arXiv 2203.11171, ICLR 2023). The doc is explicit that this is *not* self-assessment (asking one model to judge its own paths) and that the emitted prompt is a single solve-prompt ending in a parseable `FINAL ANSWER:` line — the N-sample vote is an operational step the analysis section instructs, not something one pasted prompt can do. Absolute figures (56.5% → 74.4%, PaLM 540B / GSM8K) are labeled Table 1 body figures; the +17.9% is the abstract's relative delta.
14
+ - **Chain-of-Verification** (CRITIQUE) — draft an answer, plan verification questions, answer them **independently of the draft**, then revise. That independence is the mechanism: it stops the verification from inheriting the draft's errors. Cited to Dhuliawala et al. (arXiv 2309.11495, Findings of ACL 2024). Distinct from RCoT, which reconstructs the question to catch missed *conditions*; CoVe fact-checks *claims*.
15
+ - **Few-shot / in-context learning** as a **composable technique**, not a routed framework. It lives in `references/techniques/few-shot.md` and layers onto whichever framework was chosen — covering when examples help, how many (2–5), ordering and recency effects (Lu et al., ACL 2022), what examples actually teach (format and label space over per-example correctness — Min et al., EMNLP 2022), and calibration/bias balancing (Zhao et al., ICML 2021). Brown et al. (arXiv 2005.14165) is cited only for the term, never for a number. A new "Composable Techniques" section in `SKILL.md` and the adapter routes to it.
16
+
17
+ ### Changed
18
+ - **Anti-false-narration guardrail.** Because section headers are stripped at emission, the framework choice is often invisible in the delivered prompt — so `SKILL.md` and the adapter now instruct: when two frameworks would produce the same prompt, say so, pick the simpler one, and do not manufacture a distinction the output won't show. A new drift-test invariant keeps the rule present in both files.
19
+ - Framework count 29 → 31 across all advertised sites (`package.json` description and `claudeCode.frameworks`, `plugin.json`, `marketplace.json`, `README.md`, `SKILL.md`, adapter).
20
+
21
+ ---
22
+
8
23
  ## [3.4.0] - 2026-07-22
9
24
 
10
25
  This release makes the **emission contract** explicit and consistent across the whole skill. When Prompt Architect delivers the final prompt, framework section headers (`CONTEXT:`, `ROLE:`, `BEFORE:`, …) are stripped — the user pastes a flat block of prose with no labels. Previously the framework docs and templates were written as if those headers would survive, so slots that were bare fragments ("meeting summary", "busy professionals") lost their meaning once the header was removed. Every framework now carries its own meaning in prose.
package/README.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Prompt Architect
2
2
 
3
- Transform vague prompts into expert-level, structured prompts using 29 frameworks across 7 intent categories.
3
+ Transform vague prompts into expert-level, structured prompts using 31 frameworks across 7 intent categories.
4
4
 
5
5
  Works with **Claude Code, ChatGPT, Gemini CLI, Cursor, GitHub Copilot, Windsurf, OpenAI Codex**, and [30+ Agent Skills compatible tools](https://agentskills.io).
6
6
 
@@ -60,7 +60,7 @@ Prompt Architect is an [Agent Skills](https://agentskills.io) compatible skill t
60
60
 
61
61
  ## Key Features
62
62
 
63
- ### 29 Frameworks Across 7 Intent Categories
63
+ ### 31 Frameworks Across 7 Intent Categories
64
64
 
65
65
  | Framework | Best For | Complexity |
66
66
  |-----------|----------|------------|
@@ -84,6 +84,7 @@ Prompt Architect is an [Agent Skills](https://agentskills.io) compatible skill t
84
84
  | **Least-to-Most** | Compositional multi-hop problems (simplest first) | Medium |
85
85
  | **Plan-and-Solve (PS+)** | Zero-shot numerical/calculation reasoning | Low |
86
86
  | **Chain of Thought** | Reasoning, problem-solving | Medium |
87
+ | **Self-Consistency** | Robust answers: sample many reasoning paths, majority-vote | Medium |
87
88
  | **Chain of Density** | Summarization at fixed length, maximum information density | Medium |
88
89
  | **Iterative Compression** | Shortening text toward a target length | Low |
89
90
  | **Self-Refine** | Iterative output quality improvement (any task) | Medium |
@@ -91,9 +92,12 @@ Prompt Architect is an [Agent Skills](https://agentskills.io) compatible skill t
91
92
  | **Devil's Advocate** | Strongest opposing argument against a position | Low |
92
93
  | **Pre-Mortem** | Assume failure, identify specific causes | Low |
93
94
  | **RCoT** | Verify reasoning by reconstructing the question | Medium |
95
+ | **Chain-of-Verification** | Fact-check a draft by verifying each claim independently | Medium |
94
96
  | **RPEF** | Recover/reconstruct a prompt from an existing output | Low |
95
97
  | **Reverse Role Prompting** | AI interviews you before executing | Low |
96
98
 
99
+ **Composable technique** (layered onto any framework, not one of the 31): **Few-shot / in-context examples** — the discipline of adding well-chosen, well-ordered worked examples to a prompt.
100
+
97
101
  ### Quality Scoring System
98
102
 
99
103
  Every prompt is evaluated across:
@@ -908,7 +912,7 @@ prompt-architect/
908
912
 
909
913
  **Core Components:**
910
914
  - **SKILL.md** - Main skill logic and instructions (intent-based selection system)
911
- - **Framework Reference Docs** - 28 files covering all 29 frameworks, with examples (RISE documents two variants)
915
+ - **Framework Reference Docs** - 30 files covering all 31 frameworks, with examples (RISE documents two variants)
912
916
  - **Templates** - 30 ready-to-use structures: one per framework, plus `hybrid_template.txt` for combinations
913
917
 
914
918
  ---
@@ -36,7 +36,9 @@ Score the prompt *as written*, not as you charitably interpret it — the gap be
36
36
 
37
37
  ### 2. Intent-Based Framework Selection
38
38
 
39
- With 29 frameworks, identify the user's **primary intent** first, then use the discriminating questions within that category.
39
+ With 31 frameworks, identify the user's **primary intent** first, then use the discriminating questions within that category.
40
+
41
+ **When two frameworks would produce the same prompt, say so and pick the simpler one.** Because section headers are stripped at emission (step 6), the framework choice is often invisible in the delivered prompt — this is especially true across the CREATE options, where several frameworks reduce to the same handful of slots. When you cannot point to a concrete difference the *emitted* prompt would show, do not manufacture one: name the tie plainly, choose the simpler framework, and move on. A confident rationale for an unobservable choice is exactly the overstatement this skill exists to remove.
40
42
 
41
43
  ---
42
44
 
@@ -94,6 +96,7 @@ With 29 frameworks, identify the user's **primary intent** first, then use the d
94
96
  | Multiple distinct approaches to compare | **Tree of Thought** |
95
97
  | Verify reasoning didn't overlook conditions | **RCoT** |
96
98
  | Linear step-by-step reasoning | **Chain of Thought** |
99
+ | Answer must be robust; sample many paths and majority-vote | **Self-Consistency** |
97
100
 
98
101
  ---
99
102
 
@@ -106,8 +109,9 @@ With 29 frameworks, identify the user's **primary intent** first, then use the d
106
109
  | Find the strongest opposing argument | **Devil's Advocate** |
107
110
  | Identify failure modes before they happen | **Pre-Mortem** |
108
111
  | Verify reasoning didn't miss conditions | **RCoT** |
112
+ | Draft may contain hallucinated facts; verify each claim | **Chain-of-Verification** |
109
113
 
110
- *Self-Refine = any quality. CAI = principle compliance. Devil's Advocate = opposing arguments. Pre-Mortem = failure analysis. RCoT = condition verification.*
114
+ *Self-Refine = any quality. CAI = compliance with an **explicitly stated** standard or requirement set (and aligning the artifact to it — e.g. auditing a plan against a brief's constraints). Devil's Advocate = opposing arguments. Pre-Mortem = failure analysis. RCoT = an answer or plan overlooked a condition **implicit in the problem** (units, edge cases, unstated dependencies). Chain-of-Verification = independent fact-checking of a draft's factual claims.*
111
115
 
112
116
  ---
113
117
 
@@ -129,6 +133,12 @@ Most prompts need exactly one framework. Combine only when the task genuinely ha
129
133
 
130
134
  When you combine, state plainly in your analysis which framework owns which phase. Never stack more than two — beyond that the frameworks' instructions start to overlap and contradict, and no single framework clearly owns any phase.
131
135
 
136
+ #### Composable Techniques
137
+
138
+ Some techniques are not frameworks you choose *between* — they are layers you add *on top of* whichever framework you picked, so they never appear in the routing tables above.
139
+
140
+ - **Few-shot / in-context examples** — showing 2–5 worked input→output examples inside the emitted prompt. This is the highest-leverage technique in prompting and applies to almost any framework, not just the two with a dedicated examples slot (CARE, RISE-IX). After you draft the framework prompt, decide whether examples earn their place; if they do, insert them before the final instruction, in the exact target output format, and end with the actual task. Keep examples to a handful, put the most representative one last (models weight the most recent example most heavily), keep every example in the exact format you want back, and balance the label set. Never invent examples the user or their material did not supply — fall back to a precise instruction instead.
141
+
132
142
  ---
133
143
 
134
144
  ### 3. Framework Quick Reference
@@ -137,12 +147,14 @@ When you combine, state plainly in your analysis which framework owns which phas
137
147
  **Medium:** RACE | CARE | BAB | BROKE | CRISPE
138
148
  **Comprehensive:** CO-STAR | RISEN | TIDD-EC
139
149
  **Data:** RISE-IE | RISE-IX
140
- **Reasoning:** Plan-and-Solve | Chain of Thought | Least-to-Most | Step-Back | Tree of Thought | RCoT
150
+ **Reasoning:** Plan-and-Solve | Chain of Thought | Least-to-Most | Step-Back | Tree of Thought | RCoT | Self-Consistency
141
151
  **Structure/Iteration:** Skeleton of Thought | Chain of Density | Iterative Compression
142
- **Critique/Quality:** Self-Refine | CAI Critique-Revise | Devil's Advocate | Pre-Mortem
152
+ **Critique/Quality:** Self-Refine | CAI Critique-Revise | Devil's Advocate | Pre-Mortem | Chain-of-Verification
143
153
  **Meta/Reverse:** RPEF | Reverse Role Prompting
144
154
  **Agentic:** ReAct
145
155
 
156
+ **Composable technique (layered onto any framework, not selected between):** Few-shot / in-context examples
157
+
146
158
  ### 4. Clarification Questions
147
159
 
148
160
  Ask targeted questions (3-5 at a time) based on identified gaps:
@@ -167,6 +179,7 @@ Ask targeted questions (3-5 at a time) based on identified gaps:
167
179
  **For Least-to-Most**: The full problem in one statement, paste the material the subproblems must reason over, what is the simplest thing that must be answered first, what does the final answer depend on?
168
180
  **For Plan-and-Solve**: The problem with every number, unit, and constraint written out, paste the dataset or figures the calculation runs on if any, which values are given and which must be derived?
169
181
  **For Chain of Thought**: The problem with all its conditions stated, paste the code, data, or document to reason over if any, what the reasoning steps should be?
182
+ **For Self-Consistency**: The problem with all its conditions stated, paste the data or figures it runs on if any, what the single final answer should look like so every sampled run ends in a comparable `FINAL ANSWER:` line, how many samples to run and majority-vote over (the paper uses 40; 5-10 is usually enough)?
170
183
  **For Chain of Density**: Paste the full document to summarize, the fixed word budget every summary must hit, how many densification passes (the paper uses 5)?
171
184
  **For Iterative Compression**: Paste the content to compress, where it should end up (word count, reading level, single paragraph), what should improve on each pass, how many passes and when to stop?
172
185
  **For Self-Refine**: Paste the actual draft to improve, which dimensions the critique should cover (clarity, completeness, tone), what would make this output wrong or unusable?
@@ -174,6 +187,7 @@ Ask targeted questions (3-5 at a time) based on identified gaps:
174
187
  **For Devil's Advocate**: The position, plan, or decision to attack, paste the proposal or memo that sets it out if you have one, which dimensions the attack should cover?
175
188
  **For Pre-Mortem**: The project or decision being analyzed with its team, timeline, and goals, paste the plan or proposal document if you have one, how far in the future the imagined failure should be dated?
176
189
  **For RCoT**: The question with every condition and constraint written out, paste the document those conditions come from if any, any implicit requirement not yet written into the question (units, deadlines, exclusions, edge cases) that a correct answer must still satisfy?
190
+ **For Chain-of-Verification**: Paste the draft answer to fact-check if you have one, or the factual question to answer carefully, which specific claims are most at risk of being wrong, what a correct final answer must not get wrong?
177
191
  **For RPEF**: Paste the actual output sample to reverse-engineer, paste the input that produced it or confirm it is output-only, which details are one-off specifics that should become [PLACEHOLDER] variables?
178
192
  **For Reverse Role**: What you want to achieve in one or two sentences, the domain of expertise to consult, questions one at a time or all at once, should it then do the task or synthesize a structured prompt for you to approve?
179
193
 
@@ -191,6 +205,7 @@ Using gathered information:
191
205
  2. Map user's information to framework components
192
206
  3. Fill missing elements with reasonable defaults — **with two exceptions, below**
193
207
  4. Structure according to framework format
208
+ 5. **Decide whether worked examples would materially improve the output**; if so, layer in few-shot examples as described in Composable Techniques above — this applies to any framework, not only the two with a built-in examples slot. Reach for it especially on classification, extraction, strict-format, and style-matching tasks, and only when the user or their material supplies real examples.
194
209
 
195
210
  **Never default a fact about the user's world.** Their business, metrics, history, policies, staff, customers, data, or constraints are things only they know. A plausible-sounding default here is a fabrication the user may not notice before sending — asserting "our first price increase in three years" in an email to paying customers, or inventing a phone number in a published review reply. Where such a slot is unanswered, emit a visible `[you fill this in: <what is needed>]` placeholder and list every placeholder in your analysis section.
196
211
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@ckelsoe/prompt-architect",
3
- "version": "3.4.0",
4
- "description": "Agent skill for analyzing and improving prompts using 29 frameworks across 7 intent categories. Works with Claude Code, Gemini CLI, Cursor, Copilot, and 30+ Agent Skills compatible tools.",
3
+ "version": "3.5.0",
4
+ "description": "Agent skill for analyzing and improving prompts using 31 frameworks across 7 intent categories. Works with Claude Code, Gemini CLI, Cursor, Copilot, and 30+ Agent Skills compatible tools.",
5
5
  "main": "skills/prompt-architect/SKILL.md",
6
6
  "keywords": [
7
7
  "claude-code",
@@ -81,7 +81,7 @@
81
81
  "type": "skill",
82
82
  "skillName": "prompt-architect",
83
83
  "skillPath": "skills/prompt-architect",
84
- "version": "3.4.0",
84
+ "version": "3.5.0",
85
85
  "compatibility": {
86
86
  "claudeCode": ">=1.0.0"
87
87
  },
@@ -123,7 +123,9 @@
123
123
  "Pre-Mortem",
124
124
  "RCoT",
125
125
  "RPEF",
126
- "Reverse-Role-Prompting"
126
+ "Reverse-Role-Prompting",
127
+ "Self-Consistency",
128
+ "Chain-of-Verification"
127
129
  ],
128
130
  "features": {
129
131
  "promptAnalysis": true,
package/scripts/test.js CHANGED
@@ -198,6 +198,12 @@ test('Adapter has not drifted from SKILL.md', () => {
198
198
  'Never stack more than two',
199
199
  'Never default a fact about the user',
200
200
  'Never soften or drop a prohibition',
201
+ // The anti-false-narration guardrail: the model must not invent a rationale
202
+ // for a framework choice the stripped output cannot show. Lives in both files.
203
+ 'A confident rationale for an unobservable choice',
204
+ // The few-shot hook must live in the Apply-Framework step of BOTH files, or
205
+ // the composable technique is documented but never reached at build time.
206
+ 'Decide whether worked examples would materially improve the output',
201
207
  // The worked example must demonstrate the emission rules, not violate them.
202
208
  // The adapter previously shipped a BAB example with BEFORE:/AFTER:/BRIDGE: headers.
203
209
  'Now rewrite the job posting above.',
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  name: prompt-architect
3
- description: Analyzes and improves prompts using 29 frameworks across 7 intent categories. Use when a user wants to improve, rewrite, structure, or engineer a prompt — including requests like "help me write a better prompt", "improve this prompt", "what framework should I use", "make this prompt more effective", or any prompt engineering task. Recommends the right framework based on intent (create, transform, reason, critique, recover, clarify, agentic), asks targeted questions, and delivers a structured, high-quality result.
3
+ description: Analyzes and improves prompts using 31 frameworks across 7 intent categories. Use when a user wants to improve, rewrite, structure, or engineer a prompt — including requests like "help me write a better prompt", "improve this prompt", "what framework should I use", "make this prompt more effective", or any prompt engineering task. Recommends the right framework based on intent (create, transform, reason, critique, recover, clarify, agentic), asks targeted questions, and delivers a structured, high-quality result.
4
4
  license: MIT
5
5
  compatibility: Requires no external dependencies. Works with any Agent Skills compatible tool.
6
6
  metadata:
7
7
  author: ckelsoe
8
- version: "3.4.0"
8
+ version: "3.5.0"
9
9
  homepage: https://github.com/ckelsoe/prompt-architect
10
10
  ---
11
11
 
@@ -40,7 +40,9 @@ Score the prompt *as written*, not as you charitably interpret it — the gap be
40
40
 
41
41
  ### 2. Intent-Based Framework Selection
42
42
 
43
- With 29 frameworks, identify the user's **primary intent** first, then use the discriminating questions within that category.
43
+ With 31 frameworks, identify the user's **primary intent** first, then use the discriminating questions within that category.
44
+
45
+ **When two frameworks would produce the same prompt, say so and pick the simpler one.** Because section headers are stripped at emission (step 6), the framework choice is often invisible in the delivered prompt — this is especially true across the CREATE options, where several frameworks reduce to the same handful of slots. When you cannot point to a concrete difference the *emitted* prompt would show, do not manufacture one: name the tie plainly, choose the simpler framework, and move on. A confident rationale for an unobservable choice is exactly the overstatement this skill exists to remove.
44
46
 
45
47
  ---
46
48
 
@@ -98,6 +100,7 @@ With 29 frameworks, identify the user's **primary intent** first, then use the d
98
100
  | Multiple distinct approaches to compare | **Tree of Thought** |
99
101
  | Verify reasoning didn't overlook conditions | **RCoT** |
100
102
  | Linear step-by-step reasoning | **Chain of Thought** |
103
+ | Answer must be robust; sample many paths and majority-vote | **Self-Consistency** |
101
104
 
102
105
  ---
103
106
 
@@ -110,8 +113,9 @@ With 29 frameworks, identify the user's **primary intent** first, then use the d
110
113
  | Find the strongest opposing argument | **Devil's Advocate** |
111
114
  | Identify failure modes before they happen | **Pre-Mortem** |
112
115
  | Verify reasoning didn't miss conditions | **RCoT** |
116
+ | Draft may contain hallucinated facts; verify each claim | **Chain-of-Verification** |
113
117
 
114
- *Self-Refine = any quality. CAI = principle compliance. Devil's Advocate = opposing arguments. Pre-Mortem = failure analysis. RCoT = condition verification.*
118
+ *Self-Refine = any quality. CAI = compliance with an **explicitly stated** standard or requirement set (and aligning the artifact to it — e.g. auditing a plan against a brief's constraints). Devil's Advocate = opposing arguments. Pre-Mortem = failure analysis. RCoT = an answer or plan overlooked a condition **implicit in the problem** (units, edge cases, unstated dependencies). Chain-of-Verification = independent fact-checking of a draft's factual claims.*
115
119
 
116
120
  ---
117
121
 
@@ -133,6 +137,12 @@ Most prompts need exactly one framework. Combine only when the task genuinely ha
133
137
 
134
138
  When you combine, load `assets/templates/hybrid_template.txt` and state plainly in your analysis which framework owns which phase. Never stack more than two — beyond that the frameworks' instructions start to overlap and contradict, and no single framework clearly owns any phase.
135
139
 
140
+ #### Composable Techniques
141
+
142
+ Some techniques are not frameworks you choose *between* — they are layers you add *on top of* whichever framework you picked. They answer "how should this prompt be built?", not "which shape is it?", so they never appear in the routing tables above.
143
+
144
+ - **Few-shot / in-context examples** — showing 2–5 worked input→output examples inside the emitted prompt. This is the highest-leverage technique in prompting and applies to almost any framework, not just the two with a dedicated examples slot (CARE, RISE-IX). After you draft the framework prompt, decide whether examples earn their place; if they do, insert them before the final instruction, in the exact target output format, and end with the actual task. Load `references/techniques/few-shot.md` for when to use it, how many, ordering and recency effects, and the label-space rules — and for the rule that you never invent examples the user or their material did not supply.
145
+
136
146
  ---
137
147
 
138
148
  ### 3. Framework Quick Reference
@@ -143,12 +153,14 @@ One-line per framework (load `references/frameworks/` for full detail):
143
153
  **Medium:** RACE | CARE | BAB | BROKE | CRISPE
144
154
  **Comprehensive:** CO-STAR | RISEN | TIDD-EC
145
155
  **Data:** RISE-IE | RISE-IX
146
- **Reasoning:** Plan-and-Solve | Chain of Thought | Least-to-Most | Step-Back | Tree of Thought | RCoT
156
+ **Reasoning:** Plan-and-Solve | Chain of Thought | Least-to-Most | Step-Back | Tree of Thought | RCoT | Self-Consistency
147
157
  **Structure/Iteration:** Skeleton of Thought | Chain of Density | Iterative Compression
148
- **Critique/Quality:** Self-Refine | CAI Critique-Revise | Devil's Advocate | Pre-Mortem
158
+ **Critique/Quality:** Self-Refine | CAI Critique-Revise | Devil's Advocate | Pre-Mortem | Chain-of-Verification
149
159
  **Meta/Reverse:** RPEF | Reverse Role Prompting
150
160
  **Agentic:** ReAct
151
161
 
162
+ **Composable technique (layered onto any framework, not selected between):** Few-shot / in-context examples
163
+
152
164
  ### 4. Clarification Questions
153
165
 
154
166
  Ask targeted questions (3-5 at a time) based on identified gaps:
@@ -173,6 +185,7 @@ Ask targeted questions (3-5 at a time) based on identified gaps:
173
185
  **For Least-to-Most**: The full problem in one statement, paste the material the subproblems must reason over, what is the simplest thing that must be answered first, what does the final answer depend on?
174
186
  **For Plan-and-Solve**: The problem with every number, unit, and constraint written out, paste the dataset or figures the calculation runs on if any, which values are given and which must be derived?
175
187
  **For Chain of Thought**: The problem with all its conditions stated, paste the code, data, or document to reason over if any, what the reasoning steps should be?
188
+ **For Self-Consistency**: The problem with all its conditions stated, paste the data or figures it runs on if any, what the single final answer should look like so every sampled run ends in a comparable `FINAL ANSWER:` line, how many samples to run and majority-vote over (the paper uses 40; 5-10 is usually enough)?
176
189
  **For Chain of Density**: Paste the full document to summarize, the fixed word budget every summary must hit, how many densification passes (the paper uses 5)?
177
190
  **For Iterative Compression**: Paste the content to compress, where it should end up (word count, reading level, single paragraph), what should improve on each pass, how many passes and when to stop?
178
191
  **For Self-Refine**: Paste the actual draft to improve, which dimensions the critique should cover (clarity, completeness, tone), what would make this output wrong or unusable?
@@ -180,6 +193,7 @@ Ask targeted questions (3-5 at a time) based on identified gaps:
180
193
  **For Devil's Advocate**: The position, plan, or decision to attack, paste the proposal or memo that sets it out if you have one, which dimensions the attack should cover?
181
194
  **For Pre-Mortem**: The project or decision being analyzed with its team, timeline, and goals, paste the plan or proposal document if you have one, how far in the future the imagined failure should be dated?
182
195
  **For RCoT**: The question with every condition and constraint written out, paste the document those conditions come from if any, any implicit requirement not yet written into the question (units, deadlines, exclusions, edge cases) that a correct answer must still satisfy?
196
+ **For Chain-of-Verification**: Paste the draft answer to fact-check if you have one, or the factual question to answer carefully, which specific claims are most at risk of being wrong, what a correct final answer must not get wrong?
183
197
  **For RPEF**: Paste the actual output sample to reverse-engineer, paste the input that produced it or confirm it is output-only, which details are one-off specifics that should become [PLACEHOLDER] variables?
184
198
  **For Reverse Role**: What you want to achieve in one or two sentences, the domain of expertise to consult, questions one at a time or all at once, should it then do the task or synthesize a structured prompt for you to approve?
185
199
 
@@ -197,6 +211,7 @@ Using gathered information:
197
211
  2. Map user's information to framework components
198
212
  3. Fill missing elements with reasonable defaults — **with two exceptions, below**
199
213
  4. Structure according to framework format
214
+ 5. **Decide whether worked examples would materially improve the output**; if so, layer in few-shot examples per `references/techniques/few-shot.md` — this applies to any framework, not only the two with a built-in examples slot. Reach for it especially on classification, extraction, strict-format, and style-matching tasks, and only when the user or their material supplies real examples.
200
215
 
201
216
  **Never default a fact about the user's world.** Their business, metrics, history, policies, staff, customers, data, or constraints are things only they know. A plausible-sounding default here is a fabrication the user may not notice before sending — asserting "our first price increase in three years" in an email to paying customers, or inventing a phone number in a published review reply. Where such a slot is unanswered, emit a visible `[you fill this in: <what is needed>]` placeholder and list every placeholder in your analysis section.
202
217
 
@@ -263,9 +278,14 @@ Detailed framework docs in `references/frameworks/`:
263
278
  - `rcot.md` - Reverse Chain-of-Thought: verify by reconstructing the question
264
279
  - `rpef.md` - Reverse Prompt Engineering: recover prompt from output (EMNLP 2025)
265
280
  - `reverse-role.md` - AI-Led Interview: AI asks you questions first (FATA)
281
+ - `self-consistency.md` - Sample N reasoning paths, majority-vote externally (Wang et al., ICLR 2023)
282
+ - `chain-of-verification.md` - Draft, plan checks, verify independently, revise (Dhuliawala et al., Findings of ACL 2024)
266
283
 
267
284
  Load these when applying specific frameworks for detailed component guidance, selection criteria, and examples.
268
285
 
286
+ **Composable techniques** (layered onto a framework, not selected from the routing tables) live in `references/techniques/`:
287
+ - `few-shot.md` - In-context examples: when to add them, how many, ordering, label-space rules
288
+
269
289
  ## Templates
270
290
 
271
291
  Every framework has a fill-in template at `assets/templates/<framework>_template.txt`, named after the framework's reference doc (e.g. `co-star.md` → `co-star_template.txt`). RISE has two: `rise-ie_template.txt` and `rise-ix_template.txt`. One extra template, `hybrid_template.txt`, is for combined-framework prompts (see **Combining Frameworks**).
@@ -0,0 +1,26 @@
1
+ SOURCE MATERIAL:
2
+ [Optional — paste the draft answer you want fact-checked here, OR the factual
3
+ question you want answered with verification built in. If you are asking a fresh
4
+ question rather than checking an existing draft, delete this bracketed block and
5
+ the sentence directly below it.]
6
+ Treat the material above as the subject to be verified in the steps below.
7
+
8
+ Work through the following four steps in order, in a single response.
9
+
10
+ First, write a baseline answer to the question. Do not polish it — this draft is
11
+ what you will fact-check. (If a draft was supplied above, treat that draft as your
12
+ baseline answer instead of writing a new one.)
13
+
14
+ Second, list the specific factual claims your draft makes — every name, date,
15
+ quantity, entity, and list member — and write one short verification question for
16
+ each claim. Each question must target one claim and be answerable on its own; do
17
+ not write a single broad question that re-asks the whole task.
18
+
19
+ Third, answer each verification question independently. Do not look back at your
20
+ draft while answering these questions — answer each one on its own, as if seeing
21
+ it fresh, so your checks are not biased into agreeing with whatever the draft
22
+ already says.
23
+
24
+ Fourth, write a final verified answer. Keep every claim your independent answers
25
+ confirmed, correct every claim they contradicted, and drop any claim you could
26
+ not verify. Note briefly what changed from the draft.
@@ -0,0 +1,12 @@
1
+ PROBLEM:
2
+ [Your question, calculation, or reasoning problem]
3
+
4
+ SOURCE MATERIAL:
5
+ [Optional — paste the problem data the solution runs on here (the word problem, dataset, table, figures, or constraint list). Delete this line and the one below it if the problem above already states every value needed.]
6
+ The figures and constraints in the material above are authoritative — extract from them rather than from assumed or recalled values.
7
+
8
+ Solve the problem above. Work through it step by step, showing your reasoning
9
+ at each stage. When you reach an answer, state it on its own line, in exactly
10
+ this form and with nothing else on that line:
11
+
12
+ FINAL ANSWER: [answer]
@@ -0,0 +1,231 @@
1
+ # CoVe (Chain-of-Verification)
2
+
3
+ ## Overview
4
+
5
+ Chain-of-Verification (CoVe) is a verification framework that reduces hallucinated facts by making the model check its own draft against answers it produces without looking at that draft. The model first writes a baseline response, then plans a set of targeted verification questions aimed at the discrete factual claims the draft makes, then answers those questions independently — crucially without reference to the baseline — and finally regenerates the response, correcting anything the independent answers contradicted. The whole procedure fits inside a single pasted prompt: one response that walks through all four steps in order.
6
+
7
+ **Origin:** Dhuliawala et al., "Chain-of-Verification Reduces Hallucination in Large Language Models" (arXiv 2309.11495). First posted as a preprint in September 2023, it was published in the **Findings of the Association for Computational Linguistics: ACL 2024** (anthology id 2024.findings-acl.212) — so cite the venue as Findings of ACL **2024**, not 2023. The work is a collaboration: the first author holds a dual affiliation with **ETH Zürich** alongside Meta AI, so it is not accurate to credit Meta AI alone. Note also that the paper's **abstract contains no numbers** — it states qualitatively that CoVe decreases hallucinations across tasks. The frequently quoted precision improvement on list-based Wikidata questions (0.17 to 0.36 for Llama 65B) is a **body figure from the experiments**, not an abstract claim; only present it as a body/table result, and do not attach it to the abstract.
8
+
9
+ ## Why CoVe Works
10
+
11
+ Hallucinated facts in a baseline draft are internally consistent and fluent, which is exactly why re-reading the draft does not catch them — the model that wrote the error will happily reaffirm it when asked to review its own text. CoVe breaks that loop by structural means rather than by asking harder:
12
+
13
+ - The draft is decomposed into discrete factual claims, each of which becomes a short, answerable verification question.
14
+ - Each verification question is answered **on its own**, not conditioned on the draft, so the model retrieves the fact fresh instead of echoing what it already wrote.
15
+ - The regenerated answer is forced to reconcile with those independent answers, so contradictions surface as edits.
16
+
17
+ **The independence of the verification step is the entire mechanism.** If the model answers the verification questions while looking at the baseline, it tends to repeat the baseline's errors — the check collapses into agreement with the thing being checked. The paper studies factored variants (answering verifications separately from the original draft) precisely to preserve this independence, and reports the largest gains when the verification answers are not conditioned on the initial response.
18
+
19
+ **Limitation:** CoVe targets hallucinated facts — fabricated names, dates, quantities, citations, list members. It is weaker on reasoning that is factually grounded but logically flawed, and it cannot verify a claim the model has no better knowledge of on the second pass than the first.
20
+
21
+ ## Distinction from RCoT
22
+
23
+ CoVe and RCoT are both verification frameworks in this package, and they address different failure modes with different mechanisms. Keep them separate:
24
+
25
+ - **RCoT verifies reasoning.** It reconstructs the original question from a candidate answer and cross-checks the reconstruction against the actual question to find overlooked conditions, misinterpreted constraints, and unstated assumptions. Its failure mode is a *missed condition*.
26
+ - **CoVe verifies facts.** It drafts an answer, generates targeted verification questions about the claims in that draft, answers them independently, and revises. Its failure mode is a *hallucinated fact*.
27
+
28
+ Put plainly: RCoT asks "what question would produce this answer, and does it match what was actually asked?" CoVe asks "which specific claims did I just assert, are they individually true when I check each one cold, and what must change?" Reach for RCoT on multi-condition logic and constraint problems; reach for CoVe on fact-dense output like biographies, entity lists, and cited summaries.
29
+
30
+ ## Components
31
+
32
+ ### Baseline Response
33
+ **Purpose:** Draft an initial answer to the question. This draft is expected to be fluent and may contain hallucinated facts — it is the material to be checked, not the final output.
34
+
35
+ ### Plan Verification Questions
36
+ **Purpose:** Break the draft into its discrete factual claims and turn each into a short, self-contained fact-checking question. Good verification questions are answerable on their own and target one claim each ("In what year was X founded?"), not the draft's conclusion as a whole.
37
+
38
+ **Trigger:** *"List the specific factual claims your draft makes, then write one short verification question for each claim."*
39
+
40
+ ### Answer Verifications Independently
41
+ **Purpose:** Answer every verification question on its own terms, **without** consulting the baseline draft. This is the step that does the work: answering cold prevents the model from biasing the verification into simply confirming what it already wrote.
42
+
43
+ **Trigger:** *"Answer each verification question independently. Do not look back at your draft while answering — answer as if seeing each question fresh."*
44
+
45
+ ### Final Verified Response
46
+ **Purpose:** Regenerate the answer, keeping what the independent answers confirmed and correcting anything they contradicted. Discrepancies between the draft and the independent answers become explicit edits.
47
+
48
+ ## Template Structure
49
+
50
+ Section headers are stripped at emission, so every instruction below has to carry its meaning in prose — the model receives flat text, not a labeled form. The independence requirement is the one thing most easily lost when the `ANSWER VERIFICATIONS` header disappears, so it is written as an explicit "Do not…" sentence inside the step itself, where it survives stripping.
51
+
52
+ ```
53
+ SOURCE MATERIAL:
54
+ [Optional — paste the draft answer you want fact-checked here, OR the factual
55
+ question you want answered with verification built in. If you are asking a fresh
56
+ question rather than checking an existing draft, delete this bracketed block and
57
+ the sentence directly below it.]
58
+ Treat the material above as the subject to be verified in the steps below.
59
+
60
+ Work through the following four steps in order, in a single response.
61
+
62
+ First, write a baseline answer to the question. Do not polish it — this draft is
63
+ what you will fact-check.
64
+
65
+ Second, list the specific factual claims your draft makes (names, dates,
66
+ quantities, entities, list members), and write one short verification question
67
+ for each claim.
68
+
69
+ Third, answer each verification question independently. Do not look back at your
70
+ draft while answering these questions — answer each one on its own, as if seeing
71
+ it fresh, so your checks are not biased into agreeing with the draft.
72
+
73
+ Fourth, write a final verified answer. Keep every claim your independent answers
74
+ confirmed, correct every claim they contradicted, and drop any claim you could
75
+ not verify. Note briefly what changed from the draft.
76
+ ```
77
+
78
+ The `SOURCE MATERIAL` block carries no step number because it is optional and holds either of two things: an existing draft to be checked, or the factual question to be answered from scratch with verification built in. When it holds a draft, the first step reviews that draft rather than writing a new one. When there is no draft, delete the block and the sentence beneath it, and the model produces its own baseline in step one.
79
+
80
+ ## Complete Examples
81
+
82
+ Every example below is shown in emitted form: the step headers are gone and each instruction carries its own role in prose, so nothing is lost when the scaffolding is deleted.
83
+
84
+ ### Example 1: Fact-Checking an Existing Draft (source material supplied)
85
+
86
+ **Before CoVe:**
87
+ "Is this bio accurate?"
88
+
89
+ **After CoVe:**
90
+ ```
91
+ SOURCE MATERIAL:
92
+ [Paste the draft biography you want fact-checked here]
93
+ Treat the material above as the subject to be verified in the steps below.
94
+
95
+ Work through the following four steps in order, in a single response.
96
+
97
+ First, treat the draft above as your baseline answer.
98
+
99
+ Second, list the specific factual claims the draft makes — every name, date,
100
+ title, place, and quantity — and write one short verification question for each.
101
+
102
+ Third, answer each verification question independently. Do not look back at the
103
+ draft while answering — answer each question on its own so your checks are not
104
+ biased into confirming what the draft already says.
105
+
106
+ Fourth, write a corrected version of the bio: keep every claim your independent
107
+ answers confirmed, fix every claim they contradicted, and remove any claim you
108
+ could not verify. List what you changed.
109
+ ```
110
+
111
+ ### Example 2: Named-Entity List Question (no source material)
112
+
113
+ **Before CoVe:**
114
+ "Name some politicians born in New York City."
115
+
116
+ **After CoVe** (no source material — the answer is generated from scratch, so the
117
+ `SOURCE MATERIAL` block and the sentence below it are both deleted):
118
+ ```
119
+ Work through the following four steps in order, in a single response.
120
+
121
+ First, write a baseline answer listing politicians born in New York City. Do not
122
+ polish it — this draft is what you will fact-check.
123
+
124
+ Second, for each name on your list, write one short verification question of the
125
+ form "Was [name] born in New York City?"
126
+
127
+ Third, answer each verification question independently. Do not look back at your
128
+ list while answering — answer each question on its own, so a wrong name on the
129
+ draft does not get waved through.
130
+
131
+ Fourth, write a final list that keeps only the names your independent answers
132
+ confirmed were born in New York City, and drop the rest. Note which names you
133
+ removed and why.
134
+ ```
135
+
136
+ ### Example 3: Cited Factual Summary (source material supplied)
137
+
138
+ **Before CoVe:**
139
+ "Summarize this article and make sure the numbers are right."
140
+
141
+ **After CoVe:**
142
+ ```
143
+ SOURCE MATERIAL:
144
+ [Paste the article you want summarized here]
145
+ Treat the material above as the subject to be verified in the steps below.
146
+
147
+ Work through the following four steps in order, in a single response.
148
+
149
+ First, write a baseline summary of the article above, including its key figures,
150
+ dates, and named parties.
151
+
152
+ Second, list every factual claim in your summary — each figure, date, and name —
153
+ and write one short verification question for each that can be checked against
154
+ the article.
155
+
156
+ Third, answer each verification question independently by locating the supporting
157
+ passage in the article. Do not answer from your summary — answer from the source,
158
+ so a number you misremembered while summarizing gets caught.
159
+
160
+ Fourth, write a final summary in which every figure, date, and name matches what
161
+ you verified against the article. Flag anything the article does not actually
162
+ state.
163
+ ```
164
+
165
+ ## Best Use Cases
166
+
167
+ 1. **Fact-Dense Generation**
168
+ - Biographies, company profiles, historical summaries
169
+ - Anything where fabricated names, dates, or quantities are the main risk
170
+
171
+ 2. **List and Entity Questions**
172
+ - "Name all X that Y" questions, where the paper shows the clearest gains
173
+ - Each list member becomes its own cheap verification question
174
+
175
+ 3. **Cited or Sourced Summaries**
176
+ - Summaries whose figures must match a supplied document
177
+ - Verification questions check each figure against the source
178
+
179
+ 4. **A Verification Layer for Draft Output**
180
+ - Drop CoVe in after any first-pass factual draft
181
+ - Especially before publishing or sending fact-heavy text
182
+
183
+ ## Selection Criteria
184
+
185
+ **Choose CoVe when:**
186
+ - ✅ The output is fact-dense and hallucinated details are the main risk
187
+ - ✅ The claims decompose into discrete, individually checkable questions
188
+ - ✅ You are checking a draft you already have, or generating one you don't trust
189
+ - ✅ The model plausibly knows each fact better in isolation than buried in a paragraph
190
+
191
+ **Avoid CoVe when:**
192
+ - ❌ The failure mode is a missed condition or constraint → use RCoT
193
+ - ❌ The task is reasoning or arithmetic, not factual recall → use Plan-and-Solve or CoT
194
+ - ❌ Facts genuinely require external lookup the model cannot do → use a tool-grounded method (CRITIC)
195
+ - ❌ The task is creative or generative with no facts to verify
196
+
197
+ ## CoVe vs. Other Verification Frameworks
198
+
199
+ | | CoVe | RCoT | Self-Refine | CRITIC |
200
+ |---|---|---|---|---|
201
+ | Verifies | Facts | Reasoning conditions | General quality | Facts (tool-grounded) |
202
+ | Failure mode caught | Hallucinated facts | Overlooked conditions | Vague weaknesses | Hallucinated facts |
203
+ | Mechanism | Independent re-answering of claims | Question reconstruction | Self-critique then revise | External tool checks |
204
+ | Key requirement | Verification not conditioned on draft | Reconstruct without re-reading question | Honest self-critique | Access to tools |
205
+ | Fits one prompt? | Yes | Yes | Yes | No (needs tool calls) |
206
+
207
+ ## Common Mistakes
208
+
209
+ 1. **Letting the verification see the draft**
210
+ - This is the cardinal error — if the model answers verification questions while looking at its draft, it confirms its own hallucinations
211
+ - The "Do not look back at your draft" instruction is load-bearing; do not drop it
212
+
213
+ 2. **Verification questions that are too broad**
214
+ - "Is this bio accurate?" is not a verification question — it just re-asks the whole task
215
+ - Each question must target one discrete claim: one name, one date, one figure
216
+
217
+ 3. **Skipping the correction step**
218
+ - Finding contradictions is not the goal; fixing them is
219
+ - The final answer must reconcile with the independent answers, not just report the mismatch
220
+
221
+ 4. **Using CoVe where facts aren't the risk**
222
+ - On reasoning or constraint problems, CoVe's questions have nothing factual to grip; RCoT fits those
223
+
224
+ ## Quick Reference
225
+
226
+ | Step | Purpose |
227
+ |------|---------|
228
+ | Baseline Response | Draft an initial answer (may contain hallucinations) |
229
+ | Plan Verification Questions | Turn each discrete claim into a short fact-check question |
230
+ | Answer Independently | Answer each question cold — not conditioned on the draft |
231
+ | Final Verified Response | Regenerate, correcting whatever the checks contradicted |
@@ -0,0 +1,195 @@
1
+ # Self-Consistency Framework
2
+
3
+ ## Overview
4
+
5
+ Self-Consistency is a reasoning technique that replaces a single greedy answer with a majority vote over many independently sampled reasoning paths. Instead of asking the model to think step by step once and trust the result, you run the *same* prompt several times at non-zero temperature, collect the final answer from each run, and return the answer that appears most often. The reasoning inside each run still looks like Chain-of-Thought; what changes is that the decision is made across runs rather than within one. The insight is that a correct answer tends to be reachable by several different valid lines of reasoning, while the various ways of being wrong scatter — so agreement across independent samples is itself evidence.
6
+
7
+ When prompt-architect emits a Self-Consistency prompt, its framework section headers — `PROBLEM:` and `SOURCE MATERIAL:` — are stripped and the model receives a flat block. The solve instruction is written as a complete sentence so it survives on its own, and the `FINAL ANSWER: [answer]` line is a literal output label the model is being told to produce, not a section header, so it stays in place exactly as written. The optional `SOURCE MATERIAL` block holds the data the solution runs on and is deleted when the problem is self-contained.
8
+
9
+ **Origin:** "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (Wang et al., arXiv 2203.11171, ICLR 2023). The abstract reports the technique's headline gain as a relative delta of **+17.9%** on GSM8K. The corresponding absolute figures — greedy Chain-of-Thought decoding at **56.5%** rising to **74.4%** with a majority vote over 40 sampled paths, using PaLM 540B — are **Table 1** body figures, not abstract figures. A notable finding: the plain, unweighted majority vote matched or beat every weighted aggregation scheme the authors tested, so there is no need to weight votes by probability or confidence.
10
+
11
+ ## How This Framework Is Delivered
12
+
13
+ Self-Consistency is operational, not a single instruction: the gain comes from *sampling the same prompt N times and voting across the results*. A single pasted prompt cannot run itself N times, so no one prompt can perform the vote on its own. That constraint is handled honestly rather than hidden:
14
+
15
+ - **The emitted prompt** is a clean solve-prompt. It forces step-by-step reasoning and ends with a machine-parseable `FINAL ANSWER:` line so the outputs of separate runs can be lined up and tallied. It is a single-path prompt by design.
16
+ - **The skill's ANALYSIS section** is where the user is told to run that prompt N times at non-zero temperature and take the majority vote *externally* — in a script, a notebook, or by hand. The voting lives outside the model.
17
+
18
+ Do not pretend one prompt does the voting. Stating plainly that the aggregation happens outside the model is a feature: it is what keeps the technique faithful to the paper instead of collapsing it into something weaker.
19
+
20
+ ## Components
21
+
22
+ ### The Solve Prompt (emitted)
23
+ **Purpose:** Produce one full reasoning path and one cleanly extractable answer. The prompt demands step-by-step work — the diversity of reasoning across runs is the whole engine, so each run must actually reason rather than guess — and pins the answer to a fixed `FINAL ANSWER: [answer]` line so a program can find it identically in every sample.
24
+
25
+ **What it must contain:**
26
+ - A complete solve instruction that survives header stripping ("Solve the problem above. Work through it step by step…").
27
+ - The exact terminal line `FINAL ANSWER: [answer]`, with nothing else on that line, so every run's answer sits in the same parseable position.
28
+ - Optionally, a `SOURCE MATERIAL` block naming the data the solution runs on.
29
+
30
+ ### The Sampling Parameters (analysis-side)
31
+ **Purpose:** Generate the independent reasoning paths. The paper samples **40**; in practice **5-10** is usually enough to see the benefit. Sampling must be at **non-zero temperature** — **~0.7** is a reasonable default — because temperature 0 collapses every run to the same greedy path and destroys the diversity the vote depends on.
32
+
33
+ ### The Aggregation (analysis-side)
34
+ **Purpose:** Turn N answers into one. Extract the `FINAL ANSWER:` value from each sample, count how often each distinct answer appears, and return the most frequent one. This is a **plain, unweighted majority vote** — not a confidence-weighted or probability-weighted scheme, both of which the authors found no better. A near-tie is a signal of genuine uncertainty, not an invitation to ask the model to break the tie.
35
+
36
+ ## Template Structure
37
+
38
+ Section headers — `PROBLEM:` and `SOURCE MATERIAL:` — are stripped at emission, so the solve instruction that follows is written as a complete sentence and survives on its own. The `FINAL ANSWER:` line is a literal output label, not a header, and survives verbatim. The `SOURCE MATERIAL` block is optional and is deleted when the problem already states every value needed.
39
+
40
+ ```
41
+ PROBLEM:
42
+ [Your question, calculation, or reasoning problem]
43
+
44
+ SOURCE MATERIAL:
45
+ [Optional — paste the problem data the solution runs on here (the word problem, dataset, table, figures, or constraint list). Delete this line and the one below it if the problem above already states every value needed.]
46
+ The figures and constraints in the material above are authoritative — extract from them rather than from assumed or recalled values.
47
+
48
+ Solve the problem above. Work through it step by step, showing your reasoning
49
+ at each stage. When you reach an answer, state it on its own line, in exactly
50
+ this form and with nothing else on that line:
51
+
52
+ FINAL ANSWER: [answer]
53
+ ```
54
+
55
+ ## Complete Examples
56
+
57
+ Every example below is shown in emitted form: the solve instruction and the `FINAL ANSWER:` line are the same in each — they are what survives header stripping. Where the reasoning runs on a pasted dataset or constraint list, the `SOURCE MATERIAL` block carries it and the problem refers to "the problem above"; where every value is already in the problem statement, that block is deleted. In every case the emitted artifact is a single-path solve-prompt, and the vote is run afterward per the analysis note.
58
+
59
+ ### Example 1: Math / Word Problem
60
+
61
+ **Before Self-Consistency:**
62
+ "A shipment has 3 crates of 24 units and 5 crates of 18 units. If 12% are defective, how many good units are there?"
63
+
64
+ **After Self-Consistency** (no source material — every value is stated in the problem, so the `SOURCE MATERIAL` block is deleted):
65
+ ```
66
+ PROBLEM:
67
+ A shipment has 3 crates of 24 units and 5 crates of 18 units. 12% of all
68
+ units are defective. How many good (non-defective) units are there?
69
+
70
+ Solve the problem above. Work through it step by step, showing your reasoning
71
+ at each stage. When you reach an answer, state it on its own line, in exactly
72
+ this form and with nothing else on that line:
73
+
74
+ FINAL ANSWER: [answer]
75
+ ```
76
+
77
+ **Analysis note handed to the user:** Run this prompt 5-10 times at temperature ~0.7. Extract each run's `FINAL ANSWER:` value, count them, and return the most frequent. If two answers tie, treat that as real uncertainty and inspect the reasoning rather than asking the model to pick.
78
+
79
+ ### Example 2: Logic / Reasoning Over Pasted Constraints
80
+
81
+ **Before Self-Consistency:**
82
+ "Given these seating constraints, who sits in seat 3?"
83
+
84
+ **After Self-Consistency** (source material supplied):
85
+ ```
86
+ PROBLEM:
87
+ Determine who sits in seat 3, working only from the constraints in the
88
+ problem above.
89
+
90
+ SOURCE MATERIAL:
91
+ [Paste the full list of seating constraints — people, seats, and adjacency rules — here]
92
+ The figures and constraints in the material above are authoritative — extract from them rather than from assumed or recalled values.
93
+
94
+ Solve the problem above. Work through it step by step, showing your reasoning
95
+ at each stage. When you reach an answer, state it on its own line, in exactly
96
+ this form and with nothing else on that line:
97
+
98
+ FINAL ANSWER: [answer]
99
+ ```
100
+
101
+ ### Example 3: Business Calculation
102
+
103
+ **Before Self-Consistency:**
104
+ "If CAC is $1,200, MRR per customer is $150, and gross margin is 70%, what's the payback period in months?"
105
+
106
+ **After Self-Consistency** (no source material — every value is stated, so the `SOURCE MATERIAL` block is deleted):
107
+ ```
108
+ PROBLEM:
109
+ Customer Acquisition Cost (CAC): $1,200. Average MRR per customer: $150.
110
+ Gross margin: 70%. What is the payback period in months?
111
+
112
+ Solve the problem above. Work through it step by step, showing your reasoning
113
+ at each stage. When you reach an answer, state it on its own line, in exactly
114
+ this form and with nothing else on that line:
115
+
116
+ FINAL ANSWER: [answer]
117
+ ```
118
+
119
+ Because a payback-period calculation admits several valid routes — dividing CAC by gross-margin-adjusted MRR, or building up contribution month by month — Self-Consistency is well suited here: independent runs that reach the same number through different arithmetic reinforce the result, and a run that fumbles the margin adjustment is outvoted.
120
+
121
+ ## Best Use Cases
122
+
123
+ 1. **High-Stakes Single Answers**
124
+ - Numerical problems where one wrong step flips the result
125
+ - Financial calculations where accuracy is worth the extra compute
126
+ - Answers you will act on and cannot easily re-check by hand
127
+
128
+ 2. **Problems With Multiple Valid Solution Routes**
129
+ - Word problems solvable by more than one method
130
+ - Logic puzzles with several deduction orders
131
+ - Any task where convergent independent reasoning is meaningful evidence
132
+
133
+ 3. **When Greedy Chain-of-Thought Is Almost Right**
134
+ - Prior single-path CoT gave inconsistent answers across attempts
135
+ - The model clearly can solve it but sometimes slips
136
+ - You can afford N samples and want to buy down the error rate
137
+
138
+ 4. **Discrete, Comparable Answers**
139
+ - The final answer is a number, a label, or a short choice that can be compared for equality across runs and tallied
140
+
141
+ ## Selection Criteria
142
+
143
+ **Choose Self-Consistency when:**
144
+ - ✅ The answer is discrete and comparable across runs (a number, a category, a choice)
145
+ - ✅ You can sample the prompt multiple times at non-zero temperature
146
+ - ✅ You can aggregate the results outside the model (script, notebook, or by hand)
147
+ - ✅ Accuracy justifies spending N times the compute of a single run
148
+ - ✅ Single-path CoT is close but not reliable enough
149
+
150
+ **Avoid Self-Consistency when:**
151
+ - ❌ You can only run the prompt once → use Chain-of-Thought instead
152
+ - ❌ The output is open-ended prose with no comparable "final answer" to vote on
153
+ - ❌ The task needs to explore and keep branching structure → use Tree of Thought
154
+ - ❌ The problem must be decomposed into dependent sub-problems → use Least-to-Most
155
+ - ❌ You need explicit variable extraction and planning within one run → use Plan-and-Solve
156
+
157
+ ## Self-Consistency vs. Chain-of-Thought vs. Tree of Thought
158
+
159
+ Chain-of-Thought is the single-path neighbor of Self-Consistency: they use the *same* kind of step-by-step reasoning prompt. The distinction is entirely in how the answer is chosen. CoT commits to one path (typically greedy); Self-Consistency samples many CoT paths and lets them vote. Tree of Thought is different again — it explores and prunes a branching search *within* a single run rather than voting across independent runs.
160
+
161
+ | | Chain-of-Thought | Self-Consistency | Tree of Thought |
162
+ |---|---|---|---|
163
+ | Reasoning paths | One | Many, independently sampled | One branching search |
164
+ | Temperature | Any (often greedy) | Non-zero (~0.7) required | Any |
165
+ | How the answer is chosen | The single run's output | Majority vote across runs | Best surviving branch |
166
+ | Aggregation location | None | Outside the model | Inside the run |
167
+ | Runs needed | 1 | N (5-10 typical; 40 in paper) | 1 |
168
+ | Best for | General step-by-step reasoning | Discrete answers worth N-fold compute | Problems needing lookahead/backtracking |
169
+
170
+ ## Common Mistakes
171
+
172
+ 1. **Asking the model to vote on itself**
173
+ - Self-Consistency is *not* asking the model to generate several approaches in one response and then judge which it likes best. That is self-assessment — a different and weaker technique, because the model is not a reliable judge of its own reasoning. The vote must happen across independent runs, outside the model.
174
+
175
+ 2. **Sampling at temperature 0**
176
+ - Greedy decoding produces the same path every time, so N identical runs vote unanimously for one answer and the method reduces to plain CoT with wasted compute. Sampling must be at non-zero temperature.
177
+
178
+ 3. **Weighting the votes**
179
+ - The paper found an unweighted majority vote matched or beat every weighted scheme tested. Confidence- or probability-weighting adds complexity for no gain.
180
+
181
+ 4. **Omitting or reformatting the FINAL ANSWER line**
182
+ - If runs end their answers differently, they cannot be extracted and compared programmatically. The `FINAL ANSWER: [answer]` line must appear verbatim, alone on its line, in every run.
183
+
184
+ 5. **Forcing a tie-break**
185
+ - A near-tie means genuine uncertainty. Asking the model to break it discards the very signal the method surfaced; inspect the split reasoning instead.
186
+
187
+ ## Quick Reference
188
+
189
+ | Component | Location | Purpose |
190
+ |-----------|----------|---------|
191
+ | Solve prompt | Emitted | One reasoning path plus a parseable `FINAL ANSWER:` line |
192
+ | Step-by-step instruction | Emitted | Forces reasoning so independent paths diverge meaningfully |
193
+ | Sampling (N, temperature) | Analysis-side | Generate independent paths; N≈5-10 (40 in paper), temp ~0.7 |
194
+ | Aggregation | Analysis-side | Plain unweighted majority vote over extracted answers |
195
+ | Tie handling | Analysis-side | Treat near-ties as uncertainty, not as a prompt to break the tie |
@@ -0,0 +1,121 @@
1
+ # Few-Shot / In-Context Learning (Composable Technique)
2
+
3
+ ## What this is, and why it is not a framework
4
+
5
+ Few-shot prompting means showing the model a small number of worked examples —
6
+ input paired with the desired output — inside the prompt itself, so it induces the
7
+ pattern rather than working from an instruction alone. It is the single most
8
+ load-bearing prompting technique in general use, and it is deliberately **not** one
9
+ of this skill's routed frameworks.
10
+
11
+ The reason is structural. A framework answers "how should this request be shaped?"
12
+ and you pick exactly one. Few-shot answers "should this prompt carry examples, and if
13
+ so, which ones and in what order?" — and the answer can be yes for almost any
14
+ framework. It is a **layer**, not a choice. RACE-with-examples, CO-STAR-with-examples,
15
+ and Plan-and-Solve-with-examples are all still RACE, CO-STAR, and Plan-and-Solve. So
16
+ few-shot is applied *on top of* the chosen framework's emitted prompt, never selected
17
+ *instead of* it.
18
+
19
+ Two frameworks already carry a dedicated examples slot — **CARE** (its `E`) and
20
+ **RISE-IX** (its samples). Few-shot is the discipline of choosing, ordering, and
21
+ formatting those examples well. That discipline applies equally to the many frameworks
22
+ that have no examples slot at all, which is exactly why it lives here as a technique
23
+ instead of inside any one framework.
24
+
25
+ **Origin:** In-context / few-shot learning was named and popularized by Brown et al.,
26
+ "Language Models are Few-Shot Learners" (arXiv 2005.14165, NeurIPS 2020) — cite that
27
+ paper only for the *term*, not for any benchmark number; its abstract reports model
28
+ scale ("175 billion parameters"), not prompt-construction results. The practical
29
+ guidance below rests on later work that actually measured how example choice changes
30
+ outputs.
31
+
32
+ ## When to add examples — and when not to
33
+
34
+ Add examples when:
35
+
36
+ - **The output format is easier to show than to describe.** A specific JSON shape, a
37
+ table layout, a citation style, a commit-message convention — one example settles
38
+ what a paragraph of instructions leaves ambiguous.
39
+ - **A consistent structure must repeat** across many outputs (every entry gets the same
40
+ four fields in the same order).
41
+ - **A voice or style must be matched** and the user can supply samples of it.
42
+ - **The task is classification or extraction with a fixed label set**, where examples
43
+ pin down the label space and the decision boundary.
44
+ - **Edge cases matter** that an instruction cannot cleanly enumerate — show the tricky
45
+ input handled correctly.
46
+
47
+ Do **not** add examples when:
48
+
49
+ - **The task is open-ended reasoning.** Few-shot examples can lock the model into
50
+ imitating the *surface pattern* of the examples instead of reasoning freshly;
51
+ zero-shot chain-of-thought often beats few-shot on novel multi-step problems.
52
+ - **One clear instruction already fixes the output.** Examples are tokens; every one
53
+ costs context and adds a pattern the model will over-fit to.
54
+ - **You would have to invent the examples.** A fabricated example teaches a fabricated
55
+ standard. If neither the user nor the source material supplies a real one, prefer a
56
+ precise instruction — do not manufacture a sample and present it as the target.
57
+
58
+ ## How many, and in what order
59
+
60
+ - **Count:** 2–5 is the usual working range. Returns diminish fast; more examples is
61
+ not reliably better, and past a handful the marginal example mostly adds cost and
62
+ over-fitting risk. Start with the fewest that disambiguate the task.
63
+ - **Order is not neutral.** Models are measurably sensitive to example ordering, and to
64
+ the point of a strong **recency bias** — the last example carries extra weight (Lu et
65
+ al., "Fantastically Ordered Prompts…", ACL 2022, arXiv 2104.08786, which reports a 13%
66
+ relative improvement across eleven text-classification tasks purely from ordering).
67
+ Put the most representative example last, and do not accidentally sort all of one
68
+ class to the end.
69
+
70
+ ## What the examples actually teach
71
+
72
+ - **Format and label *space* teach more than per-example correctness.** Min et al.
73
+ ("Rethinking the Role of Demonstrations…", EMNLP 2022, arXiv 2202.12837) found that
74
+ randomly replacing the labels in demonstrations barely hurt performance — what the
75
+ model reads off the examples is the *shape* of the task: the input distribution, the
76
+ set of possible answers, and the output format. Practical consequence: keep every
77
+ example in the **exact** format you want back, and make sure the examples span the
78
+ real label set. A perfectly-labeled set of examples in the wrong format teaches the
79
+ wrong thing.
80
+ - **Balance the examples to counter built-in bias.** Zhao et al. ("Calibrate Before
81
+ Use…", ICML 2021, arXiv 2102.09690) documented majority-label bias, recency bias, and
82
+ common-token bias — models drift toward whichever label appears most, appears last, or
83
+ is lexically common (their contextual calibration recovered up to 30.0% absolute on
84
+ the tasks studied). You will not calibrate logits inside a pasted prompt, but you get
85
+ most of the benefit for free by **balancing label frequencies** and not stacking
86
+ same-class examples.
87
+
88
+ ## How it composes with a framework's emitted prompt
89
+
90
+ Because section headers are stripped at emission (see SKILL.md step 6), examples cannot
91
+ live under an `EXAMPLES:` label that survives — they have to read as part of the flat
92
+ prompt. After the chosen framework's prompt is drafted, and only if examples earn their
93
+ place:
94
+
95
+ 1. Draft the framework prompt as normal.
96
+ 2. Insert the examples **before the final instruction**, introduced by a plain prose
97
+ line the stripped output can keep, e.g. *"Follow the format of these examples:"* —
98
+ each example showing input and the exact desired output.
99
+ 3. End with the actual task so the model acts after seeing the pattern, not before.
100
+
101
+ Emitted shape (the `Follow the format…` line and the examples are ordinary text in the
102
+ flat block; there are no headers):
103
+
104
+ ```
105
+ [The framework-shaped instruction and context.]
106
+
107
+ Follow the format of these examples:
108
+
109
+ Input: <real example input 1>
110
+ Output: <desired output 1, in the exact target format>
111
+
112
+ Input: <real example input 2>
113
+ Output: <desired output 2>
114
+
115
+ Now do the same for:
116
+ [Paste the actual input here]
117
+ ```
118
+
119
+ If the user has no real examples to supply and you cannot draw them from pasted source
120
+ material, say so and fall back to a precise instruction rather than inventing samples —
121
+ the same rule the frameworks follow for facts about the user's world.