@ckelsoe/prompt-architect 3.2.1 → 3.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +3 -3
- package/.claude-plugin/plugin.json +2 -2
- package/CHANGELOG.md +116 -5
- package/MIGRATION.md +1 -1
- package/README.md +42 -45
- package/adapters/README.md +1 -1
- package/adapters/system-prompt.md +41 -11
- package/package.json +5 -4
- package/scripts/install.js +52 -22
- package/scripts/test.js +62 -40
- package/scripts/validate-skill.js +184 -56
- package/skills/prompt-architect/SKILL.md +49 -47
- package/skills/prompt-architect/assets/templates/chain-of-density_template.txt +60 -30
- package/skills/prompt-architect/assets/templates/iterative-compression_template.txt +59 -0
- package/skills/prompt-architect/references/frameworks/ape.md +2 -0
- package/skills/prompt-architect/references/frameworks/bab.md +2 -0
- package/skills/prompt-architect/references/frameworks/cai-critique-revise.md +1 -1
- package/skills/prompt-architect/references/frameworks/chain-of-density.md +95 -423
- package/skills/prompt-architect/references/frameworks/chain-of-thought.md +17 -12
- package/skills/prompt-architect/references/frameworks/co-star.md +2 -0
- package/skills/prompt-architect/references/frameworks/ctf.md +3 -1
- package/skills/prompt-architect/references/frameworks/iterative-compression.md +481 -0
- package/skills/prompt-architect/references/frameworks/pre-mortem.md +6 -6
- package/skills/prompt-architect/references/frameworks/race.md +2 -0
- package/skills/prompt-architect/references/frameworks/react.md +1 -1
- package/skills/prompt-architect/references/frameworks/reverse-role.md +1 -1
- package/skills/prompt-architect/references/frameworks/rise.md +27 -8
- package/skills/prompt-architect/references/frameworks/risen.md +2 -0
- package/skills/prompt-architect/references/frameworks/rtf.md +2 -0
- package/skills/prompt-architect/references/frameworks/skeleton-of-thought.md +1 -1
- package/skills/prompt-architect/references/frameworks/step-back.md +1 -1
- package/skills/prompt-architect/references/frameworks/tidd-ec.md +19 -6
- package/skills/prompt-architect/scripts/framework_analyzer.py +0 -807
- package/skills/prompt-architect/scripts/prompt_evaluator.py +0 -336
|
@@ -2,478 +2,150 @@
|
|
|
2
2
|
|
|
3
3
|
## Overview
|
|
4
4
|
|
|
5
|
-
Chain of Density is
|
|
5
|
+
Chain of Density is a summarization technique that produces a series of increasingly **entity-dense** summaries of a source document while holding the **length fixed**. It begins with a deliberately sparse, verbose summary, then repeatedly identifies salient entities missing from the previous version and fuses them in — making room by compressing and rewriting existing text rather than by adding words or dropping content.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
**Research basis:** "From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting" (Adams, Fabbri, Ladhak, Lehman & Elhadad; arXiv 2309.04269; NewSum workshop at EMNLP 2023, ACL Anthology 2023.newsum-1.7). Entity density rose from 0.089 to 0.167 entities per token over five steps, against 0.151 for human-written summaries and 0.122 for a vanilla GPT-4 summary. In human evaluation, annotators preferred step 2 (30.8% of first-place votes) and step 3 (23.0%) over the sparse step 1 (8.3%) — density helps, but only up to a point.
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
1. Starts with a basic/verbose version
|
|
11
|
-
2. Iteratively refines through multiple passes
|
|
12
|
-
3. Increases information density each time
|
|
13
|
-
4. Removes fluff and redundancy
|
|
14
|
-
5. Enhances clarity and precision
|
|
15
|
-
6. Converges on optimal output
|
|
9
|
+
## The Defining Constraint: Length Stays Fixed
|
|
16
10
|
|
|
17
|
-
|
|
11
|
+
This is the entire mechanism, and it is what separates CoD from ordinary iterative editing:
|
|
18
12
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
**Ideal for:**
|
|
22
|
-
- Summarization tasks
|
|
23
|
-
- Content compression
|
|
24
|
-
- Iterative improvement of writing
|
|
25
|
-
- Optimizing explanations
|
|
26
|
-
- Refining complex outputs
|
|
27
|
-
- Tasks where quality improves with iteration
|
|
28
|
-
|
|
29
|
-
**Not needed for:**
|
|
30
|
-
- Simple one-shot tasks
|
|
31
|
-
- When first draft is sufficient
|
|
32
|
-
- Time-critical quick tasks
|
|
33
|
-
- Tasks with fixed formats that can't improve
|
|
34
|
-
|
|
35
|
-
## Implementation Approaches
|
|
36
|
-
|
|
37
|
-
### Approach 1: Explicit Multi-Pass
|
|
38
|
-
Request multiple iterations upfront.
|
|
39
|
-
|
|
40
|
-
**Example:**
|
|
41
|
-
```
|
|
42
|
-
Create a summary of this article using Chain of Density:
|
|
13
|
+
> "GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities **without increasing the length**." — abstract
|
|
43
14
|
|
|
44
|
-
|
|
45
|
-
Write a verbose summary hitting all main points (150-200 words)
|
|
15
|
+
> "To maintain the same length while increasing the number of entities covered, abstraction, fusion, and compression is explicitly encouraged, rather than dropping meaningful content from previous summaries." — §2
|
|
46
16
|
|
|
47
|
-
|
|
48
|
-
Refine the summary: increase information density, remove redundancy, keep it under 150 words
|
|
17
|
+
Density rises because the *same word budget* is made to carry more information. If you shorten the text on each pass, you are not doing Chain of Density — you are doing generic compression. For that, use **Iterative Compression** (`iterative-compression.md`).
|
|
49
18
|
|
|
50
|
-
|
|
51
|
-
Further refine: make every word count, increase precision, keep it under 120 words
|
|
19
|
+
The paper controls length deliberately, because length is "a strong confounder when evaluating summaries." In practice the budget holds approximately: measured token counts across the five steps were 72, 67, 67, 69, 72.
|
|
52
20
|
|
|
53
|
-
|
|
54
|
-
Final pass: maximize information density while maintaining clarity (100 words max)
|
|
21
|
+
## What Counts as a "Missing Entity"
|
|
55
22
|
|
|
56
|
-
|
|
57
|
-
Ultimate compression: distill to absolute essentials (50 words max)
|
|
23
|
+
The paper is deliberately non-prescriptive about entity *types*. An entity qualifies if it is:
|
|
58
24
|
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
**Example:**
|
|
66
|
-
```
|
|
67
|
-
PASS 1: Write a first draft explanation of quantum entanglement for a general audience.
|
|
25
|
+
- **Relevant** — to the main story
|
|
26
|
+
- **Specific** — descriptive yet concise (5 words or fewer)
|
|
27
|
+
- **Novel** — not in the previous summary
|
|
28
|
+
- **Faithful** — present in the source article
|
|
29
|
+
- **Anywhere** — located anywhere in the article, not just the opening
|
|
68
30
|
|
|
69
|
-
|
|
31
|
+
"Anywhere" is load-bearing: it is the explicit countermeasure to lead bias, and it is why later iterations start pulling content from the middle and end of the source.
|
|
70
32
|
|
|
71
|
-
|
|
33
|
+
Entity identification is done by the model itself inside the loop — there is no external extraction step.
|
|
72
34
|
|
|
73
|
-
|
|
35
|
+
## The Method
|
|
74
36
|
|
|
75
|
-
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
### Approach 3: Targeted Improvement
|
|
79
|
-
Focus each iteration on specific aspect.
|
|
80
|
-
|
|
81
|
-
**Example:**
|
|
82
|
-
```
|
|
83
|
-
Write a product description, then improve it iteratively:
|
|
37
|
+
Five iterations, issued as a **single prompt** that returns all five summaries at once (not five separate turns):
|
|
84
38
|
|
|
85
|
-
|
|
86
|
-
|
|
39
|
+
1. **Write a deliberately sparse first summary.** Long (4-5 sentences, ~80 words) but highly non-specific, padded with filler like "this article discusses." The padding is intentional — it creates the slack that later steps convert into entities.
|
|
40
|
+
2. **Identify 1-3 missing entities** from the article that are absent from the current summary.
|
|
41
|
+
3. **Rewrite at identical length**, covering everything from the previous summary *plus* the new entities. Make room through fusion, compression, and removal of uninformative phrases.
|
|
42
|
+
4. **Repeat steps 2-3** until five summaries exist.
|
|
43
|
+
5. **Choose the step you want.** Later is not automatically better — human raters preferred steps 2-3.
|
|
87
44
|
|
|
88
|
-
|
|
89
|
-
Rewrite focusing on making it clearer and easier to understand.
|
|
45
|
+
**Never drop entities from the previous summary.** If space cannot be made, add fewer new entities. Under pressure, density yields — length and prior content do not.
|
|
90
46
|
|
|
91
|
-
|
|
92
|
-
Rewrite focusing on making it more compelling and persuasive.
|
|
93
|
-
|
|
94
|
-
ITERATION 3 - CONCISION:
|
|
95
|
-
Rewrite focusing on removing every unnecessary word.
|
|
96
|
-
|
|
97
|
-
ITERATION 4 - PRECISION:
|
|
98
|
-
Rewrite focusing on being more specific and accurate.
|
|
99
|
-
|
|
100
|
-
FINAL:
|
|
101
|
-
Combine the best elements from all iterations.
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
### Approach 4: Constraint-Based Refinement
|
|
105
|
-
Add tighter constraints with each iteration.
|
|
106
|
-
|
|
107
|
-
**Example:**
|
|
108
|
-
```
|
|
109
|
-
Explain this technical concept through progressive refinement:
|
|
110
|
-
|
|
111
|
-
VERSION 1: Explain in 500 words or less
|
|
112
|
-
VERSION 2: Reduce to 300 words without losing key information
|
|
113
|
-
VERSION 3: Reduce to 150 words, keeping only essentials
|
|
114
|
-
VERSION 4: Create a 50-word version capturing the core idea
|
|
115
|
-
VERSION 5: One sentence (25 words max)
|
|
116
|
-
|
|
117
|
-
Each version should be complete and self-contained.
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
## Best Practices
|
|
121
|
-
|
|
122
|
-
### 1. Define What to Optimize
|
|
123
|
-
```
|
|
124
|
-
Good:
|
|
125
|
-
"Iterate to improve:
|
|
126
|
-
- Information density
|
|
127
|
-
- Clarity
|
|
128
|
-
- Conciseness
|
|
129
|
-
- Precision"
|
|
130
|
-
|
|
131
|
-
Poor:
|
|
132
|
-
"Make it better"
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
### 2. Specify Number of Iterations
|
|
136
|
-
```
|
|
137
|
-
Good:
|
|
138
|
-
"Refine through 4 iterations, showing each version"
|
|
139
|
-
|
|
140
|
-
Poor:
|
|
141
|
-
"Keep improving until it's good"
|
|
142
|
-
```
|
|
143
|
-
|
|
144
|
-
### 3. Show Evolution
|
|
145
|
-
```
|
|
146
|
-
Good:
|
|
147
|
-
"Show each iteration and explain what changed and why"
|
|
148
|
-
|
|
149
|
-
Poor:
|
|
150
|
-
"Just give me the final version"
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
### 4. Set Clear Constraints
|
|
154
|
-
```
|
|
155
|
-
Good:
|
|
156
|
-
"Each iteration should be 20% shorter while maintaining completeness"
|
|
157
|
-
|
|
158
|
-
Poor:
|
|
159
|
-
"Make it shorter"
|
|
160
|
-
```
|
|
161
|
-
|
|
162
|
-
## Complete Examples
|
|
163
|
-
|
|
164
|
-
### Example 1: Summarization
|
|
165
|
-
|
|
166
|
-
**Without CoD:**
|
|
167
|
-
```
|
|
168
|
-
Summarize this research paper.
|
|
169
|
-
```
|
|
170
|
-
|
|
171
|
-
**With CoD:**
|
|
172
|
-
```
|
|
173
|
-
Summarize this research paper using Chain of Density:
|
|
174
|
-
|
|
175
|
-
[Paper content]
|
|
176
|
-
|
|
177
|
-
ITERATION 1 (200 words):
|
|
178
|
-
Create a comprehensive summary covering all major points. Prioritize completeness over brevity.
|
|
179
|
-
|
|
180
|
-
ITERATION 2 (150 words):
|
|
181
|
-
Refine the summary:
|
|
182
|
-
- Remove redundant information
|
|
183
|
-
- Combine related points
|
|
184
|
-
- Increase information density
|
|
185
|
-
|
|
186
|
-
ITERATION 3 (100 words):
|
|
187
|
-
Further compression:
|
|
188
|
-
- Keep only essential information
|
|
189
|
-
- Make every sentence count
|
|
190
|
-
- Maintain accuracy
|
|
47
|
+
## When to Use Chain of Density
|
|
191
48
|
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
-
|
|
195
|
-
-
|
|
196
|
-
-
|
|
49
|
+
**Ideal for:**
|
|
50
|
+
- Summarizing an article, paper, or report where information density matters
|
|
51
|
+
- Producing a summary that must fit a fixed length but carry maximum substance
|
|
52
|
+
- Generating several density options to pick from
|
|
53
|
+
- Abstract, executive-summary, and TL;DR writing
|
|
197
54
|
|
|
198
|
-
|
|
199
|
-
|
|
55
|
+
**Not for:**
|
|
56
|
+
- General content rewriting or quality improvement → use **Self-Refine**
|
|
57
|
+
- Progressive shortening to a target word count → use **Iterative Compression**
|
|
58
|
+
- Non-summarization tasks. CoD is summarization-specific; the paper evaluates it only on news articles (CNN/DailyMail).
|
|
200
59
|
|
|
201
|
-
|
|
60
|
+
## The Prompt
|
|
202
61
|
|
|
203
|
-
|
|
204
|
-
```
|
|
205
|
-
Document this function.
|
|
206
|
-
```
|
|
62
|
+
This is the paper's actual prompt, lightly reformatted. It is reproduced closely because the specific wording — especially the fixed-length instruction and the entity criteria — is what makes the technique work.
|
|
207
63
|
|
|
208
|
-
**With CoD:**
|
|
209
64
|
```
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
[Code]
|
|
213
|
-
|
|
214
|
-
DRAFT DOCUMENTATION:
|
|
215
|
-
Write complete documentation with examples, edge cases, everything.
|
|
65
|
+
Article: {{ARTICLE}}
|
|
216
66
|
|
|
217
|
-
|
|
218
|
-
Review and rewrite confusing sections. Add examples where helpful.
|
|
67
|
+
You will generate increasingly concise, entity-dense summaries of the above Article.
|
|
219
68
|
|
|
220
|
-
|
|
221
|
-
Remove redundancy. Combine related points. Make it tighter.
|
|
69
|
+
Repeat the following 2 steps 5 times.
|
|
222
70
|
|
|
223
|
-
|
|
224
|
-
|
|
71
|
+
Step 1. Identify 1-3 informative Entities (";" delimited) from the Article which are
|
|
72
|
+
missing from the previously generated summary.
|
|
73
|
+
Step 2. Write a new, denser summary of identical length which covers every entity and
|
|
74
|
+
detail from the previous summary plus the Missing Entities.
|
|
225
75
|
|
|
226
|
-
|
|
227
|
-
|
|
76
|
+
A Missing Entity is:
|
|
77
|
+
- Relevant: to the main story.
|
|
78
|
+
- Specific: descriptive yet concise (5 words or fewer).
|
|
79
|
+
- Novel: not in the previous summary.
|
|
80
|
+
- Faithful: present in the Article.
|
|
81
|
+
- Anywhere: located anywhere in the Article.
|
|
228
82
|
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
83
|
+
Guidelines:
|
|
84
|
+
- The first summary should be long (4-5 sentences, ~80 words) yet highly non-specific,
|
|
85
|
+
containing little information beyond the entities marked as missing. Use overly verbose
|
|
86
|
+
language and fillers (e.g., "this article discusses") to reach ~80 words.
|
|
87
|
+
- Make every word count: re-write the previous summary to improve flow and make space for
|
|
88
|
+
additional entities.
|
|
89
|
+
- Make space with fusion, compression, and removal of uninformative phrases like "the
|
|
90
|
+
article discusses".
|
|
91
|
+
- The summaries should become highly dense and concise yet self-contained, e.g., easily
|
|
92
|
+
understood without the Article.
|
|
93
|
+
- Missing entities can appear anywhere in the new summary.
|
|
94
|
+
- Never drop entities from the previous summary. If space cannot be made, add fewer new
|
|
95
|
+
entities.
|
|
233
96
|
|
|
234
|
-
|
|
235
|
-
```
|
|
236
|
-
Write an email announcing this feature.
|
|
237
|
-
```
|
|
97
|
+
Remember, use the exact same number of words for each summary.
|
|
238
98
|
|
|
239
|
-
|
|
99
|
+
Answer in JSON. The JSON should be a list (length 5) of dictionaries whose keys are
|
|
100
|
+
"Missing_Entities" and "Denser_Summary".
|
|
240
101
|
```
|
|
241
|
-
Draft an announcement email, then refine it:
|
|
242
|
-
|
|
243
|
-
DRAFT:
|
|
244
|
-
Write a first draft covering all important information.
|
|
245
|
-
|
|
246
|
-
ITERATION 1 - STRUCTURE:
|
|
247
|
-
Reorganize for better flow. Most important information first.
|
|
248
|
-
|
|
249
|
-
ITERATION 2 - CLARITY:
|
|
250
|
-
Simplify language. Remove jargon. Make it scannable.
|
|
251
|
-
|
|
252
|
-
ITERATION 3 - ENGAGEMENT:
|
|
253
|
-
Strengthen the call-to-action. Make it more compelling.
|
|
254
102
|
|
|
255
|
-
|
|
256
|
-
Final edits for tone, concision, impact.
|
|
103
|
+
## Worked Example
|
|
257
104
|
|
|
258
|
-
|
|
259
|
-
```
|
|
260
|
-
|
|
261
|
-
### Example 4: Technical Explanation
|
|
105
|
+
**Source:** a 1,200-word news article about a municipal transit funding decision.
|
|
262
106
|
|
|
263
|
-
**
|
|
264
|
-
|
|
265
|
-
Explain how this algorithm works.
|
|
266
|
-
```
|
|
107
|
+
**Step 1 (sparse, ~80 words):**
|
|
108
|
+
> This article discusses a recent decision made by local government officials regarding transportation. It covers the various considerations that were taken into account during the deliberation process, as well as some of the reactions from different groups. The piece also touches on the broader context surrounding the issue and what it might mean going forward for the region and the people who live there.
|
|
267
109
|
|
|
268
|
-
|
|
269
|
-
```
|
|
270
|
-
Explain this algorithm through progressive refinement:
|
|
110
|
+
Entities: 0 meaningful. Pure filler — by design.
|
|
271
111
|
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
- What it does
|
|
275
|
-
- How it works
|
|
276
|
-
- Why it's designed this way
|
|
277
|
-
- Example walkthrough
|
|
112
|
+
**Step 2 (same length, entities fused in):**
|
|
113
|
+
> Portland's city council voted 4-1 on Tuesday to redirect $47 million from the Southwest Corridor light-rail project to bus rapid transit, citing a 30% cost overrun. Commissioner Dana Reyes cast the dissenting vote. TriMet, which operates the region's transit, had lobbied for the shift. Riders' advocacy group OPAL called the decision a setback for east-side commuters, while the Portland Business Alliance welcomed it.
|
|
278
114
|
|
|
279
|
-
|
|
280
|
-
Review Pass 1. What's unclear? What's redundant? Rewrite addressing these.
|
|
115
|
+
Same word budget. The filler is gone; ten specific entities took its place.
|
|
281
116
|
|
|
282
|
-
|
|
283
|
-
Make it maximally clear and concise. Every paragraph must earn its place.
|
|
117
|
+
**Steps 3-5** continue fusing — ridership figures, the revised completion date, the federal grant condition — each time compressing existing phrasing to make room.
|
|
284
118
|
|
|
285
|
-
|
|
286
|
-
Final pass. Perfect clarity, zero fluff, optimal examples.
|
|
119
|
+
## Best Practices
|
|
287
120
|
|
|
288
|
-
|
|
289
|
-
|
|
121
|
+
1. **Enforce the length constraint explicitly.** Models drift toward expanding. "Use the exact same number of words for each summary" is not optional boilerplate.
|
|
122
|
+
2. **Let the first summary be bad.** The instinct is to start strong; resist it. A dense first summary leaves no slack, and the chain stalls immediately.
|
|
123
|
+
3. **Ask for all iterations in one response.** The paper uses a single prompt returning a JSON list of five. Splitting it across turns loses the model's view of the prior chain.
|
|
124
|
+
4. **Pick the step, don't assume the last.** Step 5 is the densest but not the most readable. Steps 2-3 won the human evaluation.
|
|
125
|
+
5. **Watch for the readability tradeoff.** Past a point, dense summaries read as lists of facts. The paper found this explicitly.
|
|
290
126
|
|
|
291
127
|
## Combining CoD with Other Frameworks
|
|
292
128
|
|
|
293
129
|
### CoD + CO-STAR
|
|
294
|
-
|
|
295
|
-
CONTEXT: [Background]
|
|
296
|
-
OBJECTIVE: [Goal]
|
|
297
|
-
STYLE: [Writing style]
|
|
298
|
-
TONE: [Tone]
|
|
299
|
-
AUDIENCE: [Readers]
|
|
300
|
-
|
|
301
|
-
RESPONSE PROCESS (Chain of Density):
|
|
302
|
-
Draft 1: [Comprehensive version]
|
|
303
|
-
Draft 2: [Refined for clarity]
|
|
304
|
-
Draft 3: [Optimized for audience]
|
|
305
|
-
Draft 4: [Final polished version]
|
|
306
|
-
```
|
|
307
|
-
|
|
308
|
-
### CoD + RISEN
|
|
309
|
-
```
|
|
310
|
-
ROLE: [Expertise]
|
|
311
|
-
INSTRUCTIONS: Use iterative refinement
|
|
312
|
-
STEPS:
|
|
313
|
-
1. Create initial version
|
|
314
|
-
2. Review and identify weaknesses
|
|
315
|
-
3. Refine addressing weaknesses
|
|
316
|
-
4. Repeat until optimal
|
|
317
|
-
END GOAL: [Final quality criteria]
|
|
318
|
-
NARROWING: [Constraints on iterations]
|
|
319
|
-
```
|
|
130
|
+
Use CO-STAR to fix audience, tone, and format for the summary; use CoD to maximize what fits in that format. Useful when the summary has a specific reader.
|
|
320
131
|
|
|
321
132
|
### CoD + Chain of Thought
|
|
322
|
-
|
|
323
|
-
For each iteration of Chain of Density, use Chain of Thought:
|
|
324
|
-
|
|
325
|
-
ITERATION 1:
|
|
326
|
-
Think through: What should this initial version include?
|
|
327
|
-
[Create version]
|
|
328
|
-
|
|
329
|
-
ITERATION 2:
|
|
330
|
-
Think through: What weaknesses exist? How to address them?
|
|
331
|
-
[Refine version]
|
|
332
|
-
|
|
333
|
-
ITERATION 3:
|
|
334
|
-
Think through: What can be compressed or improved?
|
|
335
|
-
[Further refine]
|
|
336
|
-
```
|
|
337
|
-
|
|
338
|
-
## Advanced Techniques
|
|
339
|
-
|
|
340
|
-
### Aspect-Focused Iteration
|
|
341
|
-
Each iteration improves a different dimension:
|
|
342
|
-
|
|
343
|
-
```
|
|
344
|
-
ITERATION 1 - ACCURACY: Get the facts right
|
|
345
|
-
ITERATION 2 - CLARITY: Make it understandable
|
|
346
|
-
ITERATION 3 - CONCISION: Remove waste
|
|
347
|
-
ITERATION 4 - IMPACT: Make it compelling
|
|
348
|
-
ITERATION 5 - STYLE: Polish the voice
|
|
349
|
-
```
|
|
350
|
-
|
|
351
|
-
### A/B Iteration Branches
|
|
352
|
-
Create alternative versions and merge best parts:
|
|
353
|
-
|
|
354
|
-
```
|
|
355
|
-
DRAFT: [Initial version]
|
|
356
|
-
|
|
357
|
-
BRANCH A - CONCISION:
|
|
358
|
-
[Optimize for brevity]
|
|
359
|
-
|
|
360
|
-
BRANCH B - COMPLETENESS:
|
|
361
|
-
[Optimize for thoroughness]
|
|
362
|
-
|
|
363
|
-
MERGE:
|
|
364
|
-
Combine the best elements of both branches.
|
|
365
|
-
```
|
|
366
|
-
|
|
367
|
-
### Constrained Progressive Compression
|
|
368
|
-
```
|
|
369
|
-
VERSION 1: 1000 words - Complete explanation
|
|
370
|
-
VERSION 2: 500 words - 50% compression
|
|
371
|
-
VERSION 3: 250 words - 75% compression
|
|
372
|
-
VERSION 4: 100 words - 90% compression
|
|
373
|
-
VERSION 5: 25 words - 97.5% compression
|
|
374
|
-
|
|
375
|
-
Each version must be self-contained and accurate.
|
|
376
|
-
```
|
|
377
|
-
|
|
378
|
-
### Quality Metric Iteration
|
|
379
|
-
```
|
|
380
|
-
ITERATION 1:
|
|
381
|
-
[Create initial version]
|
|
382
|
-
Self-assess (1-10): Clarity ___ Concision ___ Accuracy ___ Impact ___
|
|
383
|
-
|
|
384
|
-
ITERATION 2:
|
|
385
|
-
[Improve lowest scoring dimension]
|
|
386
|
-
Self-assess: Clarity ___ Concision ___ Accuracy ___ Impact ___
|
|
387
|
-
|
|
388
|
-
ITERATION 3:
|
|
389
|
-
[Improve next lowest dimension]
|
|
390
|
-
Self-assess: Clarity ___ Concision ___ Accuracy ___ Impact ___
|
|
391
|
-
|
|
392
|
-
Continue until all metrics > 8
|
|
393
|
-
```
|
|
394
|
-
|
|
395
|
-
## Refinement Strategies
|
|
396
|
-
|
|
397
|
-
### What to Remove
|
|
398
|
-
- Redundant information
|
|
399
|
-
- Obvious statements
|
|
400
|
-
- Unnecessary qualifiers ("very", "really")
|
|
401
|
-
- Passive voice (often)
|
|
402
|
-
- Filler words
|
|
403
|
-
- Overly general statements
|
|
404
|
-
|
|
405
|
-
### What to Add
|
|
406
|
-
- Specific examples
|
|
407
|
-
- Precise numbers
|
|
408
|
-
- Concrete details
|
|
409
|
-
- Missing context (if essential)
|
|
410
|
-
- Clarifying examples
|
|
411
|
-
|
|
412
|
-
### What to Improve
|
|
413
|
-
- Vague language → specific language
|
|
414
|
-
- Long sentences → shorter, punchier
|
|
415
|
-
- Complex words → simpler alternatives (if clearer)
|
|
416
|
-
- Weak verbs → strong verbs
|
|
417
|
-
- Passive voice → active voice
|
|
418
|
-
|
|
419
|
-
## Quality Indicators
|
|
420
|
-
|
|
421
|
-
**Good CoD iterations show:**
|
|
422
|
-
- ✅ Measurable improvement each pass
|
|
423
|
-
- ✅ Increased information density
|
|
424
|
-
- ✅ Reduced redundancy
|
|
425
|
-
- ✅ Maintained accuracy
|
|
426
|
-
- ✅ Enhanced clarity
|
|
427
|
-
- ✅ Clear evolution path
|
|
428
|
-
|
|
429
|
-
**Poor CoD iterations show:**
|
|
430
|
-
- ❌ No meaningful change
|
|
431
|
-
- ❌ Loss of important information
|
|
432
|
-
- ❌ Decreased clarity
|
|
433
|
-
- ❌ Introduced errors
|
|
434
|
-
- ❌ Just made it shorter without improvement
|
|
435
|
-
|
|
436
|
-
## Use Cases
|
|
437
|
-
|
|
438
|
-
### 1. Content Summarization
|
|
439
|
-
Taking long documents and progressively distilling essence
|
|
440
|
-
|
|
441
|
-
### 2. Message Refinement
|
|
442
|
-
Iteratively improving emails, announcements, communications
|
|
443
|
-
|
|
444
|
-
### 3. Documentation Optimization
|
|
445
|
-
Making docs clearer and more concise through iteration
|
|
446
|
-
|
|
447
|
-
### 4. Explanation Enhancement
|
|
448
|
-
Progressively improving how concepts are explained
|
|
449
|
-
|
|
450
|
-
### 5. Writing Polish
|
|
451
|
-
Iterative editing of creative or professional writing
|
|
452
|
-
|
|
453
|
-
### 6. Presentation Optimization
|
|
454
|
-
Refining slides/talks through multiple passes
|
|
133
|
+
Ask the model to reason about which entities are most salient before selecting them. Helps when the source is long and salience is contested.
|
|
455
134
|
|
|
456
135
|
## Assessment Checklist
|
|
457
136
|
|
|
458
|
-
|
|
459
|
-
- [ ]
|
|
460
|
-
- [ ]
|
|
461
|
-
- [ ]
|
|
462
|
-
- [ ]
|
|
463
|
-
- [ ]
|
|
464
|
-
- [ ]
|
|
465
|
-
- [ ] Quality criteria established
|
|
466
|
-
- [ ] Stopping condition exists
|
|
467
|
-
- [ ] Final version is actually better
|
|
468
|
-
- [ ] Process improved understanding
|
|
137
|
+
- [ ] Is the task actually summarization? (If not, CoD is the wrong framework.)
|
|
138
|
+
- [ ] Is a fixed target length stated?
|
|
139
|
+
- [ ] Is the first summary deliberately sparse and padded?
|
|
140
|
+
- [ ] Are entity criteria included in the prompt?
|
|
141
|
+
- [ ] Does each step preserve all prior entities?
|
|
142
|
+
- [ ] Is the word count roughly constant across steps?
|
|
143
|
+
- [ ] Was the chosen step selected on merit rather than defaulting to the last?
|
|
469
144
|
|
|
470
145
|
## Tips for Success
|
|
471
146
|
|
|
472
|
-
1. **
|
|
473
|
-
2. **
|
|
474
|
-
3. **
|
|
475
|
-
4. **
|
|
476
|
-
5. **
|
|
477
|
-
6. **Don't Over-Iterate**: 3-5 iterations usually sufficient
|
|
478
|
-
7. **Preserve Accuracy**: Never sacrifice correctness for brevity
|
|
479
|
-
8. **Explain Changes**: Note what improved and why
|
|
147
|
+
1. **Fixed length, rising density** — if length is falling, it is not CoD
|
|
148
|
+
2. **Start verbose on purpose** — the filler is the raw material
|
|
149
|
+
3. **1-3 entities per step** — more causes dropped content
|
|
150
|
+
4. **Five steps is the paper's number** — fewer works; more yields diminishing returns
|
|
151
|
+
5. **Compare steps before choosing** — the densest is rarely the most useful
|
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
Chain of Thought is a prompting technique that encourages step-by-step reasoning and makes the thinking process explicit. Instead of jumping to answers, it guides Claude to break down complex problems, show intermediate steps, and verify logic along the way.
|
|
6
6
|
|
|
7
|
+
**Research basis:** "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (Wei et al., Google Research, arXiv 2201.11903, NeurIPS 2022). Prompting PaLM 540B with eight chain-of-thought exemplars raised GSM8K accuracy from 17.9% with standard prompting to 56.9%, beating the prior fine-tuned state of the art of 55% (GPT-3 plus a verifier, Cobbe et al. 2021).
|
|
8
|
+
|
|
7
9
|
## Core Concept
|
|
8
10
|
|
|
9
11
|
CoT works by:
|
|
@@ -319,24 +321,27 @@ STEPS:
|
|
|
319
321
|
### Self-Consistency
|
|
320
322
|
Generate multiple reasoning paths and choose the most common answer:
|
|
321
323
|
|
|
324
|
+
**Research basis:** "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (Wang et al., arXiv 2203.11171, ICLR 2023). Sample a diverse set of reasoning paths rather than greedy-decoding one, then "marginalize out the reasoning paths and aggregate by choosing the most consistent answer in the final answer set." On PaLM 540B / GSM8K, a majority vote over 40 sampled paths reached 74.4% versus 56.5% for greedy CoT decoding (+17.9 points); an unweighted majority vote matched or beat every weighted aggregation the authors tested.
|
|
325
|
+
|
|
326
|
+
The mechanism is **sampling the same prompt repeatedly and taking a majority vote** — not asking the model to try different methods and then judge which it likes best. Self-assessment is a different (and weaker) technique; the model is not a reliable judge of its own reasoning.
|
|
327
|
+
|
|
322
328
|
```
|
|
323
|
-
Solve this problem
|
|
329
|
+
Solve this problem. Show your step-by-step reasoning, then state your final
|
|
330
|
+
answer on its own line as:
|
|
324
331
|
|
|
325
|
-
|
|
326
|
-
[Method 1 with step-by-step reasoning]
|
|
332
|
+
FINAL ANSWER: [answer]
|
|
327
333
|
|
|
328
|
-
|
|
329
|
-
|
|
334
|
+
[Problem]
|
|
335
|
+
```
|
|
330
336
|
|
|
331
|
-
|
|
332
|
-
[Method 3 with step-by-step reasoning]
|
|
337
|
+
Run this **same prompt N times at non-zero temperature** (the paper samples 40; 5-10 is usually enough in practice, and temperature ~0.7 is a reasonable default). Then aggregate outside the model:
|
|
333
338
|
|
|
334
|
-
|
|
335
|
-
|
|
339
|
+
1. Extract the `FINAL ANSWER:` value from each of the N samples
|
|
340
|
+
2. Count how often each distinct answer appears
|
|
341
|
+
3. Return the most frequent answer — a plain, unweighted majority vote
|
|
342
|
+
4. Treat a near-tie as a signal of genuine uncertainty, not as a reason to ask the model to break the tie
|
|
336
343
|
|
|
337
|
-
|
|
338
|
-
Based on the most consistent result.
|
|
339
|
-
```
|
|
344
|
+
Do **not** ask the model to compare its own paths and pick the best one. The diversity of independently sampled reasoning is what produces the gain; collapsing it into a single self-judgment discards exactly the signal the method depends on.
|
|
340
345
|
|
|
341
346
|
### Uncertainty Quantification
|
|
342
347
|
```
|
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
CO-STAR is a comprehensive prompting framework that emphasizes context, audience, and communication style. It's particularly effective for content creation, writing tasks, and scenarios where tone and audience considerations significantly impact output quality.
|
|
6
6
|
|
|
7
|
+
**Origin:** Developed by the Data Science & AI team at GovTech (Government Technology Agency of Singapore), and popularized by Sheila Teo, who won GovTech's "Prompt Royale" — Singapore's first GPT-4 prompt engineering competition, 400+ participants, finale 8 November 2023. Teo credits the framework to GovTech rather than to herself: "The CO-STAR framework, a brainchild of GovTech Singapore's Data Science & AI team, is a handy template for structuring prompts." Practitioner framework — no controlled evaluation of CO-STAR has been published.
|
|
8
|
+
|
|
7
9
|
## Components
|
|
8
10
|
|
|
9
11
|
### C - Context
|
|
@@ -4,6 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
CTF is a simple 3-component framework focused on situational context, task clarity, and output format. It is ideal for tasks where background/situation is more important than AI persona framing, and the primary concern is getting a clearly defined output from a well-understood context.
|
|
6
6
|
|
|
7
|
+
**Origin:** No single documented originator. CTF appears in the community framework literature as a variant of RTF, with Context substituted for Role, documented alongside it in unattributed practitioner listings from 2024 onward (e.g. fvivas.com, 15 March 2024). No academic treatment; absent from The Prompt Report (arXiv 2406.06608).
|
|
8
|
+
|
|
7
9
|
## Components
|
|
8
10
|
|
|
9
11
|
### C - Context
|
|
@@ -168,7 +170,7 @@ Markdown template with:
|
|
|
168
170
|
- ❌ Audience and tone are critical (use CO-STAR)
|
|
169
171
|
- ❌ Multi-step process with methodology needed (use RISEN)
|
|
170
172
|
- ❌ Explicit dos/don'ts required (use TIDD-EC)
|
|
171
|
-
- ❌ Input→output transformation (use RISE
|
|
173
|
+
- ❌ Input→output transformation (use RISE — the Role/Input/Steps/Expectation form)
|
|
172
174
|
|
|
173
175
|
## CTF vs RTF
|
|
174
176
|
|