@ckelsoe/prompt-architect 3.2.2 → 3.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. package/.claude-plugin/marketplace.json +3 -3
  2. package/.claude-plugin/plugin.json +2 -2
  3. package/CHANGELOG.md +116 -5
  4. package/MIGRATION.md +1 -1
  5. package/README.md +32 -47
  6. package/adapters/README.md +1 -1
  7. package/adapters/system-prompt.md +41 -11
  8. package/package.json +5 -4
  9. package/scripts/install.js +52 -22
  10. package/scripts/test.js +62 -40
  11. package/scripts/validate-skill.js +184 -56
  12. package/skills/prompt-architect/SKILL.md +49 -47
  13. package/skills/prompt-architect/assets/templates/chain-of-density_template.txt +60 -30
  14. package/skills/prompt-architect/assets/templates/iterative-compression_template.txt +59 -0
  15. package/skills/prompt-architect/references/frameworks/ape.md +2 -0
  16. package/skills/prompt-architect/references/frameworks/bab.md +2 -0
  17. package/skills/prompt-architect/references/frameworks/cai-critique-revise.md +1 -1
  18. package/skills/prompt-architect/references/frameworks/chain-of-density.md +95 -423
  19. package/skills/prompt-architect/references/frameworks/chain-of-thought.md +17 -12
  20. package/skills/prompt-architect/references/frameworks/co-star.md +2 -0
  21. package/skills/prompt-architect/references/frameworks/ctf.md +3 -1
  22. package/skills/prompt-architect/references/frameworks/iterative-compression.md +481 -0
  23. package/skills/prompt-architect/references/frameworks/pre-mortem.md +6 -6
  24. package/skills/prompt-architect/references/frameworks/race.md +2 -0
  25. package/skills/prompt-architect/references/frameworks/react.md +1 -1
  26. package/skills/prompt-architect/references/frameworks/reverse-role.md +1 -1
  27. package/skills/prompt-architect/references/frameworks/rise.md +27 -8
  28. package/skills/prompt-architect/references/frameworks/risen.md +2 -0
  29. package/skills/prompt-architect/references/frameworks/rtf.md +2 -0
  30. package/skills/prompt-architect/references/frameworks/skeleton-of-thought.md +1 -1
  31. package/skills/prompt-architect/references/frameworks/step-back.md +1 -1
  32. package/skills/prompt-architect/references/frameworks/tidd-ec.md +19 -6
  33. package/skills/prompt-architect/scripts/framework_analyzer.py +0 -807
  34. package/skills/prompt-architect/scripts/prompt_evaluator.py +0 -336
@@ -2,478 +2,150 @@
2
2
 
3
3
  ## Overview
4
4
 
5
- Chain of Density is an iterative refinement technique that progressively improves outputs through multiple passes. Each iteration increases information density, removes redundancy, and enhances quality without significantly increasing length. It's based on the insight that iteration often produces better results than trying to get everything perfect in one shot.
5
+ Chain of Density is a summarization technique that produces a series of increasingly **entity-dense** summaries of a source document while holding the **length fixed**. It begins with a deliberately sparse, verbose summary, then repeatedly identifies salient entities missing from the previous version and fuses them in making room by compressing and rewriting existing text rather than by adding words or dropping content.
6
6
 
7
- ## Core Concept
7
+ **Research basis:** "From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting" (Adams, Fabbri, Ladhak, Lehman & Elhadad; arXiv 2309.04269; NewSum workshop at EMNLP 2023, ACL Anthology 2023.newsum-1.7). Entity density rose from 0.089 to 0.167 entities per token over five steps, against 0.151 for human-written summaries and 0.122 for a vanilla GPT-4 summary. In human evaluation, annotators preferred step 2 (30.8% of first-place votes) and step 3 (23.0%) over the sparse step 1 (8.3%) — density helps, but only up to a point.
8
8
 
9
- Instead of requesting the perfect output immediately, CoD:
10
- 1. Starts with a basic/verbose version
11
- 2. Iteratively refines through multiple passes
12
- 3. Increases information density each time
13
- 4. Removes fluff and redundancy
14
- 5. Enhances clarity and precision
15
- 6. Converges on optimal output
9
+ ## The Defining Constraint: Length Stays Fixed
16
10
 
17
- Think of it as "progressive enhancement" for prompts.
11
+ This is the entire mechanism, and it is what separates CoD from ordinary iterative editing:
18
12
 
19
- ## When to Use Chain of Density
20
-
21
- **Ideal for:**
22
- - Summarization tasks
23
- - Content compression
24
- - Iterative improvement of writing
25
- - Optimizing explanations
26
- - Refining complex outputs
27
- - Tasks where quality improves with iteration
28
-
29
- **Not needed for:**
30
- - Simple one-shot tasks
31
- - When first draft is sufficient
32
- - Time-critical quick tasks
33
- - Tasks with fixed formats that can't improve
34
-
35
- ## Implementation Approaches
36
-
37
- ### Approach 1: Explicit Multi-Pass
38
- Request multiple iterations upfront.
39
-
40
- **Example:**
41
- ```
42
- Create a summary of this article using Chain of Density:
13
+ > "GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities **without increasing the length**." — abstract
43
14
 
44
- ITERATION 1:
45
- Write a verbose summary hitting all main points (150-200 words)
15
+ > "To maintain the same length while increasing the number of entities covered, abstraction, fusion, and compression is explicitly encouraged, rather than dropping meaningful content from previous summaries." — §2
46
16
 
47
- ITERATION 2:
48
- Refine the summary: increase information density, remove redundancy, keep it under 150 words
17
+ Density rises because the *same word budget* is made to carry more information. If you shorten the text on each pass, you are not doing Chain of Density — you are doing generic compression. For that, use **Iterative Compression** (`iterative-compression.md`).
49
18
 
50
- ITERATION 3:
51
- Further refine: make every word count, increase precision, keep it under 120 words
19
+ The paper controls length deliberately, because length is "a strong confounder when evaluating summaries." In practice the budget holds approximately: measured token counts across the five steps were 72, 67, 67, 69, 72.
52
20
 
53
- ITERATION 4:
54
- Final pass: maximize information density while maintaining clarity (100 words max)
21
+ ## What Counts as a "Missing Entity"
55
22
 
56
- ITERATION 5:
57
- Ultimate compression: distill to absolute essentials (50 words max)
23
+ The paper is deliberately non-prescriptive about entity *types*. An entity qualifies if it is:
58
24
 
59
- After all iterations, explain what improved in each pass.
60
- ```
61
-
62
- ### Approach 2: Recursive Refinement
63
- Use output from one iteration as input to the next.
64
-
65
- **Example:**
66
- ```
67
- PASS 1: Write a first draft explanation of quantum entanglement for a general audience.
25
+ - **Relevant** to the main story
26
+ - **Specific** — descriptive yet concise (5 words or fewer)
27
+ - **Novel** — not in the previous summary
28
+ - **Faithful** present in the source article
29
+ - **Anywhere** located anywhere in the article, not just the opening
68
30
 
69
- PASS 2: Review your explanation. What's unclear? What's redundant? Rewrite it, addressing these issues.
31
+ "Anywhere" is load-bearing: it is the explicit countermeasure to lead bias, and it is why later iterations start pulling content from the middle and end of the source.
70
32
 
71
- PASS 3: Review again. Where can you be more precise? Where can you use better examples? Rewrite.
33
+ Entity identification is done by the model itself inside the loop there is no external extraction step.
72
34
 
73
- PASS 4: Final review. Every sentence should earn its place. Remove or improve anything that doesn't. Final version.
35
+ ## The Method
74
36
 
75
- Show all passes so we can see the evolution.
76
- ```
77
-
78
- ### Approach 3: Targeted Improvement
79
- Focus each iteration on specific aspect.
80
-
81
- **Example:**
82
- ```
83
- Write a product description, then improve it iteratively:
37
+ Five iterations, issued as a **single prompt** that returns all five summaries at once (not five separate turns):
84
38
 
85
- DRAFT:
86
- [Initial version]
39
+ 1. **Write a deliberately sparse first summary.** Long (4-5 sentences, ~80 words) but highly non-specific, padded with filler like "this article discusses." The padding is intentional — it creates the slack that later steps convert into entities.
40
+ 2. **Identify 1-3 missing entities** from the article that are absent from the current summary.
41
+ 3. **Rewrite at identical length**, covering everything from the previous summary *plus* the new entities. Make room through fusion, compression, and removal of uninformative phrases.
42
+ 4. **Repeat steps 2-3** until five summaries exist.
43
+ 5. **Choose the step you want.** Later is not automatically better — human raters preferred steps 2-3.
87
44
 
88
- ITERATION 1 - CLARITY:
89
- Rewrite focusing on making it clearer and easier to understand.
45
+ **Never drop entities from the previous summary.** If space cannot be made, add fewer new entities. Under pressure, density yields — length and prior content do not.
90
46
 
91
- ITERATION 2 - IMPACT:
92
- Rewrite focusing on making it more compelling and persuasive.
93
-
94
- ITERATION 3 - CONCISION:
95
- Rewrite focusing on removing every unnecessary word.
96
-
97
- ITERATION 4 - PRECISION:
98
- Rewrite focusing on being more specific and accurate.
99
-
100
- FINAL:
101
- Combine the best elements from all iterations.
102
- ```
103
-
104
- ### Approach 4: Constraint-Based Refinement
105
- Add tighter constraints with each iteration.
106
-
107
- **Example:**
108
- ```
109
- Explain this technical concept through progressive refinement:
110
-
111
- VERSION 1: Explain in 500 words or less
112
- VERSION 2: Reduce to 300 words without losing key information
113
- VERSION 3: Reduce to 150 words, keeping only essentials
114
- VERSION 4: Create a 50-word version capturing the core idea
115
- VERSION 5: One sentence (25 words max)
116
-
117
- Each version should be complete and self-contained.
118
- ```
119
-
120
- ## Best Practices
121
-
122
- ### 1. Define What to Optimize
123
- ```
124
- Good:
125
- "Iterate to improve:
126
- - Information density
127
- - Clarity
128
- - Conciseness
129
- - Precision"
130
-
131
- Poor:
132
- "Make it better"
133
- ```
134
-
135
- ### 2. Specify Number of Iterations
136
- ```
137
- Good:
138
- "Refine through 4 iterations, showing each version"
139
-
140
- Poor:
141
- "Keep improving until it's good"
142
- ```
143
-
144
- ### 3. Show Evolution
145
- ```
146
- Good:
147
- "Show each iteration and explain what changed and why"
148
-
149
- Poor:
150
- "Just give me the final version"
151
- ```
152
-
153
- ### 4. Set Clear Constraints
154
- ```
155
- Good:
156
- "Each iteration should be 20% shorter while maintaining completeness"
157
-
158
- Poor:
159
- "Make it shorter"
160
- ```
161
-
162
- ## Complete Examples
163
-
164
- ### Example 1: Summarization
165
-
166
- **Without CoD:**
167
- ```
168
- Summarize this research paper.
169
- ```
170
-
171
- **With CoD:**
172
- ```
173
- Summarize this research paper using Chain of Density:
174
-
175
- [Paper content]
176
-
177
- ITERATION 1 (200 words):
178
- Create a comprehensive summary covering all major points. Prioritize completeness over brevity.
179
-
180
- ITERATION 2 (150 words):
181
- Refine the summary:
182
- - Remove redundant information
183
- - Combine related points
184
- - Increase information density
185
-
186
- ITERATION 3 (100 words):
187
- Further compression:
188
- - Keep only essential information
189
- - Make every sentence count
190
- - Maintain accuracy
47
+ ## When to Use Chain of Density
191
48
 
192
- ITERATION 4 (75 words):
193
- Maximum density:
194
- - Distill to core findings
195
- - Remove all fluff
196
- - Preserve critical details
49
+ **Ideal for:**
50
+ - Summarizing an article, paper, or report where information density matters
51
+ - Producing a summary that must fit a fixed length but carry maximum substance
52
+ - Generating several density options to pick from
53
+ - Abstract, executive-summary, and TL;DR writing
197
54
 
198
- For each iteration, highlight what you removed and why.
199
- ```
55
+ **Not for:**
56
+ - General content rewriting or quality improvement → use **Self-Refine**
57
+ - Progressive shortening to a target word count → use **Iterative Compression**
58
+ - Non-summarization tasks. CoD is summarization-specific; the paper evaluates it only on news articles (CNN/DailyMail).
200
59
 
201
- ### Example 2: Code Documentation
60
+ ## The Prompt
202
61
 
203
- **Without CoD:**
204
- ```
205
- Document this function.
206
- ```
62
+ This is the paper's actual prompt, lightly reformatted. It is reproduced closely because the specific wording — especially the fixed-length instruction and the entity criteria — is what makes the technique work.
207
63
 
208
- **With CoD:**
209
64
  ```
210
- Document this function through iterative refinement:
211
-
212
- [Code]
213
-
214
- DRAFT DOCUMENTATION:
215
- Write complete documentation with examples, edge cases, everything.
65
+ Article: {{ARTICLE}}
216
66
 
217
- ITERATION 1 - CLARITY:
218
- Review and rewrite confusing sections. Add examples where helpful.
67
+ You will generate increasingly concise, entity-dense summaries of the above Article.
219
68
 
220
- ITERATION 2 - CONCISION:
221
- Remove redundancy. Combine related points. Make it tighter.
69
+ Repeat the following 2 steps 5 times.
222
70
 
223
- ITERATION 3 - PRECISION:
224
- Be more specific about types, behaviors, edge cases.
71
+ Step 1. Identify 1-3 informative Entities (";" delimited) from the Article which are
72
+ missing from the previously generated summary.
73
+ Step 2. Write a new, denser summary of identical length which covers every entity and
74
+ detail from the previous summary plus the Missing Entities.
225
75
 
226
- ITERATION 4 - USABILITY:
227
- Optimize for developer experience. What do they need to know most?
76
+ A Missing Entity is:
77
+ - Relevant: to the main story.
78
+ - Specific: descriptive yet concise (5 words or fewer).
79
+ - Novel: not in the previous summary.
80
+ - Faithful: present in the Article.
81
+ - Anywhere: located anywhere in the Article.
228
82
 
229
- Show each version and explain key changes.
230
- ```
231
-
232
- ### Example 3: Email Refinement
83
+ Guidelines:
84
+ - The first summary should be long (4-5 sentences, ~80 words) yet highly non-specific,
85
+ containing little information beyond the entities marked as missing. Use overly verbose
86
+ language and fillers (e.g., "this article discusses") to reach ~80 words.
87
+ - Make every word count: re-write the previous summary to improve flow and make space for
88
+ additional entities.
89
+ - Make space with fusion, compression, and removal of uninformative phrases like "the
90
+ article discusses".
91
+ - The summaries should become highly dense and concise yet self-contained, e.g., easily
92
+ understood without the Article.
93
+ - Missing entities can appear anywhere in the new summary.
94
+ - Never drop entities from the previous summary. If space cannot be made, add fewer new
95
+ entities.
233
96
 
234
- **Without CoD:**
235
- ```
236
- Write an email announcing this feature.
237
- ```
97
+ Remember, use the exact same number of words for each summary.
238
98
 
239
- **With CoD:**
99
+ Answer in JSON. The JSON should be a list (length 5) of dictionaries whose keys are
100
+ "Missing_Entities" and "Denser_Summary".
240
101
  ```
241
- Draft an announcement email, then refine it:
242
-
243
- DRAFT:
244
- Write a first draft covering all important information.
245
-
246
- ITERATION 1 - STRUCTURE:
247
- Reorganize for better flow. Most important information first.
248
-
249
- ITERATION 2 - CLARITY:
250
- Simplify language. Remove jargon. Make it scannable.
251
-
252
- ITERATION 3 - ENGAGEMENT:
253
- Strengthen the call-to-action. Make it more compelling.
254
102
 
255
- ITERATION 4 - POLISH:
256
- Final edits for tone, concision, impact.
103
+ ## Worked Example
257
104
 
258
- Show the evolution and explain major changes.
259
- ```
260
-
261
- ### Example 4: Technical Explanation
105
+ **Source:** a 1,200-word news article about a municipal transit funding decision.
262
106
 
263
- **Without CoD:**
264
- ```
265
- Explain how this algorithm works.
266
- ```
107
+ **Step 1 (sparse, ~80 words):**
108
+ > This article discusses a recent decision made by local government officials regarding transportation. It covers the various considerations that were taken into account during the deliberation process, as well as some of the reactions from different groups. The piece also touches on the broader context surrounding the issue and what it might mean going forward for the region and the people who live there.
267
109
 
268
- **With CoD:**
269
- ```
270
- Explain this algorithm through progressive refinement:
110
+ Entities: 0 meaningful. Pure filler — by design.
271
111
 
272
- PASS 1 - COMPREHENSIVE:
273
- Explain thoroughly, assuming no prior knowledge. Include:
274
- - What it does
275
- - How it works
276
- - Why it's designed this way
277
- - Example walkthrough
112
+ **Step 2 (same length, entities fused in):**
113
+ > Portland's city council voted 4-1 on Tuesday to redirect $47 million from the Southwest Corridor light-rail project to bus rapid transit, citing a 30% cost overrun. Commissioner Dana Reyes cast the dissenting vote. TriMet, which operates the region's transit, had lobbied for the shift. Riders' advocacy group OPAL called the decision a setback for east-side commuters, while the Portland Business Alliance welcomed it.
278
114
 
279
- PASS 2 - REFINED:
280
- Review Pass 1. What's unclear? What's redundant? Rewrite addressing these.
115
+ Same word budget. The filler is gone; ten specific entities took its place.
281
116
 
282
- PASS 3 - OPTIMIZED:
283
- Make it maximally clear and concise. Every paragraph must earn its place.
117
+ **Steps 3-5** continue fusing — ridership figures, the revised completion date, the federal grant condition — each time compressing existing phrasing to make room.
284
118
 
285
- PASS 4 - POLISHED:
286
- Final pass. Perfect clarity, zero fluff, optimal examples.
119
+ ## Best Practices
287
120
 
288
- Show all passes and note significant improvements.
289
- ```
121
+ 1. **Enforce the length constraint explicitly.** Models drift toward expanding. "Use the exact same number of words for each summary" is not optional boilerplate.
122
+ 2. **Let the first summary be bad.** The instinct is to start strong; resist it. A dense first summary leaves no slack, and the chain stalls immediately.
123
+ 3. **Ask for all iterations in one response.** The paper uses a single prompt returning a JSON list of five. Splitting it across turns loses the model's view of the prior chain.
124
+ 4. **Pick the step, don't assume the last.** Step 5 is the densest but not the most readable. Steps 2-3 won the human evaluation.
125
+ 5. **Watch for the readability tradeoff.** Past a point, dense summaries read as lists of facts. The paper found this explicitly.
290
126
 
291
127
  ## Combining CoD with Other Frameworks
292
128
 
293
129
  ### CoD + CO-STAR
294
- ```
295
- CONTEXT: [Background]
296
- OBJECTIVE: [Goal]
297
- STYLE: [Writing style]
298
- TONE: [Tone]
299
- AUDIENCE: [Readers]
300
-
301
- RESPONSE PROCESS (Chain of Density):
302
- Draft 1: [Comprehensive version]
303
- Draft 2: [Refined for clarity]
304
- Draft 3: [Optimized for audience]
305
- Draft 4: [Final polished version]
306
- ```
307
-
308
- ### CoD + RISEN
309
- ```
310
- ROLE: [Expertise]
311
- INSTRUCTIONS: Use iterative refinement
312
- STEPS:
313
- 1. Create initial version
314
- 2. Review and identify weaknesses
315
- 3. Refine addressing weaknesses
316
- 4. Repeat until optimal
317
- END GOAL: [Final quality criteria]
318
- NARROWING: [Constraints on iterations]
319
- ```
130
+ Use CO-STAR to fix audience, tone, and format for the summary; use CoD to maximize what fits in that format. Useful when the summary has a specific reader.
320
131
 
321
132
  ### CoD + Chain of Thought
322
- ```
323
- For each iteration of Chain of Density, use Chain of Thought:
324
-
325
- ITERATION 1:
326
- Think through: What should this initial version include?
327
- [Create version]
328
-
329
- ITERATION 2:
330
- Think through: What weaknesses exist? How to address them?
331
- [Refine version]
332
-
333
- ITERATION 3:
334
- Think through: What can be compressed or improved?
335
- [Further refine]
336
- ```
337
-
338
- ## Advanced Techniques
339
-
340
- ### Aspect-Focused Iteration
341
- Each iteration improves a different dimension:
342
-
343
- ```
344
- ITERATION 1 - ACCURACY: Get the facts right
345
- ITERATION 2 - CLARITY: Make it understandable
346
- ITERATION 3 - CONCISION: Remove waste
347
- ITERATION 4 - IMPACT: Make it compelling
348
- ITERATION 5 - STYLE: Polish the voice
349
- ```
350
-
351
- ### A/B Iteration Branches
352
- Create alternative versions and merge best parts:
353
-
354
- ```
355
- DRAFT: [Initial version]
356
-
357
- BRANCH A - CONCISION:
358
- [Optimize for brevity]
359
-
360
- BRANCH B - COMPLETENESS:
361
- [Optimize for thoroughness]
362
-
363
- MERGE:
364
- Combine the best elements of both branches.
365
- ```
366
-
367
- ### Constrained Progressive Compression
368
- ```
369
- VERSION 1: 1000 words - Complete explanation
370
- VERSION 2: 500 words - 50% compression
371
- VERSION 3: 250 words - 75% compression
372
- VERSION 4: 100 words - 90% compression
373
- VERSION 5: 25 words - 97.5% compression
374
-
375
- Each version must be self-contained and accurate.
376
- ```
377
-
378
- ### Quality Metric Iteration
379
- ```
380
- ITERATION 1:
381
- [Create initial version]
382
- Self-assess (1-10): Clarity ___ Concision ___ Accuracy ___ Impact ___
383
-
384
- ITERATION 2:
385
- [Improve lowest scoring dimension]
386
- Self-assess: Clarity ___ Concision ___ Accuracy ___ Impact ___
387
-
388
- ITERATION 3:
389
- [Improve next lowest dimension]
390
- Self-assess: Clarity ___ Concision ___ Accuracy ___ Impact ___
391
-
392
- Continue until all metrics > 8
393
- ```
394
-
395
- ## Refinement Strategies
396
-
397
- ### What to Remove
398
- - Redundant information
399
- - Obvious statements
400
- - Unnecessary qualifiers ("very", "really")
401
- - Passive voice (often)
402
- - Filler words
403
- - Overly general statements
404
-
405
- ### What to Add
406
- - Specific examples
407
- - Precise numbers
408
- - Concrete details
409
- - Missing context (if essential)
410
- - Clarifying examples
411
-
412
- ### What to Improve
413
- - Vague language → specific language
414
- - Long sentences → shorter, punchier
415
- - Complex words → simpler alternatives (if clearer)
416
- - Weak verbs → strong verbs
417
- - Passive voice → active voice
418
-
419
- ## Quality Indicators
420
-
421
- **Good CoD iterations show:**
422
- - ✅ Measurable improvement each pass
423
- - ✅ Increased information density
424
- - ✅ Reduced redundancy
425
- - ✅ Maintained accuracy
426
- - ✅ Enhanced clarity
427
- - ✅ Clear evolution path
428
-
429
- **Poor CoD iterations show:**
430
- - ❌ No meaningful change
431
- - ❌ Loss of important information
432
- - ❌ Decreased clarity
433
- - ❌ Introduced errors
434
- - ❌ Just made it shorter without improvement
435
-
436
- ## Use Cases
437
-
438
- ### 1. Content Summarization
439
- Taking long documents and progressively distilling essence
440
-
441
- ### 2. Message Refinement
442
- Iteratively improving emails, announcements, communications
443
-
444
- ### 3. Documentation Optimization
445
- Making docs clearer and more concise through iteration
446
-
447
- ### 4. Explanation Enhancement
448
- Progressively improving how concepts are explained
449
-
450
- ### 5. Writing Polish
451
- Iterative editing of creative or professional writing
452
-
453
- ### 6. Presentation Optimization
454
- Refining slides/talks through multiple passes
133
+ Ask the model to reason about which entities are most salient before selecting them. Helps when the source is long and salience is contested.
455
134
 
456
135
  ## Assessment Checklist
457
136
 
458
- When using Chain of Density:
459
- - [ ] Task benefits from iteration
460
- - [ ] Number of iterations specified
461
- - [ ] Optimization goals are clear
462
- - [ ] Each iteration has purpose
463
- - [ ] Constraints defined for each pass
464
- - [ ] Evolution is tracked/shown
465
- - [ ] Quality criteria established
466
- - [ ] Stopping condition exists
467
- - [ ] Final version is actually better
468
- - [ ] Process improved understanding
137
+ - [ ] Is the task actually summarization? (If not, CoD is the wrong framework.)
138
+ - [ ] Is a fixed target length stated?
139
+ - [ ] Is the first summary deliberately sparse and padded?
140
+ - [ ] Are entity criteria included in the prompt?
141
+ - [ ] Does each step preserve all prior entities?
142
+ - [ ] Is the word count roughly constant across steps?
143
+ - [ ] Was the chosen step selected on merit rather than defaulting to the last?
469
144
 
470
145
  ## Tips for Success
471
146
 
472
- 1. **Start Verbose**: First iteration should be complete, even if wordy
473
- 2. **One Focus Per Iteration**: Don't try to improve everything at once
474
- 3. **Show Your Work**: Display each iteration to see progress
475
- 4. **Set Constraints**: Give specific targets (word counts, focus areas)
476
- 5. **Verify Improvement**: Check that each iteration actually improves quality
477
- 6. **Don't Over-Iterate**: 3-5 iterations usually sufficient
478
- 7. **Preserve Accuracy**: Never sacrifice correctness for brevity
479
- 8. **Explain Changes**: Note what improved and why
147
+ 1. **Fixed length, rising density** if length is falling, it is not CoD
148
+ 2. **Start verbose on purpose** the filler is the raw material
149
+ 3. **1-3 entities per step** more causes dropped content
150
+ 4. **Five steps is the paper's number** fewer works; more yields diminishing returns
151
+ 5. **Compare steps before choosing** the densest is rarely the most useful
@@ -4,6 +4,8 @@
4
4
 
5
5
  Chain of Thought is a prompting technique that encourages step-by-step reasoning and makes the thinking process explicit. Instead of jumping to answers, it guides Claude to break down complex problems, show intermediate steps, and verify logic along the way.
6
6
 
7
+ **Research basis:** "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (Wei et al., Google Research, arXiv 2201.11903, NeurIPS 2022). Prompting PaLM 540B with eight chain-of-thought exemplars raised GSM8K accuracy from 17.9% with standard prompting to 56.9%, beating the prior fine-tuned state of the art of 55% (GPT-3 plus a verifier, Cobbe et al. 2021).
8
+
7
9
  ## Core Concept
8
10
 
9
11
  CoT works by:
@@ -319,24 +321,27 @@ STEPS:
319
321
  ### Self-Consistency
320
322
  Generate multiple reasoning paths and choose the most common answer:
321
323
 
324
+ **Research basis:** "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (Wang et al., arXiv 2203.11171, ICLR 2023). Sample a diverse set of reasoning paths rather than greedy-decoding one, then "marginalize out the reasoning paths and aggregate by choosing the most consistent answer in the final answer set." On PaLM 540B / GSM8K, a majority vote over 40 sampled paths reached 74.4% versus 56.5% for greedy CoT decoding (+17.9 points); an unweighted majority vote matched or beat every weighted aggregation the authors tested.
325
+
326
+ The mechanism is **sampling the same prompt repeatedly and taking a majority vote** — not asking the model to try different methods and then judge which it likes best. Self-assessment is a different (and weaker) technique; the model is not a reliable judge of its own reasoning.
327
+
322
328
  ```
323
- Solve this problem using three different approaches:
329
+ Solve this problem. Show your step-by-step reasoning, then state your final
330
+ answer on its own line as:
324
331
 
325
- APPROACH 1:
326
- [Method 1 with step-by-step reasoning]
332
+ FINAL ANSWER: [answer]
327
333
 
328
- APPROACH 2:
329
- [Method 2 with step-by-step reasoning]
334
+ [Problem]
335
+ ```
330
336
 
331
- APPROACH 3:
332
- [Method 3 with step-by-step reasoning]
337
+ Run this **same prompt N times at non-zero temperature** (the paper samples 40; 5-10 is usually enough in practice, and temperature ~0.7 is a reasonable default). Then aggregate outside the model:
333
338
 
334
- COMPARISON:
335
- Which approach is most reliable and why?
339
+ 1. Extract the `FINAL ANSWER:` value from each of the N samples
340
+ 2. Count how often each distinct answer appears
341
+ 3. Return the most frequent answer — a plain, unweighted majority vote
342
+ 4. Treat a near-tie as a signal of genuine uncertainty, not as a reason to ask the model to break the tie
336
343
 
337
- FINAL ANSWER:
338
- Based on the most consistent result.
339
- ```
344
+ Do **not** ask the model to compare its own paths and pick the best one. The diversity of independently sampled reasoning is what produces the gain; collapsing it into a single self-judgment discards exactly the signal the method depends on.
340
345
 
341
346
  ### Uncertainty Quantification
342
347
  ```
@@ -4,6 +4,8 @@
4
4
 
5
5
  CO-STAR is a comprehensive prompting framework that emphasizes context, audience, and communication style. It's particularly effective for content creation, writing tasks, and scenarios where tone and audience considerations significantly impact output quality.
6
6
 
7
+ **Origin:** Developed by the Data Science & AI team at GovTech (Government Technology Agency of Singapore), and popularized by Sheila Teo, who won GovTech's "Prompt Royale" — Singapore's first GPT-4 prompt engineering competition, 400+ participants, finale 8 November 2023. Teo credits the framework to GovTech rather than to herself: "The CO-STAR framework, a brainchild of GovTech Singapore's Data Science & AI team, is a handy template for structuring prompts." Practitioner framework — no controlled evaluation of CO-STAR has been published.
8
+
7
9
  ## Components
8
10
 
9
11
  ### C - Context
@@ -4,6 +4,8 @@
4
4
 
5
5
  CTF is a simple 3-component framework focused on situational context, task clarity, and output format. It is ideal for tasks where background/situation is more important than AI persona framing, and the primary concern is getting a clearly defined output from a well-understood context.
6
6
 
7
+ **Origin:** No single documented originator. CTF appears in the community framework literature as a variant of RTF, with Context substituted for Role, documented alongside it in unattributed practitioner listings from 2024 onward (e.g. fvivas.com, 15 March 2024). No academic treatment; absent from The Prompt Report (arXiv 2406.06608).
8
+
7
9
  ## Components
8
10
 
9
11
  ### C - Context
@@ -168,7 +170,7 @@ Markdown template with:
168
170
  - ❌ Audience and tone are critical (use CO-STAR)
169
171
  - ❌ Multi-step process with methodology needed (use RISEN)
170
172
  - ❌ Explicit dos/don'ts required (use TIDD-EC)
171
- - ❌ Input→output transformation (use RISE-IE)
173
+ - ❌ Input→output transformation (use RISE — the Role/Input/Steps/Expectation form)
172
174
 
173
175
  ## CTF vs RTF
174
176