@ckelsoe/prompt-architect 3.3.1 → 3.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +19 -0
- package/adapters/system-prompt.md +77 -51
- package/package.json +2 -2
- package/scripts/test.js +26 -0
- package/skills/prompt-architect/SKILL.md +49 -36
- package/skills/prompt-architect/assets/templates/ape_template.txt +14 -3
- package/skills/prompt-architect/assets/templates/bab_template.txt +23 -10
- package/skills/prompt-architect/assets/templates/broke_template.txt +31 -6
- package/skills/prompt-architect/assets/templates/cai-critique-revise_template.txt +16 -8
- package/skills/prompt-architect/assets/templates/care_template.txt +4 -0
- package/skills/prompt-architect/assets/templates/chain-of-density_template.txt +30 -31
- package/skills/prompt-architect/assets/templates/chain-of-thought_template.txt +8 -0
- package/skills/prompt-architect/assets/templates/co-star_template.txt +8 -0
- package/skills/prompt-architect/assets/templates/crispe_template.txt +6 -0
- package/skills/prompt-architect/assets/templates/ctf_template.txt +8 -0
- package/skills/prompt-architect/assets/templates/devils-advocate_template.txt +7 -0
- package/skills/prompt-architect/assets/templates/hybrid_template.txt +44 -24
- package/skills/prompt-architect/assets/templates/iterative-compression_template.txt +43 -28
- package/skills/prompt-architect/assets/templates/least-to-most_template.txt +4 -0
- package/skills/prompt-architect/assets/templates/plan-and-solve_template.txt +4 -0
- package/skills/prompt-architect/assets/templates/pre-mortem_template.txt +24 -14
- package/skills/prompt-architect/assets/templates/race_template.txt +9 -0
- package/skills/prompt-architect/assets/templates/rcot_template.txt +8 -0
- package/skills/prompt-architect/assets/templates/react_template.txt +38 -16
- package/skills/prompt-architect/assets/templates/rise-ie_template.txt +4 -0
- package/skills/prompt-architect/assets/templates/rise-ix_template.txt +3 -3
- package/skills/prompt-architect/assets/templates/risen_template.txt +7 -0
- package/skills/prompt-architect/assets/templates/rpef_template.txt +16 -11
- package/skills/prompt-architect/assets/templates/rtf_template.txt +9 -0
- package/skills/prompt-architect/assets/templates/skeleton-of-thought_template.txt +18 -10
- package/skills/prompt-architect/assets/templates/step-back_template.txt +4 -0
- package/skills/prompt-architect/assets/templates/tidd-ec_template.txt +54 -31
- package/skills/prompt-architect/assets/templates/tree-of-thought_template.txt +37 -21
- package/skills/prompt-architect/references/frameworks/ape.md +143 -57
- package/skills/prompt-architect/references/frameworks/bab.md +137 -58
- package/skills/prompt-architect/references/frameworks/broke.md +157 -57
- package/skills/prompt-architect/references/frameworks/cai-critique-revise.md +183 -75
- package/skills/prompt-architect/references/frameworks/care.md +67 -36
- package/skills/prompt-architect/references/frameworks/chain-of-density.md +49 -27
- package/skills/prompt-architect/references/frameworks/chain-of-thought.md +28 -7
- package/skills/prompt-architect/references/frameworks/co-star.md +100 -44
- package/skills/prompt-architect/references/frameworks/crispe.md +7 -5
- package/skills/prompt-architect/references/frameworks/ctf.md +73 -37
- package/skills/prompt-architect/references/frameworks/devils-advocate.md +105 -58
- package/skills/prompt-architect/references/frameworks/iterative-compression.md +236 -269
- package/skills/prompt-architect/references/frameworks/least-to-most.md +35 -13
- package/skills/prompt-architect/references/frameworks/plan-and-solve.md +38 -9
- package/skills/prompt-architect/references/frameworks/pre-mortem.md +176 -66
- package/skills/prompt-architect/references/frameworks/race.md +91 -38
- package/skills/prompt-architect/references/frameworks/rcot.md +1 -1
- package/skills/prompt-architect/references/frameworks/react.md +230 -97
- package/skills/prompt-architect/references/frameworks/reverse-role.md +22 -15
- package/skills/prompt-architect/references/frameworks/rise.md +94 -70
- package/skills/prompt-architect/references/frameworks/risen.md +80 -28
- package/skills/prompt-architect/references/frameworks/rpef.md +227 -58
- package/skills/prompt-architect/references/frameworks/rtf.md +2 -2
- package/skills/prompt-architect/references/frameworks/self-refine.md +4 -2
- package/skills/prompt-architect/references/frameworks/skeleton-of-thought.md +67 -21
- package/skills/prompt-architect/references/frameworks/step-back.md +47 -15
- package/skills/prompt-architect/references/frameworks/tidd-ec.md +110 -45
- package/skills/prompt-architect/references/frameworks/tree-of-thought.md +112 -37
|
@@ -19,76 +19,133 @@ Reverse Prompt Engineering (RPEF) inverts the normal prompt-to-output direction.
|
|
|
19
19
|
### Input Sample (optional but powerful)
|
|
20
20
|
**Purpose:** If the output was generated from specific input data, include it. The model uses the relationship between input and output to infer transformation rules and prompt logic.
|
|
21
21
|
|
|
22
|
+
**Template slot:** `SOURCE MATERIAL — ORIGINAL INPUT`. It is the only optional block in the template, and it is deleted together with the sentence directly beneath it when the recovery is output-only.
|
|
23
|
+
|
|
22
24
|
### Output Sample (required)
|
|
23
25
|
**Purpose:** The existing output you want to reverse-engineer. This is the primary source of truth — what you want to be able to reproduce.
|
|
24
26
|
|
|
27
|
+
**Template slot:** `SOURCE MATERIAL — OUTPUT SAMPLE`. Paste it verbatim; paraphrasing the sample destroys exactly the formatting and phrasing signals the analysis is meant to read.
|
|
28
|
+
|
|
25
29
|
### Analysis Instruction
|
|
26
|
-
**Purpose:** Directs the model to analyze the output for its implicit prompt properties
|
|
30
|
+
**Purpose:** Directs the model to analyze the output for its implicit prompt properties, and to *report* that analysis before writing anything else.
|
|
31
|
+
|
|
32
|
+
**Template slot:** `ANALYSIS INSTRUCTIONS`, which names six dimensions: implied role or persona, tone and voice, structural and formatting patterns, level of detail and scope, implicit constraints (what it did NOT do), and reasoning style. Structure and format are one merged dimension, not two. An earlier revision of *this document* listed seven by splitting them apart; that was drift in the doc's copy, not a change to the template, which has carried the merged dimension unchanged since it was introduced.
|
|
33
|
+
|
|
34
|
+
### Confidence Threshold
|
|
35
|
+
**Purpose:** Stops the model from presenting a guess as a recovered prompt when one sample is too thin to support one.
|
|
27
36
|
|
|
28
|
-
|
|
29
|
-
**Purpose:** Instructs the model to only generate the recovered prompt when confident enough. Default: 80-90%.
|
|
37
|
+
**Template slot:** the closing sentences of `RECOVERED PROMPT`. The threshold is only useful when it is paired with a defined alternative action — see the note under Template Structure on why the earlier "only proceed when 80% confident" phrasing did nothing.
|
|
30
38
|
|
|
31
39
|
### Generalization Directive
|
|
32
40
|
**Purpose:** Specifies whether the recovered prompt should be a specific single-use template or a generalized meta-prompt with fill-in placeholders.
|
|
33
41
|
|
|
42
|
+
**Template slot:** folded into `RECOVERED PROMPT` as the `[PLACEHOLDER]` instruction — the template always asks for the generalized form, and marks variability at the token level rather than asking a separate yes/no question about it.
|
|
43
|
+
|
|
34
44
|
## Template Structure
|
|
35
45
|
|
|
46
|
+
Section headers are stripped at emission, so every slot's meaning is carried by the
|
|
47
|
+
unbracketed prose around it rather than by the header above it. This template also depends
|
|
48
|
+
on sequencing — the analysis must be reported *before* the recovered prompt is written —
|
|
49
|
+
and that sequencing lives in the prose ("First, examine…", "After completing that
|
|
50
|
+
analysis…"), not in the order the headers happen to appear. Do not reintroduce
|
|
51
|
+
header-only structure.
|
|
52
|
+
|
|
36
53
|
```
|
|
37
54
|
You are an expert Prompt Engineer performing reverse prompt engineering.
|
|
38
55
|
|
|
39
56
|
TASK:
|
|
40
|
-
Analyze the
|
|
41
|
-
consistently produce this type of output.
|
|
42
|
-
reusable — replace specific details with clearly marked [PLACEHOLDER] variables.
|
|
57
|
+
Analyze the sample material below and reconstruct the prompt that would
|
|
58
|
+
consistently produce this type of output.
|
|
43
59
|
|
|
44
|
-
|
|
45
|
-
[
|
|
60
|
+
SOURCE MATERIAL — ORIGINAL INPUT:
|
|
61
|
+
[Optional — paste here, verbatim, the input that originally produced the output
|
|
62
|
+
sample below. If you have nothing to paste, delete this bracketed block along with
|
|
63
|
+
the sentence directly below it.]
|
|
64
|
+
Use the material above as the input that produced the output sample below.
|
|
46
65
|
|
|
47
|
-
OUTPUT SAMPLE:
|
|
48
|
-
[Paste the output you want to reverse-engineer]
|
|
66
|
+
SOURCE MATERIAL — OUTPUT SAMPLE:
|
|
67
|
+
[Paste here, verbatim, the output you want to reverse-engineer.]
|
|
68
|
+
Use the material above as the finished output to reverse-engineer.
|
|
49
69
|
|
|
50
70
|
ANALYSIS INSTRUCTIONS:
|
|
51
|
-
|
|
52
|
-
- Implied role or persona
|
|
53
|
-
- Tone and voice
|
|
54
|
-
- Structural
|
|
71
|
+
First, examine the output above and report what you find for each of these dimensions:
|
|
72
|
+
- Implied role or persona
|
|
73
|
+
- Tone and voice
|
|
74
|
+
- Structural and formatting patterns
|
|
55
75
|
- Level of detail and scope
|
|
56
|
-
-
|
|
57
|
-
-
|
|
58
|
-
- Reasoning style (step-by-step, direct, narrative?)
|
|
76
|
+
- Implicit constraints (what it did NOT do)
|
|
77
|
+
- Reasoning style
|
|
59
78
|
|
|
60
79
|
RECOVERED PROMPT:
|
|
61
|
-
|
|
62
|
-
type of output.
|
|
63
|
-
|
|
80
|
+
After completing that analysis, write out a reusable prompt template that reliably
|
|
81
|
+
reproduces this type of output. Mark every variable element with the literal token
|
|
82
|
+
[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
|
|
83
|
+
If you are less than 80% confident in the reconstruction, do not guess: say so, and
|
|
84
|
+
name the additional sample material that would resolve the uncertainty.
|
|
64
85
|
```
|
|
65
86
|
|
|
87
|
+
Two sentences in `RECOVERED PROMPT` do work that is easy to lose when editing this
|
|
88
|
+
template, and both replaced phrasings that failed in a specific way:
|
|
89
|
+
|
|
90
|
+
- **"written exactly that way rather than replaced with a specific value."** The earlier
|
|
91
|
+
phrasing, "Use [PLACEHOLDER] for variable elements," reads two ways: emit the literal
|
|
92
|
+
token, or fill each position with a plausible concrete value. The second reading returns
|
|
93
|
+
a filled-in example rather than the reusable template that is the entire point of the
|
|
94
|
+
framework, so the instruction is made explicit to close it off. How often the ambiguity
|
|
95
|
+
actually resolves the wrong way has not been measured here.
|
|
96
|
+
- **"If you are less than 80% confident … say so, and name the additional sample
|
|
97
|
+
material."** The earlier phrasing, "Only proceed when at least 80% confident," named no
|
|
98
|
+
alternative action, so it gated nothing: no behavior was defined for the failing branch,
|
|
99
|
+
leaving nothing in the text to stop the model proceeding anyway. A threshold is only a
|
|
100
|
+
gate when the below-threshold
|
|
101
|
+
path is spelled out, and naming the missing material makes that path useful rather than
|
|
102
|
+
merely a refusal.
|
|
103
|
+
|
|
104
|
+
Because RPEF's source material is the user's own pasted sample, the two `SOURCE MATERIAL`
|
|
105
|
+
blocks are the framework's legitimate `[...]` placeholders under SKILL.md step 5 — they are
|
|
106
|
+
material only the user can supply, not defaults the model should invent.
|
|
107
|
+
|
|
66
108
|
## Complete Examples
|
|
67
109
|
|
|
110
|
+
Every example below is shown in emitted form: each slot carries its own role in prose, so
|
|
111
|
+
nothing is lost when the headers are deleted. The output samples are filled with real
|
|
112
|
+
material rather than left as paste instructions, because an RPEF prompt with an empty
|
|
113
|
+
sample demonstrates nothing — the sample is the only thing the framework operates on.
|
|
114
|
+
|
|
68
115
|
### Example 1: Recovering a Writing Style
|
|
69
116
|
|
|
70
|
-
**Scenario:** You got a great product description once and want to reproduce that tone.
|
|
117
|
+
**Scenario:** You got a great product description once and want to reproduce that tone. Output-only, so the `ORIGINAL INPUT` block and the sentence below it are both deleted.
|
|
71
118
|
|
|
72
119
|
```
|
|
73
120
|
You are an expert Prompt Engineer performing reverse prompt engineering.
|
|
74
121
|
|
|
75
122
|
TASK:
|
|
76
|
-
Analyze the
|
|
77
|
-
|
|
123
|
+
Analyze the sample material below and reconstruct the prompt that would
|
|
124
|
+
consistently produce this type of output.
|
|
78
125
|
|
|
79
|
-
OUTPUT SAMPLE:
|
|
80
|
-
"Mira is the meeting companion that thinks ahead. While you're present
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
126
|
+
SOURCE MATERIAL — OUTPUT SAMPLE:
|
|
127
|
+
"Mira is the meeting companion that thinks ahead. While you're present in the
|
|
128
|
+
conversation, Mira is quietly capturing every decision, action item, and insight
|
|
129
|
+
— then turning them into a clean summary the moment you close your laptop. No
|
|
130
|
+
more frantic note-taking. No more 'wait, what did we decide?' Just clarity,
|
|
131
|
+
automatically."
|
|
132
|
+
Use the material above as the finished output to reverse-engineer.
|
|
85
133
|
|
|
86
134
|
ANALYSIS INSTRUCTIONS:
|
|
87
|
-
|
|
88
|
-
|
|
135
|
+
First, examine the output above and report what you find for each of these dimensions:
|
|
136
|
+
- Implied role or persona
|
|
137
|
+
- Tone and voice
|
|
138
|
+
- Structural and formatting patterns
|
|
139
|
+
- Level of detail and scope
|
|
140
|
+
- Implicit constraints (what it did NOT do)
|
|
141
|
+
- Reasoning style
|
|
89
142
|
|
|
90
143
|
RECOVERED PROMPT:
|
|
91
|
-
|
|
144
|
+
After completing that analysis, write out a reusable prompt template that reliably
|
|
145
|
+
reproduces this type of output. Mark every variable element with the literal token
|
|
146
|
+
[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
|
|
147
|
+
If you are less than 80% confident in the reconstruction, do not guess: say so, and
|
|
148
|
+
name the additional sample material that would resolve the uncertainty.
|
|
92
149
|
```
|
|
93
150
|
|
|
94
151
|
**Recovered prompt output:**
|
|
@@ -110,48 +167,159 @@ STYLE RULES:
|
|
|
110
167
|
|
|
111
168
|
### Example 2: Recovering an Analysis Format
|
|
112
169
|
|
|
113
|
-
**Scenario:** A colleague produced a competitive analysis you want to replicate.
|
|
170
|
+
**Scenario:** A colleague produced a competitive analysis you want to replicate. You have both the brief they worked from and the analysis itself, so both `SOURCE MATERIAL` blocks are used.
|
|
114
171
|
|
|
115
172
|
```
|
|
116
173
|
You are an expert Prompt Engineer performing reverse prompt engineering.
|
|
117
174
|
|
|
118
175
|
TASK:
|
|
119
|
-
Analyze the
|
|
120
|
-
|
|
176
|
+
Analyze the sample material below and reconstruct the prompt that would
|
|
177
|
+
consistently produce this type of output.
|
|
121
178
|
|
|
122
|
-
INPUT
|
|
179
|
+
SOURCE MATERIAL — ORIGINAL INPUT:
|
|
123
180
|
Companies to compare: Notion vs. Confluence vs. Linear
|
|
124
181
|
Use case: Engineering team wiki and documentation
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
182
|
+
Audience: VP Engineering, 200-person org
|
|
183
|
+
Use the material above as the input that produced the output sample below.
|
|
184
|
+
|
|
185
|
+
SOURCE MATERIAL — OUTPUT SAMPLE:
|
|
186
|
+
"**Structure and hierarchy.** Confluence wins outright here. Spaces, page trees,
|
|
187
|
+
and page-level restrictions map cleanly onto how engineering orgs already think
|
|
188
|
+
about ownership. Notion's database-plus-nesting model is more flexible but
|
|
189
|
+
degrades past roughly 500 pages without deliberate curation. Linear's documents
|
|
190
|
+
are deliberately shallow — they attach to projects rather than organizing
|
|
191
|
+
independently — so it is not a wiki candidate at all on this dimension.
|
|
192
|
+
|
|
193
|
+
**Search.** Confluence's search is the weakest of the three despite being the
|
|
194
|
+
oldest: it indexes attachments well but ranks stale pages highly. Notion's search
|
|
195
|
+
handles partial-title matching acceptably since its 2023 rebuild. Linear's search
|
|
196
|
+
is the fastest and best-scoped, but there is comparatively little to search.
|
|
197
|
+
|
|
198
|
+
**Permissioning.** Confluence again, and the gap widens with headcount. Notion's
|
|
199
|
+
permissions are workspace- and page-level but become expensive to audit past
|
|
200
|
+
about 100 seats. Linear does not attempt document-level permissioning.
|
|
201
|
+
|
|
202
|
+
**Recommendation.** For an engineering wiki proper: Confluence above roughly 150
|
|
203
|
+
engineers, where permissioning and hierarchy dominate; Notion below that, where
|
|
204
|
+
flexibility outweighs structure. Linear is complementary rather than a candidate
|
|
205
|
+
— pair it with whichever wiki you choose.
|
|
206
|
+
|
|
207
|
+
*Caveat: the page-count and seat-count thresholds above come from practitioner
|
|
208
|
+
reports, not vendor benchmarks.*"
|
|
209
|
+
Use the material above as the finished output to reverse-engineer.
|
|
128
210
|
|
|
129
211
|
ANALYSIS INSTRUCTIONS:
|
|
130
|
-
|
|
131
|
-
|
|
212
|
+
First, examine the output above and report what you find for each of these dimensions:
|
|
213
|
+
- Implied role or persona
|
|
214
|
+
- Tone and voice
|
|
215
|
+
- Structural and formatting patterns
|
|
216
|
+
- Level of detail and scope
|
|
217
|
+
- Implicit constraints (what it did NOT do)
|
|
218
|
+
- Reasoning style
|
|
132
219
|
|
|
133
220
|
RECOVERED PROMPT:
|
|
134
|
-
|
|
221
|
+
After completing that analysis, write out a reusable prompt template that reliably
|
|
222
|
+
reproduces this type of output. Mark every variable element with the literal token
|
|
223
|
+
[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
|
|
224
|
+
If you are less than 80% confident in the reconstruction, do not guess: say so, and
|
|
225
|
+
name the additional sample material that would resolve the uncertainty.
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
**Recovered prompt output:**
|
|
229
|
+
```
|
|
230
|
+
ROLE: You are a senior technical evaluator who compares tools for engineering
|
|
231
|
+
leadership audiences.
|
|
232
|
+
|
|
233
|
+
TASK: Compare [TOOL A], [TOOL B], and [TOOL C] for [USE CASE], writing for
|
|
234
|
+
[AUDIENCE AND ORG SIZE].
|
|
235
|
+
|
|
236
|
+
METHOD:
|
|
237
|
+
- Choose 3-4 dimensions that actually matter for the stated use case. Use the same
|
|
238
|
+
dimensions for every tool, as bold headings.
|
|
239
|
+
- Under each dimension, name the winner in the first sentence, then justify it.
|
|
240
|
+
- Where a tool is not a genuine candidate on a dimension, say so plainly instead of
|
|
241
|
+
scoring it — do not manufacture a three-way comparison that isn't there.
|
|
242
|
+
- Close with a recommendation conditional on a named threshold (org size, seat count,
|
|
243
|
+
volume) rather than a single unqualified pick.
|
|
244
|
+
- Mark any figure sourced from practitioner reports rather than vendor benchmarks
|
|
245
|
+
with an explicit caveat line at the end.
|
|
246
|
+
|
|
247
|
+
TONE: Direct and evaluative — verdict first, reasoning second. No feature-matrix
|
|
248
|
+
tables, no vendor marketing language, no hedging on the recommendation.
|
|
135
249
|
```
|
|
136
250
|
|
|
137
|
-
### Example 3: Output-Only Recovery
|
|
251
|
+
### Example 3: Output-Only Recovery, Where the Confidence Gate Fires
|
|
138
252
|
|
|
139
|
-
**Scenario:** You have
|
|
253
|
+
**Scenario:** You have an email you're happy with but lost the prompt. One sample is often too thin to generalize from — this example shows what the template's confidence rule is for.
|
|
140
254
|
|
|
141
255
|
```
|
|
142
|
-
You are an expert Prompt Engineer.
|
|
256
|
+
You are an expert Prompt Engineer performing reverse prompt engineering.
|
|
143
257
|
|
|
144
258
|
TASK:
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
259
|
+
Analyze the sample material below and reconstruct the prompt that would
|
|
260
|
+
consistently produce this type of output.
|
|
261
|
+
|
|
262
|
+
SOURCE MATERIAL — OUTPUT SAMPLE:
|
|
263
|
+
"Hi Dana,
|
|
264
|
+
|
|
265
|
+
Quick update on the Q3 rollout: we're on track for the Sept 15 launch, but I want
|
|
266
|
+
to flag something early rather than at the last minute. The data migration is
|
|
267
|
+
running about 30% longer than we scoped, which eats most of our buffer. That
|
|
268
|
+
doesn't put the date at risk yet — it means we've lost the room to absorb a
|
|
269
|
+
second surprise.
|
|
270
|
+
|
|
271
|
+
Two options. We hold the date and move the reporting dashboard to a fast-follow
|
|
272
|
+
in early October, or we hold scope and move the launch to Sept 29. I'd recommend
|
|
273
|
+
the first: the dashboard is the piece your team flagged as least useful in month
|
|
274
|
+
one.
|
|
275
|
+
|
|
276
|
+
Happy to talk it through Thursday if that's easier. Either way you'll have a firm
|
|
277
|
+
recommendation from me by Friday.
|
|
148
278
|
|
|
149
|
-
|
|
150
|
-
|
|
279
|
+
Best,
|
|
280
|
+
Ray"
|
|
281
|
+
Use the material above as the finished output to reverse-engineer.
|
|
282
|
+
|
|
283
|
+
ANALYSIS INSTRUCTIONS:
|
|
284
|
+
First, examine the output above and report what you find for each of these dimensions:
|
|
285
|
+
- Implied role or persona
|
|
286
|
+
- Tone and voice
|
|
287
|
+
- Structural and formatting patterns
|
|
288
|
+
- Level of detail and scope
|
|
289
|
+
- Implicit constraints (what it did NOT do)
|
|
290
|
+
- Reasoning style
|
|
151
291
|
|
|
152
292
|
RECOVERED PROMPT:
|
|
153
|
-
|
|
154
|
-
|
|
293
|
+
After completing that analysis, write out a reusable prompt template that reliably
|
|
294
|
+
reproduces this type of output. Mark every variable element with the literal token
|
|
295
|
+
[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
|
|
296
|
+
If you are less than 80% confident in the reconstruction, do not guess: say so, and
|
|
297
|
+
name the additional sample material that would resolve the uncertainty.
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
**Recovered prompt output** — here the gate fires, which is the correct behavior rather than a failure:
|
|
301
|
+
```
|
|
302
|
+
Confidence in a reusable reconstruction: roughly 60%. That is below the 80% bar you
|
|
303
|
+
set, so I am not going to hand you a template and let it look more reliable than it is.
|
|
304
|
+
|
|
305
|
+
What the sample supports confidently: the structure is stable and readable — status
|
|
306
|
+
line, early flag, explicit statement of what the impact is and is not, exactly two
|
|
307
|
+
named options, a recommendation justified by the recipient's own stated priority, an
|
|
308
|
+
offer to discuss, and a committed date for the firm answer. The register is candid and
|
|
309
|
+
non-defensive; the bad news arrives in the second sentence rather than buried below a
|
|
310
|
+
paragraph of reassurance.
|
|
311
|
+
|
|
312
|
+
What one sample cannot settle, and why each matters:
|
|
313
|
+
- Whether "exactly two options" is a rule or an artifact of this situation. A template
|
|
314
|
+
that hardcodes two will force false binaries where the honest answer is one or three.
|
|
315
|
+
- Whether the recommendation systematically favors protecting the date, or whether that
|
|
316
|
+
was specific to this trade-off.
|
|
317
|
+
- Whether this register holds for an external client, or only for an internal
|
|
318
|
+
stakeholder like Dana with whom the writer has standing rapport.
|
|
319
|
+
|
|
320
|
+
Send two or three more emails you were happy with — ideally one carrying
|
|
321
|
+
straightforwardly good news, and one written to a client rather than a colleague — and
|
|
322
|
+
I can separate the fixed structure from this instance's specifics.
|
|
155
323
|
```
|
|
156
324
|
|
|
157
325
|
## Best Use Cases
|
|
@@ -187,9 +355,10 @@ as placeholder variables. Capture the tone, structure, and approach precisely.
|
|
|
187
355
|
|
|
188
356
|
## Quick Reference
|
|
189
357
|
|
|
190
|
-
| Component | Purpose |
|
|
191
|
-
|
|
192
|
-
| Input Sample | The input data (if any) that produced the output |
|
|
193
|
-
| Output Sample | The output to reverse-engineer (required) |
|
|
194
|
-
| Analysis
|
|
195
|
-
|
|
|
358
|
+
| Component | Template slot | Purpose |
|
|
359
|
+
|-----------|---------------|---------|
|
|
360
|
+
| Input Sample | `SOURCE MATERIAL — ORIGINAL INPUT` | The input data (if any) that produced the output — optional, deleted with its trailing sentence when unused |
|
|
361
|
+
| Output Sample | `SOURCE MATERIAL — OUTPUT SAMPLE` | The output to reverse-engineer, pasted verbatim (required) |
|
|
362
|
+
| Analysis Instruction | `ANALYSIS INSTRUCTIONS` | The six dimensions to examine, reported before anything else is written |
|
|
363
|
+
| Generalization Directive | `RECOVERED PROMPT` | Reusable template marking variables with the literal token `[PLACEHOLDER]` |
|
|
364
|
+
| Confidence Threshold | `RECOVERED PROMPT` (closing) | Below 80%, say so and name the sample material that would resolve it |
|
|
@@ -156,7 +156,7 @@ FORMAT: Provide as:
|
|
|
156
156
|
- ❌ Rich context needed (use CO-STAR)
|
|
157
157
|
- ❌ Audience/tone matters (use CO-STAR)
|
|
158
158
|
- ❌ Specific methodology required (use RISEN)
|
|
159
|
-
- ❌ Input transformation needed (use RISE)
|
|
159
|
+
- ❌ Input transformation needed (use RISE-IE)
|
|
160
160
|
|
|
161
161
|
## Common Mistakes
|
|
162
162
|
|
|
@@ -221,7 +221,7 @@ FORMAT: [Follow this structure]
|
|
|
221
221
|
|
|
222
222
|
## When to Upgrade from RTF
|
|
223
223
|
|
|
224
|
-
**Upgrade to RISE if:**
|
|
224
|
+
**Upgrade to RISE-IE if:**
|
|
225
225
|
- Need to specify input characteristics
|
|
226
226
|
- Processing steps matter
|
|
227
227
|
- Input → output transformation
|
|
@@ -4,7 +4,9 @@
|
|
|
4
4
|
|
|
5
5
|
Self-Refine is an iterative output improvement framework where a single model acts as its own generator, critic, and refiner — with no external training, no additional models, and no labeled data required. The core loop: generate an initial output → produce specific, actionable feedback → refine using that feedback → repeat until a stopping criterion is met.
|
|
6
6
|
|
|
7
|
-
**Research basis:** "Self-Refine: Iterative Refinement with Self-Feedback" (Madaan et al., arXiv 2303.17651, NeurIPS 2023).
|
|
7
|
+
**Research basis:** "Self-Refine: Iterative Refinement with Self-Feedback" (Madaan et al., arXiv 2303.17651, NeurIPS 2023). Evaluated across 7 diverse tasks using GPT-3.5, ChatGPT, and GPT-4. The abstract reports that Self-Refine outputs are "preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance." That ~20% is an *average across the 7 tasks* — per-task gains vary substantially, and the abstract gives no range. Project site: selfrefine.info.
|
|
8
|
+
|
|
9
|
+
*Correction:* an earlier version of this file reported "improvement of 5-40% over single-pass generation from GPT-3.5 and GPT-4." Neither bound appears in the paper, and ChatGPT was omitted from the model list. Verified against the arXiv abstract.
|
|
8
10
|
|
|
9
11
|
## The Core Loop
|
|
10
12
|
|
|
@@ -190,7 +192,7 @@ biggest single point of failure identified.
|
|
|
190
192
|
|
|
191
193
|
| | Self-Refine | CAI Critique-Revise | BAB | Chain of Density |
|
|
192
194
|
|---|---|---|---|---|
|
|
193
|
-
| Direction | Any output → improvement | Output vs. explicit principle | Before state → after state |
|
|
195
|
+
| Direction | Any output → improvement | Output vs. explicit principle | Before state → after state | Same length → higher entity density |
|
|
194
196
|
| Critique type | Multi-dimensional quality | Principle-aligned alignment | Transformation rules | Density/information |
|
|
195
197
|
| Iterations | Configurable loop | One cycle (or more) | Single transform | N passes |
|
|
196
198
|
| Best for | General quality improvement | Alignment/compliance | Rewriting existing content | Summarization |
|
|
@@ -34,22 +34,48 @@ Point X: [point name] — [one-line description]
|
|
|
34
34
|
|
|
35
35
|
## Template Structure
|
|
36
36
|
|
|
37
|
+
Section headers are stripped at emission, so every header's meaning is carried by
|
|
38
|
+
unbracketed prose that ships with the prompt. Do not reintroduce header-only structure.
|
|
39
|
+
|
|
37
40
|
```
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
41
|
+
SOURCE MATERIAL:
|
|
42
|
+
[Optional — paste the document, data, or notes the answer must be drawn from here. If you
|
|
43
|
+
have nothing to paste, delete this bracketed block along with the sentence directly below
|
|
44
|
+
it.]
|
|
45
|
+
Use the material above as the factual basis for the work described below.
|
|
42
46
|
|
|
43
|
-
|
|
44
|
-
|
|
47
|
+
Answer the topic or question below in two phases: first a skeleton outline, then a full
|
|
48
|
+
expansion of every point in that skeleton.
|
|
45
49
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
50
|
+
TOPIC / QUESTION:
|
|
51
|
+
[Write out the full topic or question as a complete, self-contained sentence rather than a
|
|
52
|
+
bare noun phrase, including any audience, scope, or depth of coverage specified]
|
|
49
53
|
|
|
50
|
-
|
|
54
|
+
PHASE 1 — SKELETON:
|
|
55
|
+
First, outline the topic above. List only the key points, one per line, in this format:
|
|
56
|
+
N. [Point name] | [One-sentence description of what this point covers]
|
|
57
|
+
Replace N with the point's number and fill in both bracketed parts, keeping that line's
|
|
58
|
+
format exactly. Produce the skeleton and nothing else — do not expand any point yet.
|
|
59
|
+
|
|
60
|
+
PHASE 2 — EXPAND:
|
|
61
|
+
Once the skeleton is complete, expand each of its points into [2-4 sentences / a paragraph /
|
|
62
|
+
detailed coverage]. Restate a point's name before expanding it, and make each expansion
|
|
63
|
+
self-contained.
|
|
51
64
|
```
|
|
52
65
|
|
|
66
|
+
The two-phase framing sentence sits before `TOPIC / QUESTION` so that both phases are
|
|
67
|
+
announced before the topic appears; the topic in turn sits before both phases so that
|
|
68
|
+
Phase 1's "the topic above" resolves once the headers are gone.
|
|
69
|
+
|
|
70
|
+
**Parallel expansion is not part of the emitted prompt.** The template runs both phases in
|
|
71
|
+
a single self-contained prompt. Splitting Phase 2 into one request per skeleton point — the
|
|
72
|
+
source of the paper's latency gain — is a workflow the sender builds around the model, not
|
|
73
|
+
an instruction the model can act on. A line such as "[Optional: expand points in parallel by
|
|
74
|
+
sending each as a separate request]" addresses the human holding the prompt, and a
|
|
75
|
+
header-stripped prompt has nowhere to put a note to the sender: the model would read it as
|
|
76
|
+
an instruction it cannot execute. If that note is ever added for the sender's benefit, it
|
|
77
|
+
must be deleted before the prompt is sent.
|
|
78
|
+
|
|
53
79
|
### Minimal Single-Pass SoT
|
|
54
80
|
For use in a single prompt when parallel API calls aren't available:
|
|
55
81
|
```
|
|
@@ -67,14 +93,23 @@ Present as: Skeleton → then Full Expansion.
|
|
|
67
93
|
|
|
68
94
|
**After SoT:**
|
|
69
95
|
```
|
|
96
|
+
Answer the topic or question below in two phases: first a skeleton outline, then a full
|
|
97
|
+
expansion of every point in that skeleton.
|
|
98
|
+
|
|
99
|
+
TOPIC / QUESTION:
|
|
100
|
+
Explain microservices architecture to a senior engineer who is evaluating whether to
|
|
101
|
+
migrate an existing monolith, covering the trade-offs that bear on that decision.
|
|
102
|
+
|
|
70
103
|
PHASE 1 — SKELETON:
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
104
|
+
First, outline the topic above. List 5-7 key points, one per line, in this format:
|
|
105
|
+
N. [Point name] | [One-sentence description of what this point covers]
|
|
106
|
+
Replace N with the point's number and fill in both bracketed parts, keeping that line's
|
|
107
|
+
format exactly. Produce the skeleton and nothing else — do not expand any point yet.
|
|
74
108
|
|
|
75
109
|
PHASE 2 — EXPAND:
|
|
76
|
-
|
|
77
|
-
|
|
110
|
+
Once the skeleton is complete, expand each of its points into 2-3 paragraphs. Restate a
|
|
111
|
+
point's name before expanding it, and make each expansion self-contained. Include the
|
|
112
|
+
specific trade-offs that bear on a migration decision.
|
|
78
113
|
```
|
|
79
114
|
|
|
80
115
|
**Skeleton output (Phase 1):**
|
|
@@ -104,15 +139,26 @@ risk of scope creep, integration failures, and a multi-month feature freeze...
|
|
|
104
139
|
|
|
105
140
|
**After SoT:**
|
|
106
141
|
```
|
|
142
|
+
Answer the topic or question below in two phases: first a skeleton outline, then a full
|
|
143
|
+
expansion of every point in that skeleton.
|
|
144
|
+
|
|
145
|
+
TOPIC / QUESTION:
|
|
146
|
+
Write a competitive analysis comparing our product against [you fill this in: name of the
|
|
147
|
+
first competitor] and [you fill this in: name of the second competitor], for an internal
|
|
148
|
+
audience deciding where to invest next.
|
|
149
|
+
|
|
107
150
|
PHASE 1 — SKELETON:
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
N. [
|
|
151
|
+
First, outline the topic above. List 6-8 comparison dimensions, one per line, in this
|
|
152
|
+
format:
|
|
153
|
+
N. [Dimension name] | [One-sentence description of what to compare on that dimension]
|
|
154
|
+
Replace N with the dimension's number and fill in both bracketed parts, keeping that line's
|
|
155
|
+
format exactly. Produce the skeleton and nothing else — do not write the analysis yet.
|
|
111
156
|
|
|
112
157
|
PHASE 2 — EXPAND:
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
158
|
+
Once the skeleton is complete, expand each dimension into one paragraph analyzing how our
|
|
159
|
+
product compares on it. Restate the dimension's name before expanding it, and make each
|
|
160
|
+
expansion self-contained. Use specific, concrete observations, and flag any dimension where
|
|
161
|
+
we have a clear advantage or a significant gap.
|
|
116
162
|
```
|
|
117
163
|
|
|
118
164
|
### Example 3: Learning Content
|