@ckelsoe/prompt-architect 3.3.1 → 3.5.0

package/skills/prompt-architect/references/frameworks/rpef.md

@@ -19,76 +19,133 @@ Reverse Prompt Engineering (RPEF) inverts the normal prompt-to-output direction.
 ### Input Sample (optional but powerful)
 **Purpose:** If the output was generated from specific input data, include it. The model uses the relationship between input and output to infer transformation rules and prompt logic.
 
+**Template slot:** `SOURCE MATERIAL — ORIGINAL INPUT`. It is the only optional block in the template, and it is deleted together with the sentence directly beneath it when the recovery is output-only.
+
 ### Output Sample (required)
 **Purpose:** The existing output you want to reverse-engineer. This is the primary source of truth — what you want to be able to reproduce.
 
+**Template slot:** `SOURCE MATERIAL — OUTPUT SAMPLE`. Paste it verbatim; paraphrasing the sample destroys exactly the formatting and phrasing signals the analysis is meant to read.
+
 ### Analysis Instruction
-**Purpose:** Directs the model to analyze the output for its implicit prompt properties: tone, structure, reasoning style, constraints, format, level of detail, persona, and assumptions.
+**Purpose:** Directs the model to analyze the output for its implicit prompt properties, and to *report* that analysis before writing anything else.
+
+**Template slot:** `ANALYSIS INSTRUCTIONS`, which names six dimensions: implied role or persona, tone and voice, structural and formatting patterns, level of detail and scope, implicit constraints (what it did NOT do), and reasoning style. Structure and format are one merged dimension, not two. An earlier revision of *this document* listed seven by splitting them apart; that was drift in the doc's copy, not a change to the template, which has carried the merged dimension unchanged since it was introduced.
+
+### Confidence Threshold
+**Purpose:** Stops the model from presenting a guess as a recovered prompt when one sample is too thin to support one.
 
-### Confidence Threshold (optional)
-**Purpose:** Instructs the model to only generate the recovered prompt when confident enough. Default: 80-90%.
+**Template slot:** the closing sentences of `RECOVERED PROMPT`. The threshold is only useful when it is paired with a defined alternative action — see the note under Template Structure on why the earlier "only proceed when 80% confident" phrasing did nothing.
 
 ### Generalization Directive
 **Purpose:** Specifies whether the recovered prompt should be a specific single-use template or a generalized meta-prompt with fill-in placeholders.
 
+**Template slot:** folded into `RECOVERED PROMPT` as the `[PLACEHOLDER]` instruction — the template always asks for the generalized form, and marks variability at the token level rather than asking a separate yes/no question about it.
+
 ## Template Structure
 
+Section headers are stripped at emission, so every slot's meaning is carried by the
+unbracketed prose around it rather than by the header above it. This template also depends
+on sequencing — the analysis must be reported *before* the recovered prompt is written —
+and that sequencing lives in the prose ("First, examine…", "After completing that
+analysis…"), not in the order the headers happen to appear. Do not reintroduce
+header-only structure.
+
 ```
 You are an expert Prompt Engineer performing reverse prompt engineering.
 
 TASK:
-Analyze the following [input and] output. Reconstruct the prompt that would
-consistently produce this type of output. Your recovered prompt should be
-reusable — replace specific details with clearly marked [PLACEHOLDER] variables.
+Analyze the sample material below and reconstruct the prompt that would
+consistently produce this type of output.
 
-[INPUT DATA (if applicable):]
-[Paste input here, or omit if output-only]
+SOURCE MATERIAL — ORIGINAL INPUT:
+[Optional — paste here, verbatim, the input that originally produced the output
+sample below. If you have nothing to paste, delete this bracketed block along with
+the sentence directly below it.]
+Use the material above as the input that produced the output sample below.
 
-OUTPUT SAMPLE:
-[Paste the output you want to reverse-engineer]
+SOURCE MATERIAL — OUTPUT SAMPLE:
+[Paste here, verbatim, the output you want to reverse-engineer.]
+Use the material above as the finished output to reverse-engineer.
 
 ANALYSIS INSTRUCTIONS:
-Examine the output for:
-- Implied role or persona (who generated this?)
-- Tone and voice (formal, casual, technical, etc.)
-- Structural patterns (how is it organized?)
+First, examine the output above and report what you find for each of these dimensions:
+- Implied role or persona
+- Tone and voice
+- Structural and formatting patterns
 - Level of detail and scope
-- Any implicit constraints (what did it not do?)
-- Format conventions (length, sections, bullet style, etc.)
-- Reasoning style (step-by-step, direct, narrative?)
+- Implicit constraints (what it did NOT do)
+- Reasoning style
 
 RECOVERED PROMPT:
-Generate a reusable prompt template that would reliably reproduce this
-type of output. Use [PLACEHOLDER] notation for variable elements.
-Only proceed when at least 80% confident in the reconstruction.
+After completing that analysis, write out a reusable prompt template that reliably
+reproduces this type of output. Mark every variable element with the literal token
+[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
+If you are less than 80% confident in the reconstruction, do not guess: say so, and
+name the additional sample material that would resolve the uncertainty.
 ```
 
+Two sentences in `RECOVERED PROMPT` do work that is easy to lose when editing this
+template, and both replaced phrasings that failed in a specific way:
+
+- **"written exactly that way rather than replaced with a specific value."** The earlier
+  phrasing, "Use [PLACEHOLDER] for variable elements," reads two ways: emit the literal
+  token, or fill each position with a plausible concrete value. The second reading returns
+  a filled-in example rather than the reusable template that is the entire point of the
+  framework, so the instruction is made explicit to close it off. How often the ambiguity
+  actually resolves the wrong way has not been measured here.
+- **"If you are less than 80% confident … say so, and name the additional sample
+  material."** The earlier phrasing, "Only proceed when at least 80% confident," named no
+  alternative action, so it gated nothing: no behavior was defined for the failing branch,
+  leaving nothing in the text to stop the model proceeding anyway. A threshold is only a
+  gate when the below-threshold
+  path is spelled out, and naming the missing material makes that path useful rather than
+  merely a refusal.
+
+Because RPEF's source material is the user's own pasted sample, the two `SOURCE MATERIAL`
+blocks are the framework's legitimate `[...]` placeholders under SKILL.md step 5 — they are
+material only the user can supply, not defaults the model should invent.
+
 ## Complete Examples
 
+Every example below is shown in emitted form: each slot carries its own role in prose, so
+nothing is lost when the headers are deleted. The output samples are filled with real
+material rather than left as paste instructions, because an RPEF prompt with an empty
+sample demonstrates nothing — the sample is the only thing the framework operates on.
+
 ### Example 1: Recovering a Writing Style
 
-**Scenario:** You got a great product description once and want to reproduce that tone.
+**Scenario:** You got a great product description once and want to reproduce that tone. Output-only, so the `ORIGINAL INPUT` block and the sentence below it are both deleted.
 
 ```
 You are an expert Prompt Engineer performing reverse prompt engineering.
 
 TASK:
-Analyze the following product description. Reconstruct the prompt that
-would consistently produce this style of writing for other products.
+Analyze the sample material below and reconstruct the prompt that would
+consistently produce this type of output.
 
-OUTPUT SAMPLE:
-"Mira is the meeting companion that thinks ahead. While you're present
-in the conversation, Mira is quietly capturing every decision, action
-item, and insight — then turning them into a clean summary the moment
-you close your laptop. No more frantic note-taking. No more 'wait, what
-did we decide?' Just clarity, automatically."
+SOURCE MATERIAL — OUTPUT SAMPLE:
+"Mira is the meeting companion that thinks ahead. While you're present in the
+conversation, Mira is quietly capturing every decision, action item, and insight
+— then turning them into a clean summary the moment you close your laptop. No
+more frantic note-taking. No more 'wait, what did we decide?' Just clarity,
+automatically."
+Use the material above as the finished output to reverse-engineer.
 
 ANALYSIS INSTRUCTIONS:
-Examine for: implied persona, tone, structure, what it does vs. doesn't
-say, sentence length patterns, opening hook strategy, implicit audience.
+First, examine the output above and report what you find for each of these dimensions:
+- Implied role or persona
+- Tone and voice
+- Structural and formatting patterns
+- Level of detail and scope
+- Implicit constraints (what it did NOT do)
+- Reasoning style
 
 RECOVERED PROMPT:
-Generate a reusable template for writing product descriptions in this style.
+After completing that analysis, write out a reusable prompt template that reliably
+reproduces this type of output. Mark every variable element with the literal token
+[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
+If you are less than 80% confident in the reconstruction, do not guess: say so, and
+name the additional sample material that would resolve the uncertainty.
 ```
 
 **Recovered prompt output:**
@@ -110,48 +167,159 @@ STYLE RULES:
 
 ### Example 2: Recovering an Analysis Format
 
-**Scenario:** A colleague produced a competitive analysis you want to replicate.
+**Scenario:** A colleague produced a competitive analysis you want to replicate. You have both the brief they worked from and the analysis itself, so both `SOURCE MATERIAL` blocks are used.
 
 ```
 You are an expert Prompt Engineer performing reverse prompt engineering.
 
 TASK:
-Analyze the following competitive analysis excerpt. Reconstruct the prompt
-template that produced this format and analytical approach.
+Analyze the sample material below and reconstruct the prompt that would
+consistently produce this type of output.
 
-INPUT DATA:
+SOURCE MATERIAL — ORIGINAL INPUT:
 Companies to compare: Notion vs. Confluence vs. Linear
 Use case: Engineering team wiki and documentation
-
-OUTPUT SAMPLE:
-[Paste the analysis here]
+Audience: VP Engineering, 200-person org
+Use the material above as the input that produced the output sample below.
+
+SOURCE MATERIAL — OUTPUT SAMPLE:
+"**Structure and hierarchy.** Confluence wins outright here. Spaces, page trees,
+and page-level restrictions map cleanly onto how engineering orgs already think
+about ownership. Notion's database-plus-nesting model is more flexible but
+degrades past roughly 500 pages without deliberate curation. Linear's documents
+are deliberately shallow — they attach to projects rather than organizing
+independently — so it is not a wiki candidate at all on this dimension.
+
+**Search.** Confluence's search is the weakest of the three despite being the
+oldest: it indexes attachments well but ranks stale pages highly. Notion's search
+handles partial-title matching acceptably since its 2023 rebuild. Linear's search
+is the fastest and best-scoped, but there is comparatively little to search.
+
+**Permissioning.** Confluence again, and the gap widens with headcount. Notion's
+permissions are workspace- and page-level but become expensive to audit past
+about 100 seats. Linear does not attempt document-level permissioning.
+
+**Recommendation.** For an engineering wiki proper: Confluence above roughly 150
+engineers, where permissioning and hierarchy dominate; Notion below that, where
+flexibility outweighs structure. Linear is complementary rather than a candidate
+— pair it with whichever wiki you choose.
+
+*Caveat: the page-count and seat-count thresholds above come from practitioner
+reports, not vendor benchmarks.*"
+Use the material above as the finished output to reverse-engineer.
 
 ANALYSIS INSTRUCTIONS:
-Identify: comparison dimensions, how trade-offs are framed, what's
-prioritized, how conclusions are drawn, what data is cited vs. inferred.
+First, examine the output above and report what you find for each of these dimensions:
+- Implied role or persona
+- Tone and voice
+- Structural and formatting patterns
+- Level of detail and scope
+- Implicit constraints (what it did NOT do)
+- Reasoning style
 
 RECOVERED PROMPT:
-Template should work for any set of tools in any category.
+After completing that analysis, write out a reusable prompt template that reliably
+reproduces this type of output. Mark every variable element with the literal token
+[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
+If you are less than 80% confident in the reconstruction, do not guess: say so, and
+name the additional sample material that would resolve the uncertainty.
+```
+
+**Recovered prompt output:**
+```
+ROLE: You are a senior technical evaluator who compares tools for engineering
+leadership audiences.
+
+TASK: Compare [TOOL A], [TOOL B], and [TOOL C] for [USE CASE], writing for
+[AUDIENCE AND ORG SIZE].
+
+METHOD:
+- Choose 3-4 dimensions that actually matter for the stated use case. Use the same
+  dimensions for every tool, as bold headings.
+- Under each dimension, name the winner in the first sentence, then justify it.
+- Where a tool is not a genuine candidate on a dimension, say so plainly instead of
+  scoring it — do not manufacture a three-way comparison that isn't there.
+- Close with a recommendation conditional on a named threshold (org size, seat count,
+  volume) rather than a single unqualified pick.
+- Mark any figure sourced from practitioner reports rather than vendor benchmarks
+  with an explicit caveat line at the end.
+
+TONE: Direct and evaluative — verdict first, reasoning second. No feature-matrix
+tables, no vendor marketing language, no hedging on the recommendation.
 ```
 
-### Example 3: Output-Only Recovery
+### Example 3: Output-Only Recovery, Where the Confidence Gate Fires
 
-**Scenario:** You have a great email you wrote with AI assistance but can't find the original prompt.
+**Scenario:** You have an email you're happy with but lost the prompt. One sample is often too thin to generalize from — this example shows what the template's confidence rule is for.
 
 ```
-You are an expert Prompt Engineer.
+You are an expert Prompt Engineer performing reverse prompt engineering.
 
 TASK:
-I have an email I'm happy with but lost the prompt that generated it.
-Reverse-engineer a reusable prompt from the email so I can produce
-similar emails for other clients.
+Analyze the sample material below and reconstruct the prompt that would
+consistently produce this type of output.
+
+SOURCE MATERIAL — OUTPUT SAMPLE:
+"Hi Dana,
+
+Quick update on the Q3 rollout: we're on track for the Sept 15 launch, but I want
+to flag something early rather than at the last minute. The data migration is
+running about 30% longer than we scoped, which eats most of our buffer. That
+doesn't put the date at risk yet — it means we've lost the room to absorb a
+second surprise.
+
+Two options. We hold the date and move the reporting dashboard to a fast-follow
+in early October, or we hold scope and move the launch to Sept 29. I'd recommend
+the first: the dashboard is the piece your team flagged as least useful in month
+one.
+
+Happy to talk it through Thursday if that's easier. Either way you'll have a firm
+recommendation from me by Friday.
 
-OUTPUT SAMPLE:
-[Paste the email]
+Best,
+Ray"
+Use the material above as the finished output to reverse-engineer.
+
+ANALYSIS INSTRUCTIONS:
+First, examine the output above and report what you find for each of these dimensions:
+- Implied role or persona
+- Tone and voice
+- Structural and formatting patterns
+- Level of detail and scope
+- Implicit constraints (what it did NOT do)
+- Reasoning style
 
 RECOVERED PROMPT:
-Create a template with [CLIENT NAME], [SITUATION], and [SPECIFIC REQUEST]
-as placeholder variables. Capture the tone, structure, and approach precisely.
+After completing that analysis, write out a reusable prompt template that reliably
+reproduces this type of output. Mark every variable element with the literal token
+[PLACEHOLDER], written exactly that way rather than replaced with a specific value.
+If you are less than 80% confident in the reconstruction, do not guess: say so, and
+name the additional sample material that would resolve the uncertainty.
+```
+
+**Recovered prompt output** — here the gate fires, which is the correct behavior rather than a failure:
+```
+Confidence in a reusable reconstruction: roughly 60%. That is below the 80% bar you
+set, so I am not going to hand you a template and let it look more reliable than it is.
+
+What the sample supports confidently: the structure is stable and readable — status
+line, early flag, explicit statement of what the impact is and is not, exactly two
+named options, a recommendation justified by the recipient's own stated priority, an
+offer to discuss, and a committed date for the firm answer. The register is candid and
+non-defensive; the bad news arrives in the second sentence rather than buried below a
+paragraph of reassurance.
+
+What one sample cannot settle, and why each matters:
+- Whether "exactly two options" is a rule or an artifact of this situation. A template
+  that hardcodes two will force false binaries where the honest answer is one or three.
+- Whether the recommendation systematically favors protecting the date, or whether that
+  was specific to this trade-off.
+- Whether this register holds for an external client, or only for an internal
+  stakeholder like Dana with whom the writer has standing rapport.
+
+Send two or three more emails you were happy with — ideally one carrying
+straightforwardly good news, and one written to a client rather than a colleague — and
+I can separate the fixed structure from this instance's specifics.
 ```
 
 ## Best Use Cases
@@ -187,9 +355,10 @@ as placeholder variables. Capture the tone, structure, and approach precisely.
 
 ## Quick Reference
 
-| Component | Purpose |
-|-----------|---------|
-| Input Sample | The input data (if any) that produced the output |
-| Output Sample | The output to reverse-engineer (required) |
-| Analysis Instructions | What dimensions to examine |
-| Recovered Prompt | Reusable template with [PLACEHOLDER] variables |
+| Component | Template slot | Purpose |
+|-----------|---------------|---------|
+| Input Sample | `SOURCE MATERIAL — ORIGINAL INPUT` | The input data (if any) that produced the output — optional, deleted with its trailing sentence when unused |
+| Output Sample | `SOURCE MATERIAL — OUTPUT SAMPLE` | The output to reverse-engineer, pasted verbatim (required) |
+| Analysis Instruction | `ANALYSIS INSTRUCTIONS` | The six dimensions to examine, reported before anything else is written |
+| Generalization Directive | `RECOVERED PROMPT` | Reusable template marking variables with the literal token `[PLACEHOLDER]` |
+| Confidence Threshold | `RECOVERED PROMPT` (closing) | Below 80%, say so and name the sample material that would resolve it |

package/skills/prompt-architect/references/frameworks/rtf.md

@@ -156,7 +156,7 @@ FORMAT: Provide as:
 - ❌ Rich context needed (use CO-STAR)
 - ❌ Audience/tone matters (use CO-STAR)
 - ❌ Specific methodology required (use RISEN)
-- ❌ Input transformation needed (use RISE)
+- ❌ Input transformation needed (use RISE-IE)
 
 ## Common Mistakes
 
@@ -221,7 +221,7 @@ FORMAT: [Follow this structure]
 
 ## When to Upgrade from RTF
 
-**Upgrade to RISE if:**
+**Upgrade to RISE-IE if:**
 - Need to specify input characteristics
 - Processing steps matter
 - Input → output transformation

package/skills/prompt-architect/references/frameworks/self-consistency.md

@@ -0,0 +1,195 @@
+# Self-Consistency Framework
+
+## Overview
+
+Self-Consistency is a reasoning technique that replaces a single greedy answer with a majority vote over many independently sampled reasoning paths. Instead of asking the model to think step by step once and trust the result, you run the *same* prompt several times at non-zero temperature, collect the final answer from each run, and return the answer that appears most often. The reasoning inside each run still looks like Chain-of-Thought; what changes is that the decision is made across runs rather than within one. The insight is that a correct answer tends to be reachable by several different valid lines of reasoning, while the various ways of being wrong scatter — so agreement across independent samples is itself evidence.
+
+When prompt-architect emits a Self-Consistency prompt, its framework section headers — `PROBLEM:` and `SOURCE MATERIAL:` — are stripped and the model receives a flat block. The solve instruction is written as a complete sentence so it survives on its own, and the `FINAL ANSWER: [answer]` line is a literal output label the model is being told to produce, not a section header, so it stays in place exactly as written. The optional `SOURCE MATERIAL` block holds the data the solution runs on and is deleted when the problem is self-contained.
+
+**Origin:** "Self-Consistency Improves Chain of Thought Reasoning in Language Models" (Wang et al., arXiv 2203.11171, ICLR 2023). The abstract reports the technique's headline gain as a relative delta of **+17.9%** on GSM8K. The corresponding absolute figures — greedy Chain-of-Thought decoding at **56.5%** rising to **74.4%** with a majority vote over 40 sampled paths, using PaLM 540B — are **Table 1** body figures, not abstract figures. A notable finding: the plain, unweighted majority vote matched or beat every weighted aggregation scheme the authors tested, so there is no need to weight votes by probability or confidence.
+
+## How This Framework Is Delivered
+
+Self-Consistency is operational, not a single instruction: the gain comes from *sampling the same prompt N times and voting across the results*. A single pasted prompt cannot run itself N times, so no one prompt can perform the vote on its own. That constraint is handled honestly rather than hidden:
+
+- **The emitted prompt** is a clean solve-prompt. It forces step-by-step reasoning and ends with a machine-parseable `FINAL ANSWER:` line so the outputs of separate runs can be lined up and tallied. It is a single-path prompt by design.
+- **The skill's ANALYSIS section** is where the user is told to run that prompt N times at non-zero temperature and take the majority vote *externally* — in a script, a notebook, or by hand. The voting lives outside the model.
+
+Do not pretend one prompt does the voting. Stating plainly that the aggregation happens outside the model is a feature: it is what keeps the technique faithful to the paper instead of collapsing it into something weaker.
+
+## Components
+
+### The Solve Prompt (emitted)
+**Purpose:** Produce one full reasoning path and one cleanly extractable answer. The prompt demands step-by-step work — the diversity of reasoning across runs is the whole engine, so each run must actually reason rather than guess — and pins the answer to a fixed `FINAL ANSWER: [answer]` line so a program can find it identically in every sample.
+
+**What it must contain:**
+- A complete solve instruction that survives header stripping ("Solve the problem above. Work through it step by step…").
+- The exact terminal line `FINAL ANSWER: [answer]`, with nothing else on that line, so every run's answer sits in the same parseable position.
+- Optionally, a `SOURCE MATERIAL` block naming the data the solution runs on.
+
+### The Sampling Parameters (analysis-side)
+**Purpose:** Generate the independent reasoning paths. The paper samples **40**; in practice **5-10** is usually enough to see the benefit. Sampling must be at **non-zero temperature** — **~0.7** is a reasonable default — because temperature 0 collapses every run to the same greedy path and destroys the diversity the vote depends on.
+
+### The Aggregation (analysis-side)
+**Purpose:** Turn N answers into one. Extract the `FINAL ANSWER:` value from each sample, count how often each distinct answer appears, and return the most frequent one. This is a **plain, unweighted majority vote** — not a confidence-weighted or probability-weighted scheme, both of which the authors found no better. A near-tie is a signal of genuine uncertainty, not an invitation to ask the model to break the tie.
+
+## Template Structure
+
+Section headers — `PROBLEM:` and `SOURCE MATERIAL:` — are stripped at emission, so the solve instruction that follows is written as a complete sentence and survives on its own. The `FINAL ANSWER:` line is a literal output label, not a header, and survives verbatim. The `SOURCE MATERIAL` block is optional and is deleted when the problem already states every value needed.
+
+```
+PROBLEM:
+[Your question, calculation, or reasoning problem]
+
+SOURCE MATERIAL:
+[Optional — paste the problem data the solution runs on here (the word problem, dataset, table, figures, or constraint list). Delete this line and the one below it if the problem above already states every value needed.]
+The figures and constraints in the material above are authoritative — extract from them rather than from assumed or recalled values.
+
+Solve the problem above. Work through it step by step, showing your reasoning
+at each stage. When you reach an answer, state it on its own line, in exactly
+this form and with nothing else on that line:
+
+FINAL ANSWER: [answer]
+```
+
+## Complete Examples
+
+Every example below is shown in emitted form: the solve instruction and the `FINAL ANSWER:` line are the same in each — they are what survives header stripping. Where the reasoning runs on a pasted dataset or constraint list, the `SOURCE MATERIAL` block carries it and the problem refers to "the problem above"; where every value is already in the problem statement, that block is deleted. In every case the emitted artifact is a single-path solve-prompt, and the vote is run afterward per the analysis note.
+
+### Example 1: Math / Word Problem
+
+**Before Self-Consistency:**
+"A shipment has 3 crates of 24 units and 5 crates of 18 units. If 12% are defective, how many good units are there?"
+
+**After Self-Consistency** (no source material — every value is stated in the problem, so the `SOURCE MATERIAL` block is deleted):
+```
+PROBLEM:
+A shipment has 3 crates of 24 units and 5 crates of 18 units. 12% of all
+units are defective. How many good (non-defective) units are there?
+
+Solve the problem above. Work through it step by step, showing your reasoning
+at each stage. When you reach an answer, state it on its own line, in exactly
+this form and with nothing else on that line:
+
+FINAL ANSWER: [answer]
+```
+
+**Analysis note handed to the user:** Run this prompt 5-10 times at temperature ~0.7. Extract each run's `FINAL ANSWER:` value, count them, and return the most frequent. If two answers tie, treat that as real uncertainty and inspect the reasoning rather than asking the model to pick.
+
+### Example 2: Logic / Reasoning Over Pasted Constraints
+
+**Before Self-Consistency:**
+"Given these seating constraints, who sits in seat 3?"
+
+**After Self-Consistency** (source material supplied):
+```
+PROBLEM:
+Determine who sits in seat 3, working only from the constraints in the
+problem above.
+
+SOURCE MATERIAL:
+[Paste the full list of seating constraints — people, seats, and adjacency rules — here]
+The figures and constraints in the material above are authoritative — extract from them rather than from assumed or recalled values.
+
+Solve the problem above. Work through it step by step, showing your reasoning
+at each stage. When you reach an answer, state it on its own line, in exactly
+this form and with nothing else on that line:
+
+FINAL ANSWER: [answer]
+```
+
+### Example 3: Business Calculation
+
+**Before Self-Consistency:**
+"If CAC is $1,200, MRR per customer is $150, and gross margin is 70%, what's the payback period in months?"
+
+**After Self-Consistency** (no source material — every value is stated, so the `SOURCE MATERIAL` block is deleted):
+```
+PROBLEM:
+Customer Acquisition Cost (CAC): $1,200. Average MRR per customer: $150.
+Gross margin: 70%. What is the payback period in months?
+
+Solve the problem above. Work through it step by step, showing your reasoning
+at each stage. When you reach an answer, state it on its own line, in exactly
+this form and with nothing else on that line:
+
+FINAL ANSWER: [answer]
+```
+
+Because a payback-period calculation admits several valid routes — dividing CAC by gross-margin-adjusted MRR, or building up contribution month by month — Self-Consistency is well suited here: independent runs that reach the same number through different arithmetic reinforce the result, and a run that fumbles the margin adjustment is outvoted.
+
+## Best Use Cases
+
+1. **High-Stakes Single Answers**
+   - Numerical problems where one wrong step flips the result
+   - Financial calculations where accuracy is worth the extra compute
+   - Answers you will act on and cannot easily re-check by hand
+
+2. **Problems With Multiple Valid Solution Routes**
+   - Word problems solvable by more than one method
+   - Logic puzzles with several deduction orders
+   - Any task where convergent independent reasoning is meaningful evidence
+
+3. **When Greedy Chain-of-Thought Is Almost Right**
+   - Prior single-path CoT gave inconsistent answers across attempts
+   - The model clearly can solve it but sometimes slips
+   - You can afford N samples and want to buy down the error rate
+
+4. **Discrete, Comparable Answers**
+   - The final answer is a number, a label, or a short choice that can be compared for equality across runs and tallied
+
+## Selection Criteria
+
+**Choose Self-Consistency when:**
+- ✅ The answer is discrete and comparable across runs (a number, a category, a choice)
+- ✅ You can sample the prompt multiple times at non-zero temperature
+- ✅ You can aggregate the results outside the model (script, notebook, or by hand)
+- ✅ Accuracy justifies spending N times the compute of a single run
+- ✅ Single-path CoT is close but not reliable enough
+
+**Avoid Self-Consistency when:**
+- ❌ You can only run the prompt once → use Chain-of-Thought instead
+- ❌ The output is open-ended prose with no comparable "final answer" to vote on
+- ❌ The task needs to explore and keep branching structure → use Tree of Thought
+- ❌ The problem must be decomposed into dependent sub-problems → use Least-to-Most
+- ❌ You need explicit variable extraction and planning within one run → use Plan-and-Solve
+
+## Self-Consistency vs. Chain-of-Thought vs. Tree of Thought
+
+Chain-of-Thought is the single-path neighbor of Self-Consistency: they use the *same* kind of step-by-step reasoning prompt. The distinction is entirely in how the answer is chosen. CoT commits to one path (typically greedy); Self-Consistency samples many CoT paths and lets them vote. Tree of Thought is different again — it explores and prunes a branching search *within* a single run rather than voting across independent runs.
+
+| | Chain-of-Thought | Self-Consistency | Tree of Thought |
+|---|---|---|---|
+| Reasoning paths | One | Many, independently sampled | One branching search |
+| Temperature | Any (often greedy) | Non-zero (~0.7) required | Any |
+| How the answer is chosen | The single run's output | Majority vote across runs | Best surviving branch |
+| Aggregation location | None | Outside the model | Inside the run |
+| Runs needed | 1 | N (5-10 typical; 40 in paper) | 1 |
+| Best for | General step-by-step reasoning | Discrete answers worth N-fold compute | Problems needing lookahead/backtracking |
+
+## Common Mistakes
+
+1. **Asking the model to vote on itself**
+   - Self-Consistency is *not* asking the model to generate several approaches in one response and then judge which it likes best. That is self-assessment — a different and weaker technique, because the model is not a reliable judge of its own reasoning. The vote must happen across independent runs, outside the model.
+
+2. **Sampling at temperature 0**
+   - Greedy decoding produces the same path every time, so N identical runs vote unanimously for one answer and the method reduces to plain CoT with wasted compute. Sampling must be at non-zero temperature.
+
+3. **Weighting the votes**
+   - The paper found an unweighted majority vote matched or beat every weighted scheme tested. Confidence- or probability-weighting adds complexity for no gain.
+
+4. **Omitting or reformatting the FINAL ANSWER line**
+   - If runs end their answers differently, they cannot be extracted and compared programmatically. The `FINAL ANSWER: [answer]` line must appear verbatim, alone on its line, in every run.
+
+5. **Forcing a tie-break**
+   - A near-tie means genuine uncertainty. Asking the model to break it discards the very signal the method surfaced; inspect the split reasoning instead.
+
+## Quick Reference
+
+| Component | Location | Purpose |
+|-----------|----------|---------|
+| Solve prompt | Emitted | One reasoning path plus a parseable `FINAL ANSWER:` line |
+| Step-by-step instruction | Emitted | Forces reasoning so independent paths diverge meaningfully |
+| Sampling (N, temperature) | Analysis-side | Generate independent paths; N≈5-10 (40 in paper), temp ~0.7 |
+| Aggregation | Analysis-side | Plain unweighted majority vote over extracted answers |
+| Tie handling | Analysis-side | Treat near-ties as uncertainty, not as a prompt to break the tie |

package/skills/prompt-architect/references/frameworks/self-refine.md

@@ -4,7 +4,9 @@
 
 Self-Refine is an iterative output improvement framework where a single model acts as its own generator, critic, and refiner — with no external training, no additional models, and no labeled data required. The core loop: generate an initial output → produce specific, actionable feedback → refine using that feedback → repeat until a stopping criterion is met.
 
-**Research basis:** "Self-Refine: Iterative Refinement with Self-Feedback" (Madaan et al., arXiv 2303.17651, NeurIPS 2023). Tested across 7 task types — improvement of 5-40% over single-pass generation from GPT-3.5 and GPT-4. Humans preferred Self-Refined outputs in blind evaluation. Project site: selfrefine.info.
+**Research basis:** "Self-Refine: Iterative Refinement with Self-Feedback" (Madaan et al., arXiv 2303.17651, NeurIPS 2023). Evaluated across 7 diverse tasks using GPT-3.5, ChatGPT, and GPT-4. The abstract reports that Self-Refine outputs are "preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance." That ~20% is an *average across the 7 tasks* — per-task gains vary substantially, and the abstract gives no range. Project site: selfrefine.info.
+
+*Correction:* an earlier version of this file reported "improvement of 5-40% over single-pass generation from GPT-3.5 and GPT-4." Neither bound appears in the paper, and ChatGPT was omitted from the model list. Verified against the arXiv abstract.
 
 ## The Core Loop
 
@@ -190,7 +192,7 @@ biggest single point of failure identified.
 
 | | Self-Refine | CAI Critique-Revise | BAB | Chain of Density |
 |---|---|---|---|---|
-| Direction | Any output → improvement | Output vs. explicit principle | Before state → after state | Content → denser compression |
+| Direction | Any output → improvement | Output vs. explicit principle | Before state → after state | Same length → higher entity density |
 | Critique type | Multi-dimensional quality | Principle-aligned alignment | Transformation rules | Density/information |
 | Iterations | Configurable loop | One cycle (or more) | Single transform | N passes |
 | Best for | General quality improvement | Alignment/compliance | Rewriting existing content | Summarization |