@bastani/atomic 0.7.17 → 0.8.0

package/dist/builtin/workflows/skills/prompt-engineer/references/advanced_patterns.md

@@ -0,0 +1,271 @@
+# Advanced Prompting Patterns
+
+This document covers sophisticated prompt engineering techniques for complex tasks requiring structured reasoning, long-form content, or multi-step processing.
+
+## Chain of Thought (CoT) Prompting
+
+### What is Chain of Thought?
+
+Chain of thought prompting encourages the model to break down complex problems systematically. Giving the model space to think can dramatically improve its performance on research, analysis, and problem-solving tasks.
+
+### Key Benefits
+
+- **Accuracy**: Stepping through problems reduces errors, especially in math, logic, analysis, or generally complex tasks
+- **Coherence**: Structured reasoning produces more organized responses
+- **Debugging**: Observing the model's thought process reveals unclear prompt areas
+
+### When to Use CoT
+
+Apply CoT for tasks that a human would need to think through, like:
+
+- Complex math or logic problems
+- Multi-step analysis
+- Writing complex documents
+- Decisions with many factors
+- Planning specs
+
+**Trade-off**: Increased output length may impact latency, so avoid using CoT for straightforward tasks.
+
+### Three CoT Techniques (Least to Most Complex)
+
+**1. Basic Prompt**
+Include "Think step-by-step" in your request. Simple but lacks specific guidance.
+
+**2. Guided Prompt**
+Outline specific steps for the model's reasoning process. Provides direction without structuring the output format, making answer extraction more difficult.
+
+**3. Structured Prompt**
+Use XML tags like `<thinking>` and `<answer>` to separate reasoning from final answers. This enables easy parsing of both thought process and conclusions.
+
+Example:
+
+```
+Please analyze this problem and provide your reasoning.
+
+Put your step-by-step thinking in <thinking> tags.
+Put your final answer in <answer> tags.
+```
+
+### Critical Implementation Note
+
+**"Always have the model output its thinking. Without outputting its thought process, no thinking occurs!"** Visible reasoning is essential for CoT effectiveness.
+
+---
+
+## Multishot Prompting
+
+### Core Concept
+
+Multishot prompting (also called few-shot prompting) involves providing a few well-crafted examples in your prompt to improve the model's output quality. This technique is particularly effective for tasks requiring structured outputs or adherence to specific formats.
+
+### Key Benefits
+
+- **Accuracy**: Examples reduce misinterpretation of instructions
+- **Consistency**: Examples enforce uniform structure and style
+- **Performance**: Well-chosen examples boost the model's ability to handle complex tasks
+
+### Crafting Effective Examples
+
+Examples should be:
+
+1. **Relevant** — Mirror your actual use case
+2. **Diverse** — Cover edge cases and vary sufficiently to avoid unintended pattern recognition
+3. **Clear** — Wrapped in `<example>` tags (multiple examples nested in `<examples>` tags)
+
+### Optimal Quantity
+
+Include 3-5 diverse, relevant examples. More examples = better performance, especially for complex tasks.
+
+### Template Structure
+
+```xml
+<examples>
+  <example>
+    <input>Sample input 1</input>
+    <output>Expected output 1</output>
+  </example>
+
+  <example>
+    <input>Sample input 2</input>
+    <output>Expected output 2</output>
+  </example>
+
+  <example>
+    <input>Sample input 3</input>
+    <output>Expected output 3</output>
+  </example>
+</examples>
+```
+
+---
+
+## Prompt Chaining
+
+### Core Concept
+
+Prompt chaining breaks complex tasks into smaller, sequential subtasks, with each step receiving the model's focused attention. This approach improves accuracy, clarity, and traceability compared to handling everything in a single prompt.
+
+### Key Benefits
+
+1. **Accuracy**: Each subtask gets full attention, reducing errors
+2. **Clarity**: Simpler instructions produce clearer outputs
+3. **Traceability**: Issues can be pinpointed and fixed in specific steps
+
+### When to Use Chaining
+
+Apply this technique for multi-step tasks involving:
+
+- Research synthesis and document analysis
+- Iterative content creation
+- Multiple transformations or citations
+- Tasks where the model might miss or mishandle steps
+
+### Core Techniques
+
+**1. Identify Subtasks**
+Break work into distinct, sequential steps with single, clear objectives.
+
+**2. Structure with XML**
+Use XML tags to pass outputs between prompts for clear handoffs between steps.
+
+**3. Single-Task Goals**
+Each subtask should focus on one objective to maintain clarity.
+
+**4. Iterate & Refine**
+Adjust subtasks based on the model's performance.
+
+### Workflow Examples
+
+- **Content pipelines**: Research → Outline → Draft → Edit → Format
+- **Data processing**: Extract → Transform → Analyze → Visualize
+- **Decision-making**: Gather info → List options → Analyze → Recommend
+- **Verification loops**: Generate → Review → Refine → Re-review
+- **Writing Specs**: Research → Plan → Implement (see detailed example below)
+
+### Complex Example: Spec Workflow
+
+This workflow represents a research-driven, AI-augmented software development process that emphasizes thorough planning and human oversight before implementation. It's designed to maximize quality and alignment by incorporating both AI assistance and human feedback at critical decision points.
+
+**Phase 1: Research & Requirements**
+
+1. **Deep Research** — Begin with comprehensive research into the problem space: understanding user needs, exploring existing solutions, reviewing relevant technologies, and identifying constraints. Build a solid foundation of knowledge before defining what to build.
+
+2. **Product Requirements Document (PRD)** — Distill research findings into a formal PRD that articulates the _what_ and _why_. Define the problem statement, target users, success metrics, user stories, and business objectives. Remain technology-agnostic, focusing purely on outcomes rather than implementation details.
+
+**Phase 2: AI-Assisted Design**
+
+3. **Brainstorm with Coding Agent** — This is where the workflow diverges from traditional approaches. Engineers collaborate with an AI coding agent to explore technical possibilities. This brainstorming session generates multiple implementation approaches, identifies potential challenges, discusses trade-offs, and leverages AI's knowledge of patterns and best practices. It's an exploratory phase that surfaces ideas that might not emerge from human-only brainstorming.
+
+4. **Technical Design/Spec** — Formalize the brainstorming output into a technical specification describing the _how_: architecture decisions, API designs, data models, technology stack choices, system components and their interactions, scalability considerations, and security/performance requirements. This becomes the engineering blueprint for implementation.
+
+**Phase 3: Human Validation Loop**
+
+5. **Human Feedback** — A critical checkpoint where experienced engineers, architects, or technical leads review the spec. This human oversight ensures the AI-assisted design is sound, catches edge cases or concerns, validates assumptions, and aligns the technical approach with organizational standards and long-term architecture. This phase acknowledges that AI assistance needs human verification.
+
+6. **Refined Technical Design/Spec** — Incorporate feedback to improve the specification. This might involve adjusting the architecture, adding clarifications, addressing edge cases, or reconsidering technology choices. The refined spec represents the agreed-upon technical approach with human validation baked in.
+
+**Phase 4: Execution**
+
+7. **Implementation Plan Doc** — Break down the refined spec into an actionable plan. Include task decomposition, effort estimates, dependency mapping, milestone definitions, and sprint/timeline planning. This bridges the gap between "what we'll build" and "how we'll actually execute it."
+
+8. **Implementation** — Engineers build the solution according to the plan and spec. The detailed planning from previous phases helps implementation proceed smoothly, though real-world discoveries may still require spec updates.
+
+9. **Testing** — The final validation phase ensures the implementation meets requirements through unit tests, integration tests, QA validation, performance testing, and verification against both the PRD objectives and technical spec requirements.
+
+**Key Characteristics:**
+
+- **AI-Augmented but Human-Validated**: The workflow embraces AI assistance for exploration and design while maintaining human oversight at critical junctures. This balances the speed and breadth of AI with the judgment and experience of senior engineers.
+
+- **Separation of Concerns**: The workflow clearly distinguishes between product requirements (PRD), technical design (Spec), and execution planning (Plan Doc). This separation ensures each artifact serves its specific purpose without conflation.
+
+- **Feedback Integration**: Unlike linear waterfall processes, this workflow explicitly includes a feedback loop after the initial spec, acknowledging that first drafts benefit from review and iteration.
+
+- **Research-Driven**: Starting with deep research rather than jumping straight to requirements ensures decisions are grounded in solid understanding of the problem space.
+
+This workflow is particularly well-suited for complex projects where upfront investment in planning pays dividends, teams working with AI coding tools, and organizations that want to leverage AI capabilities while maintaining human control over critical technical decisions.
+
+### Advanced: Self-Correction Chains
+
+Chain prompts so the model reviews its own work, catching errors and refining outputs—especially valuable for high-stakes tasks.
+
+### Optimization Tip
+
+For independent subtasks (like analyzing multiple documents), create separate prompts and run them in parallel for speed.
+
+---
+
+## Long Context Tips
+
+### Key Techniques
+
+**1. Document Placement**
+Place lengthy documents (100K+ tokens) at the beginning of prompts rather than at the end. Queries at the end can improve response quality by up to 30% in tests, especially with complex, multi-document inputs.
+
+**2. Structural Organization**
+Implement XML tags to organize multiple documents clearly. The recommended approach wraps each item in `<document>` tags containing `<document_content>` and `<source>` subtags, enabling better information retrieval.
+
+Example:
+
+```xml
+<documents>
+  <document>
+    <source>Report A</source>
+    <document_content>
+      Content here...
+    </document_content>
+  </document>
+
+  <document>
+    <source>Report B</source>
+    <document_content>
+      Content here...
+    </document_content>
+  </document>
+</documents>
+
+Now analyze these documents and answer: [Your question here]
+```
+
+**3. Quote Grounding**
+Request that the model extract relevant quotes from source materials before completing the primary task. This method helps the model navigate through extraneous content and focus on pertinent information.
+
+### Practical Example
+
+For medical diagnostics, request quotes from patient records placed in `<quotes>` tags, followed by diagnostic analysis in `<info>` tags. This two-step approach ensures responses remain anchored to specific document passages.
+
+### Context Window Advantage
+
+Modern language models support large context windows (100K–1M+ tokens), enabling complex, data-rich analysis across multiple documents simultaneously—making these organizational techniques particularly valuable for sophisticated tasks.
+
+---
+
+## Extended Thinking Tips
+
+### Core Prompting Techniques
+
+**General Over Prescriptive Instructions**
+Rather than providing step-by-step guidance, models often perform better with high-level directives. Ask the model to "think about this thoroughly and in great detail" and "consider multiple approaches" rather than numbering specific steps it must follow.
+
+**Multishot Prompting**
+When you provide examples using XML tags like `<thinking>` or `<scratchpad>`, the model generalizes these patterns to its formal extended thinking process. This helps the model follow similar reasoning trajectories for new problems.
+
+**Instruction Following Enhancement**
+Extended thinking significantly improves how well the model follows instructions by allowing it to reason about them internally before executing them in responses. For complex instructions, breaking them into numbered steps that the model can methodically work through yields better results.
+
+### Advanced Strategies
+
+**Debugging and Steering**
+You can examine the model's thinking output to understand its logic, though this method isn't perfectly reliable. Importantly, you should not pass the model's thinking back as user input, as this degrades performance.
+
+**Long-Form Output Optimization**
+For extensive content generation, explicitly request detailed outputs and increase both thinking budget and maximum token limits. For very long pieces (20,000+ words), request detailed outlines with paragraph-level word counts.
+
+**Verification and Error Reduction**
+Prompt the model to verify its work with test cases before completion. For coding tasks, ask it to run through test scenarios within extended thinking itself.
+
+### Technical Considerations
+
+- Thinking tokens require a minimum budget of 1,024 tokens
+- Extended thinking functions optimally in English
+- With large context windows, thinking budgets can scale significantly higher
+- Traditional chain-of-thought prompting with XML tags works for smaller thinking requirements

package/dist/builtin/workflows/skills/prompt-engineer/references/core_prompting.md

@@ -0,0 +1,137 @@
+# Core Prompting Techniques
+
+This document covers fundamental prompt engineering techniques that form the foundation of effective language model interactions.
+
+## Be Clear and Direct
+
+### Core Principle
+
+Think of the model as "a brilliant but very new employee (with amnesia) who needs explicit instructions." The better you explain what you want, the better the model performs.
+
+### The Golden Rule
+
+Show your prompt to a colleague with minimal context and ask them to follow the instructions. If they're confused, the model likely will be too.
+
+### Key Techniques
+
+**1. Provide Context**
+
+- Explain what the results will be used for
+- Identify the intended audience
+- Describe where the task fits in your workflow
+- Define what successful completion looks like
+
+**2. Be Specific About Output**
+Explicitly state formatting requirements (e.g., "output only code and nothing else")
+
+**3. Use Sequential Instructions**
+Structure requests with numbered lists or bullet points to ensure the model follows your exact process.
+
+### Practical Examples
+
+**Anonymizing Feedback**
+
+- ❌ Vague: "Remove personal information"
+- ✅ Specific: "Replace all names with [NAME], email addresses with [EMAIL], phone numbers with [PHONE], and locations with [LOCATION]"
+
+**Marketing Emails**
+
+- ❌ Unclear: "Write a marketing email"
+- ✅ Detailed: "Write a marketing email to enterprise customers about our new security features. Tone: professional but approachable. Highlight: SSO, audit logs, and compliance certifications. Include a CTA to schedule a demo."
+
+**Incident Reports**
+
+- ❌ Generic: "Summarize this incident"
+- ✅ Terse: "Extract: timestamp, severity, affected systems, root cause, resolution. Output as bullet points only."
+
+### Key Insight
+
+Precision prevents hallucination and ensures the model delivers exactly what you need.
+
+---
+
+## System Prompts and Role Prompting
+
+### Core Technique
+
+Use the `system` parameter to assign the model a specific professional identity. This transforms it from a general assistant into a specialized expert in a particular domain.
+
+### Key Benefits
+
+- **Enhanced accuracy** in complex domains like legal analysis or financial modeling
+- **Tailored tone** adjusted to match the assigned role's communication style
+- **Improved focus** keeping the model aligned with task-specific requirements
+
+### Best Practice
+
+"Use the `system` parameter to set the model's role. Put everything else, like task-specific instructions, in the `user` turn instead."
+
+### Experimentation is Key
+
+Roles can significantly impact outputs. A "data scientist" provides different insights than a "marketing strategist" analyzing identical information. Adding specificity—such as "data scientist specializing in customer insight analysis for Fortune 500 companies"—yields even more tailored results.
+
+### Real-World Examples
+
+**Legal Contract Analysis**
+
+- Without role: Surface-level summaries
+- With role (General Counsel at Fortune 500 tech company): Identifies critical risks like unfavorable indemnification clauses, inadequate liability caps, IP ownership concerns
+
+**Financial Analysis**
+
+- Without role: Basic observations
+- With role (CFO of high-growth SaaS company): Strategic insights including segment performance, margin implications, cash runway calculations, actionable recommendations
+
+---
+
+## Using XML Tags
+
+### Core Purpose
+
+XML tags help the model parse prompts more accurately by clearly separating different components like context, instructions, and examples.
+
+### Key Benefits
+
+1. **Clarity** - Clearly separate different parts of your prompt and ensure your prompt is well structured
+2. **Accuracy** - Reduces misinterpretation errors in prompt components
+3. **Flexibility** - Simplifies modifying or reorganizing prompt sections
+4. **Parseability** - Makes extracting specific response sections easier through post-processing
+
+### Best Practices
+
+**1. Maintain Consistency**
+Apply identical tag names throughout and reference them when discussing content
+
+**2. Utilize Nesting**
+Arrange tags hierarchically for complex information structures
+
+**3. Common Tag Patterns**
+
+```xml
+<context>Background information</context>
+<instructions>What to do</instructions>
+<examples>Sample inputs/outputs</examples>
+<documents>Long-form content</documents>
+<thinking>Model reasoning process</thinking>
+<answer>Final response</answer>
+```
+
+### Advanced Technique
+
+Combining XML tags with multishot prompting or chain of thought methods creates super-structured, high-performance prompts.
+
+### Practical Impact
+
+**Financial Reporting**
+
+- Without tags: Disorganized narrative
+- With tags: Concise, list-formatted reports
+
+**Legal Analysis**
+
+- Without tags: Scattered observations
+- With tags: Organized findings and actionable recommendations
+
+### Important Note
+
+No specific XML tags are canonically required—tag names should align logically with their content.

package/dist/builtin/workflows/skills/prompt-engineer/references/quality_improvement.md

@@ -0,0 +1,193 @@
+# Quality Improvement Techniques
+
+This document covers techniques for improving specific aspects of language model output quality: consistency, factual accuracy, and security.
+
+## Reducing Hallucinations
+
+### Core Definition
+
+Language models can generate factually incorrect or contextually inconsistent text, a problem termed "hallucination." This guide provides strategies to minimize such issues.
+
+### Basic Strategies
+
+**1. Permission to Admit Uncertainty**
+Allow the model to say "I don't know" by explicitly granting permission to acknowledge uncertainty. This straightforward approach substantially reduces false information generation.
+
+Example:
+
+```
+If you don't know the answer or are uncertain, please say so rather than guessing.
+```
+
+**2. Direct Quotation Grounding**
+For very lengthy documents (100K+ tokens) or when working with multiple large documents, request that the model extract verbatim passages before proceeding with analysis. This anchors responses to actual source material rather than inferred content.
+
+Example:
+
+```
+First, find and quote the relevant passages from the document.
+Then, based only on those quotes, provide your analysis.
+```
+
+**3. Citation Verification**
+Make outputs traceable by requiring the model to cite supporting quotes for each claim. The model should then verify claims by locating corroborating evidence; unsupported statements must be removed.
+
+Example:
+
+```
+For each claim you make, provide a direct quote from the source material.
+After drafting your response, verify that each claim has supporting evidence.
+Remove any claims that cannot be substantiated with quotes.
+```
+
+### Advanced Approaches
+
+**Step-by-step reasoning**
+Request the model explain its logic before providing final answers, exposing potentially flawed assumptions
+
+**Multiple-run comparison**
+Execute identical prompts several times and analyze outputs for inconsistencies suggesting hallucinations
+
+**Progressive validation**
+Use prior responses as foundation for follow-up queries asking for verification or expansion of statements
+
+**Information source limitation**
+Explicitly restrict the model to provided materials, excluding general knowledge access
+
+Example:
+
+```
+Use ONLY the information provided in the attached documents.
+Do not use any external knowledge or general information.
+If the documents don't contain the information needed to answer, say so.
+```
+
+### Important Caveat
+
+While these techniques significantly reduce hallucinations, they don't eliminate them entirely. Always validate critical information, especially for high-stakes decisions.
+
+---
+
+## Increasing Consistency
+
+### Core Techniques
+
+**1. Format Specification**
+Define desired output structures using JSON, XML, or custom templates. This approach ensures the model understands all formatting requirements before generating responses.
+
+Example JSON:
+
+```json
+{
+    "sentiment": "positive|negative|neutral",
+    "confidence": "high|medium|low",
+    "key_themes": ["theme1", "theme2"],
+    "summary": "Brief summary here"
+}
+```
+
+Example XML:
+
+```xml
+<analysis>
+  <sentiment>positive|negative|neutral</sentiment>
+  <confidence>high|medium|low</confidence>
+  <key_themes>
+    <theme>theme1</theme>
+    <theme>theme2</theme>
+  </key_themes>
+  <summary>Brief summary here</summary>
+</analysis>
+```
+
+**2. Response Prefilling**
+Begin the Assistant turn with your desired structure. This technique "bypasses the model's default preamble and enforces your structure," making it particularly effective for standardized reports.
+
+Example:
+
+```
+User: Analyze this customer feedback.
+Assistant: {
+```
+
+This forces the model to immediately start with the JSON structure.
+
+**3. Example-Based Constraints**
+Supply concrete examples of desired output. Examples train the model's understanding better than abstract instructions alone.
+
+**4. Retrieval-Grounded Responses**
+For knowledge-dependent tasks, use retrieval mechanisms to anchor the model's replies in fixed information sets. This maintains contextual consistency across multiple interactions.
+
+**5. Prompt Chaining**
+Decompose intricate workflows into sequential, focused subtasks. This prevents inconsistency errors by ensuring "each subtask gets the model's full attention."
+
+### Practical Applications
+
+The guide demonstrates these techniques through real-world scenarios:
+
+- **Customer feedback analysis**: Using JSON structures for consistent categorization
+- **Sales report generation**: Via XML templates for standardized formatting
+- **Competitive intelligence**: With structured formats for comparable analysis
+- **IT support systems**: Leveraging knowledge bases for consistent responses
+
+Each example illustrates how precise specifications and contextual grounding produce reliable, repeatable outputs suitable for scaled operations.
+
+---
+
+## Mitigating Jailbreaks and Prompt Injections
+
+### Core Strategies
+
+**1. Harmlessness Screening**
+Pre-screen user inputs using a lightweight model for content moderation. Have the model evaluate whether submitted content "refers to harmful, illegal, or explicit activities" and respond with Y or N accordingly.
+
+Example:
+
+```
+Evaluate the following user input. Does it refer to harmful, illegal, or explicit activities?
+Respond with only Y or N.
+
+User input: <user_input>{USER_INPUT}</user_input>
+```
+
+**2. Input Validation**
+Filter prompts for jailbreaking patterns. You can use an LLM to create a generalized validation screen by providing known jailbreaking language as examples.
+
+**3. Prompt Engineering**
+Design system prompts that establish clear ethical boundaries. For instance, define organizational values including:
+
+- "Integrity: Never deceive or aid in deception"
+- "Compliance: Refuse any request that violates laws or our policies"
+
+Example system prompt:
+
+```
+You are an AI assistant for [Company]. You must adhere to these values:
+
+1. Integrity: Never deceive users or help them deceive others
+2. Safety: Refuse requests for harmful, illegal, or explicit content
+3. Compliance: Follow all applicable laws and company policies
+4. Privacy: Protect user data and confidential information
+
+If a request violates these values, politely explain why you cannot help and suggest an alternative approach if possible.
+```
+
+**4. User Accountability**
+Monitor for repeated abuse attempts. If a user "triggers the same kind of refusal multiple times," communicate that their actions violate usage policies and take appropriate enforcement action.
+
+**5. Continuous Monitoring**
+Regularly analyze outputs for jailbreaking indicators and use findings to refine your validation strategies iteratively.
+
+### Advanced Approach: Layered Protection
+
+Combine multiple safeguards for enterprise applications. For example, in a financial services context, the system should sequentially:
+
+1. Screen queries for compliance
+2. Process legitimate requests
+3. Refuse non-compliant ones with specific explanations
+
+This multi-layered approach creates comprehensive defense without relying on any single security mechanism.
+
+### Important Note
+
+No single technique provides complete protection. A defense-in-depth approach combining multiple strategies provides the most robust security against jailbreaks and prompt injections.