@sylix/coworker 2.0.11 → 2.0.12

package/dist/skills/defaults/llm-application/prompt-engineering-patterns.md

@@ -0,0 +1,449 @@
+---
+name: prompt-engineering-patterns
+description: Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability in production
+---
+
+# Prompt Engineering Patterns
+
+Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability.
+
+## When to Use This Skill
+
+- Designing complex prompts for production LLM applications
+- Optimizing prompt performance and consistency
+- Implementing structured reasoning patterns (chain-of-thought, tree-of-thought)
+- Building few-shot learning systems with dynamic example selection
+- Creating reusable prompt templates with variable interpolation
+- Debugging and refining prompts that produce inconsistent outputs
+- Implementing system prompts for specialized AI assistants
+- Using structured outputs (JSON mode) for reliable parsing
+
+## Core Capabilities
+
+### 1. Few-Shot Learning
+
+- Example selection strategies (semantic similarity, diversity sampling)
+- Balancing example count with context window constraints
+- Constructing effective demonstrations with input-output pairs
+- Dynamic example retrieval from knowledge bases
+- Handling edge cases through strategic example selection
+
+### 2. Chain-of-Thought Prompting
+
+- Step-by-step reasoning elicitation
+- Zero-shot CoT with "Let's think step by step"
+- Few-shot CoT with reasoning traces
+- Self-consistency techniques (sampling multiple reasoning paths)
+- Verification and validation steps
+
+### 3. Structured Outputs
+
+- JSON mode for reliable parsing
+- Pydantic schema enforcement
+- Type-safe response handling
+- Error handling for malformed outputs
+
+### 4. Prompt Optimization
+
+- Iterative refinement workflows
+- A/B testing prompt variations
+- Measuring prompt performance metrics (accuracy, consistency, latency)
+- Reducing token usage while maintaining quality
+- Handling edge cases and failure modes
+
+### 5. Template Systems
+
+- Variable interpolation and formatting
+- Conditional prompt sections
+- Multi-turn conversation templates
+- Role-based prompt composition
+- Modular prompt components
+
+### 6. System Prompt Design
+
+- Setting model behavior and constraints
+- Defining output formats and structure
+- Establishing role and expertise
+- Safety guidelines and content policies
+- Context setting and background information
+
+## Quick Start
+
+```python
+from langchain_anthropic import ChatAnthropic
+from langchain_core.prompts import ChatPromptTemplate
+from pydantic import BaseModel, Field
+
+class SQLQuery(BaseModel):
+    query: str = Field(description="The SQL query")
+    explanation: str = Field(description="Brief explanation of what the query does")
+    tables_used: list[str] = Field(description="List of tables referenced")
+
+llm = ChatAnthropic(model="claude-sonnet-4-6")
+structured_llm = llm.with_structured_output(SQLQuery)
+
+prompt = ChatPromptTemplate.from_messages([
+    ("system", """You are an expert SQL developer. Generate efficient, secure SQL queries.
+    Always use parameterized queries to prevent SQL injection.
+    Explain your reasoning briefly."""),
+    ("user", "Convert this to SQL: {query}")
+])
+
+chain = prompt | structured_llm
+
+result = await chain.ainvoke({
+    "query": "Find all users who registered in the last 30 days"
+})
+print(result.query)
+```
+
+## Key Patterns
+
+### Pattern 1: Structured Output with Pydantic
+
+```python
+from anthropic import Anthropic
+from pydantic import BaseModel, Field
+from typing import Literal
+import json
+
+class SentimentAnalysis(BaseModel):
+    sentiment: Literal["positive", "negative", "neutral"]
+    confidence: float = Field(ge=0, le=1)
+    key_phrases: list[str]
+    reasoning: str
+
+async def analyze_sentiment(text: str) -> SentimentAnalysis:
+    client = Anthropic()
+
+    message = client.messages.create(
+        model="claude-sonnet-4-6",
+        max_tokens=500,
+        messages=[{
+            "role": "user",
+            "content": f"""Analyze the sentiment of this text.
+
+Text: {text}
+
+Respond with JSON matching this schema:
+{{
+    "sentiment": "positive" | "negative" | "neutral",
+    "confidence": 0.0-1.0,
+    "key_phrases": ["phrase1", "phrase2"],
+    "reasoning": "brief explanation"
+}}"""
+        }]
+    )
+
+    return SentimentAnalysis(**json.loads(message.content[0].text))
+```
+
+### Pattern 2: Chain-of-Thought with Self-Verification
+
+```python
+from langchain_core.prompts import ChatPromptTemplate
+
+cot_prompt = ChatPromptTemplate.from_template("""
+Solve this problem step by step.
+
+Problem: {problem}
+
+Instructions:
+1. Break down the problem into clear steps
+2. Work through each step showing your reasoning
+3. State your final answer
+4. Verify your answer by checking it against the original problem
+
+Format your response as:
+## Steps
+[Your step-by-step reasoning]
+
+## Answer
+[Your final answer]
+
+## Verification
+[Check that your answer is correct]
+""")
+```
+
+### Pattern 3: Few-Shot with Dynamic Example Selection
+
+```python
+from langchain_voyageai import VoyageAIEmbeddings
+from langchain_core.example_selectors import SemanticSimilarityExampleSelector
+from langchain_chroma import Chroma
+
+example_selector = SemanticSimilarityExampleSelector.from_examples(
+    examples=[
+        {"input": "How do I reset my password?", "output": "Go to Settings > Security > Reset Password"},
+        {"input": "Where can I see my order history?", "output": "Navigate to Account > Orders"},
+        {"input": "How do I contact support?", "output": "Click Help > Contact Us or email support@example.com"},
+    ],
+    embeddings=VoyageAIEmbeddings(model="voyage-3-large"),
+    vectorstore_cls=Chroma,
+    k=2
+)
+
+async def get_few_shot_prompt(query: str) -> str:
+    examples = await example_selector.aselect_examples({"input": query})
+
+    examples_text = "\n".join(
+        f"User: {ex['input']}\nAssistant: {ex['output']}"
+        for ex in examples
+    )
+
+    return f"""You are a helpful customer support assistant.
+
+Here are some example interactions:
+{examples_text}
+
+Now respond to this query:
+User: {query}
+Assistant:"""
+```
+
+### Pattern 4: Progressive Disclosure
+
+```python
+PROMPT_LEVELS = {
+    "simple": "Summarize this article: {text}",
+
+    "constrained": """Summarize this article in 3 bullet points, focusing on:
+- Key findings
+- Main conclusions
+- Practical implications
+
+Article: {text}""",
+
+    "reasoning": """Read this article carefully.
+1. First, identify the main topic and thesis
+2. Then, extract the key supporting points
+3. Finally, summarize in 3 bullet points
+
+Article: {text}
+
+Summary:""",
+
+    "few_shot": """Read articles and provide concise summaries.
+
+Example:
+Article: "New research shows that regular exercise can reduce anxiety by up to 40%..."
+Summary:
+• Regular exercise reduces anxiety by up to 40%
+• 30 minutes of moderate activity 3x/week is sufficient
+• Benefits appear within 2 weeks of starting
+
+Now summarize this article:
+Article: {text}
+
+Summary:"""
+}
+```
+
+### Pattern 5: Error Recovery and Fallback
+
+```python
+from pydantic import BaseModel, ValidationError
+import json
+
+class ResponseWithConfidence(BaseModel):
+    answer: str
+    confidence: float
+    sources: list[str]
+    alternative_interpretations: list[str] = []
+
+ERROR_RECOVERY_PROMPT = """
+Answer the question based on the context provided.
+
+Context: {context}
+Question: {question}
+
+Instructions:
+1. If you can answer confidently (>0.8), provide a direct answer
+2. If you're somewhat confident (0.5-0.8), provide your best answer with caveats
+3. If you're uncertain (<0.5), explain what information is missing
+4. Always provide alternative interpretations if the question is ambiguous
+
+Respond in JSON:
+{{
+    "answer": "your answer or 'I cannot determine this from the context'",
+    "confidence": 0.0-1.0,
+    "sources": ["relevant context excerpts"],
+    "alternative_interpretations": ["if question is ambiguous"]
+}}
+"""
+
+async def answer_with_fallback(context: str, question: str, llm) -> ResponseWithConfidence:
+    prompt = ERROR_RECOVERY_PROMPT.format(context=context, question=question)
+
+    try:
+        response = await llm.ainvoke(prompt)
+        return ResponseWithConfidence(**json.loads(response.content))
+    except (json.JSONDecodeError, ValidationError) as e:
+        simple_prompt = f"Based on: {context}\n\nAnswer: {question}"
+        simple_response = await llm.ainvoke(simple_prompt)
+        return ResponseWithConfidence(
+            answer=simple_response.content,
+            confidence=0.5,
+            sources=["fallback extraction"],
+            alternative_interpretations=[]
+        )
+```
+
+### Pattern 6: Role-Based System Prompts
+
+```python
+SYSTEM_PROMPTS = {
+    "analyst": """You are a senior data analyst with expertise in SQL, Python, and business intelligence.
+
+Your responsibilities:
+- Write efficient, well-documented queries
+- Explain your analysis methodology
+- Highlight key insights and recommendations
+- Flag any data quality concerns
+
+Communication style:
+- Be precise and technical when discussing methodology
+- Translate technical findings into business impact
+- Use clear visualizations when helpful""",
+
+    "assistant": """You are a helpful AI assistant focused on accuracy and clarity.
+
+Core principles:
+- Always cite sources when making factual claims
+- Acknowledge uncertainty rather than guessing
+- Ask clarifying questions when the request is ambiguous
+- Provide step-by-step explanations for complex topics
+
+Constraints:
+- Do not provide medical, legal, or financial advice
+- Redirect harmful requests appropriately
+- Protect user privacy""",
+
+    "code_reviewer": """You are a senior software engineer conducting code reviews.
+
+Review criteria:
+- Correctness: Does the code work as intended?
+- Security: Are there any vulnerabilities?
+- Performance: Are there efficiency concerns?
+- Maintainability: Is the code readable and well-structured?
+- Best practices: Does it follow language idioms?
+
+Output format:
+1. Summary assessment (approve/request changes)
+2. Critical issues (must fix)
+3. Suggestions (nice to have)
+4. Positive feedback (what's done well)"""
+}
+```
+
+## Integration Patterns
+
+### With RAG Systems
+
+```python
+RAG_PROMPT = """You are a knowledgeable assistant that answers questions based on provided context.
+
+Context (retrieved from knowledge base):
+{context}
+
+Instructions:
+1. Answer ONLY based on the provided context
+2. If the context doesn't contain the answer, say "I don't have information about that in my knowledge base"
+3. Cite specific passages using [1], [2] notation
+4. If the question is ambiguous, ask for clarification
+
+Question: {question}
+
+Answer:"""
+```
+
+### With Validation and Verification
+
+```python
+VALIDATED_PROMPT = """Complete the following task:
+
+Task: {task}
+
+After generating your response, verify it meets ALL these criteria:
+✓ Directly addresses the original request
+✓ Contains no factual errors
+✓ Is appropriately detailed (not too brief, not too verbose)
+✓ Uses proper formatting
+✓ Is safe and appropriate
+
+If verification fails on any criterion, revise before responding.
+
+Response:"""
+```
+
+## Performance Optimization
+
+### Token Efficiency
+
+```python
+# Before: Verbose prompt (150+ tokens)
+verbose_prompt = """
+I would like you to please take the following text and provide me with a comprehensive
+summary of the main points. The summary should capture the key ideas and important details
+while being concise and easy to understand.
+"""
+
+# After: Concise prompt (30 tokens)
+concise_prompt = """Summarize the key points concisely:
+
+{text}
+
+Summary:"""
+```
+
+### Caching Common Prefixes
+
+```python
+from anthropic import Anthropic
+
+client = Anthropic()
+
+response = client.messages.create(
+    model="claude-sonnet-4-6",
+    max_tokens=1000,
+    system=[
+        {
+            "type": "text",
+            "text": LONG_SYSTEM_PROMPT,
+            "cache_control": {"type": "ephemeral"}
+        }
+    ],
+    messages=[{"role": "user", "content": user_query}]
+)
+```
+
+## Best Practices
+
+1. **Be Specific**: Vague prompts produce inconsistent results
+2. **Show, Don't Tell**: Examples are more effective than descriptions
+3. **Use Structured Outputs**: Enforce schemas with Pydantic for reliability
+4. **Test Extensively**: Evaluate on diverse, representative inputs
+5. **Iterate Rapidly**: Small changes can have large impacts
+6. **Monitor Performance**: Track metrics in production
+7. **Version Control**: Treat prompts as code with proper versioning
+8. **Document Intent**: Explain why prompts are structured as they are
+
+## Common Pitfalls
+
+- **Over-engineering**: Starting with complex prompts before trying simple ones
+- **Example pollution**: Using examples that don't match the target task
+- **Context overflow**: Exceeding token limits with excessive examples
+- **Ambiguous instructions**: Leaving room for multiple interpretations
+- **Ignoring edge cases**: Not testing on unusual or boundary inputs
+- **No error handling**: Assuming outputs will always be well-formed
+- **Hardcoded values**: Not parameterizing prompts for reuse
+
+## Success Metrics
+
+- **Accuracy**: Correctness of outputs
+- **Consistency**: Reproducibility across similar inputs
+- **Latency**: Response time (P50, P95, P99)
+- **Token Usage**: Average tokens per request
+- **Success Rate**: Percentage of valid, parseable outputs
+- **User Satisfaction**: Ratings and feedback

package/dist/skills/defaults/llm-application/prompt-engineering.md

@@ -0,0 +1,219 @@
+---
+name: prompt-engineering-patterns
+description: Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability in production. Use when optimizing prompts, improving LLM outputs, or designing production prompt templates.
+---
+
+# Prompt Engineering Patterns
+
+Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability.
+
+## When to Use This Skill
+
+- Designing complex prompts for production LLM applications
+- Optimizing prompt performance and consistency
+- Implementing structured reasoning patterns (chain-of-thought, tree-of-thought)
+- Building few-shot learning systems with dynamic example selection
+- Creating reusable prompt templates with variable interpolation
+- Debugging and refining prompts that produce inconsistent outputs
+- Implementing system prompts for specialized AI assistants
+- Using structured outputs (JSON mode) for reliable parsing
+
+## Core Capabilities
+
+### 1. Few-Shot Learning
+
+- Example selection strategies (semantic similarity, diversity sampling)
+- Balancing example count with context window constraints
+- Constructing effective demonstrations with input-output pairs
+- Dynamic example retrieval from knowledge bases
+- Handling edge cases through strategic example selection
+
+### 2. Chain-of-Thought Prompting
+
+- Step-by-step reasoning elicitation
+- Zero-shot CoT with "Let's think step by step"
+- Few-shot CoT with reasoning traces
+- Self-consistency techniques (sampling multiple reasoning paths)
+- Verification and validation steps
+
+### 3. Structured Outputs
+
+- JSON mode for reliable parsing
+- Pydantic schema enforcement
+- Type-safe response handling
+- Error handling for malformed outputs
+
+### 4. Prompt Optimization
+
+- Iterative refinement workflows
+- A/B testing prompt variations
+- Measuring prompt performance metrics (accuracy, consistency, latency)
+- Reducing token usage while maintaining quality
+- Handling edge cases and failure modes
+
+### 5. Template Systems
+
+- Variable interpolation and formatting
+- Conditional prompt sections
+- Multi-turn conversation templates
+- Role-based prompt composition
+- Modular prompt components
+
+### 6. System Prompt Design
+
+- Setting model behavior and constraints
+- Defining output formats and structure
+- Establishing role and expertise
+- Safety guidelines and content policies
+- Context setting and background information
+
+## Quick Start
+
+```python
+from langchain_anthropic import ChatAnthropic
+from langchain_core.prompts import ChatPromptTemplate
+from pydantic import BaseModel, Field
+
+class SQLQuery(BaseModel):
+    query: str = Field(description="The SQL query")
+    explanation: str = Field(description="Brief explanation of what the query does")
+    tables_used: list[str] = Field(description="List of tables referenced")
+
+llm = ChatAnthropic(model="claude-sonnet-4-6")
+structured_llm = llm.with_structured_output(SQLQuery)
+
+prompt = ChatPromptTemplate.from_messages([
+    ("system", """You are an expert SQL developer. Generate efficient, secure SQL queries.
+    Always use parameterized queries to prevent SQL injection.
+    Explain your reasoning briefly."""),
+    ("user", "Convert this SQL: {query}")
+])
+
+chain = prompt | structured_llm
+
+result = await chain.ainvoke({
+    "query": "Find all users who registered in the last 30 days"
+})
+```
+
+## Key Patterns
+
+### Pattern 1: Structured Output with Pydantic
+
+```python
+from anthropic import Anthropic
+from pydantic import BaseModel, Field
+from typing import Literal
+import json
+
+class SentimentAnalysis(BaseModel):
+    sentiment: Literal["positive", "negative", "neutral"]
+    confidence: float = Field(ge=0, le=1)
+    key_phrases: list[str]
+    reasoning: str
+
+async def analyze_sentiment(text: str) -> SentimentAnalysis:
+    client = Anthropic()
+    message = client.messages.create(
+        model="claude-sonnet-4-6",
+        max_tokens=500,
+        messages=[{
+            "role": "user",
+            "content": f"""Analyze sentiment of: {text}
+            Respond with JSON:
+            {{
+                "sentiment": "positive" | "negative" | "neutral",
+                "confidence": 0.0-1.0,
+                "key_phrases": ["phrase1"],
+                "reasoning": "explanation"
+            }}"""
+        }]
+    )
+    return SentimentAnalysis(**json.loads(message.content[0].text))
+```
+
+### Pattern 2: Chain-of-Thought with Self-Verification
+
+```python
+cot_prompt = """
+Solve this problem step by step.
+
+Problem: {problem}
+
+Instructions:
+1. Break down into clear steps
+2. Work through each step showing reasoning
+3. State final answer
+4. Verify by checking against original problem
+
+Format:
+## Steps
+[reasoning]
+
+## Answer
+[final answer]
+
+## Verification
+[check correctness]
+"""
+```
+
+### Pattern 3: Few-Shot with Dynamic Example Selection
+
+```python
+from langchain_voyageai import VoyageAIEmbeddings
+from langchain_core.example_selectors import SemanticSimilarityExampleSelector
+
+example_selector = SemanticSimilarityExampleSelector.from_examples(
+    examples=[
+        {"input": "How do I reset password?", "output": "Settings > Security > Reset Password"},
+        {"input": "Where is my order?", "output": "Account > Orders"},
+    ],
+    embeddings=VoyageAIEmbeddings(model="voyage-3-large"),
+    vectorstore_cls=Chroma,
+    k=2
+)
+```
+
+### Pattern 4: Progressive Disclosure
+
+```python
+PROMPT_LEVELS = {
+    "simple": "Summarize: {text}",
+    "constrained": "Summarize in 3 bullets: {text}",
+    "reasoning": "Read carefully, identify thesis, summarize: {text}",
+    "few_shot": """Example: Article: "Research shows exercise reduces anxiety..."
+Summary: • Reduces anxiety up to 40%\n• 30 min 3x/week sufficient\n• Benefits within 2 weeks
+
+Now: {text}"""
+}
+```
+
+### Pattern 5: Error Recovery and Fallback
+
+```python
+ERROR_RECOVERY_PROMPT = """
+Answer based on context.
+
+Context: {context}
+Question: {question}
+
+Instructions:
+1. Confident (>0.8): direct answer
+2. Somewhat (0.5-0.8): best answer with caveats
+3. Uncertain (<0.5): explain missing info
+
+JSON response:
+{{"answer": "...", "confidence": 0.0-1.0, "sources": []}}
+"""
+```
+
+## Best Practices
+
+1. **Use structured outputs** - Pydantic schemas for reliable parsing
+2. **Implement chain-of-thought** - For complex reasoning tasks
+3. **Few-shot with examples** - But limit to 3-5 to save tokens
+4. **Iterate and test** - A/B test prompt variations
+5. **Handle failures gracefully** - Parse errors, fallbacks
+6. **Token budgeting** - Balance quality vs context length
+7. **System prompts matter** - Set behavior explicitly