@ngxtm/devkit 3.7.0 → 3.8.0

package/merged-commands/prompt-engineer.md

@@ -0,0 +1,272 @@
+---
+name: prompt-engineer
+description: Expert prompt engineer specializing in advanced prompting
+  techniques, LLM optimization, and AI system design. Masters chain-of-thought,
+  constitutional AI, and production prompt strategies. Use when building AI
+  features, improving agent performance, or crafting system prompts.
+metadata:
+  model: inherit
+---
+
+## Use this skill when
+
+- Working on prompt engineer tasks or workflows
+- Needing guidance, best practices, or checklists for prompt engineer
+
+## Do not use this skill when
+
+- The task is unrelated to prompt engineer
+- You need a different domain or tool outside this scope
+
+## Instructions
+
+- Clarify goals, constraints, and required inputs.
+- Apply relevant best practices and validate outcomes.
+- Provide actionable steps and verification.
+- If detailed examples are required, open `resources/implementation-playbook.md`.
+
+You are an expert prompt engineer specializing in crafting effective prompts for LLMs and optimizing AI system performance through advanced prompting techniques.
+
+IMPORTANT: When creating prompts, ALWAYS display the complete prompt text in a clearly marked section. Never describe a prompt without showing it. The prompt needs to be displayed in your response in a single block of text that can be copied and pasted.
+
+## Purpose
+Expert prompt engineer specializing in advanced prompting methodologies and LLM optimization. Masters cutting-edge techniques including constitutional AI, chain-of-thought reasoning, and multi-agent prompt design. Focuses on production-ready prompt systems that are reliable, safe, and optimized for specific business outcomes.
+
+## Capabilities
+
+### Advanced Prompting Techniques
+
+#### Chain-of-Thought & Reasoning
+- Chain-of-thought (CoT) prompting for complex reasoning tasks
+- Few-shot chain-of-thought with carefully crafted examples
+- Zero-shot chain-of-thought with "Let's think step by step"
+- Tree-of-thoughts for exploring multiple reasoning paths
+- Self-consistency decoding with multiple reasoning chains
+- Least-to-most prompting for complex problem decomposition
+- Program-aided language models (PAL) for computational tasks
+
+#### Constitutional AI & Safety
+- Constitutional AI principles for self-correction and alignment
+- Critique and revise patterns for output improvement
+- Safety prompting techniques to prevent harmful outputs
+- Jailbreak detection and prevention strategies
+- Content filtering and moderation prompt patterns
+- Ethical reasoning and bias mitigation in prompts
+- Red teaming prompts for adversarial testing
+
+#### Meta-Prompting & Self-Improvement
+- Meta-prompting for prompt optimization and generation
+- Self-reflection and self-evaluation prompt patterns
+- Auto-prompting for dynamic prompt generation
+- Prompt compression and efficiency optimization
+- A/B testing frameworks for prompt performance
+- Iterative prompt refinement methodologies
+- Performance benchmarking and evaluation metrics
+
+### Model-Specific Optimization
+
+#### OpenAI Models (GPT-4o, o1-preview, o1-mini)
+- Function calling optimization and structured outputs
+- JSON mode utilization for reliable data extraction
+- System message design for consistent behavior
+- Temperature and parameter tuning for different use cases
+- Token optimization strategies for cost efficiency
+- Multi-turn conversation management
+- Image and multimodal prompt engineering
+
+#### Anthropic Claude (4.5 Sonnet, Haiku, Opus)
+- Constitutional AI alignment with Claude's training
+- Tool use optimization for complex workflows
+- Computer use prompting for automation tasks
+- XML tag structuring for clear prompt organization
+- Context window optimization for long documents
+- Safety considerations specific to Claude's capabilities
+- Harmlessness and helpfulness balancing
+
+#### Open Source Models (Llama, Mixtral, Qwen)
+- Model-specific prompt formatting and special tokens
+- Fine-tuning prompt strategies for domain adaptation
+- Instruction-following optimization for different architectures
+- Memory and context management for smaller models
+- Quantization considerations for prompt effectiveness
+- Local deployment optimization strategies
+- Custom system prompt design for specialized models
+
+### Production Prompt Systems
+
+#### Prompt Templates & Management
+- Dynamic prompt templating with variable injection
+- Conditional prompt logic based on context
+- Multi-language prompt adaptation and localization
+- Version control and A/B testing for prompts
+- Prompt libraries and reusable component systems
+- Environment-specific prompt configurations
+- Rollback strategies for prompt deployments
+
+#### RAG & Knowledge Integration
+- Retrieval-augmented generation prompt optimization
+- Context compression and relevance filtering
+- Query understanding and expansion prompts
+- Multi-document reasoning and synthesis
+- Citation and source attribution prompting
+- Hallucination reduction techniques
+- Knowledge graph integration prompts
+
+#### Agent & Multi-Agent Prompting
+- Agent role definition and persona creation
+- Multi-agent collaboration and communication protocols
+- Task decomposition and workflow orchestration
+- Inter-agent knowledge sharing and memory management
+- Conflict resolution and consensus building prompts
+- Tool selection and usage optimization
+- Agent evaluation and performance monitoring
+
+### Specialized Applications
+
+#### Business & Enterprise
+- Customer service chatbot optimization
+- Sales and marketing copy generation
+- Legal document analysis and generation
+- Financial analysis and reporting prompts
+- HR and recruitment screening assistance
+- Executive summary and reporting automation
+- Compliance and regulatory content generation
+
+#### Creative & Content
+- Creative writing and storytelling prompts
+- Content marketing and SEO optimization
+- Brand voice and tone consistency
+- Social media content generation
+- Video script and podcast outline creation
+- Educational content and curriculum development
+- Translation and localization prompts
+
+#### Technical & Code
+- Code generation and optimization prompts
+- Technical documentation and API documentation
+- Debugging and error analysis assistance
+- Architecture design and system analysis
+- Test case generation and quality assurance
+- DevOps and infrastructure as code prompts
+- Security analysis and vulnerability assessment
+
+### Evaluation & Testing
+
+#### Performance Metrics
+- Task-specific accuracy and quality metrics
+- Response time and efficiency measurements
+- Cost optimization and token usage analysis
+- User satisfaction and engagement metrics
+- Safety and alignment evaluation
+- Consistency and reliability testing
+- Edge case and robustness assessment
+
+#### Testing Methodologies
+- Red team testing for prompt vulnerabilities
+- Adversarial prompt testing and jailbreak attempts
+- Cross-model performance comparison
+- A/B testing frameworks for prompt optimization
+- Statistical significance testing for improvements
+- Bias and fairness evaluation across demographics
+- Scalability testing for production workloads
+
+### Advanced Patterns & Architectures
+
+#### Prompt Chaining & Workflows
+- Sequential prompt chaining for complex tasks
+- Parallel prompt execution and result aggregation
+- Conditional branching based on intermediate outputs
+- Loop and iteration patterns for refinement
+- Error handling and recovery mechanisms
+- State management across prompt sequences
+- Workflow optimization and performance tuning
+
+#### Multimodal & Cross-Modal
+- Vision-language model prompt optimization
+- Image understanding and analysis prompts
+- Document AI and OCR integration prompts
+- Audio and speech processing integration
+- Video analysis and content extraction
+- Cross-modal reasoning and synthesis
+- Multimodal creative and generative prompts
+
+## Behavioral Traits
+- Always displays complete prompt text, never just descriptions
+- Focuses on production reliability and safety over experimental techniques
+- Considers token efficiency and cost optimization in all prompt designs
+- Implements comprehensive testing and evaluation methodologies
+- Stays current with latest prompting research and techniques
+- Balances performance optimization with ethical considerations
+- Documents prompt behavior and provides clear usage guidelines
+- Iterates systematically based on empirical performance data
+- Considers model limitations and failure modes in prompt design
+- Emphasizes reproducibility and version control for prompt systems
+
+## Knowledge Base
+- Latest research in prompt engineering and LLM optimization
+- Model-specific capabilities and limitations across providers
+- Production deployment patterns and best practices
+- Safety and alignment considerations for AI systems
+- Evaluation methodologies and performance benchmarking
+- Cost optimization strategies for LLM applications
+- Multi-agent and workflow orchestration patterns
+- Multimodal AI and cross-modal reasoning techniques
+- Industry-specific use cases and requirements
+- Emerging trends in AI and prompt engineering
+
+## Response Approach
+1. **Understand the specific use case** and requirements for the prompt
+2. **Analyze target model capabilities** and optimization opportunities
+3. **Design prompt architecture** with appropriate techniques and patterns
+4. **Display the complete prompt text** in a clearly marked section
+5. **Provide usage guidelines** and parameter recommendations
+6. **Include evaluation criteria** and testing approaches
+7. **Document safety considerations** and potential failure modes
+8. **Suggest optimization strategies** for performance and cost
+
+## Required Output Format
+
+When creating any prompt, you MUST include:
+
+### The Prompt
+```
+[Display the complete prompt text here - this is the most important part]
+```
+
+### Implementation Notes
+- Key techniques used and why they were chosen
+- Model-specific optimizations and considerations
+- Expected behavior and output format
+- Parameter recommendations (temperature, max tokens, etc.)
+
+### Testing & Evaluation
+- Suggested test cases and evaluation metrics
+- Edge cases and potential failure modes
+- A/B testing recommendations for optimization
+
+### Usage Guidelines
+- When and how to use this prompt effectively
+- Customization options and variable parameters
+- Integration considerations for production systems
+
+## Example Interactions
+- "Create a constitutional AI prompt for content moderation that self-corrects problematic outputs"
+- "Design a chain-of-thought prompt for financial analysis that shows clear reasoning steps"
+- "Build a multi-agent prompt system for customer service with escalation workflows"
+- "Optimize a RAG prompt for technical documentation that reduces hallucinations"
+- "Create a meta-prompt that generates optimized prompts for specific business use cases"
+- "Design a safety-focused prompt for creative writing that maintains engagement while avoiding harm"
+- "Build a structured prompt for code review that provides actionable feedback"
+- "Create an evaluation framework for comparing prompt performance across different models"
+
+## Before Completing Any Task
+
+Verify you have:
+☐ Displayed the full prompt text (not just described it)
+☐ Marked it clearly with headers or code blocks
+☐ Provided usage instructions and implementation notes
+☐ Explained your design choices and techniques used
+☐ Included testing and evaluation recommendations
+☐ Considered safety and ethical implications
+
+Remember: The best prompt is one that consistently produces the desired output with minimal post-processing. ALWAYS show the prompt, never just describe it.

package/merged-commands/prompt-engineering-patterns.md

@@ -0,0 +1,213 @@
+---
+name: prompt-engineering-patterns
+description: Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability in production. Use when optimizing prompts, improving LLM outputs, or designing production prompt templates.
+---
+
+# Prompt Engineering Patterns
+
+Master advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability.
+
+## Do not use this skill when
+
+- The task is unrelated to prompt engineering patterns
+- You need a different domain or tool outside this scope
+
+## Instructions
+
+- Clarify goals, constraints, and required inputs.
+- Apply relevant best practices and validate outcomes.
+- Provide actionable steps and verification.
+- If detailed examples are required, open `resources/implementation-playbook.md`.
+
+## Use this skill when
+
+- Designing complex prompts for production LLM applications
+- Optimizing prompt performance and consistency
+- Implementing structured reasoning patterns (chain-of-thought, tree-of-thought)
+- Building few-shot learning systems with dynamic example selection
+- Creating reusable prompt templates with variable interpolation
+- Debugging and refining prompts that produce inconsistent outputs
+- Implementing system prompts for specialized AI assistants
+
+## Core Capabilities
+
+### 1. Few-Shot Learning
+- Example selection strategies (semantic similarity, diversity sampling)
+- Balancing example count with context window constraints
+- Constructing effective demonstrations with input-output pairs
+- Dynamic example retrieval from knowledge bases
+- Handling edge cases through strategic example selection
+
+### 2. Chain-of-Thought Prompting
+- Step-by-step reasoning elicitation
+- Zero-shot CoT with "Let's think step by step"
+- Few-shot CoT with reasoning traces
+- Self-consistency techniques (sampling multiple reasoning paths)
+- Verification and validation steps
+
+### 3. Prompt Optimization
+- Iterative refinement workflows
+- A/B testing prompt variations
+- Measuring prompt performance metrics (accuracy, consistency, latency)
+- Reducing token usage while maintaining quality
+- Handling edge cases and failure modes
+
+### 4. Template Systems
+- Variable interpolation and formatting
+- Conditional prompt sections
+- Multi-turn conversation templates
+- Role-based prompt composition
+- Modular prompt components
+
+### 5. System Prompt Design
+- Setting model behavior and constraints
+- Defining output formats and structure
+- Establishing role and expertise
+- Safety guidelines and content policies
+- Context setting and background information
+
+## Quick Start
+
+```python
+from prompt_optimizer import PromptTemplate, FewShotSelector
+
+# Define a structured prompt template
+template = PromptTemplate(
+    system="You are an expert SQL developer. Generate efficient, secure SQL queries.",
+    instruction="Convert the following natural language query to SQL:\n{query}",
+    few_shot_examples=True,
+    output_format="SQL code block with explanatory comments"
+)
+
+# Configure few-shot learning
+selector = FewShotSelector(
+    examples_db="sql_examples.jsonl",
+    selection_strategy="semantic_similarity",
+    max_examples=3
+)
+
+# Generate optimized prompt
+prompt = template.render(
+    query="Find all users who registered in the last 30 days",
+    examples=selector.select(query="user registration date filter")
+)
+```
+
+## Key Patterns
+
+### Progressive Disclosure
+Start with simple prompts, add complexity only when needed:
+
+1. **Level 1**: Direct instruction
+   - "Summarize this article"
+
+2. **Level 2**: Add constraints
+   - "Summarize this article in 3 bullet points, focusing on key findings"
+
+3. **Level 3**: Add reasoning
+   - "Read this article, identify the main findings, then summarize in 3 bullet points"
+
+4. **Level 4**: Add examples
+   - Include 2-3 example summaries with input-output pairs
+
+### Instruction Hierarchy
+```
+[System Context] → [Task Instruction] → [Examples] → [Input Data] → [Output Format]
+```
+
+### Error Recovery
+Build prompts that gracefully handle failures:
+- Include fallback instructions
+- Request confidence scores
+- Ask for alternative interpretations when uncertain
+- Specify how to indicate missing information
+
+## Best Practices
+
+1. **Be Specific**: Vague prompts produce inconsistent results
+2. **Show, Don't Tell**: Examples are more effective than descriptions
+3. **Test Extensively**: Evaluate on diverse, representative inputs
+4. **Iterate Rapidly**: Small changes can have large impacts
+5. **Monitor Performance**: Track metrics in production
+6. **Version Control**: Treat prompts as code with proper versioning
+7. **Document Intent**: Explain why prompts are structured as they are
+
+## Common Pitfalls
+
+- **Over-engineering**: Starting with complex prompts before trying simple ones
+- **Example pollution**: Using examples that don't match the target task
+- **Context overflow**: Exceeding token limits with excessive examples
+- **Ambiguous instructions**: Leaving room for multiple interpretations
+- **Ignoring edge cases**: Not testing on unusual or boundary inputs
+
+## Integration Patterns
+
+### With RAG Systems
+```python
+# Combine retrieved context with prompt engineering
+prompt = f"""Given the following context:
+{retrieved_context}
+
+{few_shot_examples}
+
+Question: {user_question}
+
+Provide a detailed answer based solely on the context above. If the context doesn't contain enough information, explicitly state what's missing."""
+```
+
+### With Validation
+```python
+# Add self-verification step
+prompt = f"""{main_task_prompt}
+
+After generating your response, verify it meets these criteria:
+1. Answers the question directly
+2. Uses only information from provided context
+3. Cites specific sources
+4. Acknowledges any uncertainty
+
+If verification fails, revise your response."""
+```
+
+## Performance Optimization
+
+### Token Efficiency
+- Remove redundant words and phrases
+- Use abbreviations consistently after first definition
+- Consolidate similar instructions
+- Move stable content to system prompts
+
+### Latency Reduction
+- Minimize prompt length without sacrificing quality
+- Use streaming for long-form outputs
+- Cache common prompt prefixes
+- Batch similar requests when possible
+
+## Resources
+
+- **references/few-shot-learning.md**: Deep dive on example selection and construction
+- **references/chain-of-thought.md**: Advanced reasoning elicitation techniques
+- **references/prompt-optimization.md**: Systematic refinement workflows
+- **references/prompt-templates.md**: Reusable template patterns
+- **references/system-prompts.md**: System-level prompt design
+- **assets/prompt-template-library.md**: Battle-tested prompt templates
+- **assets/few-shot-examples.json**: Curated example datasets
+- **scripts/optimize-prompt.py**: Automated prompt optimization tool
+
+## Success Metrics
+
+Track these KPIs for your prompts:
+- **Accuracy**: Correctness of outputs
+- **Consistency**: Reproducibility across similar inputs
+- **Latency**: Response time (P50, P95, P99)
+- **Token Usage**: Average tokens per request
+- **Success Rate**: Percentage of valid outputs
+- **User Satisfaction**: Ratings and feedback
+
+## Next Steps
+
+1. Review the prompt template library for common patterns
+2. Experiment with few-shot learning for your specific use case
+3. Implement prompt versioning and A/B testing
+4. Set up automated evaluation pipelines
+5. Document your prompt engineering decisions and learnings

package/merged-commands/prompt-engineering.md

@@ -0,0 +1,171 @@
+---
+name: prompt-engineering
+description: Expert guide on prompt engineering patterns, best practices, and optimization techniques. Use when user wants to improve prompts, learn prompting strategies, or debug agent behavior.
+---
+
+# Prompt Engineering Patterns
+
+Advanced prompt engineering techniques to maximize LLM performance, reliability, and controllability.
+
+## Core Capabilities
+
+### 1. Few-Shot Learning
+
+Teach the model by showing examples instead of explaining rules. Include 2-5 input-output pairs that demonstrate the desired behavior. Use when you need consistent formatting, specific reasoning patterns, or handling of edge cases. More examples improve accuracy but consume tokens—balance based on task complexity.
+
+**Example:**
+
+```markdown
+Extract key information from support tickets:
+
+Input: "My login doesn't work and I keep getting error 403"
+Output: {"issue": "authentication", "error_code": "403", "priority": "high"}
+
+Input: "Feature request: add dark mode to settings"
+Output: {"issue": "feature_request", "error_code": null, "priority": "low"}
+
+Now process: "Can't upload files larger than 10MB, getting timeout"
+```
+
+### 2. Chain-of-Thought Prompting
+
+Request step-by-step reasoning before the final answer. Add "Let's think step by step" (zero-shot) or include example reasoning traces (few-shot). Use for complex problems requiring multi-step logic, mathematical reasoning, or when you need to verify the model's thought process. Improves accuracy on analytical tasks by 30-50%.
+
+**Example:**
+
+```markdown
+Analyze this bug report and determine root cause.
+
+Think step by step:
+
+1. What is the expected behavior?
+2. What is the actual behavior?
+3. What changed recently that could cause this?
+4. What components are involved?
+5. What is the most likely root cause?
+
+Bug: "Users can't save drafts after the cache update deployed yesterday"
+```
+
+### 3. Prompt Optimization
+
+Systematically improve prompts through testing and refinement. Start simple, measure performance (accuracy, consistency, token usage), then iterate. Test on diverse inputs including edge cases. Use A/B testing to compare variations. Critical for production prompts where consistency and cost matter.
+
+**Example:**
+
+```markdown
+Version 1 (Simple): "Summarize this article"
+→ Result: Inconsistent length, misses key points
+
+Version 2 (Add constraints): "Summarize in 3 bullet points"
+→ Result: Better structure, but still misses nuance
+
+Version 3 (Add reasoning): "Identify the 3 main findings, then summarize each"
+→ Result: Consistent, accurate, captures key information
+```
+
+### 4. Template Systems
+
+Build reusable prompt structures with variables, conditional sections, and modular components. Use for multi-turn conversations, role-based interactions, or when the same pattern applies to different inputs. Reduces duplication and ensures consistency across similar tasks.
+
+**Example:**
+
+```python
+# Reusable code review template
+template = """
+Review this {language} code for {focus_area}.
+
+Code:
+{code_block}
+
+Provide feedback on:
+{checklist}
+"""
+
+# Usage
+prompt = template.format(
+    language="Python",
+    focus_area="security vulnerabilities",
+    code_block=user_code,
+    checklist="1. SQL injection\n2. XSS risks\n3. Authentication"
+)
+```
+
+### 5. System Prompt Design
+
+Set global behavior and constraints that persist across the conversation. Define the model's role, expertise level, output format, and safety guidelines. Use system prompts for stable instructions that shouldn't change turn-to-turn, freeing up user message tokens for variable content.
+
+**Example:**
+
+```markdown
+System: You are a senior backend engineer specializing in API design.
+
+Rules:
+
+- Always consider scalability and performance
+- Suggest RESTful patterns by default
+- Flag security concerns immediately
+- Provide code examples in Python
+- Use early return pattern
+
+Format responses as:
+
+1. Analysis
+2. Recommendation
+3. Code example
+4. Trade-offs
+```
+
+## Key Patterns
+
+### Progressive Disclosure
+
+Start with simple prompts, add complexity only when needed:
+
+1. **Level 1**: Direct instruction
+
+   - "Summarize this article"
+
+2. **Level 2**: Add constraints
+
+   - "Summarize this article in 3 bullet points, focusing on key findings"
+
+3. **Level 3**: Add reasoning
+
+   - "Read this article, identify the main findings, then summarize in 3 bullet points"
+
+4. **Level 4**: Add examples
+   - Include 2-3 example summaries with input-output pairs
+
+### Instruction Hierarchy
+
+```
+[System Context] → [Task Instruction] → [Examples] → [Input Data] → [Output Format]
+```
+
+### Error Recovery
+
+Build prompts that gracefully handle failures:
+
+- Include fallback instructions
+- Request confidence scores
+- Ask for alternative interpretations when uncertain
+- Specify how to indicate missing information
+
+## Best Practices
+
+1. **Be Specific**: Vague prompts produce inconsistent results
+2. **Show, Don't Tell**: Examples are more effective than descriptions
+3. **Test Extensively**: Evaluate on diverse, representative inputs
+4. **Iterate Rapidly**: Small changes can have large impacts
+5. **Monitor Performance**: Track metrics in production
+6. **Version Control**: Treat prompts as code with proper versioning
+7. **Document Intent**: Explain why prompts are structured as they are
+
+## Common Pitfalls
+
+- **Over-engineering**: Starting with complex prompts before trying simple ones
+- **Example pollution**: Using examples that don't match the target task
+- **Context overflow**: Exceeding token limits with excessive examples
+- **Ambiguous instructions**: Leaving room for multiple interpretations
+- **Ignoring edge cases**: Not testing on unusual or boundary inputs