bps-kit 1.2.2 → 1.3.1

package/templates/skills_normal/n8n-workflow-patterns/README.md

@@ -0,0 +1,251 @@
+# n8n Workflow Patterns
+
+Proven architectural patterns for building n8n workflows.
+
+---
+
+## Purpose
+
+Teaches architectural patterns for building n8n workflows. Provides structure, best practices, and proven approaches for common use cases.
+
+## Activates On
+
+- build workflow
+- workflow pattern
+- workflow architecture
+- workflow structure
+- webhook processing
+- http api
+- api integration
+- database sync
+- ai agent
+- chatbot
+- scheduled task
+- automation pattern
+
+## File Count
+
+7 files, ~3,700 lines total
+
+## Priority
+
+**HIGH** - Addresses 813 webhook searches (most common use case)
+
+## Dependencies
+
+**n8n-mcp tools**:
+- search_nodes (find nodes for patterns)
+- get_node (understand node operations)
+- search_templates (find example workflows)
+- ai_agents_guide (AI pattern guidance)
+
+**Related skills**:
+- n8n MCP Tools Expert (find and configure nodes)
+- n8n Expression Syntax (write expressions in patterns)
+- n8n Node Configuration (configure pattern nodes)
+- n8n Validation Expert (validate pattern implementations)
+
+## Coverage
+
+### The 5 Core Patterns
+
+1. **Webhook Processing** (Most Common - 813 searches)
+   - Receive HTTP requests → Process → Respond
+   - Critical gotcha: Data under $json.body
+   - Authentication, validation, error handling
+
+2. **HTTP API Integration** (892 templates)
+   - Fetch from REST APIs → Transform → Store/Use
+   - Authentication methods, pagination, rate limiting
+   - Error handling and retries
+
+3. **Database Operations** (456 templates)
+   - Read/Write/Sync database data
+   - Batch processing, transactions, performance
+   - Security: parameterized queries, read-only access
+
+4. **AI Agent Workflow** (234 templates, 270 AI nodes)
+   - AI agents with tool access and memory
+   - 8 AI connection types
+   - ANY node can be an AI tool
+
+5. **Scheduled Tasks** (28% of all workflows)
+   - Recurring automation workflows
+   - Cron schedules, timezone handling
+   - Monitoring and error handling
+
+### Cross-Cutting Concerns
+
+- Data flow patterns (linear, branching, parallel, loops)
+- Error handling strategies
+- Performance optimization
+- Security best practices
+- Testing approaches
+- Monitoring and logging
+
+## Evaluations
+
+5 scenarios (100% coverage expected):
+1. **eval-001**: Webhook workflow structure
+2. **eval-002**: HTTP API integration pattern
+3. **eval-003**: Database sync pattern
+4. **eval-004**: AI agent workflow with tools
+5. **eval-005**: Scheduled report generation
+
+## Key Features
+
+✅ **5 Proven Patterns**: Webhook, HTTP API, Database, AI Agent, Scheduled tasks
+✅ **Complete Examples**: Working workflow configurations for each pattern
+✅ **Best Practices**: Proven approaches from real-world n8n usage
+✅ **Common Gotchas**: Documented mistakes and their fixes
+✅ **Integration Guide**: How patterns work with other skills
+✅ **Template Examples**: Real examples from 2,653+ n8n templates
+
+## Files
+
+- **SKILL.md** (486 lines) - Pattern overview, selection guide, checklist
+- **webhook_processing.md** (554 lines) - Webhook patterns, data structure, auth
+- **http_api_integration.md** (763 lines) - REST APIs, pagination, rate limiting
+- **database_operations.md** (854 lines) - DB operations, batch processing, security
+- **ai_agent_workflow.md** (918 lines) - AI agents, tools, memory, 8 connection types
+- **scheduled_tasks.md** (845 lines) - Cron schedules, timezone, monitoring
+- **README.md** (this file) - Skill metadata
+
+## Success Metrics
+
+**Expected outcomes**:
+- Users select appropriate pattern for their use case
+- Workflows follow proven structural patterns
+- Common gotchas avoided (webhook $json.body, SQL injection, etc.)
+- Proper error handling implemented
+- Security best practices followed
+
+## Pattern Selection Stats
+
+Common workflow composition:
+
+**Trigger Distribution**:
+- Webhook: 35% (most common)
+- Schedule: 28%
+- Manual: 22%
+- Service triggers: 15%
+
+**Transformation Nodes**:
+- Set: 68%
+- Code: 42%
+- IF: 38%
+- Switch: 18%
+
+**Output Channels**:
+- HTTP Request: 45%
+- Slack: 32%
+- Database: 28%
+- Email: 24%
+
+**Complexity**:
+- Simple (3-5 nodes): 42%
+- Medium (6-10 nodes): 38%
+- Complex (11+ nodes): 20%
+
+## Critical Insights
+
+**Webhook Processing**:
+- 813 searches (most common use case!)
+- #1 gotcha: Data under $json.body (not $json directly)
+- Must choose response mode: onReceived vs lastNode
+
+**API Integration**:
+- Authentication via credentials (never hardcode!)
+- Pagination essential for large datasets
+- Rate limiting prevents API bans
+- continueOnFail: true for error handling
+
+**Database Operations**:
+- Always use parameterized queries (SQL injection prevention)
+- Batch processing for large datasets
+- Read-only access for AI tools
+- Transaction handling for multi-step operations
+
+**AI Agents**:
+- 8 AI connection types (ai_languageModel, ai_tool, ai_memory, etc.)
+- ANY node can be an AI tool (connect to ai_tool port)
+- Memory essential for conversations (Window Buffer recommended)
+- Tool descriptions critical (AI uses them to decide when to call)
+
+**Scheduled Tasks**:
+- Set workflow timezone explicitly (DST handling)
+- Prevent overlapping executions (use locks)
+- Error Trigger workflow for alerts
+- Batch processing for large data
+
+## Workflow Creation Checklist
+
+Every pattern follows this checklist:
+
+### Planning Phase
+- [ ] Identify the pattern (webhook, API, database, AI, scheduled)
+- [ ] List required nodes (use search_nodes)
+- [ ] Understand data flow (input → transform → output)
+- [ ] Plan error handling strategy
+
+### Implementation Phase
+- [ ] Create workflow with appropriate trigger
+- [ ] Add data source nodes
+- [ ] Configure authentication/credentials
+- [ ] Add transformation nodes (Set, Code, IF)
+- [ ] Add output/action nodes
+- [ ] Configure error handling
+
+### Validation Phase
+- [ ] Validate each node configuration
+- [ ] Validate complete workflow
+- [ ] Test with sample data
+- [ ] Handle edge cases
+
+### Deployment Phase
+- [ ] Review workflow settings
+- [ ] Activate workflow
+- [ ] Monitor first executions
+- [ ] Document workflow
+
+## Real Template Examples
+
+**Weather to Slack** (Template #2947):
+```
+Schedule (daily 8 AM) → HTTP Request (weather) → Set → Slack
+```
+
+**Webhook Processing**: 1,085 templates
+**HTTP API Integration**: 892 templates
+**Database Operations**: 456 templates
+**AI Workflows**: 234 templates
+
+Use `search_templates` to find examples for your use case!
+
+## Integration with Other Skills
+
+**Pattern Selection** (this skill):
+1. Identify use case
+2. Select appropriate pattern
+3. Follow pattern structure
+
+**Node Discovery** (n8n MCP Tools Expert):
+4. Find nodes for pattern (search_nodes)
+5. Understand node operations (get_node)
+
+**Implementation** (n8n Expression Syntax + Node Configuration):
+6. Write expressions ({{$json.body.field}})
+7. Configure nodes properly
+
+**Validation** (n8n Validation Expert):
+8. Validate workflow structure
+9. Fix validation errors
+
+## Last Updated
+
+2025-10-20
+
+---
+
+**Part of**: n8n-skills repository
+**Conceived by**: Romuald Członkowski - [www.aiadvisors.pl/en](https://www.aiadvisors.pl/en)

package/templates/skills_normal/n8n-workflow-patterns/SKILL.md

@@ -0,0 +1,411 @@
+---
+name: n8n-workflow-patterns
+description: Proven workflow architectural patterns from real n8n workflows. Use when building new workflows, designing workflow structure, choosing workflow patterns, planning workflow architecture, or asking about webhook processing, HTTP API integration, database operations, AI agent workflows, or scheduled tasks.
+---
+
+# n8n Workflow Patterns
+
+Proven architectural patterns for building n8n workflows.
+
+---
+
+## The 5 Core Patterns
+
+Based on analysis of real workflow usage:
+
+1. **[Webhook Processing](webhook_processing.md)** (Most Common)
+   - Receive HTTP requests → Process → Output
+   - Pattern: Webhook → Validate → Transform → Respond/Notify
+
+2. **[HTTP API Integration](http_api_integration.md)**
+   - Fetch from REST APIs → Transform → Store/Use
+   - Pattern: Trigger → HTTP Request → Transform → Action → Error Handler
+
+3. **[Database Operations](database_operations.md)**
+   - Read/Write/Sync database data
+   - Pattern: Schedule → Query → Transform → Write → Verify
+
+4. **[AI Agent Workflow](ai_agent_workflow.md)**
+   - AI agents with tools and memory
+   - Pattern: Trigger → AI Agent (Model + Tools + Memory) → Output
+
+5. **[Scheduled Tasks](scheduled_tasks.md)**
+   - Recurring automation workflows
+   - Pattern: Schedule → Fetch → Process → Deliver → Log
+
+---
+
+## Pattern Selection Guide
+
+### When to use each pattern:
+
+**Webhook Processing** - Use when:
+- Receiving data from external systems
+- Building integrations (Slack commands, form submissions, GitHub webhooks)
+- Need instant response to events
+- Example: "Receive Stripe payment webhook → Update database → Send confirmation"
+
+**HTTP API Integration** - Use when:
+- Fetching data from external APIs
+- Synchronizing with third-party services
+- Building data pipelines
+- Example: "Fetch GitHub issues → Transform → Create Jira tickets"
+
+**Database Operations** - Use when:
+- Syncing between databases
+- Running database queries on schedule
+- ETL workflows
+- Example: "Read Postgres records → Transform → Write to MySQL"
+
+**AI Agent Workflow** - Use when:
+- Building conversational AI
+- Need AI with tool access
+- Multi-step reasoning tasks
+- Example: "Chat with AI that can search docs, query database, send emails"
+
+**Scheduled Tasks** - Use when:
+- Recurring reports or summaries
+- Periodic data fetching
+- Maintenance tasks
+- Example: "Daily: Fetch analytics → Generate report → Email team"
+
+---
+
+## Common Workflow Components
+
+All patterns share these building blocks:
+
+### 1. Triggers
+- **Webhook** - HTTP endpoint (instant)
+- **Schedule** - Cron-based timing (periodic)
+- **Manual** - Click to execute (testing)
+- **Polling** - Check for changes (intervals)
+
+### 2. Data Sources
+- **HTTP Request** - REST APIs
+- **Database nodes** - Postgres, MySQL, MongoDB
+- **Service nodes** - Slack, Google Sheets, etc.
+- **Code** - Custom JavaScript/Python
+
+### 3. Transformation
+- **Set** - Map/transform fields
+- **Code** - Complex logic
+- **IF/Switch** - Conditional routing
+- **Merge** - Combine data streams
+
+### 4. Outputs
+- **HTTP Request** - Call APIs
+- **Database** - Write data
+- **Communication** - Email, Slack, Discord
+- **Storage** - Files, cloud storage
+
+### 5. Error Handling
+- **Error Trigger** - Catch workflow errors
+- **IF** - Check for error conditions
+- **Stop and Error** - Explicit failure
+- **Continue On Fail** - Per-node setting
+
+---
+
+## Workflow Creation Checklist
+
+When building ANY workflow, follow this checklist:
+
+### Planning Phase
+- [ ] Identify the pattern (webhook, API, database, AI, scheduled)
+- [ ] List required nodes (use search_nodes)
+- [ ] Understand data flow (input → transform → output)
+- [ ] Plan error handling strategy
+
+### Implementation Phase
+- [ ] Create workflow with appropriate trigger
+- [ ] Add data source nodes
+- [ ] Configure authentication/credentials
+- [ ] Add transformation nodes (Set, Code, IF)
+- [ ] Add output/action nodes
+- [ ] Configure error handling
+
+### Validation Phase
+- [ ] Validate each node configuration (validate_node)
+- [ ] Validate complete workflow (validate_workflow)
+- [ ] Test with sample data
+- [ ] Handle edge cases (empty data, errors)
+
+### Deployment Phase
+- [ ] Review workflow settings (execution order, timeout, error handling)
+- [ ] Activate workflow using `activateWorkflow` operation
+- [ ] Monitor first executions
+- [ ] Document workflow purpose and data flow
+
+---
+
+## Data Flow Patterns
+
+### Linear Flow
+```
+Trigger → Transform → Action → End
+```
+**Use when**: Simple workflows with single path
+
+### Branching Flow
+```
+Trigger → IF → [True Path]
+             └→ [False Path]
+```
+**Use when**: Different actions based on conditions
+
+### Parallel Processing
+```
+Trigger → [Branch 1] → Merge
+       └→ [Branch 2] ↗
+```
+**Use when**: Independent operations that can run simultaneously
+
+### Loop Pattern
+```
+Trigger → Split in Batches → Process → Loop (until done)
+```
+**Use when**: Processing large datasets in chunks
+
+### Error Handler Pattern
+```
+Main Flow → [Success Path]
+         └→ [Error Trigger → Error Handler]
+```
+**Use when**: Need separate error handling workflow
+
+---
+
+## Common Gotchas
+
+### 1. Webhook Data Structure
+**Problem**: Can't access webhook payload data
+
+**Solution**: Data is nested under `$json.body`
+```javascript
+❌ {{$json.email}}
+✅ {{$json.body.email}}
+```
+See: n8n Expression Syntax skill
+
+### 2. Multiple Input Items
+**Problem**: Node processes all input items, but I only want one
+
+**Solution**: Use "Execute Once" mode or process first item only
+```javascript
+{{$json[0].field}}  // First item only
+```
+
+### 3. Authentication Issues
+**Problem**: API calls failing with 401/403
+
+**Solution**:
+- Configure credentials properly
+- Use the "Credentials" section, not parameters
+- Test credentials before workflow activation
+
+### 4. Node Execution Order
+**Problem**: Nodes executing in unexpected order
+
+**Solution**: Check workflow settings → Execution Order
+- v0: Top-to-bottom (legacy)
+- v1: Connection-based (recommended)
+
+### 5. Expression Errors
+**Problem**: Expressions showing as literal text
+
+**Solution**: Use {{}} around expressions
+- See n8n Expression Syntax skill for details
+
+---
+
+## Integration with Other Skills
+
+These skills work together with Workflow Patterns:
+
+**n8n MCP Tools Expert** - Use to:
+- Find nodes for your pattern (search_nodes)
+- Understand node operations (get_node)
+- Create workflows (n8n_create_workflow)
+- Deploy templates (n8n_deploy_template)
+- Use ai_agents_guide for AI pattern guidance
+
+**n8n Expression Syntax** - Use to:
+- Write expressions in transformation nodes
+- Access webhook data correctly ({{$json.body.field}})
+- Reference previous nodes ({{$node["Node Name"].json.field}})
+
+**n8n Node Configuration** - Use to:
+- Configure specific operations for pattern nodes
+- Understand node-specific requirements
+
+**n8n Validation Expert** - Use to:
+- Validate workflow structure
+- Fix validation errors
+- Ensure workflow correctness before deployment
+
+---
+
+## Pattern Statistics
+
+Common workflow patterns:
+
+**Most Common Triggers**:
+1. Webhook - 35%
+2. Schedule (periodic tasks) - 28%
+3. Manual (testing/admin) - 22%
+4. Service triggers (Slack, email, etc.) - 15%
+
+**Most Common Transformations**:
+1. Set (field mapping) - 68%
+2. Code (custom logic) - 42%
+3. IF (conditional routing) - 38%
+4. Switch (multi-condition) - 18%
+
+**Most Common Outputs**:
+1. HTTP Request (APIs) - 45%
+2. Slack - 32%
+3. Database writes - 28%
+4. Email - 24%
+
+**Average Workflow Complexity**:
+- Simple (3-5 nodes): 42%
+- Medium (6-10 nodes): 38%
+- Complex (11+ nodes): 20%
+
+---
+
+## Quick Start Examples
+
+### Example 1: Simple Webhook → Slack
+```
+1. Webhook (path: "form-submit", POST)
+2. Set (map form fields)
+3. Slack (post message to #notifications)
+```
+
+### Example 2: Scheduled Report
+```
+1. Schedule (daily at 9 AM)
+2. HTTP Request (fetch analytics)
+3. Code (aggregate data)
+4. Email (send formatted report)
+5. Error Trigger → Slack (notify on failure)
+```
+
+### Example 3: Database Sync
+```
+1. Schedule (every 15 minutes)
+2. Postgres (query new records)
+3. IF (check if records exist)
+4. MySQL (insert records)
+5. Postgres (update sync timestamp)
+```
+
+### Example 4: AI Assistant
+```
+1. Webhook (receive chat message)
+2. AI Agent
+   ├─ OpenAI Chat Model (ai_languageModel)
+   ├─ HTTP Request Tool (ai_tool)
+   ├─ Database Tool (ai_tool)
+   └─ Window Buffer Memory (ai_memory)
+3. Webhook Response (send AI reply)
+```
+
+### Example 5: API Integration
+```
+1. Manual Trigger (for testing)
+2. HTTP Request (GET /api/users)
+3. Split In Batches (process 100 at a time)
+4. Set (transform user data)
+5. Postgres (upsert users)
+6. Loop (back to step 3 until done)
+```
+
+---
+
+## Detailed Pattern Files
+
+For comprehensive guidance on each pattern:
+
+- **[webhook_processing.md](webhook_processing.md)** - Webhook patterns, data structure, response handling
+- **[http_api_integration.md](http_api_integration.md)** - REST APIs, authentication, pagination, retries
+- **[database_operations.md](database_operations.md)** - Queries, sync, transactions, batch processing
+- **[ai_agent_workflow.md](ai_agent_workflow.md)** - AI agents, tools, memory, langchain nodes
+- **[scheduled_tasks.md](scheduled_tasks.md)** - Cron schedules, reports, maintenance tasks
+
+---
+
+## Real Template Examples
+
+From n8n template library:
+
+**Template #2947**: Weather to Slack
+- Pattern: Scheduled Task
+- Nodes: Schedule → HTTP Request (weather API) → Set → Slack
+- Complexity: Simple (4 nodes)
+
+**Webhook Processing**: Most common pattern
+- Most common: Form submissions, payment webhooks, chat integrations
+
+**HTTP API**: Common pattern
+- Most common: Data fetching, third-party integrations
+
+**Database Operations**: Common pattern
+- Most common: ETL, data sync, backup workflows
+
+**AI Agents**: Growing in usage
+- Most common: Chatbots, content generation, data analysis
+
+Use `search_templates` and `get_template` from n8n-mcp tools to find examples!
+
+---
+
+## Best Practices
+
+### ✅ Do
+
+- Start with the simplest pattern that solves your problem
+- Plan your workflow structure before building
+- Use error handling on all workflows
+- Test with sample data before activation
+- Follow the workflow creation checklist
+- Use descriptive node names
+- Document complex workflows (notes field)
+- Monitor workflow executions after deployment
+
+### ❌ Don't
+
+- Build workflows in one shot (iterate! avg 56s between edits)
+- Skip validation before activation
+- Ignore error scenarios
+- Use complex patterns when simple ones suffice
+- Hardcode credentials in parameters
+- Forget to handle empty data cases
+- Mix multiple patterns without clear boundaries
+- Deploy without testing
+
+---
+
+## Summary
+
+**Key Points**:
+1. **5 core patterns** cover 90%+ of workflow use cases
+2. **Webhook processing** is the most common pattern
+3. Use the **workflow creation checklist** for every workflow
+4. **Plan pattern** → **Select nodes** → **Build** → **Validate** → **Deploy**
+5. Integrate with other skills for complete workflow development
+
+**Next Steps**:
+1. Identify your use case pattern
+2. Read the detailed pattern file
+3. Use n8n MCP Tools Expert to find nodes
+4. Follow the workflow creation checklist
+5. Use n8n Validation Expert to validate
+
+**Related Skills**:
+- n8n MCP Tools Expert - Find and configure nodes
+- n8n Expression Syntax - Write expressions correctly
+- n8n Validation Expert - Validate and fix errors
+- n8n Node Configuration - Configure specific operations