bps-kit 1.2.2 → 1.3.1

package/templates/skills_normal/n8n-code-python/DATA_ACCESS.md

@@ -0,0 +1,702 @@
+# Data Access Patterns - Python Code Node
+
+Complete guide to accessing data in n8n Code nodes using Python.
+
+---
+
+## Overview
+
+In n8n Python Code nodes, you access data using **underscore-prefixed** variables: `_input`, `_json`, `_node`.
+
+**Data Access Priority** (by common usage):
+1. **`_input.all()`** - Most common - Batch operations, aggregations
+2. **`_input.first()`** - Very common - Single item operations
+3. **`_input.item`** - Common - Each Item mode only
+4. **`_node["NodeName"]["json"]`** - Specific node references
+5. **`_json`** - Direct current item (use `_input` instead)
+
+**Python vs JavaScript**:
+| JavaScript | Python (Beta) | Python (Native) |
+|------------|---------------|-----------------|
+| `$input.all()` | `_input.all()` | `_items` |
+| `$input.first()` | `_input.first()` | `_items[0]` |
+| `$input.item` | `_input.item` | `_item` |
+| `$json` | `_json` | `_item["json"]` |
+| `$node["Name"]` | `_node["Name"]` | Not available |
+
+---
+
+## Pattern 1: _input.all() - Process All Items
+
+**Usage**: Most common pattern for batch processing
+
+**When to use:**
+- Processing multiple records
+- Aggregating data (sum, count, average)
+- Filtering lists
+- Transforming datasets
+
+### Basic Usage
+
+```python
+# Get all items from previous node
+all_items = _input.all()
+
+# all_items is a list of dictionaries like:
+# [
+#   {"json": {"id": 1, "name": "Alice"}},
+#   {"json": {"id": 2, "name": "Bob"}}
+# ]
+
+print(f"Received {len(all_items)} items")
+
+return all_items
+```
+
+### Example 1: Filter Active Items
+
+```python
+all_items = _input.all()
+
+# Filter only active items
+active_items = [
+    item for item in all_items
+    if item["json"].get("status") == "active"
+]
+
+return active_items
+```
+
+### Example 2: Transform All Items
+
+```python
+all_items = _input.all()
+
+# Transform to new structure
+transformed = []
+for item in all_items:
+    transformed.append({
+        "json": {
+            "id": item["json"].get("id"),
+            "full_name": f"{item['json'].get('first_name', '')} {item['json'].get('last_name', '')}",
+            "email": item["json"].get("email"),
+            "processed_at": datetime.now().isoformat()
+        }
+    })
+
+return transformed
+```
+
+### Example 3: Aggregate Data
+
+```python
+all_items = _input.all()
+
+# Calculate total
+total = sum(item["json"].get("amount", 0) for item in all_items)
+
+return [{
+    "json": {
+        "total": total,
+        "count": len(all_items),
+        "average": total / len(all_items) if all_items else 0
+    }
+}]
+```
+
+### Example 4: Sort and Limit
+
+```python
+all_items = _input.all()
+
+# Get top 5 by score
+sorted_items = sorted(
+    all_items,
+    key=lambda item: item["json"].get("score", 0),
+    reverse=True
+)
+top_five = sorted_items[:5]
+
+return [{"json": item["json"]} for item in top_five]
+```
+
+### Example 5: Group By Category
+
+```python
+all_items = _input.all()
+
+# Group items by category
+grouped = {}
+for item in all_items:
+    category = item["json"].get("category", "Uncategorized")
+
+    if category not in grouped:
+        grouped[category] = []
+
+    grouped[category].append(item["json"])
+
+# Convert to list format
+return [
+    {
+        "json": {
+            "category": category,
+            "items": items,
+            "count": len(items)
+        }
+    }
+    for category, items in grouped.items()
+]
+```
+
+### Example 6: Deduplicate by ID
+
+```python
+all_items = _input.all()
+
+# Remove duplicates by ID
+seen = set()
+unique = []
+
+for item in all_items:
+    item_id = item["json"].get("id")
+
+    if item_id and item_id not in seen:
+        seen.add(item_id)
+        unique.append(item)
+
+return unique
+```
+
+---
+
+## Pattern 2: _input.first() - Get First Item
+
+**Usage**: Very common for single-item operations
+
+**When to use:**
+- Previous node returns single object
+- Working with API responses
+- Getting initial/first data point
+
+### Basic Usage
+
+```python
+# Get first item from previous node
+first_item = _input.first()
+
+# Access the JSON data
+data = first_item["json"]
+
+print(f"First item: {data}")
+
+return [{"json": data}]
+```
+
+### Example 1: Process Single API Response
+
+```python
+# Get API response (typically single object)
+response = _input.first()["json"]
+
+# Extract what you need
+return [{
+    "json": {
+        "user_id": response.get("data", {}).get("user", {}).get("id"),
+        "user_name": response.get("data", {}).get("user", {}).get("name"),
+        "status": response.get("status"),
+        "fetched_at": datetime.now().isoformat()
+    }
+}]
+```
+
+### Example 2: Transform Single Object
+
+```python
+data = _input.first()["json"]
+
+# Transform structure
+return [{
+    "json": {
+        "id": data.get("id"),
+        "contact": {
+            "email": data.get("email"),
+            "phone": data.get("phone")
+        },
+        "address": {
+            "street": data.get("street"),
+            "city": data.get("city"),
+            "zip": data.get("zip")
+        }
+    }
+}]
+```
+
+### Example 3: Validate Single Item
+
+```python
+item = _input.first()["json"]
+
+# Validation logic
+is_valid = bool(item.get("email") and "@" in item.get("email", ""))
+
+return [{
+    "json": {
+        **item,
+        "valid": is_valid,
+        "validated_at": datetime.now().isoformat()
+    }
+}]
+```
+
+### Example 4: Extract Nested Data
+
+```python
+response = _input.first()["json"]
+
+# Navigate nested structure
+users = response.get("data", {}).get("users", [])
+
+return [
+    {
+        "json": {
+            "id": user.get("id"),
+            "name": user.get("profile", {}).get("name", "Unknown"),
+            "email": user.get("contact", {}).get("email", "no-email")
+        }
+    }
+    for user in users
+]
+```
+
+---
+
+## Pattern 3: _input.item - Current Item (Each Item Mode)
+
+**Usage**: Common in "Run Once for Each Item" mode
+
+**When to use:**
+- Mode is set to "Run Once for Each Item"
+- Need to process items independently
+- Per-item API calls or validations
+
+**IMPORTANT**: Only use in "Each Item" mode. Will be undefined in "All Items" mode.
+
+### Basic Usage
+
+```python
+# In "Run Once for Each Item" mode
+current_item = _input.item
+data = current_item["json"]
+
+print(f"Processing item: {data.get('id')}")
+
+return [{
+    "json": {
+        **data,
+        "processed": True
+    }
+}]
+```
+
+### Example 1: Add Processing Metadata
+
+```python
+item = _input.item
+
+return [{
+    "json": {
+        **item["json"],
+        "processed": True,
+        "processed_at": datetime.now().isoformat(),
+        "processing_duration": random.random() * 1000  # Simulated
+    }
+}]
+```
+
+### Example 2: Per-Item Validation
+
+```python
+item = _input.item
+data = item["json"]
+
+# Validate this specific item
+errors = []
+
+if not data.get("email"):
+    errors.append("Email required")
+if not data.get("name"):
+    errors.append("Name required")
+if data.get("age") and data["age"] < 18:
+    errors.append("Must be 18+")
+
+return [{
+    "json": {
+        **data,
+        "valid": len(errors) == 0,
+        "errors": errors if errors else None
+    }
+}]
+```
+
+### Example 3: Conditional Processing
+
+```python
+item = _input.item
+data = item["json"]
+
+# Process based on item type
+if data.get("type") == "premium":
+    return [{
+        "json": {
+            **data,
+            "discount": 0.20,
+            "tier": "premium"
+        }
+    }]
+else:
+    return [{
+        "json": {
+            **data,
+            "discount": 0.05,
+            "tier": "standard"
+        }
+    }]
+```
+
+---
+
+## Pattern 4: _node - Reference Other Nodes
+
+**Usage**: Less common, but powerful for specific scenarios
+
+**When to use:**
+- Need data from specific named node
+- Combining data from multiple nodes
+
+### Basic Usage
+
+```python
+# Get output from specific node
+webhook_data = _node["Webhook"]["json"]
+api_data = _node["HTTP Request"]["json"]
+
+return [{
+    "json": {
+        "from_webhook": webhook_data,
+        "from_api": api_data
+    }
+}]
+```
+
+### Example 1: Combine Multiple Sources
+
+```python
+# Reference multiple nodes
+webhook = _node["Webhook"]["json"]
+database = _node["Postgres"]["json"]
+api = _node["HTTP Request"]["json"]
+
+return [{
+    "json": {
+        "combined": {
+            "webhook": webhook.get("body", {}),
+            "db_records": len(database) if isinstance(database, list) else 1,
+            "api_response": api.get("status")
+        },
+        "processed_at": datetime.now().isoformat()
+    }
+}]
+```
+
+### Example 2: Compare Across Nodes
+
+```python
+old_data = _node["Get Old Data"]["json"]
+new_data = _node["Get New Data"]["json"]
+
+# Simple comparison
+changes = {
+    "added": [n for n in new_data if n.get("id") not in [o.get("id") for o in old_data]],
+    "removed": [o for o in old_data if o.get("id") not in [n.get("id") for n in new_data]]
+}
+
+return [{
+    "json": {
+        "changes": changes,
+        "summary": {
+            "added": len(changes["added"]),
+            "removed": len(changes["removed"])
+        }
+    }
+}]
+```
+
+---
+
+## Critical: Webhook Data Structure
+
+**MOST COMMON MISTAKE**: Forgetting webhook data is nested under `["body"]`
+
+### The Problem
+
+Webhook node wraps all incoming data under a `"body"` property.
+
+### Structure
+
+```python
+# Webhook node output structure:
+{
+    "headers": {
+        "content-type": "application/json",
+        "user-agent": "..."
+    },
+    "params": {},
+    "query": {},
+    "body": {
+        # ← YOUR DATA IS HERE
+        "name": "Alice",
+        "email": "alice@example.com",
+        "message": "Hello!"
+    }
+}
+```
+
+### Wrong vs Right
+
+```python
+# ❌ WRONG: Trying to access directly
+name = _json["name"]  # KeyError!
+email = _json["email"]  # KeyError!
+
+# ✅ CORRECT: Access via ["body"]
+name = _json["body"]["name"]  # "Alice"
+email = _json["body"]["email"]  # "alice@example.com"
+
+# ✅ SAFER: Use .get() for safe access
+webhook_data = _json.get("body", {})
+name = webhook_data.get("name")  # None if missing
+email = webhook_data.get("email", "no-email")  # Default value
+```
+
+### Example: Full Webhook Processing
+
+```python
+# Get webhook data from previous node
+webhook_output = _input.first()["json"]
+
+# Access the actual payload
+payload = webhook_output.get("body", {})
+
+# Access headers if needed
+content_type = webhook_output.get("headers", {}).get("content-type")
+
+# Access query parameters if needed
+api_key = webhook_output.get("query", {}).get("api_key")
+
+# Process the actual data
+return [{
+    "json": {
+        # Data from webhook body
+        "user_name": payload.get("name"),
+        "user_email": payload.get("email"),
+        "message": payload.get("message"),
+
+        # Metadata
+        "received_at": datetime.now().isoformat(),
+        "content_type": content_type,
+        "authenticated": bool(api_key)
+    }
+}]
+```
+
+### POST Data, Query Params, and Headers
+
+```python
+webhook = _input.first()["json"]
+
+return [{
+    "json": {
+        # POST body data
+        "form_data": webhook.get("body", {}),
+
+        # Query parameters (?key=value)
+        "query_params": webhook.get("query", {}),
+
+        # HTTP headers
+        "user_agent": webhook.get("headers", {}).get("user-agent"),
+        "content_type": webhook.get("headers", {}).get("content-type"),
+
+        # Request metadata
+        "method": webhook.get("method"),  # POST, GET, etc.
+        "url": webhook.get("url")
+    }
+}]
+```
+
+---
+
+## Choosing the Right Pattern
+
+### Decision Tree
+
+```
+Do you need ALL items from previous node?
+├─ YES → Use _input.all()
+│
+└─ NO → Do you need just the FIRST item?
+    ├─ YES → Use _input.first()
+    │
+    └─ NO → Are you in "Each Item" mode?
+        ├─ YES → Use _input.item
+        │
+        └─ NO → Do you need specific node data?
+            ├─ YES → Use _node["NodeName"]
+            └─ NO → Use _input.first() (default)
+```
+
+### Quick Reference Table
+
+| Scenario | Use This | Example |
+|----------|----------|---------|
+| Sum all amounts | `_input.all()` | `sum(i["json"].get("amount", 0) for i in items)` |
+| Get API response | `_input.first()` | `_input.first()["json"].get("data")` |
+| Process each independently | `_input.item` | `_input.item["json"]` (Each Item mode) |
+| Combine two nodes | `_node["Name"]` | `_node["API"]["json"]` |
+| Filter list | `_input.all()` | `[i for i in items if i["json"].get("active")]` |
+| Transform single object | `_input.first()` | `{**_input.first()["json"], "new": True}` |
+| Webhook data | `_input.first()` | `_input.first()["json"]["body"]` |
+
+---
+
+## Common Mistakes
+
+### Mistake 1: Using _json Without Context
+
+```python
+# ❌ RISKY: _json is ambiguous
+value = _json["field"]
+
+# ✅ CLEAR: Be explicit
+value = _input.first()["json"]["field"]
+```
+
+### Mistake 2: Forgetting ["json"] Property
+
+```python
+# ❌ WRONG: Trying to access fields on item dictionary
+items = _input.all()
+names = [item["name"] for item in items]  # KeyError!
+
+# ✅ CORRECT: Access via ["json"]
+names = [item["json"]["name"] for item in items]
+```
+
+### Mistake 3: Using _input.item in All Items Mode
+
+```python
+# ❌ WRONG: _input.item is None in "All Items" mode
+data = _input.item["json"]  # AttributeError!
+
+# ✅ CORRECT: Use appropriate method
+data = _input.first()["json"]  # Or _input.all()
+```
+
+### Mistake 4: Not Handling Empty Lists
+
+```python
+# ❌ WRONG: Crashes if no items
+first = _input.all()[0]["json"]  # IndexError!
+
+# ✅ CORRECT: Check length first
+items = _input.all()
+if items:
+    first = items[0]["json"]
+else:
+    return []
+
+# ✅ ALSO CORRECT: Use _input.first()
+first = _input.first()["json"]  # Built-in safety
+```
+
+### Mistake 5: Direct Dictionary Access (KeyError)
+
+```python
+# ❌ RISKY: Crashes if key missing
+value = item["json"]["field"]  # KeyError!
+
+# ✅ SAFE: Use .get()
+value = item["json"].get("field", "default")
+```
+
+---
+
+## Advanced Patterns
+
+### Pattern: Safe Nested Access
+
+```python
+# Deep nested access with .get()
+value = (
+    _input.first()["json"]
+    .get("level1", {})
+    .get("level2", {})
+    .get("level3", "default")
+)
+```
+
+### Pattern: List Comprehension with Filtering
+
+```python
+items = _input.all()
+
+# Filter and transform in one step
+result = [
+    {
+        "json": {
+            "id": item["json"]["id"],
+            "name": item["json"]["name"].upper()
+        }
+    }
+    for item in items
+    if item["json"].get("active") and item["json"].get("verified")
+]
+
+return result
+```
+
+### Pattern: Dictionary Comprehension
+
+```python
+items = _input.all()
+
+# Create lookup dictionary
+lookup = {
+    item["json"]["id"]: item["json"]
+    for item in items
+    if "id" in item["json"]
+}
+
+return [{"json": lookup}]
+```
+
+---
+
+## Summary
+
+**Most Common Patterns**:
+1. `_input.all()` - Process multiple items, batch operations
+2. `_input.first()` - Single item, API responses
+3. `_input.item` - Each Item mode processing
+
+**Critical Rule**:
+- Webhook data is under `["body"]` property
+
+**Best Practice**:
+- Use `.get()` for dictionary access to avoid KeyError
+- Always check for empty lists
+- Be explicit: Use `_input.first()["json"]["field"]` instead of `_json["field"]`
+
+**See Also**:
+- [SKILL.md](SKILL.md) - Overview and quick start
+- [COMMON_PATTERNS.md](COMMON_PATTERNS.md) - Python-specific patterns
+- [ERROR_PATTERNS.md](ERROR_PATTERNS.md) - Avoid common mistakes