@mohanscodex/spectra-code 0.4.8 → 0.5.0

package/skills/cloudflare-deploy/references/tail-workers/gotchas.md

@@ -0,0 +1,192 @@
+# Tail Workers Gotchas & Debugging
+
+## Critical Pitfalls
+
+### 1. Not Using `ctx.waitUntil()`
+
+**Problem:** Async work doesn't complete or tail Worker times out  
+**Cause:** Handlers exit immediately; awaiting blocks processing  
+**Solution:**
+
+```typescript
+// ❌ WRONG - fire and forget
+export default {
+  async tail(events) {
+    fetch(endpoint, { body: JSON.stringify(events) });
+  }
+};
+
+// ❌ WRONG - blocking await
+export default {
+  async tail(events, env, ctx) {
+    await fetch(endpoint, { body: JSON.stringify(events) });
+  }
+};
+
+// ✅ CORRECT
+export default {
+  async tail(events, env, ctx) {
+    ctx.waitUntil(
+      (async () => {
+        await fetch(endpoint, { body: JSON.stringify(events) });
+        await processMore();
+      })()
+    );
+  }
+};
+```
+
+### 2. Missing `tail()` Handler
+
+**Problem:** Producer deployment fails  
+**Cause:** Worker in `tail_consumers` doesn't export `tail()` handler  
+**Solution:** Ensure `export default { async tail(events, env, ctx) { ... } }`
+
+### 3. Outcome vs HTTP Status
+
+**Problem:** Filtering by wrong status  
+**Cause:** `outcome` is script execution status, not HTTP status
+
+```typescript
+// ❌ WRONG
+if (event.outcome === 500) { /* never matches */ }
+
+// ✅ CORRECT
+if (event.outcome === 'exception') { /* script threw */ }
+if (event.event?.response?.status === 500) { /* HTTP 500 */ }
+```
+
+### 4. Timestamp Units
+
+**Problem:** Dates off by 1000x  
+**Cause:** Timestamps are epoch milliseconds, not seconds
+
+```typescript
+// ❌ WRONG: const date = new Date(event.eventTimestamp * 1000);
+// ✅ CORRECT: const date = new Date(event.eventTimestamp);
+```
+
+### 5. Type Name Mismatch
+
+**Problem:** Using `TailItem` type  
+**Cause:** Old docs used `TailItem`, SDK uses `TraceItem`
+
+```typescript
+import type { TraceItem } from '@cloudflare/workers-types';
+export default {
+  async tail(events: TraceItem[], env, ctx) { /* ... */ }
+};
+```
+
+### 6. Excessive Logging Volume
+
+**Problem:** Unexpected high costs  
+**Cause:** Invoked on EVERY producer request  
+**Solution:** Sample events
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    if (Math.random() > 0.1) return;  // 10% sample
+    ctx.waitUntil(sendToEndpoint(events));
+  }
+};
+```
+
+### 7. Serialization Issues
+
+**Problem:** `JSON.stringify()` fails  
+**Cause:** `log.message` is `unknown[]` with non-serializable values  
+**Solution:**
+
+```typescript
+const safePayload = events.map(e => ({
+  ...e,
+  logs: e.logs.map(log => ({
+    ...log,
+    message: log.message.map(m => {
+      try { return JSON.parse(JSON.stringify(m)); }
+      catch { return String(m); }
+    })
+  }))
+}));
+```
+
+### 8. Missing Error Handling
+
+**Problem:** Tail Worker silently fails  
+**Cause:** No try/catch  
+**Solution:**
+
+```typescript
+ctx.waitUntil((async () => {
+  try {
+    await fetch(env.ENDPOINT, { body: JSON.stringify(events) });
+  } catch (error) {
+    console.error("Tail error:", error);
+    await env.FALLBACK_KV.put(`failed:${Date.now()}`, JSON.stringify(events));
+  }
+})());
+```
+
+### 9. Deployment Order
+
+**Problem:** Producer deployment fails  
+**Cause:** Tail consumer not deployed yet  
+**Solution:** Deploy tail consumer FIRST
+
+```bash
+cd tail-worker && wrangler deploy
+cd ../producer && wrangler deploy
+```
+
+### 10. No Event Retry
+
+**Problem:** Events lost when handler fails  
+**Cause:** Failed invocations NOT retried  
+**Solution:** Implement fallback storage (see #8)
+
+## Debugging
+
+**View logs:** `wrangler tail my-tail-worker`
+
+**Incremental testing:**
+1. Verify receipt: `console.log('Events:', events.length)`
+2. Inspect structure: `console.log(JSON.stringify(events[0], null, 2))`
+3. Add external call with `ctx.waitUntil()`
+
+**Monitor dashboard:** Check invocation count (matches producer?), error rate, CPU time
+
+## Testing
+
+Add test endpoint to producer:
+
+```typescript
+export default {
+  async fetch(request) {
+    if (request.url.includes('/test')) {
+      console.log('Test log');
+      throw new Error('Test error');
+    }
+    return new Response('OK');
+  }
+};
+```
+
+Trigger: `curl https://producer.example.workers.dev/test`
+
+## Common Errors
+
+| Error | Cause | Solution |
+|-------|-------|----------|
+| "Tail consumer not found" | Not deployed | Deploy tail Worker first |
+| "No tail handler" | Missing `tail()` | Add to default export |
+| "waitUntil is not a function" | Missing `ctx` | Add `ctx` parameter |
+| Timeout | Blocking await | Use `ctx.waitUntil()` |
+
+## Performance Notes
+
+- Max 100 events per invocation
+- Each consumer receives all events independently
+- CPU limits same as regular Workers
+- For high volume, use Durable Objects batching

package/skills/cloudflare-deploy/references/tail-workers/patterns.md

@@ -0,0 +1,180 @@
+# Tail Workers Common Patterns
+
+## Community Libraries
+
+While most tail Worker implementations are custom, these libraries may help:
+
+**Logging/Observability:**
+- **Axiom** - `axiom-cloudflare-workers` (npm) - Direct Axiom integration
+- **Baselime** - SDK for Baselime observability platform
+- **LogFlare** - Structured log aggregation
+
+**Type Definitions:**
+- **@cloudflare/workers-types** - Official TypeScript types (use `TraceItem`)
+
+**Note:** Most integrations require custom tail handler implementation. See integration examples below.
+
+## Basic Patterns
+
+### HTTP Endpoint Logging
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const payload = events.map(event => ({
+      script: event.scriptName,
+      timestamp: event.eventTimestamp,
+      outcome: event.outcome,
+      url: event.event?.request?.url,
+      status: event.event?.response?.status,
+      logs: event.logs,
+      exceptions: event.exceptions,
+    }));
+    
+    ctx.waitUntil(
+      fetch(env.LOG_ENDPOINT, {
+        method: "POST",
+        body: JSON.stringify(payload),
+      })
+    );
+  }
+};
+```
+
+### Error Tracking Only
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const errors = events.filter(e => 
+      e.outcome === 'exception' || e.exceptions.length > 0
+    );
+    
+    if (errors.length === 0) return;
+    
+    ctx.waitUntil(
+      fetch(env.ERROR_ENDPOINT, {
+        method: "POST",
+        body: JSON.stringify(errors),
+      })
+    );
+  }
+};
+```
+
+## Storage Integration
+
+### KV Storage with TTL
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    ctx.waitUntil(
+      Promise.all(events.map(event =>
+        env.LOGS_KV.put(
+          `log:${event.scriptName}:${event.eventTimestamp}`,
+          JSON.stringify(event),
+          { expirationTtl: 86400 }  // 24 hours
+        )
+      ))
+    );
+  }
+};
+```
+
+### Analytics Engine Metrics
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    ctx.waitUntil(
+      Promise.all(events.map(event =>
+        env.ANALYTICS.writeDataPoint({
+          blobs: [event.scriptName, event.outcome],
+          doubles: [1, event.event?.response?.status ?? 0],
+          indexes: [event.event?.request?.cf?.colo ?? 'unknown'],
+        })
+      ))
+    );
+  }
+};
+```
+
+## Filtering & Routing
+
+Filter by route, outcome, or other criteria:
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    // Route filtering
+    const apiEvents = events.filter(e => 
+      e.event?.request?.url?.includes('/api/')
+    );
+    
+    // Multi-destination routing
+    const errors = events.filter(e => e.outcome === 'exception');
+    const success = events.filter(e => e.outcome === 'ok');
+    
+    const tasks = [];
+    if (errors.length > 0) {
+      tasks.push(fetch(env.ERROR_ENDPOINT, {
+        method: "POST",
+        body: JSON.stringify(errors),
+      }));
+    }
+    if (success.length > 0) {
+      tasks.push(fetch(env.SUCCESS_ENDPOINT, {
+        method: "POST",
+        body: JSON.stringify(success),
+      }));
+    }
+    
+    ctx.waitUntil(Promise.all(tasks));
+  }
+};
+```
+
+## Sampling
+
+Reduce costs by processing only a percentage of events:
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    if (Math.random() > 0.1) return;  // 10% sample rate
+    ctx.waitUntil(fetch(env.LOG_ENDPOINT, {
+      method: "POST",
+      body: JSON.stringify(events),
+    }));
+  }
+};
+```
+
+## Advanced Patterns
+
+### Batching with Durable Objects
+
+Accumulate events before sending:
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const batch = env.BATCH_DO.get(env.BATCH_DO.idFromName("batch"));
+    ctx.waitUntil(batch.fetch("https://batch/add", {
+      method: "POST",
+      body: JSON.stringify(events),
+    }));
+  }
+};
+```
+
+See durable-objects skill for full implementation.
+
+### Workers for Platforms
+
+Dynamic dispatch sends TWO events per request. Filter by `scriptName` to distinguish dispatch vs user Worker events.
+
+### Error Handling
+
+Always wrap external calls. See gotchas.md for fallback storage pattern.

package/skills/cloudflare-deploy/references/terraform/README.md

@@ -0,0 +1,102 @@
+# Cloudflare Terraform Provider
+
+**Expert guidance for Cloudflare Terraform Provider - infrastructure as code for Cloudflare resources.**
+
+## Core Principles
+
+- **Provider-first**: Use Terraform provider for ALL infrastructure - never mix with wrangler.jsonc for the same resources
+- **State management**: Always use remote state (S3, Terraform Cloud, etc.) for team environments
+- **Modular architecture**: Create reusable modules for common patterns (zones, workers, pages)
+- **Version pinning**: Always pin provider version with `~>` for predictable upgrades
+- **Secret management**: Use variables + environment vars for sensitive data - never hardcode API tokens
+
+## Provider Version
+
+| Version | Status | Notes |
+|---------|--------|-------|
+| 5.x | Current | Auto-generated from OpenAPI, breaking changes from v4 |
+| 4.x | Legacy | Manual maintenance, deprecated |
+
+**Critical:** v5 renamed many resources (`cloudflare_record` → `cloudflare_dns_record`, `cloudflare_worker_*` → `cloudflare_workers_*`). See [gotchas.md](./gotchas.md#v5-breaking-changes) for migration details.
+
+## Provider Setup
+
+### Basic Configuration
+
+```hcl
+terraform {
+  required_version = ">= 1.0"
+  
+  required_providers {
+    cloudflare = {
+      source  = "cloudflare/cloudflare"
+      version = "~> 5.15.0"
+    }
+  }
+}
+
+provider "cloudflare" {
+  api_token = var.cloudflare_api_token  # or CLOUDFLARE_API_TOKEN env var
+}
+```
+
+### Authentication Methods (priority order)
+
+1. **API Token** (RECOMMENDED): `api_token` or `CLOUDFLARE_API_TOKEN`
+   - Create: Dashboard → My Profile → API Tokens
+   - Scope to specific accounts/zones for security
+   
+2. **Global API Key** (LEGACY): `api_key` + `api_email` or `CLOUDFLARE_API_KEY` + `CLOUDFLARE_EMAIL`
+   - Less secure, use tokens instead
+   
+3. **User Service Key**: `user_service_key` for Origin CA certificates
+
+
+
+## Quick Reference: Common Commands
+
+```bash
+terraform init          # Initialize provider
+terraform plan          # Plan changes
+terraform apply         # Apply changes
+terraform destroy       # Destroy resources
+terraform import cloudflare_zone.example <zone-id>  # Import existing
+terraform state list    # List resources in state
+terraform output        # Show outputs
+terraform fmt -recursive  # Format code
+terraform validate      # Validate configuration
+```
+
+## Import Existing Resources
+
+Use cf-terraforming to generate configs from existing Cloudflare resources:
+
+```bash
+# Install
+brew install cloudflare/cloudflare/cf-terraforming
+
+# Generate HCL from existing resources
+cf-terraforming generate --resource-type cloudflare_dns_record --zone <zone-id>
+
+# Import into Terraform state
+cf-terraforming import --resource-type cloudflare_dns_record --zone <zone-id>
+```
+
+## Reading Order
+
+1. Start with [README.md](./README.md) for provider setup and authentication
+2. Review [configuration.md](./configuration.md) for resource configurations
+3. Check [api.md](./api.md) for data sources and existing resource queries
+4. See [patterns.md](./patterns.md) for multi-environment and CI/CD patterns
+5. Read [gotchas.md](./gotchas.md) for state drift, v5 breaking changes, and troubleshooting
+
+## In This Reference
+- [configuration.md](./configuration.md) - Resources for zones, DNS, workers, KV, R2, D1, Pages, rulesets
+- [api.md](./api.md) - Data sources for existing resources
+- [patterns.md](./patterns.md) - Architecture patterns, multi-env setup, CI/CD integration
+- [gotchas.md](./gotchas.md) - Common issues, security, best practices
+
+## See Also
+- [pulumi](../pulumi/) - Alternative IaC tool for Cloudflare
+- [wrangler](../wrangler/) - CLI deployment alternative
+- [workers](../workers/) - Worker runtime documentation

package/skills/cloudflare-deploy/references/terraform/api.md

@@ -0,0 +1,178 @@
+# Terraform Data Sources Reference
+
+Query existing Cloudflare resources to reference in your configurations.
+
+## v5 Data Source Names
+
+| v4 Name | v5 Name | Notes |
+|---------|---------|-------|
+| `cloudflare_record` | `cloudflare_dns_record` | |
+| `cloudflare_worker_script` | `cloudflare_workers_script` | Note: plural |
+| `cloudflare_access_*` | `cloudflare_zero_trust_*` | Access → Zero Trust |
+
+## Zone Data Sources
+
+```hcl
+# Get zone by name
+data "cloudflare_zone" "example" {
+  name = "example.com"
+}
+
+# Use in resources
+resource "cloudflare_dns_record" "www" {
+  zone_id = data.cloudflare_zone.example.id
+  name = "www"
+  # ...
+}
+```
+
+## Account Data Sources
+
+```hcl
+# List all accounts
+data "cloudflare_accounts" "main" {
+  name = "My Account"
+}
+
+# Use account ID
+resource "cloudflare_worker_script" "api" {
+  account_id = data.cloudflare_accounts.main.accounts[0].id
+  # ...
+}
+```
+
+## Worker Data Sources
+
+```hcl
+# Get existing worker script (v5: cloudflare_workers_script)
+data "cloudflare_workers_script" "existing" {
+  account_id = var.account_id
+  name = "existing-worker"
+}
+
+# Reference in service bindings
+resource "cloudflare_workers_script" "consumer" {
+  service_binding {
+    name = "UPSTREAM"
+    service = data.cloudflare_workers_script.existing.name
+  }
+}
+```
+
+## KV Data Sources
+
+```hcl
+# Get KV namespace
+data "cloudflare_workers_kv_namespace" "existing" {
+  account_id = var.account_id
+  namespace_id = "abc123"
+}
+
+# Use in worker binding
+resource "cloudflare_workers_script" "api" {
+  kv_namespace_binding {
+    name = "KV"
+    namespace_id = data.cloudflare_workers_kv_namespace.existing.id
+  }
+}
+```
+
+## Lists Data Source
+
+```hcl
+# Get IP lists for WAF rules
+data "cloudflare_list" "blocked_ips" {
+  account_id = var.account_id
+  name = "blocked_ips"
+}
+```
+
+## IP Ranges Data Source
+
+```hcl
+# Get Cloudflare IP ranges (for firewall rules)
+data "cloudflare_ip_ranges" "cloudflare" {}
+
+output "ipv4_cidrs" {
+  value = data.cloudflare_ip_ranges.cloudflare.ipv4_cidr_blocks
+}
+
+output "ipv6_cidrs" {
+  value = data.cloudflare_ip_ranges.cloudflare.ipv6_cidr_blocks
+}
+
+# Use in security group rules (AWS example)
+resource "aws_security_group_rule" "allow_cloudflare" {
+  type = "ingress"
+  from_port = 443
+  to_port = 443
+  protocol = "tcp"
+  cidr_blocks = data.cloudflare_ip_ranges.cloudflare.ipv4_cidr_blocks
+  security_group_id = aws_security_group.web.id
+}
+```
+
+## Common Patterns
+
+### Import ID Formats
+
+| Resource | Import ID Format |
+|----------|------------------|
+| `cloudflare_zone` | `<zone-id>` |
+| `cloudflare_dns_record` | `<zone-id>/<record-id>` |
+| `cloudflare_workers_script` | `<account-id>/<script-name>` |
+| `cloudflare_workers_kv_namespace` | `<account-id>/<namespace-id>` |
+| `cloudflare_r2_bucket` | `<account-id>/<bucket-name>` |
+| `cloudflare_d1_database` | `<account-id>/<database-id>` |
+| `cloudflare_pages_project` | `<account-id>/<project-name>` |
+
+```bash
+# Example: Import DNS record
+terraform import cloudflare_dns_record.example <zone-id>/<record-id>
+```
+
+### Reference Across Modules
+
+```hcl
+# modules/worker/main.tf
+data "cloudflare_zone" "main" {
+  name = var.domain
+}
+
+resource "cloudflare_worker_route" "api" {
+  zone_id = data.cloudflare_zone.main.id
+  pattern = "api.${var.domain}/*"
+  script_name = cloudflare_worker_script.api.name
+}
+```
+
+### Output Important Values
+
+```hcl
+output "zone_id" {
+  value = cloudflare_zone.main.id
+  description = "Zone ID for DNS management"
+}
+
+output "worker_url" {
+  value = "https://${cloudflare_worker_domain.api.hostname}"
+  description = "Worker API endpoint"
+}
+
+output "kv_namespace_id" {
+  value = cloudflare_workers_kv_namespace.app.id
+  sensitive = false
+}
+
+output "name_servers" {
+  value = cloudflare_zone.main.name_servers
+  description = "Name servers for domain registration"
+}
+```
+
+## See Also
+
+- [README](./README.md) - Provider setup
+- [Configuration Reference](./configuration.md) - All resource types
+- [Patterns](./patterns.md) - Architecture patterns
+- [Troubleshooting](./gotchas.md) - Common issues