@namch/agent-assistant 1.0.0

package/skills/cloudflare-expert/references/tail-workers/README.md

@@ -0,0 +1,640 @@
+# Cloudflare Tail Workers Skill
+
+## Purpose
+Expert guidance on Cloudflare Tail Workers—specialized Workers that consume execution events from producer Workers for logging, debugging, analytics, and observability.
+
+## When to Use
+- User implements observability/logging for Cloudflare Workers
+- User needs to process Worker execution events, logs, exceptions
+- User builds custom analytics or error tracking
+- User configures real-time event streaming
+- User mentions tail handlers, tail consumers, or producer Workers
+
+## Core Concepts
+
+### What Are Tail Workers?
+Tail Workers automatically process events from producer Workers (the Workers being monitored). They receive:
+- HTTP request/response info
+- Console logs (console.log/error/warn/debug)
+- Uncaught exceptions
+- Execution outcomes (ok, exception, exceededCpu, etc.)
+- Diagnostic channel events
+
+**Key characteristics:**
+- Invoked AFTER producer finishes executing
+- Capture entire request lifecycle including Service Bindings and Dynamic Dispatch sub-requests
+- Billed by CPU time, not request count
+- Available on Workers Paid and Enterprise tiers
+
+### Alternative: OpenTelemetry Export
+For batch exports to observability tools (Sentry, Grafana, Honeycomb):
+- Consider OTEL export instead of Tail Workers
+- OTEL sends logs/traces in batches (more efficient)
+- Tail Workers = advanced mode for custom processing
+
+## Implementation Patterns
+
+### Basic Tail Handler Structure
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    // Process events from producer Worker
+  }
+};
+```
+
+**Parameters:**
+- `events`: Array of `TailItem` objects (one per producer invocation)
+- `env`: Bindings (KV, D1, R2, env vars, etc.)
+- `ctx`: Context with `waitUntil()` for async work
+
+**CRITICAL:** Tail handlers don't return values. Use `ctx.waitUntil()` for async operations.
+
+### Event Structure (`TailItem`)
+
+```typescript
+interface TailItem {
+  scriptName: string;           // Producer Worker name
+  eventTimestamp: number;        // Epoch time
+  outcome: 'ok' | 'exception' | 'exceededCpu' | 'exceededMemory' 
+         | 'canceled' | 'scriptNotFound' | 'responseStreamDisconnected' | 'unknown';
+  
+  event: {
+    request?: {
+      url: string;               // Redacted by default
+      method: string;
+      headers: Record<string, string>;  // Sensitive headers redacted
+      cf: IncomingRequestCfProperties;
+      getUnredacted(): TailRequest;     // Bypass redaction (use carefully)
+    };
+    response?: {
+      status: number;
+    };
+  };
+  
+  logs: Array<{
+    timestamp: number;
+    level: 'debug' | 'info' | 'log' | 'warn' | 'error';
+    message: any[];              // Args passed to console function
+  }>;
+  
+  exceptions: Array<{
+    timestamp: number;
+    name: string;                // Error type (Error, TypeError, etc.)
+    message: string;             // Error description
+  }>;
+  
+  diagnosticsChannelEvents: Array<{
+    channel: string;
+    message: any;
+    timestamp: number;
+  }>;
+}
+```
+
+### Configuration
+
+**Producer Worker wrangler.toml:**
+```toml
+name = "my-producer-worker"
+tail_consumers = [{service = "my-tail-worker"}]
+```
+
+**Producer Worker wrangler.jsonc:**
+```json
+{
+  "name": "my-producer-worker",
+  "tail_consumers": [
+    {
+      "service": "my-tail-worker"
+    }
+  ]
+}
+```
+
+**Tail Worker wrangler.toml:**
+```toml
+name = "my-tail-worker"
+# No special config needed, just must have tail() handler
+```
+
+## Common Use Cases
+
+### 1. Send Logs to HTTP Endpoint
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const payload = events.map(event => ({
+      script: event.scriptName,
+      timestamp: event.eventTimestamp,
+      outcome: event.outcome,
+      url: event.event?.request?.url,
+      status: event.event?.response?.status,
+      logs: event.logs,
+      exceptions: event.exceptions,
+    }));
+    
+    ctx.waitUntil(
+      fetch(env.LOG_ENDPOINT, {
+        method: "POST",
+        headers: { "Content-Type": "application/json" },
+        body: JSON.stringify(payload),
+      })
+    );
+  }
+};
+```
+
+### 2. Error Tracking to External Service
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    for (const event of events) {
+      // Only process errors
+      if (event.outcome === 'exception' || event.exceptions.length > 0) {
+        ctx.waitUntil(
+          fetch("https://error-tracker.example.com/errors", {
+            method: "POST",
+            headers: {
+              "Authorization": `Bearer ${env.ERROR_TRACKER_TOKEN}`,
+              "Content-Type": "application/json",
+            },
+            body: JSON.stringify({
+              script: event.scriptName,
+              timestamp: event.eventTimestamp,
+              exceptions: event.exceptions,
+              request: event.event?.request?.getUnredacted?.(),  // If needed
+              logs: event.logs,
+            }),
+          })
+        );
+      }
+    }
+  }
+};
+```
+
+### 3. Store Logs in KV
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const promises = events.map(event => {
+      const key = `log:${event.scriptName}:${event.eventTimestamp}`;
+      const value = JSON.stringify({
+        outcome: event.outcome,
+        logs: event.logs,
+        exceptions: event.exceptions,
+      });
+      
+      // TTL: 24 hours
+      return env.LOGS_KV.put(key, value, { expirationTtl: 86400 });
+    });
+    
+    ctx.waitUntil(Promise.all(promises));
+  }
+};
+```
+
+### 4. Analytics Engine for Aggregated Metrics
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const writes = events.map(event => 
+      env.ANALYTICS.writeDataPoint({
+        // String dimensions
+        blobs: [
+          event.scriptName,
+          event.outcome,
+          event.event?.request?.method ?? 'unknown',
+        ],
+        // Numeric metrics
+        doubles: [
+          1,  // Count
+          event.event?.response?.status ?? 0,
+        ],
+        // Indexed fields for filtering
+        indexes: [
+          event.event?.request?.cf?.colo ?? 'unknown',
+        ],
+      })
+    );
+    
+    ctx.waitUntil(Promise.all(writes));
+  }
+};
+```
+
+### 5. Filter Specific Routes/Patterns
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    // Only process API routes
+    const apiEvents = events.filter(event => 
+      event.event?.request?.url?.includes('/api/')
+    );
+    
+    if (apiEvents.length === 0) return;
+    
+    ctx.waitUntil(
+      fetch(env.API_LOGS_ENDPOINT, {
+        method: "POST",
+        body: JSON.stringify(apiEvents),
+      })
+    );
+  }
+};
+```
+
+### 6. Multi-Destination Logging
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    // Send errors to one place, everything else to another
+    const errors = events.filter(e => e.outcome === 'exception');
+    const success = events.filter(e => e.outcome === 'ok');
+    
+    const tasks = [];
+    
+    if (errors.length > 0) {
+      tasks.push(
+        fetch(env.ERROR_ENDPOINT, {
+          method: "POST",
+          body: JSON.stringify(errors),
+        })
+      );
+    }
+    
+    if (success.length > 0) {
+      tasks.push(
+        fetch(env.SUCCESS_ENDPOINT, {
+          method: "POST",
+          body: JSON.stringify(success),
+        })
+      );
+    }
+    
+    ctx.waitUntil(Promise.all(tasks));
+  }
+};
+```
+
+### 7. Performance Monitoring
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const metrics = events.map(event => ({
+      script: event.scriptName,
+      timestamp: event.eventTimestamp,
+      duration: calculateDuration(event),  // Custom logic
+      outcome: event.outcome,
+      status: event.event?.response?.status,
+      colo: event.event?.request?.cf?.colo,
+    }));
+    
+    ctx.waitUntil(
+      fetch(env.METRICS_ENDPOINT, {
+        method: "POST",
+        headers: { "X-API-Key": env.METRICS_API_KEY },
+        body: JSON.stringify(metrics),
+      })
+    );
+  }
+};
+```
+
+## Security & Privacy
+
+### Automatic Redaction
+By default, sensitive data is redacted from `TailRequest`:
+
+**Header redaction:**
+- Headers containing: `auth`, `key`, `secret`, `token`, `jwt` (case-insensitive)
+- `cookie` and `set-cookie` headers
+- Redacted values show as `"REDACTED"`
+
+**URL redaction:**
+- Hex IDs: 32+ hex digits → `"REDACTED"`
+- Base-64 IDs: 21+ chars with 2+ upper, 2+ lower, 2+ digits → `"REDACTED"`
+
+### Bypassing Redaction
+```typescript
+// Use with extreme caution
+const unredacted = event.event?.request?.getUnredacted();
+// unredacted.url and unredacted.headers contain raw values
+```
+
+**Best practices:**
+- Only call `getUnredacted()` when absolutely necessary
+- Never log unredacted sensitive data
+- Implement additional filtering before external transmission
+- Use environment variables for API keys, never hardcode
+
+## Wrangler CLI Usage
+
+### Deploy Tail Worker
+```bash
+wrangler deploy
+```
+
+### View Live Tail Locally (NOT Tail Workers)
+```bash
+# This streams logs to terminal, different from Tail Workers
+wrangler tail <producer-worker-name>
+```
+
+### Update Producer Configuration
+```bash
+# Edit wrangler.toml to add tail_consumers
+wrangler deploy
+```
+
+### Remove Tail Consumer
+```toml
+# Remove from wrangler.toml or set empty array
+tail_consumers = []
+```
+
+## TypeScript Types
+
+```typescript
+// Add to your Tail Worker
+export default {
+  async tail(
+    events: TailItem[],
+    env: Env,
+    ctx: ExecutionContext
+  ): Promise<void> {
+    // Implementation
+  }
+} satisfies ExportedHandler<Env>;
+
+interface Env {
+  // Your bindings
+  LOGS_KV: KVNamespace;
+  ANALYTICS: AnalyticsEngineDataset;
+  LOG_ENDPOINT: string;
+  API_TOKEN: string;
+}
+```
+
+## Testing & Development
+
+### Local Testing
+Tail Workers cannot be fully tested locally with `wrangler dev`. Deploy to staging environment for testing.
+
+### Testing Strategy
+1. Deploy producer Worker to staging
+2. Deploy Tail Worker to staging
+3. Configure `tail_consumers` in producer
+4. Trigger producer Worker requests
+5. Verify Tail Worker receives events (check destination logs/storage)
+
+### Debugging Tips
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    // Log to console for debugging (won't be captured by self)
+    console.log('Received events:', events.length);
+    
+    try {
+      // Your logic
+      await processEvents(events, env);
+    } catch (error) {
+      // Log errors
+      console.error('Tail Worker error:', error);
+      // Consider sending errors to monitoring service
+    }
+  }
+};
+```
+
+## Advanced Patterns
+
+### Batching Events
+```typescript
+// Use KV or Durable Objects to batch events before sending
+export default {
+  async tail(events, env, ctx) {
+    const batch = await env.BATCH_DO.get(env.BATCH_DO.idFromName("batch"));
+    ctx.waitUntil(batch.addEvents(events));
+  }
+};
+```
+
+### Sampling
+```typescript
+// Only process a percentage of events
+export default {
+  async tail(events, env, ctx) {
+    const sampleRate = 0.1;  // 10%
+    const sampledEvents = events.filter(() => Math.random() < sampleRate);
+    
+    if (sampledEvents.length > 0) {
+      ctx.waitUntil(sendToEndpoint(sampledEvents, env));
+    }
+  }
+};
+```
+
+### Workers for Platforms
+For dynamic dispatch Workers, `events` array contains TWO elements:
+1. Dynamic dispatch Worker event
+2. User Worker event
+
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    for (const event of events) {
+      // Distinguish between dispatch and user Worker
+      if (event.scriptName === 'dispatch-worker') {
+        // Handle dispatch Worker event
+      } else {
+        // Handle user Worker event
+      }
+    }
+  }
+};
+```
+
+## Common Pitfalls
+
+1. **Not using `ctx.waitUntil()`:**
+   ```typescript
+   // ❌ WRONG - async work may not complete
+   export default {
+     async tail(events) {
+       fetch(endpoint, { body: JSON.stringify(events) });
+     }
+   };
+   
+   // ✅ CORRECT
+   export default {
+     async tail(events, env, ctx) {
+       ctx.waitUntil(
+         fetch(endpoint, { body: JSON.stringify(events) })
+       );
+     }
+   };
+   ```
+
+2. **Missing tail() handler:**
+   Producer Worker deployment will fail if tail_consumers references a Worker without tail() handler.
+
+3. **Outcome vs HTTP Status:**
+   `outcome` is script execution status, NOT HTTP status. A Worker can return 500 but have outcome='ok' if script completed successfully.
+
+4. **Excessive logging:**
+   Tail Workers are invoked on EVERY producer invocation. Be mindful of volume and costs.
+
+5. **Blocking operations:**
+   Don't await in tail handler unless necessary. Use `ctx.waitUntil()` for fire-and-forget operations.
+
+## Integration Examples
+
+### Sentry
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const errors = events.filter(e => 
+      e.outcome === 'exception' || e.exceptions.length > 0
+    );
+    
+    for (const event of errors) {
+      ctx.waitUntil(
+        fetch(`https://sentry.io/api/${env.SENTRY_PROJECT}/store/`, {
+          method: "POST",
+          headers: {
+            "X-Sentry-Auth": `Sentry sentry_key=${env.SENTRY_KEY}`,
+            "Content-Type": "application/json",
+          },
+          body: JSON.stringify({
+            message: event.exceptions[0]?.message,
+            level: "error",
+            tags: { worker: event.scriptName },
+            extra: { event },
+          }),
+        })
+      );
+    }
+  }
+};
+```
+
+### Datadog
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    const logs = events.flatMap(event => 
+      event.logs.map(log => ({
+        ddsource: "cloudflare-worker",
+        ddtags: `worker:${event.scriptName},outcome:${event.outcome}`,
+        hostname: event.event?.request?.cf?.colo,
+        message: log.message.join(" "),
+        status: log.level,
+        timestamp: log.timestamp,
+      }))
+    );
+    
+    ctx.waitUntil(
+      fetch("https://http-intake.logs.datadoghq.com/v1/input", {
+        method: "POST",
+        headers: {
+          "DD-API-KEY": env.DATADOG_API_KEY,
+          "Content-Type": "application/json",
+        },
+        body: JSON.stringify(logs),
+      })
+    );
+  }
+};
+```
+
+## Related Resources
+- Tail Workers Docs: https://developers.cloudflare.com/workers/observability/logs/tail-workers/
+- Tail Handler API: https://developers.cloudflare.com/workers/runtime-apis/handlers/tail/
+- Analytics Engine: https://developers.cloudflare.com/analytics/analytics-engine/
+- OpenTelemetry Export: https://developers.cloudflare.com/workers/observability/exporting-opentelemetry-data/
+
+## Decision Tree
+
+```
+Need observability for Workers?
+├─ Batch export to known tools (Sentry/Grafana/Honeycomb)?
+│  └─ Use OpenTelemetry export (not Tail Workers)
+├─ Custom real-time processing needed?
+│  ├─ Aggregated metrics?
+│  │  └─ Use Tail Worker + Analytics Engine
+│  ├─ Error tracking?
+│  │  └─ Use Tail Worker + external service
+│  ├─ Custom logging/debugging?
+│  │  └─ Use Tail Worker + KV/HTTP endpoint
+│  └─ Complex event processing?
+│     └─ Use Tail Worker + Durable Objects
+└─ Quick debugging?
+   └─ Use `wrangler tail` (different from Tail Workers)
+```
+
+## Code Quality Guidelines
+
+### Type Safety
+```typescript
+// ✅ Use proper types
+interface Env {
+  LOG_ENDPOINT: string;
+  API_TOKEN: string;
+}
+
+export default {
+  async tail(
+    events: TailItem[],
+    env: Env,
+    ctx: ExecutionContext
+  ): Promise<void> {
+    // Type-safe implementation
+  }
+} satisfies ExportedHandler<Env>;
+
+// ❌ Avoid any
+export default {
+  async tail(events: any, env: any, ctx: any) {
+    // Unsafe
+  }
+};
+```
+
+### Error Handling
+```typescript
+export default {
+  async tail(events, env, ctx) {
+    ctx.waitUntil(
+      (async () => {
+        try {
+          await fetch(env.ENDPOINT, {
+            method: "POST",
+            body: JSON.stringify(events),
+          });
+        } catch (error) {
+          // Log to console or send to fallback
+          console.error("Failed to send events:", error);
+        }
+      })()
+    );
+  }
+};
+```
+
+### Minimal, Surgical Changes
+- Process only necessary events (filter early)
+- Avoid unnecessary data transformations
+- Keep handlers focused and simple
+
+## Summary
+Tail Workers provide real-time, custom event processing for Cloudflare Workers. Use them when you need fine-grained control over logging, error tracking, or analytics that goes beyond standard OTEL export. Always use `ctx.waitUntil()` for async work, be mindful of sensitive data redaction, and consider Analytics Engine for aggregated metrics.

package/skills/cloudflare-expert/references/terraform/README.md

@@ -0,0 +1,76 @@
+# Cloudflare Terraform Provider
+
+**Expert guidance for Cloudflare Terraform Provider - infrastructure as code for Cloudflare resources.**
+
+## Core Principles
+
+- **Provider-first**: Use Terraform provider for ALL infrastructure - never mix with wrangler.toml for the same resources
+- **State management**: Always use remote state (S3, Terraform Cloud, etc.) for team environments
+- **Modular architecture**: Create reusable modules for common patterns (zones, workers, pages)
+- **Version pinning**: Always pin provider version with `~>` for predictable upgrades
+- **Secret management**: Use variables + environment vars for sensitive data - never hardcode API tokens
+
+## Provider Setup
+
+### Basic Configuration
+
+```hcl
+terraform {
+  required_version = ">= 1.0"
+  
+  required_providers {
+    cloudflare = {
+      source  = "cloudflare/cloudflare"
+      version = "~> 5.15.0"
+    }
+  }
+}
+
+provider "cloudflare" {
+  api_token = var.cloudflare_api_token  # or CLOUDFLARE_API_TOKEN env var
+}
+```
+
+### Authentication Methods (priority order)
+
+1. **API Token** (RECOMMENDED): `api_token` or `CLOUDFLARE_API_TOKEN`
+   - Create: Dashboard → My Profile → API Tokens
+   - Scope to specific accounts/zones for security
+   
+2. **Global API Key** (LEGACY): `api_key` + `api_email` or `CLOUDFLARE_API_KEY` + `CLOUDFLARE_EMAIL`
+   - Less secure, use tokens instead
+   
+3. **User Service Key**: `user_service_key` for Origin CA certificates
+
+### Backend Configuration
+
+```hcl
+terraform {
+  backend "s3" {
+    bucket = "terraform-state"
+    key    = "cloudflare/terraform.tfstate"
+    region = "us-east-1"
+  }
+}
+```
+
+## Quick Reference: Common Commands
+
+```bash
+terraform init          # Initialize provider
+terraform plan          # Plan changes
+terraform apply         # Apply changes
+terraform destroy       # Destroy resources
+terraform import cloudflare_zone.example <zone-id>  # Import existing
+terraform state list    # List resources in state
+terraform output        # Show outputs
+terraform fmt -recursive  # Format code
+terraform validate      # Validate configuration
+```
+
+## See Also
+
+- [Configuration Reference](./configuration.md) - Resources for zones, DNS, workers, KV, R2, D1, Pages, rulesets
+- [API Reference](./api.md) - Data sources for existing resources
+- [Patterns & Use Cases](./patterns.md) - Architecture patterns, multi-env setup, CI/CD integration
+- [Troubleshooting & Best Practices](./gotchas.md) - Common issues, security, best practices