devstack 0.1.0

package/vendor/skills/skills/cloudflare/references/pages-functions/configuration.md

@@ -0,0 +1,130 @@
+# Configuration
+
+## TypeScript Setup
+
+**Generate types from wrangler.jsonc** (replaces deprecated `@cloudflare/workers-types`):
+
+```bash
+npx wrangler types
+```
+
+Creates `worker-configuration.d.ts` with typed `Env` interface based on your bindings.
+
+```typescript
+// functions/api.ts
+export const onRequest: PagesFunction<Env> = async ctx => {
+  // ctx.env.KV, ctx.env.DB, etc. are fully typed
+  return Response.json({ ok: true });
+};
+```
+
+**Manual types** (if not using wrangler types):
+
+```typescript
+interface Env {
+  KV: KVNamespace;
+  DB: D1Database;
+  API_KEY: string;
+}
+export const onRequest: PagesFunction<Env> = async ctx => {
+  /* ... */
+};
+```
+
+## wrangler.jsonc
+
+```jsonc
+{
+  "$schema": "./node_modules/wrangler/config-schema.json",
+  "name": "my-pages-app",
+  "pages_build_output_dir": "./dist",
+  "compatibility_date": "2025-01-01",
+  "compatibility_flags": ["nodejs_compat"],
+
+  "vars": { "API_URL": "https://api.example.com" },
+  "kv_namespaces": [{ "binding": "KV", "id": "abc123" }],
+  "d1_databases": [{ "binding": "DB", "database_name": "prod-db", "database_id": "xyz789" }],
+  "r2_buckets": [{ "binding": "BUCKET", "bucket_name": "my-bucket" }],
+  "durable_objects": {
+    "bindings": [{ "name": "COUNTER", "class_name": "Counter", "script_name": "counter-worker" }],
+  },
+  "services": [{ "binding": "AUTH", "service": "auth-worker" }],
+  "ai": { "binding": "AI" },
+  "vectorize": [{ "binding": "VECTORIZE", "index_name": "my-index" }],
+  "analytics_engine_datasets": [{ "binding": "ANALYTICS" }],
+}
+```
+
+## Environment Overrides
+
+Top-level → local dev, `env.preview` → preview, `env.production` → production
+
+```jsonc
+{
+  "vars": { "API_URL": "http://localhost:8787" },
+  "env": {
+    "production": { "vars": { "API_URL": "https://api.example.com" } },
+  },
+}
+```
+
+**Note:** If overriding `vars`, `kv_namespaces`, `d1_databases`, etc., ALL must be redefined (non-inheritable)
+
+## Local Secrets (.dev.vars)
+
+**Local dev only** - NOT deployed:
+
+```bash
+# .dev.vars (add to .gitignore)
+SECRET_KEY="my-secret-value"
+```
+
+Accessed via `ctx.env.SECRET_KEY`. Set production secrets:
+
+```bash
+echo "value" | npx wrangler pages secret put SECRET_KEY --project-name=my-app
+```
+
+## Static Config Files
+
+**\_routes.json** - Custom routing:
+
+```json
+{ "version": 1, "include": ["/api/*"], "exclude": ["/static/*"] }
+```
+
+**\_headers** - Static headers:
+
+```
+/static/*
+  Cache-Control: public, max-age=31536000
+```
+
+**\_redirects** - Redirects:
+
+```
+/old  /new  301
+```
+
+## Local Dev & Deployment
+
+```bash
+# Dev server
+npx wrangler pages dev ./dist
+
+# With bindings
+npx wrangler pages dev ./dist --kv=KV --d1=DB=db-id --r2=BUCKET
+
+# Durable Objects (2 terminals)
+cd do-worker && npx wrangler dev
+cd pages-project && npx wrangler pages dev ./dist --do COUNTER=Counter@do-worker
+
+# Deploy
+npx wrangler pages deploy ./dist
+npx wrangler pages deploy ./dist --branch preview
+
+# Download config
+npx wrangler pages download config my-project
+```
+
+**See also:** [api.md](./api.md) for binding usage examples

package/vendor/skills/skills/cloudflare/references/pages-functions/gotchas.md

@@ -0,0 +1,102 @@
+# Gotchas & Debugging
+
+## Error Diagnosis
+
+| Symptom                            | Likely Cause                                                                  | Solution                                                                                  |
+| ---------------------------------- | ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
+| **Function not invoking**          | Wrong `/functions` location, wrong extension, or `_routes.json` excludes path | Check `pages_build_output_dir`, use `.js`/`.ts`, verify `_routes.json`                    |
+| **`ctx.env.BINDING` undefined**    | Binding not configured or name mismatch                                       | Add to `wrangler.jsonc`, verify exact name (case-sensitive), redeploy                     |
+| **TypeScript errors on `ctx.env`** | Missing type definition                                                       | Run `wrangler types` or define `interface Env {}`                                         |
+| **Middleware not running**         | Wrong filename/location or missing `ctx.next()`                               | Name exactly `_middleware.js`, export `onRequest`, call `ctx.next()`                      |
+| **Secrets missing in production**  | `.dev.vars` not deployed                                                      | `.dev.vars` is local only - set production secrets via dashboard or `wrangler secret put` |
+| **Type mismatch on binding**       | Wrong interface type                                                          | See [api.md](./api.md) bindings table for correct types                                   |
+| **"KV key not found" but exists**  | Key in wrong namespace or env                                                 | Verify namespace binding, check preview vs production env                                 |
+| **Function times out**             | Synchronous wait or missing `await`                                           | All I/O must be async/await, use `ctx.waitUntil()` for background tasks                   |
+
+## Common Errors
+
+### TypeScript type errors
+
+**Problem:** `ctx.env.MY_BINDING` shows type error  
+**Cause:** No type definition for `Env`  
+**Solution:** Run `npx wrangler types` or manually define:
+
+```typescript
+interface Env {
+  MY_BINDING: KVNamespace;
+}
+export const onRequest: PagesFunction<Env> = async ctx => {
+  /* ... */
+};
+```
+
+### Secrets not available in production
+
+**Problem:** `ctx.env.SECRET_KEY` is undefined in production  
+**Cause:** `.dev.vars` is local-only, not deployed  
+**Solution:** Set production secrets:
+
+```bash
+echo "value" | npx wrangler pages secret put SECRET_KEY --project-name=my-app
+```
+
+## Debugging
+
+```typescript
+// Console logging
+export async function onRequest(ctx) {
+  console.log("Request:", ctx.request.method, ctx.request.url);
+  const res = await ctx.next();
+  console.log("Status:", res.status);
+  return res;
+}
+```
+
+```bash
+# Stream real-time logs
+npx wrangler pages deployment tail
+npx wrangler pages deployment tail --status error
+```
+
+```jsonc
+// Source maps (wrangler.jsonc)
+{ "upload_source_maps": true }
+```
+
+## Limits
+
+| Resource    | Free                 | Paid                      |
+| ----------- | -------------------- | ------------------------- |
+| CPU time    | 10ms                 | 50ms                      |
+| Memory      | 128 MB               | 128 MB                    |
+| Script size | 10 MB compressed     | 10 MB compressed          |
+| Env vars    | 5 KB per var, 64 max | 5 KB per var, 64 max      |
+| Requests    | 100k/day             | Unlimited ($0.50/million) |
+
+## Best Practices
+
+**Performance:** Minimize deps (cold start), use KV for cache/D1 for relational/R2 for large files, set `Cache-Control` headers, batch DB ops, handle errors gracefully
+
+**Security:** Never commit secrets (use `.dev.vars` + gitignore), validate input, sanitize before DB, implement auth middleware, set CORS headers, rate limit per-IP
+
+## Migration
+
+**Workers → Pages Functions:**
+
+- `export default { fetch(req, env) {} }` → `export function onRequest(ctx) { const { request, env } = ctx; }`
+- Use `_worker.js` for complex routing: `env.ASSETS.fetch(request)` for static files
+
+**Other platforms → Pages:**
+
+- File-based routing: `/functions/api/users.js` → `/api/users`
+- Dynamic routes: `[param]` not `:param`
+- Replace Node.js deps with Workers APIs or add `nodejs_compat` flag
+
+## Resources
+
+- [Official Docs](https://developers.cloudflare.com/pages/functions/)
+- [Workers APIs](https://developers.cloudflare.com/workers/runtime-apis/)
+- [Examples](https://github.com/cloudflare/pages-example-projects)
+- [Discord](https://discord.gg/cloudflaredev)
+
+**See also:** [configuration.md](./configuration.md) for TypeScript setup | [patterns.md](./patterns.md) for middleware/auth | [api.md](./api.md) for bindings

package/vendor/skills/skills/cloudflare/references/pages-functions/patterns.md

@@ -0,0 +1,148 @@
+# Common Patterns
+
+## Background Tasks (waitUntil)
+
+Non-blocking tasks after response sent (analytics, cleanup, webhooks):
+
+```typescript
+export async function onRequest(ctx: EventContext<Env>) {
+  const res = Response.json({ success: true });
+
+  ctx.waitUntil(ctx.env.KV.put("last-visit", new Date().toISOString()));
+  ctx.waitUntil(
+    Promise.all([
+      ctx.env.ANALYTICS.writeDataPoint({ event: "view" }),
+      fetch("https://webhook.site/...", { method: "POST" }),
+    ])
+  );
+
+  return res; // Returned immediately
+}
+```
+
+## Middleware & Auth
+
+```typescript
+// functions/_middleware.js (global) or functions/users/_middleware.js (scoped)
+export async function onRequest(ctx) {
+  try {
+    return await ctx.next();
+  } catch (err) {
+    return new Response(err.message, { status: 500 });
+  }
+}
+
+// Chained: export const onRequest = [errorHandler, auth, logger];
+
+// Auth
+async function auth(ctx: EventContext<Env>) {
+  const token = ctx.request.headers.get("authorization")?.replace("Bearer ", "");
+  if (!token) return new Response("Unauthorized", { status: 401 });
+  const session = await ctx.env.KV.get(`session:${token}`);
+  if (!session) return new Response("Invalid", { status: 401 });
+  ctx.data.user = JSON.parse(session);
+  return ctx.next();
+}
+```
+
+## CORS & Rate Limiting
+
+```typescript
+// CORS middleware
+const cors = { "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "GET, POST" };
+export async function onRequestOptions() {
+  return new Response(null, { headers: cors });
+}
+export async function onRequest(ctx) {
+  const res = await ctx.next();
+  Object.entries(cors).forEach(([k, v]) => res.headers.set(k, v));
+  return res;
+}
+
+// Rate limiting (KV-based)
+async function rateLimit(ctx: EventContext<Env>) {
+  const ip = ctx.request.headers.get("CF-Connecting-IP") || "unknown";
+  const count = parseInt((await ctx.env.KV.get(`rate:${ip}`)) || "0");
+  if (count >= 100) return new Response("Rate limited", { status: 429 });
+  await ctx.env.KV.put(`rate:${ip}`, (count + 1).toString(), { expirationTtl: 3600 });
+  return ctx.next();
+}
+```
+
+## Forms, Caching, Redirects
+
+```typescript
+// JSON & file upload
+export async function onRequestPost(ctx) {
+  const ct = ctx.request.headers.get("content-type") || "";
+  if (ct.includes("application/json")) return Response.json(await ctx.request.json());
+  if (ct.includes("multipart/form-data")) {
+    const file = (await ctx.request.formData()).get("file") as File;
+    await ctx.env.BUCKET.put(file.name, file.stream());
+    return Response.json({ uploaded: file.name });
+  }
+}
+
+// Cache API
+export async function onRequest(ctx) {
+  let res = await caches.default.match(ctx.request);
+  if (!res) {
+    res = new Response("Data");
+    res.headers.set("Cache-Control", "public, max-age=3600");
+    ctx.waitUntil(caches.default.put(ctx.request, res.clone()));
+  }
+  return res;
+}
+
+// Redirects
+export async function onRequest(ctx) {
+  if (new URL(ctx.request.url).pathname === "/old") {
+    return Response.redirect(new URL("/new", ctx.request.url), 301);
+  }
+  return ctx.next();
+}
+```
+
+## Testing
+
+**Unit tests** (Vitest + cloudflare:test):
+
+```typescript
+import { env } from "cloudflare:test";
+import { it, expect } from "vitest";
+import { onRequest } from "../functions/api";
+
+it("returns JSON", async () => {
+  const req = new Request("http://localhost/api");
+  const ctx = { request: req, env, params: {}, data: {} } as EventContext;
+  const res = await onRequest(ctx);
+  expect(res.status).toBe(200);
+});
+```
+
+**Integration:** `wrangler pages dev` + Playwright/Cypress
+
+## Advanced Mode (\_worker.js)
+
+Use `_worker.js` for complex routing (replaces `/functions`):
+
+```typescript
+interface Env {
+  ASSETS: Fetcher;
+  KV: KVNamespace;
+}
+
+export default {
+  async fetch(request: Request, env: Env): Promise<Response> {
+    const url = new URL(request.url);
+    if (url.pathname.startsWith("/api/")) {
+      return Response.json({ data: await env.KV.get("key") });
+    }
+    return env.ASSETS.fetch(request); // Static files
+  },
+} satisfies ExportedHandler<Env>;
+```
+
+**When:** Existing Worker, framework-generated (Next.js/SvelteKit), custom routing logic
+
+**See also:** [api.md](./api.md) for `env.ASSETS.fetch()` | [gotchas.md](./gotchas.md) for debugging

package/vendor/skills/skills/cloudflare/references/pipelines/README.md

@@ -0,0 +1,109 @@
+# Cloudflare Pipelines
+
+ETL streaming platform for ingesting, transforming, and loading data into R2 with SQL transformations.
+
+## Overview
+
+Pipelines provides:
+
+- **Streams**: Durable event buffers (HTTP/Workers ingestion)
+- **Pipelines**: SQL-based transformations
+- **Sinks**: R2 destinations (Iceberg tables or Parquet/JSON files)
+
+**Status**: Open beta (Workers Paid plan)  
+**Pricing**: No charge beyond standard R2 storage/operations
+
+## Architecture
+
+```
+Data Sources → Streams → Pipelines (SQL) → Sinks → R2
+                 ↑          ↓                ↓
+            HTTP/Workers  Transform     Iceberg/Parquet
+```
+
+| Component | Purpose            | Key Feature                            |
+| --------- | ------------------ | -------------------------------------- |
+| Streams   | Event ingestion    | Structured (validated) or unstructured |
+| Pipelines | Transform with SQL | Immutable after creation               |
+| Sinks     | Write to R2        | Exactly-once delivery                  |
+
+## Quick Start
+
+```bash
+# Interactive setup (recommended)
+npx wrangler pipelines setup
+```
+
+**Minimal Worker example:**
+
+```typescript
+interface Env {
+  STREAM: Pipeline;
+}
+
+export default {
+  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
+    const event = { user_id: "123", event_type: "purchase", amount: 29.99 };
+
+    // Fire-and-forget pattern
+    ctx.waitUntil(env.STREAM.send([event]));
+
+    return new Response("OK");
+  },
+} satisfies ExportedHandler<Env>;
+```
+
+## Which Sink Type?
+
+```
+Need SQL queries on data?
+  → R2 Data Catalog (Iceberg)
+    ✅ ACID transactions, time-travel, schema evolution
+    ❌ More setup complexity (namespace, table, catalog token)
+
+Just file storage/archival?
+  → R2 Storage (Parquet)
+    ✅ Simple, direct file access
+    ❌ No built-in SQL queries
+
+Using external tools (Spark/Athena)?
+  → R2 Storage (Parquet with partitioning)
+    ✅ Standard format, partition pruning for performance
+    ❌ Must manage schema compatibility yourself
+```
+
+## Common Use Cases
+
+- **Analytics pipelines**: Clickstream, telemetry, server logs
+- **Data warehousing**: ETL into queryable Iceberg tables
+- **Event processing**: Mobile/IoT with enrichment
+- **Ecommerce analytics**: User events, purchases, views
+
+## Reading Order
+
+**New to Pipelines?** Start here:
+
+1. [configuration.md](./configuration.md) - Setup streams, sinks, pipelines
+2. [api.md](./api.md) - Send events, TypeScript types, SQL functions
+3. [patterns.md](./patterns.md) - Best practices, integrations, complete example
+4. [gotchas.md](./gotchas.md) - Critical warnings, troubleshooting
+
+**Task-based routing:**
+
+- Setup pipeline → [configuration.md](./configuration.md)
+- Send/query data → [api.md](./api.md)
+- Implement pattern → [patterns.md](./patterns.md)
+- Debug issue → [gotchas.md](./gotchas.md)
+
+## In This Reference
+
+- [configuration.md](./configuration.md) - wrangler.jsonc bindings, schema definition, sink options, CLI commands
+- [api.md](./api.md) - Pipeline binding interface, send() method, HTTP ingest, SQL function reference
+- [patterns.md](./patterns.md) - Fire-and-forget, schema validation with Zod, integrations, performance tuning
+- [gotchas.md](./gotchas.md) - Silent validation failures, immutable pipelines, latency expectations, limits
+
+## See Also
+
+- [r2](../r2/) - R2 storage backend for sinks
+- [queues](../queues/) - Compare with Queues for async processing
+- [workers](../workers/) - Worker runtime for event ingestion

package/vendor/skills/skills/cloudflare/references/pipelines/api.md

@@ -0,0 +1,214 @@
+# Pipelines API Reference
+
+## Pipeline Binding Interface
+
+```typescript
+// From @cloudflare/workers-types
+interface Pipeline {
+  send(data: object | object[]): Promise<void>;
+}
+
+interface Env {
+  STREAM: Pipeline;
+}
+
+export default {
+  async fetch(request: Request, env: Env): Promise<Response> {
+    // send() returns Promise<void> - no result data
+    await env.STREAM.send([event]);
+    return new Response("OK");
+  },
+} satisfies ExportedHandler<Env>;
+```
+
+**Key points:**
+
+- `send()` accepts single object or array
+- Always returns `Promise<void>` (no confirmation data)
+- Throws on network/validation errors (wrap in try/catch)
+- Use `ctx.waitUntil()` for fire-and-forget pattern
+
+## Writing Events
+
+### Single Event
+
+```typescript
+await env.STREAM.send([
+  {
+    user_id: "12345",
+    event_type: "purchase",
+    product_id: "widget-001",
+    amount: 29.99,
+  },
+]);
+```
+
+### Batch Events
+
+```typescript
+const events = [
+  { user_id: "user1", event_type: "view" },
+  { user_id: "user2", event_type: "purchase", amount: 50 },
+];
+await env.STREAM.send(events);
+```
+
+**Limits:**
+
+- Max 1 MB per request
+- 5 MB/s per stream
+
+### Fire-and-Forget Pattern
+
+```typescript
+export default {
+  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
+    const event = {
+      /* ... */
+    };
+
+    // Don't block response on send
+    ctx.waitUntil(env.STREAM.send([event]));
+
+    return new Response("OK");
+  },
+};
+```
+
+### Error Handling
+
+```typescript
+try {
+  await env.STREAM.send([event]);
+} catch (error) {
+  console.error("Pipeline send failed:", error);
+  // Log to another system, retry, or return error response
+  return new Response("Failed to track event", { status: 500 });
+}
+```
+
+## HTTP Ingest API
+
+### Endpoint Format
+
+```
+https://{stream-id}.ingest.cloudflare.com
+```
+
+Get `{stream-id}` from: `npx wrangler pipelines streams list`
+
+### Request Format
+
+**CRITICAL:** Must send array, not single object
+
+```bash
+# ✅ Correct
+curl -X POST https://{stream-id}.ingest.cloudflare.com \
+  -H "Content-Type: application/json" \
+  -d '[{"user_id": "123", "event_type": "purchase"}]'
+
+# ❌ Wrong - will fail
+curl -X POST https://{stream-id}.ingest.cloudflare.com \
+  -H "Content-Type: application/json" \
+  -d '{"user_id": "123", "event_type": "purchase"}'
+```
+
+### Authentication
+
+```bash
+curl -X POST https://{stream-id}.ingest.cloudflare.com \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer YOUR_API_TOKEN" \
+  -d '[{"event": "data"}]'
+```
+
+**Required permission:** Workers Pipeline Send
+
+Create token: Dashboard → Workers → API tokens → Create with Pipeline Send permission
+
+### Response Codes
+
+| Code | Meaning           | Action                             |
+| ---- | ----------------- | ---------------------------------- |
+| 200  | Accepted          | Success                            |
+| 400  | Invalid format    | Check JSON array, schema match     |
+| 401  | Auth failed       | Verify token valid                 |
+| 413  | Payload too large | Split into smaller batches (<1 MB) |
+| 429  | Rate limited      | Back off, retry with delay         |
+| 5xx  | Server error      | Retry with exponential backoff     |
+
+## SQL Functions Quick Reference
+
+Available in `INSERT INTO sink SELECT ... FROM stream` transformations:
+
+| Function                     | Example                                             | Use Case                  |
+| ---------------------------- | --------------------------------------------------- | ------------------------- |
+| `UPPER(s)`                   | `UPPER(event_type)`                                 | Normalize strings         |
+| `LOWER(s)`                   | `LOWER(email)`                                      | Case-insensitive matching |
+| `CONCAT(...)`                | `CONCAT(user_id, '_', product_id)`                  | Generate composite keys   |
+| `CASE WHEN ... THEN ... END` | `CASE WHEN amount > 100 THEN 'high' ELSE 'low' END` | Conditional enrichment    |
+| `CAST(x AS type)`            | `CAST(timestamp AS string)`                         | Type conversion           |
+| `COALESCE(x, y)`             | `COALESCE(amount, 0.0)`                             | Default values            |
+| Math operators               | `amount * 1.1`, `price / quantity`                  | Calculations              |
+| Comparison                   | `amount > 100`, `status IN ('active', 'pending')`   | Filtering                 |
+
+**String types for CAST:** `string`, `int32`, `int64`, `float32`, `float64`, `bool`, `timestamp`
+
+Full reference: [Pipelines SQL Reference](https://developers.cloudflare.com/pipelines/sql-reference/)
+
+## SQL Transform Examples
+
+### Filter Events
+
+```sql
+INSERT INTO my_sink
+SELECT * FROM my_stream
+WHERE event_type = 'purchase' AND amount > 100
+```
+
+### Select Specific Fields
+
+```sql
+INSERT INTO my_sink
+SELECT user_id, event_type, timestamp, amount
+FROM my_stream
+```
+
+### Transform and Enrich
+
+```sql
+INSERT INTO my_sink
+SELECT
+  user_id,
+  UPPER(event_type) as event_type,
+  timestamp,
+  amount * 1.1 as amount_with_tax,
+  CONCAT(user_id, '_', product_id) as unique_key,
+  CASE
+    WHEN amount > 1000 THEN 'high_value'
+    WHEN amount > 100 THEN 'medium_value'
+    ELSE 'low_value'
+  END as customer_tier
+FROM my_stream
+WHERE event_type IN ('purchase', 'refund')
+```
+
+## Querying Results (R2 Data Catalog)
+
+```bash
+export WRANGLER_R2_SQL_AUTH_TOKEN=YOUR_CATALOG_TOKEN
+
+npx wrangler r2 sql query "warehouse_name" "
+SELECT
+  event_type,
+  COUNT(*) as event_count,
+  SUM(amount) as total_revenue
+FROM default.my_table
+WHERE event_type = 'purchase'
+  AND timestamp >= '2025-01-01'
+GROUP BY event_type
+ORDER BY total_revenue DESC
+LIMIT 100"
+```
+
+**Note:** Iceberg tables support standard SQL queries with GROUP BY, JOINs, WHERE, ORDER BY, etc.