@vodailoc/kilo-kit-mcp 1.1.0

package/skills/engineering/web-frameworks/references/turborepo-pipelines.md

@@ -0,0 +1,517 @@
+# Turborepo Task Pipelines
+
+Task orchestration, dependencies, and parallel execution strategies.
+
+## Pipeline Configuration
+
+Define tasks in `turbo.json`:
+
+```json
+{
+  "$schema": "https://turbo.build/schema.json",
+  "pipeline": {
+    "build": {
+      "dependsOn": ["^build"],
+      "outputs": ["dist/**", ".next/**"]
+    },
+    "test": {
+      "dependsOn": ["build"],
+      "outputs": ["coverage/**"]
+    },
+    "lint": {},
+    "dev": {
+      "cache": false,
+      "persistent": true
+    }
+  }
+}
+```
+
+## Task Dependencies
+
+### Topological Dependencies (^)
+
+`^` means "run this task in dependencies first":
+
+```json
+{
+  "pipeline": {
+    "build": {
+      "dependsOn": ["^build"]
+    }
+  }
+}
+```
+
+Example flow:
+```
+packages/ui (dependency)
+  ↓ builds first
+apps/web (depends on @repo/ui)
+  ↓ builds second
+```
+
+### Internal Dependencies
+
+Run tasks in same package first:
+
+```json
+{
+  "pipeline": {
+    "deploy": {
+      "dependsOn": ["build", "test"]
+    }
+  }
+}
+```
+
+Execution order in same package:
+1. Run `build`
+2. Run `test`
+3. Run `deploy`
+
+### Combined Dependencies
+
+Mix topological and internal:
+
+```json
+{
+  "pipeline": {
+    "test": {
+      "dependsOn": ["^build", "lint"]
+    }
+  }
+}
+```
+
+Execution order:
+1. Build all dependencies (`^build`)
+2. Lint current package (`lint`)
+3. Run tests (`test`)
+
+## Task Configuration Options
+
+### outputs
+
+Define what gets cached:
+
+```json
+{
+  "build": {
+    "outputs": [
+      "dist/**",           // All files in dist
+      ".next/**",          // Next.js build
+      "!.next/cache/**",   // Exclude Next.js cache
+      "build/**",          // Build directory
+      "public/dist/**"     // Public assets
+    ]
+  }
+}
+```
+
+### cache
+
+Enable/disable caching:
+
+```json
+{
+  "dev": {
+    "cache": false        // Don't cache dev server
+  },
+  "build": {
+    "cache": true         // Cache build (default)
+  }
+}
+```
+
+### persistent
+
+Keep task running (for dev servers):
+
+```json
+{
+  "dev": {
+    "cache": false,
+    "persistent": true    // Don't kill after completion
+  }
+}
+```
+
+### env
+
+Environment variables affecting output:
+
+```json
+{
+  "build": {
+    "env": [
+      "NODE_ENV",
+      "NEXT_PUBLIC_API_URL",
+      "DATABASE_URL"
+    ]
+  }
+}
+```
+
+### passThroughEnv
+
+Pass env vars without affecting cache:
+
+```json
+{
+  "build": {
+    "passThroughEnv": [
+      "DEBUG",            // Pass through but don't invalidate cache
+      "LOG_LEVEL"
+    ]
+  }
+}
+```
+
+### inputs
+
+Override default input detection:
+
+```json
+{
+  "build": {
+    "inputs": [
+      "src/**/*.ts",
+      "!src/**/*.test.ts", // Exclude test files
+      "package.json"
+    ]
+  }
+}
+```
+
+### outputMode
+
+Control output display:
+
+```json
+{
+  "build": {
+    "outputMode": "full"        // Show all output
+  },
+  "dev": {
+    "outputMode": "hash-only"   // Show cache hash only
+  },
+  "test": {
+    "outputMode": "new-only"    // Show new output only
+  },
+  "lint": {
+    "outputMode": "errors-only" // Show errors only
+  }
+}
+```
+
+## Running Tasks
+
+### Basic Execution
+
+```bash
+# Run build in all packages
+turbo run build
+
+# Run multiple tasks
+turbo run build test lint
+
+# Run with specific package manager
+pnpm turbo run build
+```
+
+### Filtering
+
+Run tasks in specific packages:
+
+```bash
+# Single package
+turbo run build --filter=web
+turbo run build --filter=@repo/ui
+
+# Multiple packages
+turbo run build --filter=web --filter=api
+
+# All apps
+turbo run build --filter='./apps/*'
+
+# Pattern matching
+turbo run test --filter='*-api'
+```
+
+### Dependency Filtering
+
+```bash
+# Package and its dependencies
+turbo run build --filter='...web'
+
+# Package's dependencies only (exclude package itself)
+turbo run build --filter='...^web'
+
+# Package and its dependents
+turbo run test --filter='ui...'
+
+# Package's dependents only
+turbo run test --filter='^ui...'
+```
+
+### Git-Based Filtering
+
+Run only on changed packages:
+
+```bash
+# Changed since main branch
+turbo run build --filter='[main]'
+
+# Changed since HEAD~1
+turbo run build --filter='[HEAD~1]'
+
+# Changed in working directory
+turbo run test --filter='...[HEAD]'
+
+# Package and dependencies, only if changed
+turbo run build --filter='...[origin/main]'
+```
+
+## Concurrency Control
+
+### Parallel Execution (Default)
+
+Turborepo runs tasks in parallel when safe:
+
+```bash
+# Run with default parallelism
+turbo run build
+```
+
+### Limit Concurrency
+
+```bash
+# Max 3 tasks at once
+turbo run build --concurrency=3
+
+# 50% of CPU cores
+turbo run build --concurrency=50%
+
+# No parallelism (sequential)
+turbo run build --concurrency=1
+```
+
+### Continue on Error
+
+```bash
+# Don't stop on first error
+turbo run test --continue
+```
+
+## Task Execution Order
+
+Example monorepo:
+```
+apps/
+├── web (depends on @repo/ui, @repo/utils)
+└── docs (depends on @repo/ui)
+packages/
+├── ui (depends on @repo/utils)
+└── utils (no dependencies)
+```
+
+With config:
+```json
+{
+  "pipeline": {
+    "build": {
+      "dependsOn": ["^build"]
+    }
+  }
+}
+```
+
+Execution order for `turbo run build`:
+1. **Wave 1** (parallel): `@repo/utils` (no dependencies)
+2. **Wave 2** (parallel): `@repo/ui` (depends on utils)
+3. **Wave 3** (parallel): `web` and `docs` (both depend on ui)
+
+## Complex Pipeline Examples
+
+### Full-Stack Application
+
+```json
+{
+  "pipeline": {
+    "build": {
+      "dependsOn": ["^build"],
+      "outputs": [".next/**", "dist/**"]
+    },
+    "test": {
+      "dependsOn": ["^build"],
+      "outputs": ["coverage/**"]
+    },
+    "lint": {
+      "dependsOn": ["^build"]
+    },
+    "typecheck": {
+      "dependsOn": ["^build"]
+    },
+    "dev": {
+      "cache": false,
+      "persistent": true
+    },
+    "deploy": {
+      "dependsOn": ["build", "test", "lint", "typecheck"]
+    }
+  }
+}
+```
+
+### Monorepo with Code Generation
+
+```json
+{
+  "pipeline": {
+    "generate": {
+      "cache": false,
+      "outputs": ["src/generated/**"]
+    },
+    "build": {
+      "dependsOn": ["^build", "generate"],
+      "outputs": ["dist/**"]
+    },
+    "test": {
+      "dependsOn": ["generate"],
+      "outputs": ["coverage/**"]
+    }
+  }
+}
+```
+
+### Database-Dependent Pipeline
+
+```json
+{
+  "pipeline": {
+    "db:generate": {
+      "cache": false
+    },
+    "db:migrate": {
+      "cache": false
+    },
+    "build": {
+      "dependsOn": ["^build", "db:generate"],
+      "outputs": ["dist/**"]
+    },
+    "test:unit": {
+      "dependsOn": ["build"]
+    },
+    "test:integration": {
+      "dependsOn": ["db:migrate"],
+      "cache": false
+    }
+  }
+}
+```
+
+## Dry Run
+
+Preview execution without running:
+
+```bash
+# See what would run
+turbo run build --dry-run
+
+# JSON output for scripts
+turbo run build --dry-run=json
+
+# Show full task graph
+turbo run build --graph
+```
+
+## Force Execution
+
+Ignore cache and run tasks:
+
+```bash
+# Force rebuild everything
+turbo run build --force
+
+# Force specific package
+turbo run build --filter=web --force
+```
+
+## Output Control
+
+```bash
+# Show only errors
+turbo run build --output-logs=errors-only
+
+# Show new logs only
+turbo run build --output-logs=new-only
+
+# Show cache hash only
+turbo run build --output-logs=hash-only
+
+# Show full output
+turbo run build --output-logs=full
+```
+
+## Best Practices
+
+1. **Use topological dependencies** - `^build` ensures correct build order
+2. **Cache build outputs** - Define `outputs` for faster rebuilds
+3. **Disable cache for dev** - Set `cache: false` for dev servers
+4. **Mark persistent tasks** - Use `persistent: true` for long-running tasks
+5. **Filter strategically** - Use filters to run only affected tasks
+6. **Control concurrency** - Limit parallelism for resource-intensive tasks
+7. **Configure env vars** - Include vars that affect output in `env`
+8. **Use dry-run** - Preview execution plan before running
+9. **Continue on error in CI** - Use `--continue` to see all errors
+10. **Leverage git filtering** - Run only on changed packages in CI
+
+## Common Patterns
+
+### CI/CD Pipeline
+
+```yaml
+# .github/workflows/ci.yml
+jobs:
+  build:
+    steps:
+      - run: turbo run build test lint --filter='...[origin/main]'
+```
+
+Only build/test/lint changed packages and their dependents.
+
+### Development Workflow
+
+```bash
+# Start all dev servers
+turbo run dev
+
+# Start specific app with dependencies
+turbo run dev --filter=web...
+```
+
+### Pre-commit Hook
+
+```json
+// package.json
+{
+  "scripts": {
+    "pre-commit": "turbo run lint test --filter='...[HEAD]'"
+  }
+}
+```
+
+Only lint/test changed packages.
+
+### Deployment
+
+```bash
+# Build and test specific app
+turbo run build test --filter=web...
+
+# Deploy if successful
+turbo run deploy --filter=web
+```
+
+Build app and its dependencies, then deploy.