@nurix/etna 0.15.6 → 0.15.8

package/data/harnesses/code-fable/skills/workflow-mapper/SKILL.md

@@ -0,0 +1,430 @@
+---
+name: workflow-mapper
+description: Discover and document all major workflows in a codebase by tracing call chains from user action to database/external system. Produces mermaid sequence diagrams and markdown documentation. Use when the user asks to map workflows, trace flows, document sequences, understand how features work end-to-end, or generate architecture diagrams. Triggers on "map workflows", "trace workflows", "workflow diagram", "sequence diagram", "how does X work end-to-end", "document flows", "codebase workflows", "how does this feature work", "trace the request flow", "show me the call chain", "what happens when a user does X", "end-to-end flow".
+---
+
+# Workflow Mapper Skill
+
+Identify and document every major workflow in a codebase by tracing execution paths from trigger to terminal effect. Output is a single markdown file with mermaid sequence diagrams.
+
+---
+
+## Decision Tree
+
+```
+User request
+    │
+    ├─ "Map all workflows" ────────────▶ Full Discovery (all phases)
+    │
+    ├─ "Map workflow for {feature}" ───▶ Targeted Discovery (skip Phase 1 broad scan)
+    │
+    ├─ "Update workflow map" ──────────▶ Incremental (diff against existing map)
+    │
+    └─ "How does X work end-to-end" ──▶ Single Workflow Trace (Phase 2-4 only)
+```
+
+---
+
+## Phase 1: Entry Point Discovery
+
+Find all places where a multi-step process begins. Use parallel sub-agents — one per technique.
+
+### Technique 1: Route Harvesting
+
+Scan all API route definitions. These are the primary entry points for backend workflows.
+
+```
+Search patterns (adapt to framework):
+  Express:    router.(get|post|put|patch|delete)\(
+  Fastify:    fastify.(get|post|put|patch|delete)\(
+  Next.js:    export (async )?function (GET|POST|PUT|PATCH|DELETE)
+  Hono:       app.(get|post|put|patch|delete)\(
+  tRPC:       .query\(|.mutation\(
+```
+
+For each route, record: `{ method, path, handler_function, file }`.
+
+Filter out trivial CRUD (single DB call + response). Keep routes that:
+- Call `startWorkflow`, `addJob`, `enqueue`, `emit`, `publish` (async dispatch)
+- Call 2+ service functions sequentially
+- Have transaction blocks (`$transaction`, `BEGIN`, `withTransaction`)
+- Return a job/workflow ID for polling
+- Trigger WebSocket/SSE events
+
+### Technique 2: Background Job Discovery
+
+Find all registered background jobs, workflows, and queues.
+
+```
+Search patterns:
+  Temporal:   export async function .*Workflow|workflowModules|startChild\(
+  BullMQ:     new Queue\(|new Worker\(|.add\(|.process\(
+  Celery:     @app.task|@shared_task
+  Inngest:    createFunction\(|inngest.send\(
+  Cron:       cron.schedule\(|@Cron\(|node-cron
+  EventEmitter: .on\(|.emit\(|EventEmitter
+```
+
+For Temporal specifically, also check:
+- `temporal/config/workflowModules.js` — workflow registry with queues and dependencies
+- `temporal/config/activityModules.js` — activity aggregation
+
+### Technique 3: Frontend Action Tracing
+
+Find UI-initiated multi-step flows.
+
+```
+Search patterns:
+  Redux thunks:   createAsyncThunk\(|dispatch\(
+  API calls:      api\.(post|put|patch|delete)\(|axios\.(post|put|patch|delete)\(|fetch\(
+  Form submits:   onSubmit|handleSubmit
+  Custom hooks:   use[A-Z].*\(\) that contain API calls
+```
+
+Focus on actions that:
+- POST/PUT/PATCH/DELETE to backend (not just GETs)
+- Dispatch multiple Redux actions in sequence
+- Start polling loops (`setInterval`, recursive `setTimeout`, retry patterns)
+- Open WebSocket connections or listen for server-sent events
+
+### Technique 4: Event & Message Tracing
+
+Find event-driven workflows (WebSocket, pub/sub, SSE).
+
+```
+Search patterns:
+  WebSocket:  socket.on\(|io.on\(|ws.on\(
+  SSE:        EventSource|text/event-stream
+  Redis pub:  .publish\(|.subscribe\(
+  Message:    .sendMessage\(|.postMessage\(
+```
+
+### Technique 5: Scheduled & Lifecycle Workflows
+
+```
+Search patterns:
+  App startup:    app.listen|bootstrap|onModuleInit|main\(\)
+  Shutdown:       SIGTERM|SIGINT|gracefulShutdown|onModuleDestroy
+  Migrations:     migrate|seed|up\(\)|down\(\)
+  Health checks:  /health|/ping|/ready|/live
+```
+
+---
+
+## Phase 2: Call Chain Tracing
+
+For each discovered entry point, trace the full execution path. Use one sub-agent per workflow for parallelism.
+
+### Tracing Algorithm
+
+```
+1. Start at the entry point function
+2. Read the function body
+3. For each function call or await:
+   a. Is it a DB operation? → Record as "Database" participant action
+   b. Is it an external API call? → Record as "{ServiceName}" participant action
+   c. Is it a queue/workflow dispatch? → Record as "{QueueSystem}" participant, then recurse into the dispatched handler
+   d. Is it a Redux dispatch? → Record as "Redux" participant action
+   e. Is it an internal function in the same service? → Recurse (but cap depth at 4)
+   f. Is it a WebSocket emit? → Record as "WebSocket" participant action
+4. Record branching (if/else, try/catch, switch) as alt/opt blocks
+5. Record loops (polling, retry, pagination) as loop blocks
+6. Stop when you hit: a return to the caller, a terminal DB write, or an external system response
+7. If a workflow trace exceeds the configured depth cap, stop tracing and note the overflow point with a comment like "→ [continues deeper, capped at depth N]". Do not silently truncate.
+```
+
+### Participant Identification
+
+Map code actors to diagram participants using this hierarchy:
+
+| Code Pattern | Participant Name |
+|---|---|
+| React component, hook, event handler | `User` + `Frontend` |
+| Express/API route handler | `API` |
+| Temporal workflow | `Temporal` |
+| BullMQ worker/processor | `Queue` |
+| Prisma/Sequelize/Knex call | `Database` |
+| External HTTP call (axios, fetch to external URL) | `{ServiceName}` (derive from URL or config) |
+| Redis operation | `Cache` |
+| WebSocket emit/broadcast | `WebSocket` |
+| File system / S3 / storage | `Storage` |
+| Git API calls | `Git` |
+| Email/SMS send | `Notifications` |
+| Auth provider (Keycloak, Better Auth, Auth0) | `Auth` |
+
+### Branching Detection
+
+Look for conditional logic that represents meaningful workflow forks:
+
+```
+- Version conflicts (optimistic locking)     → alt block
+- Permission/auth checks                      → opt block
+- Feature flags                               → opt block
+- Error handling with different outcomes       → alt block
+- Retry with backoff                          → loop block
+- Polling for completion                      → loop block
+- Batch processing                            → loop block
+```
+
+---
+
+## Phase 3: Classification & Filtering
+
+Not every API route is a "workflow". Classify and filter.
+
+### Workflow Categories
+
+| Category | Criteria | Include? |
+|---|---|---|
+| **Orchestration** | Multi-step, multi-service, async handoffs (save, publish, deploy) | Always |
+| **Auth & Session** | Login, logout, token refresh, SSO flows | Always |
+| **Real-time** | WebSocket connections, live updates, presence | Always |
+| **Event-driven** | Pub/sub chains, webhook handlers, event sourcing | Always |
+| **Scheduled** | Cron jobs, recurring workflows, cleanup tasks | Always |
+| **Data Pipeline** | ETL, import/export, migration, sync between systems | Always |
+| **Simple CRUD** | Single DB call + response, no branching | Skip |
+| **Static Serving** | File serving, asset delivery | Skip |
+| **Health/Config** | Health checks, config endpoints | Skip |
+
+### Complexity Scoring
+
+Score each workflow to determine diagram detail level:
+
+- **Steps**: Count distinct operations (DB calls, API calls, dispatches). 1-2 = simple, 3-5 = moderate, 6+ = complex
+- **Participants**: Count distinct systems involved. 2 = simple, 3-4 = moderate, 5+ = complex
+- **Branching**: Count alt/opt blocks. 0 = linear, 1-2 = moderate, 3+ = complex
+- **Async**: Has polling, callbacks, child workflows? +1 complexity level
+
+Simple workflows get a compact diagram. Complex workflows get full detail with alt/opt/loop blocks.
+
+---
+
+## Phase 4: Mermaid Diagram Generation
+
+### Diagram Template
+
+````markdown
+### {Workflow Name}
+
+**Trigger:** {What initiates this — button click, API call, cron schedule, event}
+**Participants:** {List of systems involved}
+**Key files:** {3-5 most important files in the chain}
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant Frontend
+    participant API
+    participant Database
+
+    User->>Frontend: {User action}
+    Frontend->>API: {HTTP method} {path}
+    API->>Database: {Operation}
+    Database-->>API: {Result}
+    API-->>Frontend: {Response}
+    Frontend-->>User: {UI update}
+```
+
+**Notes:**
+- {Important implementation detail not visible in diagram}
+- {Performance consideration, timeout, retry policy}
+- {Known edge cases or failure modes}
+````
+
+### Diagram Rules
+
+1. **Arrow types**: Solid (`->>`) for calls/requests, dashed (`-->>`) for returns/responses
+2. **Activation bars**: Use `activate`/`deactivate` only for long-running operations (Temporal workflows, polling loops)
+3. **Alt blocks**: Use for real workflow branching (conflict vs success), not for error handling
+4. **Loop blocks**: Use for polling, retry, batch processing — include the condition
+5. **Opt blocks**: Use for conditional steps (feature flags, optional integrations)
+6. **Notes**: Use `Note over` for context that doesn't fit in arrow labels
+7. **Participant order**: Left-to-right follows the data flow direction (User → Frontend → API → Backend → Database)
+8. **Label brevity**: Arrow labels should be ≤8 words. Use the action, not the implementation detail
+9. **Max participants**: 7 per diagram. If more, split into sub-diagrams or collapse internal services
+10. **Max steps**: 15 visible interactions per diagram. For longer workflows, group related steps or use `rect` blocks
+
+### Grouping Related Steps
+
+For workflows with many sequential same-type operations, collapse them:
+
+```mermaid
+sequenceDiagram
+    rect rgb(240, 240, 240)
+        Note over Temporal,Database: Save all entities (10 types)
+        Temporal->>Database: Upsert objects, attributes, pages...
+        Database-->>Temporal: Saved
+    end
+```
+
+---
+
+## Phase 5: Output Assembly
+
+### Output File Structure
+
+Write to the user-specified path, or default to `docs/architecture/workflows.md`. Before writing output files, ensure the target directory exists. Create it with `mkdir -p` if needed.
+
+````markdown
+---
+generated: {YYYY-MM-DD}
+scope: {full | feature-name}
+workflows_found: {count}
+---
+
+# Codebase Workflows
+
+Auto-generated workflow map. Traces execution paths from user action to terminal effect.
+
+## Table of Contents
+
+| Workflow | Category | Trigger | Participants | Complexity |
+|---|---|---|---|---|
+| [{Name}](#{anchor}) | {Category} | {Trigger} | {Count} | {Simple/Moderate/Complex} |
+
+---
+
+## Orchestration Workflows
+
+### {Workflow Name}
+{Diagram + metadata as defined in Phase 4}
+
+---
+
+## Authentication & Session Workflows
+
+### {Workflow Name}
+{...}
+
+---
+
+## Real-time Workflows
+
+### {Workflow Name}
+{...}
+
+---
+
+## Event-driven Workflows
+
+### {Workflow Name}
+{...}
+
+---
+
+## Scheduled Workflows
+
+### {Workflow Name}
+{...}
+
+---
+
+## Data Pipeline Workflows
+
+### {Workflow Name}
+{...}
+````
+
+### Section Ordering
+
+1. Orchestration (the core business logic — save, publish, deploy)
+2. Authentication & Session
+3. Real-time (WebSocket, SSE)
+4. Event-driven (pub/sub, webhooks)
+5. Scheduled (cron, recurring)
+6. Data Pipeline (import/export, sync, migration)
+
+Skip empty sections.
+
+---
+
+## Execution Strategy
+
+### Full Discovery (default)
+
+Launch 5 parallel sub-agents for Phase 1 (one per technique). Collect results, deduplicate entry points, then launch parallel sub-agents for Phase 2 (one per workflow, max 8 concurrent). Run Phase 3-5 sequentially in the main agent.
+
+### Targeted Discovery
+
+User specifies a feature name. Skip broad Phase 1 scan. Instead:
+1. Find the feature's directory/files (grep for feature name in routes, components, workflows)
+2. Trace from those entry points only
+3. Output a single-feature workflow document
+
+### Incremental Update
+
+Read the existing workflow map file. For each workflow listed, check if the key files have changed (git diff). Re-trace only changed workflows. Preserve unchanged sections.
+
+---
+
+## Advanced Techniques
+
+### Payload Storage Detection
+
+Look for patterns where large payloads are stored separately and referenced by ID:
+```
+storePayload|savePayload|workflow_payload|jobData
+→ then: retrievePayload|fetchPayload|getPayload (in the worker/activity)
+```
+Note this in the diagram as a "Store payload → reference by ID" pattern.
+
+### Optimistic Locking Detection
+
+Look for version comparison patterns:
+```
+version.*!==|version.*conflict|StaleObjectError|OptimisticLock
+if.*serverVersion.*localVersion|if.*version.*mismatch
+```
+These indicate conflict branching that should appear as `alt` blocks.
+
+### Polling Loop Detection
+
+Look for patterns where the frontend waits for async completion:
+```
+setInterval|setTimeout.*recursive|poll|retry.*status
+GET.*status\?.*workflowId|GET.*status\?.*jobId
+```
+Note the polling interval and timeout in the diagram.
+
+### Child Workflow / Sub-job Detection
+
+Look for spawned sub-processes:
+```
+startChild\(|spawnChild|fork\(|.add\(.*queue
+Promise\.all.*map.*startWorkflow
+```
+These should appear as nested sequences or separate linked diagrams.
+
+### Transaction Boundary Detection
+
+Look for explicit transaction markers:
+```
+\$transaction|\$executeRaw|BEGIN|COMMIT|ROLLBACK
+withTransaction|runInTransaction|atomic
+```
+Group operations within a transaction using `rect` blocks in the diagram.
+
+### Rate Limiting & Middleware Detection
+
+Look for middleware applied to route groups:
+```
+rateLimit|rateLimiter|throttle
+authenticate|authorize|requireAuth|requireRole
+validate|validateBody|validateParams
+```
+Note these as guards in the diagram using `opt` blocks or notes.
+
+---
+
+## Quality Checks
+
+Before outputting, verify:
+
+- [ ] Every diagram has ≥3 participants (otherwise it's probably too simple to document)
+- [ ] Every diagram has both request and response arrows (no dangling calls)
+- [ ] Participant names are consistent across all diagrams (same DB = same name everywhere)
+- [ ] Alt/opt/loop blocks have clear conditions written in the block label
+- [ ] Key files listed actually exist in the codebase (verify with glob)
+- [ ] No workflow is documented twice under different names
+- [ ] Mermaid syntax is valid (no unclosed blocks, balanced activate/deactivate)
+- [ ] Table of contents anchors match actual heading anchors

package/data/harnesses/code-fable/skills/workflow-mapper/references/patterns.md

@@ -0,0 +1,214 @@
+# Search Pattern Reference
+
+Concrete grep/glob patterns for each discovery technique. Agents should adapt these to the actual framework detected in the codebase.
+
+---
+
+## Framework Detection (run first)
+
+Identify the tech stack to select the right patterns:
+
+| Check | Pattern | Means |
+|---|---|---|
+| `package.json` contains `express` | `"express"` | Express backend |
+| `package.json` contains `@temporalio` | `"@temporalio"` | Temporal workflows |
+| `package.json` contains `bullmq` | `"bullmq"` | BullMQ job queues |
+| `package.json` contains `@reduxjs/toolkit` | `"@reduxjs/toolkit"` | Redux state management |
+| `package.json` contains `socket.io` | `"socket.io"` | WebSocket via Socket.IO |
+| `app/workflows/` exists | glob `app/workflows/` | Workflow directory convention |
+| `temporal/config/` exists | glob `temporal/config/` | Temporal config convention |
+| Files matching `*.workflow.ts` | glob `**/*.workflow.*` | Workflow file convention |
+| Files matching `*.queue.ts` | glob `**/*.queue.*` | Queue file convention |
+
+---
+
+## Route Harvesting Patterns
+
+### Express
+
+```
+# Route definitions
+grep: router\.(get|post|put|patch|delete)\(
+glob: **/routes/**/*.{js,ts}, **/router/**/*.{js,ts}
+
+# Route mounting
+grep: app\.use\(.*/api
+```
+
+### Next.js App Router
+
+```
+glob: app/**/route.{js,ts}
+grep: export (async )?function (GET|POST|PUT|PATCH|DELETE)
+```
+
+### Next.js Pages API
+
+```
+glob: pages/api/**/*.{js,ts}
+grep: export default (async )?function handler
+```
+
+### Fastify
+
+```
+grep: fastify\.(get|post|put|patch|delete)\(
+grep: (register|plugin).*routes
+```
+
+---
+
+## Temporal Workflow Patterns
+
+```
+# Workflow definitions
+grep: export async function \w+Workflow
+glob: **/workflow.{js,ts}, **/*.workflow.{js,ts}
+
+# Activity definitions
+grep: export async function \w+Activity
+glob: **/activities.{js,ts}, **/*.activities.{js,ts}
+
+# Workflow registry
+glob: **/workflowModules.{js,ts}
+grep: workflows\s*=\s*\{
+
+# Activity registry
+glob: **/activityModules.{js,ts}
+grep: activities\s*=\s*\{
+
+# Workflow dispatch
+grep: startWorkflow\(|startChild\(|executeChild\(
+
+# Activity proxy
+grep: proxyActivities\(
+
+# Signals and queries
+grep: defineSignal\(|defineQuery\(|setHandler\(
+
+# Continue-as-new
+grep: continueAsNew\(
+```
+
+---
+
+## BullMQ Patterns
+
+```
+# Queue definitions
+grep: new Queue\(|new FlowProducer\(
+glob: **/queue.{js,ts}, **/*.queue.{js,ts}
+
+# Worker definitions
+grep: new Worker\(
+glob: **/worker.{js,ts}, **/*.worker.{js,ts}
+
+# Job dispatch
+grep: \.add\(|\.addBulk\(|\.addFlow\(
+
+# Job processing
+grep: \.process\(|processor:
+```
+
+---
+
+## Redux / Frontend Action Patterns
+
+```
+# Async thunks (dispatched actions that call APIs)
+grep: createAsyncThunk\(
+glob: **/*Slice.{js,ts}, **/*slice.{js,ts}, **/slices/**/*.{js,ts}
+
+# API calls from frontend
+grep: api\.(post|put|patch|delete)\(|axios\.(post|put|patch|delete)\(
+glob: **/hooks/use*.{js,ts,jsx,tsx}, **/services/**/*.{js,ts}
+
+# Dispatch sequences
+grep: dispatch\(.*Actions?\.|dispatch\(.*thunk
+```
+
+---
+
+## WebSocket Patterns
+
+```
+# Socket.IO server
+grep: io\.on\(|socket\.on\(|socket\.emit\(
+glob: **/socket/**/*.{js,ts}, **/ws/**/*.{js,ts}
+
+# Socket.IO client
+grep: socket\.on\(|socket\.emit\(|useSocket
+glob: **/hooks/useSocket*.{js,ts,jsx,tsx}
+
+# Native WebSocket
+grep: new WebSocket\(|ws\.on\(
+```
+
+---
+
+## Database Patterns
+
+```
+# Prisma
+grep: prisma\.\w+\.(findMany|findFirst|findUnique|create|update|delete|upsert|aggregate|groupBy)
+grep: \$transaction\(|\$executeRaw\(|\$queryRaw\(
+
+# Sequelize
+grep: \.findAll\(|\.findOne\(|\.create\(|\.update\(|\.destroy\(
+grep: sequelize\.transaction\(
+
+# Raw SQL
+grep: \.query\(|\.execute\(|BEGIN|COMMIT|ROLLBACK
+```
+
+---
+
+## External Service Patterns
+
+```
+# Outbound HTTP (to other services)
+grep: axios\.(get|post|put|patch|delete)\(|fetch\(.*https?://
+grep: got\.(get|post|put|patch|delete)\(|request\.(get|post|put|patch|delete)\(
+
+# Git operations
+grep: git.*push|git.*commit|createBlob|createTree|createCommit|updateRef
+grep: octokit\.|@octokit/
+
+# Email
+grep: sendMail|transporter\.send|sgMail\.send|resend\.send
+
+# File storage
+grep: s3\.upload|s3\.putObject|uploadToS3|writeFile.*storage
+```
+
+---
+
+## Workflow Complexity Indicators
+
+Patterns that suggest a route is a real workflow (not simple CRUD):
+
+```
+# Async dispatch (work happens later)
+grep: startWorkflow\(|\.add\(.*queue|emit\(|publish\(
+
+# Multi-step (sequential calls)
+# Look for functions with 3+ await statements to different services
+
+# Transaction blocks
+grep: \$transaction|\$executeRaw|BEGIN.*COMMIT|withTransaction
+
+# Polling/status endpoints
+grep: /status\?|workflowId|jobId|runId
+
+# Version/conflict checks
+grep: version.*conflict|optimistic|stale|409
+
+# Retry/backoff
+grep: maximumAttempts|retry|backoff|exponential
+
+# Child/sub workflows
+grep: startChild\(|spawnChild|fork\(
+
+# Feature flags / conditional steps
+grep: featureFlag|isEnabled|canAccess
+```