vibes-plug 2.14.1 → 3.9.0

package/skills/async-queue-temporal-expert/SKILL.md

@@ -1,217 +1,218 @@
----
-name: async-queue-temporal-expert
-description: "Unified expert guide for async job queues & durable workflows: BullMQ v5 (Redis queues), Trigger.dev v3 (serverless tasks), Inngest, and Temporal.io (distributed sagas) / Panduan ahli terpadu untuk antrean job asinkron & workflow tahan-gagal: BullMQ v5, Trigger.dev v3, Inngest, dan Temporal.io."
-author: "Roedy Rustam"
----
-
-# Async Queue & Durable Workflow Expert (2026 Unified Edition)
-
-[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
-
----
-
-<a name="english"></a>
-## English
-
-### Purpose & Overview
-Unified production-grade guide for background execution pipelines, async job queues, and distributed state machines. Covers the full spectrum from simple Redis-backed task queues to complex multi-service sagas with compensating rollbacks.
-
-### 3-Tier Execution Model
-| Tier | Engine | Best For |
-|------|--------|----------|
-| **Tier 1: Redis Task Queues** | BullMQ v5 | High-throughput worker jobs, priority queues, rate limiting, DLQ |
-| **Tier 2: Serverless Durable Tasks** | Trigger.dev v3 / Inngest | Step-checkpointed tasks, automatic resume across crashes, zero infra |
-| **Tier 3: Distributed Sagas** | Temporal.io | Multi-service orchestration, compensating rollbacks, long-running workflows |
-
-### Core Capabilities
-1. **Idempotency & Deduplication**: Deterministic `jobId` keys prevent double billing or duplicate emails.
-2. **Dead Letter Queues (DLQ)**: Auto-relocate permanently failing jobs for audit and alerting.
-3. **Exponential Backoff with Jitter**: Prevents thundering herds on upstream services.
-4. **Tenant Priority Queues**: VIP/enterprise tiers get lower BullMQ priority numbers (higher throughput).
-5. **Durable State Machines**: Workflows survive restarts, deployments, and network partitions.
-6. **Saga Compensations**: Multi-step transactions with automated reverse-order rollbacks.
-
----
-
-### Tier 1: BullMQ v5 — Redis Task Queues (TypeScript)
-
-```typescript
-import { Queue, Worker, Job } from 'bullmq';
-import Redis from 'ioredis';
-
-const redisConnection = new Redis(process.env.REDIS_URL!, {
-  maxRetriesPerRequest: null, // Required by BullMQ
-});
-
-export interface NotificationPayload {
-  tenantId: string;
-  userId: string;
-  type: 'email' | 'webhook';
-  payload: Record<string, unknown>;
-  idempotencyKey: string;
-}
-
-// Main Queue
-export const notificationQueue = new Queue<NotificationPayload>('notifications', {
-  connection: redisConnection,
-  defaultJobOptions: {
-    attempts: 5,
-    backoff: { type: 'exponential', delay: 1500 },
-    removeOnComplete: { age: 86400, count: 5000 },
-    removeOnFail: false, // Preserved for DLQ audit
-  },
-});
-
-// Dead Letter Queue
-export const notificationDLQ = new Queue('notifications-dlq', {
-  connection: redisConnection,
-});
-
-// Enqueue with deduplication & priority
-export async function enqueueNotification(data: NotificationPayload, isVip = false) {
-  return await notificationQueue.add('send_notification', data, {
-    jobId: `notif_${data.idempotencyKey}`, // Deterministic dedup key
-    priority: isVip ? 1 : 10,
-  });
-}
-
-// Worker with concurrency & rate limiting
-export const notificationWorker = new Worker<NotificationPayload>(
-  'notifications',
-  async (job: Job<NotificationPayload>) => {
-    if (job.data.type === 'email') await deliverEmail(job.data);
-  },
-  {
-    connection: redisConnection,
-    concurrency: 20,
-    limiter: { max: 100, duration: 1000 },
-  }
-);
-
-// DLQ forwarding on retry exhaustion
-notificationWorker.on('failed', async (job, error) => {
-  if (job && job.attemptsMade >= (job.opts.attempts || 5)) {
-    await notificationDLQ.add('failed_notification', {
-      originalJobId: job.id, failedReason: error.message,
-      data: job.data, exhaustedAt: new Date().toISOString(),
-    });
-  }
-});
-```
-
----
-
-### Tier 2: Trigger.dev v3 — Serverless Durable Tasks
-
-```typescript
-import { task } from '@trigger.dev/sdk/v3';
-
-export const generateReport = task({
-  id: 'generate-enterprise-report',
-  retry: { maxAttempts: 4, minTimeoutInMs: 2000, factor: 2, randomize: true },
-  run: async (payload: { tenantId: string; month: string }, { ctx }) => {
-    // Each step is a durable checkpoint — survives crashes
-    const data = await ctx.run('fetch-telemetry', async () => {
-      return await fetchTelemetryFromWarehouse(payload.tenantId, payload.month);
-    });
-    const pdfUrl = await ctx.run('render-pdf', async () => {
-      return await generateReportPdf(data);
-    });
-    await ctx.run('dispatch-webhook', async () => {
-      return await sendWebhookNotification(payload.tenantId, pdfUrl);
-    });
-    return { success: true, pdfUrl };
-  },
-});
-```
-
----
-
-### Tier 3: Temporal.io — Distributed Saga with Compensations
-
-```typescript
-import { proxyActivities, ApplicationFailure } from '@temporalio/workflow';
-import type * as activities from './activities';
-
-const { chargeCustomer, provisionLicense, sendWelcomeEmail, refundCustomer, revokeLicense } =
-  proxyActivities<typeof activities>({
-    startToCloseTimeout: '1 minute',
-    retry: {
-      initialInterval: '1s', backoffCoefficient: 2, maximumAttempts: 5,
-      nonRetryableErrorTypes: ['InvalidCardError', 'AccountSuspendedError'],
-    },
-  });
-
-export async function subscriptionSagaWorkflow(input: {
-  customerId: string; planId: string; amountCents: number;
-}) {
-  const compensations: Array<() => Promise<void>> = [];
-  try {
-    const charge = await chargeCustomer(input.customerId, input.amountCents);
-    compensations.unshift(() => refundCustomer(charge.chargeId));
-
-    const license = await provisionLicense(input.customerId, input.planId);
-    compensations.unshift(() => revokeLicense(license.licenseId));
-
-    await sendWelcomeEmail(input.customerId, license.licenseKey);
-    return { status: 'COMPLETED' };
-  } catch (error) {
-    for (const compensate of compensations) {
-      try { await compensate(); } catch (e) { console.error('Compensation failed:', e); }
-    }
-    throw ApplicationFailure.create({
-      message: `Saga rolled back: ${(error as Error).message}`, nonRetryable: true,
-    });
-  }
-}
-```
-
-> **Temporal Determinism Rule**: Never use `Math.random()`, `Date.now()`, or direct DB calls inside workflow files. Run them inside activities.
-
----
-
-### Implementation Checklist
-- [ ] Configure `maxRetriesPerRequest: null` on ioredis for BullMQ v5.
-- [ ] Use deterministic `jobId` from business logic (`order_${orderId}`) for deduplication.
-- [ ] Forward permanently dead jobs to DLQ via `worker.on('failed')` listener.
-- [ ] Implement rate limiting via worker `limiter` to protect third-party APIs.
-- [ ] Enforce deterministic code inside Temporal workflows (activities for side-effects).
-- [ ] Store large payloads in S3/R2; pass only IDs through queues.
-- [ ] Implement Saga rollback handlers for multi-step distributed payments.
-
-## Orchestration & Integration
-- Integrates with: `js-backend-expert`, `cron-scheduler-expert`, `error-resilience-expert`, `saas-billing`, `doku-payment-gateway`, `data-telemetry-expert`.
-
----
-
-<a name="bahasa-indonesia"></a>
-## Bahasa Indonesia
-
-### Tujuan & Gambaran Umum
-Panduan terpadu tingkat produksi untuk pipeline eksekusi latar belakang, antrean job asinkron, dan state machine terdistribusi. Mencakup spektrum penuh dari antrean Redis sederhana hingga saga multi-layanan dengan rollback kompensasi.
-
-### Model Eksekusi 3-Tier
-| Tier | Engine | Cocok Untuk |
-|------|--------|-------------|
-| **Tier 1: Antrean Redis** | BullMQ v5 | Job worker throughput tinggi, prioritas, rate limiting, DLQ |
-| **Tier 2: Task Serverless** | Trigger.dev v3 / Inngest | Task dengan checkpoint, resume otomatis, tanpa infra |
-| **Tier 3: Saga Terdistribusi** | Temporal.io | Orkestrasi multi-layanan, rollback kompensasi, workflow jangka panjang |
-
-### Kemampuan Utama
-1. **Idempotensi & Deduplikasi**: Kunci `jobId` deterministik mencegah duplikasi penagihan atau email.
-2. **Dead Letter Queue (DLQ)**: Pemindahan otomatis job gagal total untuk audit.
-3. **Backoff Eksponensial + Jitter**: Mencegah thundering herd pada server hilir.
-4. **Prioritas Tenant**: Tier VIP/enterprise mendapat prioritas lebih tinggi (angka lebih kecil di BullMQ).
-5. **State Machine Tahan-Gagal**: Workflow bertahan saat restart, deployment, dan partisi jaringan.
-6. **Kompensasi Saga**: Transaksi multi-langkah dengan rollback otomatis urutan mundur.
-
-### Checklist Implementasi
-- [ ] Atur `maxRetriesPerRequest: null` pada ioredis untuk BullMQ v5.
-- [ ] Gunakan `jobId` deterministik dari ID bisnis (`invoice_${invoiceId}`) untuk deduplikasi.
-- [ ] Pasang listener `worker.on('failed')` untuk forward job gagal ke DLQ.
-- [ ] Terapkan rate limiter pada worker untuk stabilitas API eksternal.
-- [ ] Pastikan kode Temporal selalu deterministik (side-effect hanya di activities).
-- [ ] Simpan file besar di S3/R2; kirim hanya referensi ID melalui queue.
-
-## Integrasi Orkestrasi
-- Terintegrasi dengan: `js-backend-expert`, `cron-scheduler-expert`, `error-resilience-expert`, `saas-billing`, `doku-payment-gateway`, `data-telemetry-expert`.
+---
+name: async-queue-temporal-expert
+description: "Unified expert guide for async job queues & durable workflows: BullMQ v5 (Redis queues), Trigger.dev v3 (serverless tasks), Inngest, and Temporal.io (distributed sagas) / Panduan ahli terpadu untuk antrean job asinkron & workflow tahan-gagal: BullMQ v5, Trigger.dev v3, Inngest, dan Temporal.io."
+author: "Roedy Rustam"
+version: "3.0.0"
+---
+
+# Async Queue & Durable Workflow Expert (2026 Unified Edition)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Purpose & Overview
+Unified production-grade guide for background execution pipelines, async job queues, and distributed state machines. Covers the full spectrum from simple Redis-backed task queues to complex multi-service sagas with compensating rollbacks.
+
+### 3-Tier Execution Model
+| Tier | Engine | Best For |
+|------|--------|----------|
+| **Tier 1: Redis Task Queues** | BullMQ v5 | High-throughput worker jobs, priority queues, rate limiting, DLQ |
+| **Tier 2: Serverless Durable Tasks** | Trigger.dev v3 / Inngest | Step-checkpointed tasks, automatic resume across crashes, zero infra |
+| **Tier 3: Distributed Sagas** | Temporal.io | Multi-service orchestration, compensating rollbacks, long-running workflows |
+
+### Core Capabilities
+1. **Idempotency & Deduplication**: Deterministic `jobId` keys prevent double billing or duplicate emails.
+2. **Dead Letter Queues (DLQ)**: Auto-relocate permanently failing jobs for audit and alerting.
+3. **Exponential Backoff with Jitter**: Prevents thundering herds on upstream services.
+4. **Tenant Priority Queues**: VIP/enterprise tiers get lower BullMQ priority numbers (higher throughput).
+5. **Durable State Machines**: Workflows survive restarts, deployments, and network partitions.
+6. **Saga Compensations**: Multi-step transactions with automated reverse-order rollbacks.
+
+---
+
+### Tier 1: BullMQ v5 — Redis Task Queues (TypeScript)
+
+```typescript
+import { Queue, Worker, Job } from 'bullmq';
+import Redis from 'ioredis';
+
+const redisConnection = new Redis(process.env.REDIS_URL!, {
+  maxRetriesPerRequest: null, // Required by BullMQ
+});
+
+export interface NotificationPayload {
+  tenantId: string;
+  userId: string;
+  type: 'email' | 'webhook';
+  payload: Record<string, unknown>;
+  idempotencyKey: string;
+}
+
+// Main Queue
+export const notificationQueue = new Queue<NotificationPayload>('notifications', {
+  connection: redisConnection,
+  defaultJobOptions: {
+    attempts: 5,
+    backoff: { type: 'exponential', delay: 1500 },
+    removeOnComplete: { age: 86400, count: 5000 },
+    removeOnFail: false, // Preserved for DLQ audit
+  },
+});
+
+// Dead Letter Queue
+export const notificationDLQ = new Queue('notifications-dlq', {
+  connection: redisConnection,
+});
+
+// Enqueue with deduplication & priority
+export async function enqueueNotification(data: NotificationPayload, isVip = false) {
+  return await notificationQueue.add('send_notification', data, {
+    jobId: `notif_${data.idempotencyKey}`, // Deterministic dedup key
+    priority: isVip ? 1 : 10,
+  });
+}
+
+// Worker with concurrency & rate limiting
+export const notificationWorker = new Worker<NotificationPayload>(
+  'notifications',
+  async (job: Job<NotificationPayload>) => {
+    if (job.data.type === 'email') await deliverEmail(job.data);
+  },
+  {
+    connection: redisConnection,
+    concurrency: 20,
+    limiter: { max: 100, duration: 1000 },
+  }
+);
+
+// DLQ forwarding on retry exhaustion
+notificationWorker.on('failed', async (job, error) => {
+  if (job && job.attemptsMade >= (job.opts.attempts || 5)) {
+    await notificationDLQ.add('failed_notification', {
+      originalJobId: job.id, failedReason: error.message,
+      data: job.data, exhaustedAt: new Date().toISOString(),
+    });
+  }
+});
+```
+
+---
+
+### Tier 2: Trigger.dev v3 — Serverless Durable Tasks
+
+```typescript
+import { task } from '@trigger.dev/sdk/v3';
+
+export const generateReport = task({
+  id: 'generate-enterprise-report',
+  retry: { maxAttempts: 4, minTimeoutInMs: 2000, factor: 2, randomize: true },
+  run: async (payload: { tenantId: string; month: string }, { ctx }) => {
+    // Each step is a durable checkpoint — survives crashes
+    const data = await ctx.run('fetch-telemetry', async () => {
+      return await fetchTelemetryFromWarehouse(payload.tenantId, payload.month);
+    });
+    const pdfUrl = await ctx.run('render-pdf', async () => {
+      return await generateReportPdf(data);
+    });
+    await ctx.run('dispatch-webhook', async () => {
+      return await sendWebhookNotification(payload.tenantId, pdfUrl);
+    });
+    return { success: true, pdfUrl };
+  },
+});
+```
+
+---
+
+### Tier 3: Temporal.io — Distributed Saga with Compensations
+
+```typescript
+import { proxyActivities, ApplicationFailure } from '@temporalio/workflow';
+import type * as activities from './activities';
+
+const { chargeCustomer, provisionLicense, sendWelcomeEmail, refundCustomer, revokeLicense } =
+  proxyActivities<typeof activities>({
+    startToCloseTimeout: '1 minute',
+    retry: {
+      initialInterval: '1s', backoffCoefficient: 2, maximumAttempts: 5,
+      nonRetryableErrorTypes: ['InvalidCardError', 'AccountSuspendedError'],
+    },
+  });
+
+export async function subscriptionSagaWorkflow(input: {
+  customerId: string; planId: string; amountCents: number;
+}) {
+  const compensations: Array<() => Promise<void>> = [];
+  try {
+    const charge = await chargeCustomer(input.customerId, input.amountCents);
+    compensations.unshift(() => refundCustomer(charge.chargeId));
+
+    const license = await provisionLicense(input.customerId, input.planId);
+    compensations.unshift(() => revokeLicense(license.licenseId));
+
+    await sendWelcomeEmail(input.customerId, license.licenseKey);
+    return { status: 'COMPLETED' };
+  } catch (error) {
+    for (const compensate of compensations) {
+      try { await compensate(); } catch (e) { console.error('Compensation failed:', e); }
+    }
+    throw ApplicationFailure.create({
+      message: `Saga rolled back: ${(error as Error).message}`, nonRetryable: true,
+    });
+  }
+}
+```
+
+> **Temporal Determinism Rule**: Never use `Math.random()`, `Date.now()`, or direct DB calls inside workflow files. Run them inside activities.
+
+---
+
+### Implementation Checklist
+- [ ] Configure `maxRetriesPerRequest: null` on ioredis for BullMQ v5.
+- [ ] Use deterministic `jobId` from business logic (`order_${orderId}`) for deduplication.
+- [ ] Forward permanently dead jobs to DLQ via `worker.on('failed')` listener.
+- [ ] Implement rate limiting via worker `limiter` to protect third-party APIs.
+- [ ] Enforce deterministic code inside Temporal workflows (activities for side-effects).
+- [ ] Store large payloads in S3/R2; pass only IDs through queues.
+- [ ] Implement Saga rollback handlers for multi-step distributed payments.
+
+## Orchestration & Integration
+- Integrates with: `js-backend-expert`, `cron-scheduler-expert`, `error-resilience-expert`, `saas-billing`, `doku-payment-gateway`, `data-telemetry-expert`.
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Tujuan & Gambaran Umum
+Panduan terpadu tingkat produksi untuk pipeline eksekusi latar belakang, antrean job asinkron, dan state machine terdistribusi. Mencakup spektrum penuh dari antrean Redis sederhana hingga saga multi-layanan dengan rollback kompensasi.
+
+### Model Eksekusi 3-Tier
+| Tier | Engine | Cocok Untuk |
+|------|--------|-------------|
+| **Tier 1: Antrean Redis** | BullMQ v5 | Job worker throughput tinggi, prioritas, rate limiting, DLQ |
+| **Tier 2: Task Serverless** | Trigger.dev v3 / Inngest | Task dengan checkpoint, resume otomatis, tanpa infra |
+| **Tier 3: Saga Terdistribusi** | Temporal.io | Orkestrasi multi-layanan, rollback kompensasi, workflow jangka panjang |
+
+### Kemampuan Utama
+1. **Idempotensi & Deduplikasi**: Kunci `jobId` deterministik mencegah duplikasi penagihan atau email.
+2. **Dead Letter Queue (DLQ)**: Pemindahan otomatis job gagal total untuk audit.
+3. **Backoff Eksponensial + Jitter**: Mencegah thundering herd pada server hilir.
+4. **Prioritas Tenant**: Tier VIP/enterprise mendapat prioritas lebih tinggi (angka lebih kecil di BullMQ).
+5. **State Machine Tahan-Gagal**: Workflow bertahan saat restart, deployment, dan partisi jaringan.
+6. **Kompensasi Saga**: Transaksi multi-langkah dengan rollback otomatis urutan mundur.
+
+### Checklist Implementasi
+- [ ] Atur `maxRetriesPerRequest: null` pada ioredis untuk BullMQ v5.
+- [ ] Gunakan `jobId` deterministik dari ID bisnis (`invoice_${invoiceId}`) untuk deduplikasi.
+- [ ] Pasang listener `worker.on('failed')` untuk forward job gagal ke DLQ.
+- [ ] Terapkan rate limiter pada worker untuk stabilitas API eksternal.
+- [ ] Pastikan kode Temporal selalu deterministik (side-effect hanya di activities).
+- [ ] Simpan file besar di S3/R2; kirim hanya referensi ID melalui queue.
+
+## Integrasi Orkestrasi
+- Terintegrasi dengan: `js-backend-expert`, `cron-scheduler-expert`, `error-resilience-expert`, `saas-billing`, `doku-payment-gateway`, `data-telemetry-expert`.