vibes-plug 1.0.0 → 2.11.0

package/skills/cron-scheduler-expert/SKILL.md

@@ -0,0 +1,304 @@
+---
+name: cron-scheduler-expert
+description: "Expert guide for scheduled tasks, cron jobs, recurring background work (Vercel Cron, Cloudflare Workers Cron, Inngest, node-cron), and distributed scheduling / Panduan ahli untuk tugas terjadwal, cron job, pekerjaan latar belakang berulang, dan penjadwalan terdistribusi."
+author: "Roedy Rustam"
+---
+
+# Cron & Scheduler Expert (2026 Edition)
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Orchestration & Integration
+Connects and orchestrates with relevant domain skills like `brainstorming`, `zero-to-prod-orchestrator`, and `project-context-mapper` to ensure cohesive execution.
+
+### Description
+Production-grade guide for implementing scheduled tasks, cron jobs, and recurring background work in modern web applications. Covers **Vercel Cron Jobs**, **Cloudflare Workers Cron Triggers**, **Inngest scheduled functions**, **node-cron**, **BullMQ repeatable jobs**, **distributed cron locking** (Redis-based), **timezone-aware scheduling**, and common use cases (cleanup jobs, report generation, health checks, subscription renewals).
+
+### Trigger Conditions
+Activate this skill when:
+- Setting up scheduled/recurring tasks (daily reports, cleanup, health checks).
+- Configuring Vercel Cron Jobs or Cloudflare Workers Cron Triggers.
+- Implementing subscription renewal checks or billing cycle jobs.
+- Building data aggregation or ETL pipelines on a schedule.
+- Needing distributed cron locking to prevent duplicate execution.
+- Scheduling email digests or notification batches.
+
+---
+
+### Platform Selection Guide
+
+| Platform | Best For | Max Frequency | Invocation Model |
+|---|---|---|---|
+| **Vercel Cron** | Vercel-hosted Next.js apps | 1/min (Pro), 1/day (Hobby) | HTTP endpoint trigger |
+| **Cloudflare Workers Cron** | Edge-first apps | 1/min | Worker script trigger |
+| **Inngest** | Complex workflows, fan-out | Any (event-driven) | Serverless function |
+| **BullMQ Repeatable** | Self-hosted, Redis-backed | Any | Worker process |
+| **node-cron** | Simple Node.js servers | Any | In-process |
+| **Trigger.dev v3** | Durable scheduled tasks | Any | Serverless function |
+
+**Recommendation**: Use **Vercel Cron** for Next.js apps on Vercel. Use **Inngest** for complex multi-step scheduled workflows. Use **BullMQ repeatable jobs** for self-hosted environments.
+
+---
+
+### 1. Vercel Cron Jobs
+
+```json
+// vercel.json
+{
+  "crons": [
+    {
+      "path": "/api/cron/daily-cleanup",
+      "schedule": "0 3 * * *"
+    },
+    {
+      "path": "/api/cron/hourly-health-check",
+      "schedule": "0 * * * *"
+    },
+    {
+      "path": "/api/cron/weekly-report",
+      "schedule": "0 9 * * 1"
+    }
+  ]
+}
+```
+
+```typescript
+// app/api/cron/daily-cleanup/route.ts
+import { NextResponse } from 'next/server';
+
+export async function GET(request: Request) {
+  // Verify the request is from Vercel Cron (not public)
+  const authHeader = request.headers.get('authorization');
+  if (authHeader !== `Bearer ${process.env.CRON_SECRET}`) {
+    return NextResponse.json({ error: 'Unauthorized' }, { status: 401 });
+  }
+
+  try {
+    // Cleanup expired sessions
+    const deletedSessions = await db.delete(sessions)
+      .where(lt(sessions.expiresAt, new Date()));
+
+    // Cleanup orphaned uploads (older than 24h, not linked to any record)
+    const deletedUploads = await db.delete(uploads)
+      .where(and(
+        isNull(uploads.linkedRecordId),
+        lt(uploads.createdAt, new Date(Date.now() - 86400000)),
+      ));
+
+    return NextResponse.json({
+      ok: true,
+      cleaned: { sessions: deletedSessions.rowCount, uploads: deletedUploads.rowCount },
+    });
+  } catch (error) {
+    console.error('Daily cleanup failed:', error);
+    return NextResponse.json({ ok: false, error: 'Cleanup failed' }, { status: 500 });
+  }
+}
+
+export const runtime = 'nodejs';
+export const maxDuration = 60; // Allow up to 60s for cleanup
+```
+
+---
+
+### 2. Inngest Scheduled Functions
+
+```typescript
+// inngest/functions/weekly-report.ts
+import { inngest } from '@/lib/inngest';
+
+export const weeklyReportJob = inngest.createFunction(
+  {
+    id: 'weekly-usage-report',
+    name: 'Generate Weekly Usage Report',
+  },
+  { cron: 'TZ=Asia/Jakarta 0 9 * * 1' }, // Every Monday 9 AM WIB
+  async ({ step }) => {
+    // Step 1: Gather data
+    const metrics = await step.run('gather-metrics', async () => {
+      return await db.query.usageMetrics.findMany({
+        where: gte(usageMetrics.createdAt, subDays(new Date(), 7)),
+      });
+    });
+
+    // Step 2: Generate report
+    const report = await step.run('generate-report', async () => {
+      return generateWeeklyReport(metrics);
+    });
+
+    // Step 3: Send to admins
+    await step.run('send-report', async () => {
+      const admins = await db.query.users.findMany({
+        where: eq(users.role, 'admin'),
+      });
+      for (const admin of admins) {
+        await emailQueue.add('weekly-report', { to: admin.email, report });
+      }
+    });
+
+    return { sent: true, reportId: report.id };
+  },
+);
+```
+
+---
+
+### 3. BullMQ Repeatable Jobs
+
+```typescript
+// queues/scheduled-jobs.ts
+import { Queue, Worker } from 'bullmq';
+
+const scheduledQueue = new Queue('scheduled', {
+  connection: { host: process.env.REDIS_HOST },
+});
+
+// Add repeatable jobs on startup
+export async function registerScheduledJobs() {
+  // Daily cleanup at 3 AM
+  await scheduledQueue.add('daily-cleanup', {}, {
+    repeat: { pattern: '0 3 * * *', tz: 'Asia/Jakarta' },
+    jobId: 'daily-cleanup', // Prevent duplicates on restart
+  });
+
+  // Every 5 minutes: health check
+  await scheduledQueue.add('health-check', {}, {
+    repeat: { every: 5 * 60 * 1000 }, // 5 minutes in ms
+    jobId: 'health-check',
+  });
+
+  // Monthly: subscription renewal check (1st of each month at midnight)
+  await scheduledQueue.add('subscription-renewal', {}, {
+    repeat: { pattern: '0 0 1 * *', tz: 'Asia/Jakarta' },
+    jobId: 'subscription-renewal',
+  });
+}
+
+// Worker
+const worker = new Worker('scheduled', async (job) => {
+  switch (job.name) {
+    case 'daily-cleanup':
+      await performDailyCleanup();
+      break;
+    case 'health-check':
+      await performHealthCheck();
+      break;
+    case 'subscription-renewal':
+      await checkSubscriptionRenewals();
+      break;
+  }
+}, { connection: { host: process.env.REDIS_HOST } });
+```
+
+---
+
+### 4. Distributed Cron Locking (Redis)
+
+```typescript
+// lib/cron-lock.ts
+import { Redis } from 'ioredis';
+
+const redis = new Redis(process.env.REDIS_URL!);
+
+/**
+ * Acquire a distributed lock to prevent duplicate cron execution
+ * across multiple server instances.
+ */
+export async function acquireCronLock(
+  jobName: string,
+  ttlSeconds: number = 300,
+): Promise<boolean> {
+  const lockKey = `cron-lock:${jobName}`;
+  // SET NX (only if not exists) with TTL
+  const result = await redis.set(lockKey, Date.now().toString(), 'EX', ttlSeconds, 'NX');
+  return result === 'OK';
+}
+
+export async function releaseCronLock(jobName: string): Promise<void> {
+  await redis.del(`cron-lock:${jobName}`);
+}
+
+// Usage in cron handler
+export async function GET(request: Request) {
+  const locked = await acquireCronLock('daily-cleanup', 300);
+  if (!locked) {
+    return Response.json({ skipped: true, reason: 'Already running on another instance' });
+  }
+
+  try {
+    await performDailyCleanup();
+    return Response.json({ ok: true });
+  } finally {
+    await releaseCronLock('daily-cleanup');
+  }
+}
+```
+
+---
+
+### Common Cron Patterns
+
+| Schedule | Cron Expression | Description |
+|---|---|---|
+| Every minute | `* * * * *` | Health checks, queue monitoring |
+| Every 5 minutes | `*/5 * * * *` | Cache warming, metric aggregation |
+| Every hour | `0 * * * *` | Data sync, report generation |
+| Daily at 3 AM | `0 3 * * *` | Cleanup, backups |
+| Weekly Monday 9 AM | `0 9 * * 1` | Weekly reports |
+| Monthly 1st at midnight | `0 0 1 * *` | Billing cycles, subscription renewals |
+| Weekdays at 8 AM | `0 8 * * 1-5` | Business notifications |
+
+---
+
+### Common Pitfalls to Avoid
+
+| Anti-Pattern | Problem | Correct Approach |
+|---|---|---|
+| No distributed locking | Duplicate execution across instances | Use Redis-based NX lock |
+| Long-running cron without timeout | Resource exhaustion, overlapping runs | Set `maxDuration` and use lock TTL |
+| No monitoring of cron failures | Silent failures go unnoticed | Log results, alert on failures |
+| Hardcoded timezone | Jobs run at wrong time after DST | Always use explicit `tz` parameter |
+| Cron job does too much work | Timeout, partial completion | Break into smaller steps (Inngest/Temporal) |
+
+---
+
+### Integration with Other Skills
+
+- `async-queue-temporal-expert` — BullMQ repeatable jobs, Inngest scheduled functions
+- `cloud-hosting-expert` — Vercel Cron, Cloudflare Workers Cron configuration
+- `saas-billing` — Subscription renewal checks, billing cycle jobs
+- `email-notification-expert` — Scheduled email digests, daily summaries
+- `data-telemetry-expert` — Scheduled metric aggregation and reporting
+- `logging-error-tracking-expert` — Cron job failure monitoring
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Integrasi Orkestrasi
+Terhubung dan mengorkestrasi skill domain yang relevan seperti `brainstorming`, `zero-to-prod-orchestrator`, dan `project-context-mapper` untuk memastikan eksekusi yang kohesif.
+
+### Deskripsi
+Panduan tingkat produksi untuk mengimplementasikan tugas terjadwal, cron job, dan pekerjaan latar belakang berulang di aplikasi web modern. Mencakup **Vercel Cron Jobs**, **Cloudflare Workers Cron Triggers**, **Inngest scheduled functions**, **node-cron**, **BullMQ repeatable jobs**, **distributed cron locking** (berbasis Redis), **penjadwalan sadar zona waktu**, dan kasus penggunaan umum (pembersihan data, pembuatan laporan, health check, perpanjangan langganan).
+
+### Kondisi Pemicu
+Aktifkan skill ini ketika:
+- Menyiapkan tugas terjadwal/berulang (laporan harian, pembersihan, health check).
+- Mengonfigurasi Vercel Cron Jobs atau Cloudflare Workers Cron Triggers.
+- Mengimplementasikan pemeriksaan perpanjangan langganan atau siklus billing.
+- Membangun pipeline agregasi data atau ETL pada jadwal tertentu.
+- Membutuhkan penguncian cron terdistribusi untuk mencegah eksekusi duplikat.
+
+### Integrasi dengan Skill Lain
+
+- `async-queue-temporal-expert` — BullMQ repeatable jobs, Inngest scheduled functions
+- `cloud-hosting-expert` — Konfigurasi Vercel Cron, Cloudflare Workers Cron
+- `saas-billing` — Pemeriksaan perpanjangan langganan, siklus billing
+- `email-notification-expert` — Digest email terjadwal, ringkasan harian
+- `data-telemetry-expert` — Agregasi metrik terjadwal dan pelaporan

package/skills/data-pipeline-etl-expert/SKILL.md

@@ -0,0 +1,84 @@
+---
+name: data-pipeline-etl-expert
+description: "Expert guide for Data Pipelines, ETL/ELT, and Analytics Engineering. Covers dbt, Apache Airflow, Dagster, BigQuery, ClickHouse, and DuckDB / Panduan ahli untuk Data Pipelines, ETL/ELT. Mencakup dbt, Airflow, Dagster, BigQuery, ClickHouse, dan DuckDB."
+author: "Roedy Rustam"
+---
+
+# Data Pipeline & ETL Expert
+
+[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
+
+---
+
+<a name="english"></a>
+## English
+
+### Description
+A specialized skill for building robust data architectures, Analytics Engineering, and ETL (Extract, Transform, Load) or ELT pipelines. It covers modern data stack orchestration (Airflow, Dagster), transformation tools (dbt), and high-performance OLAP databases (BigQuery, Snowflake, ClickHouse, DuckDB).
+
+### Trigger Conditions
+- When designing reporting dashboards or analytics infrastructure for a SaaS.
+- When moving large volumes of data from transactional databases (PostgreSQL/MySQL) to a data warehouse.
+- When the user asks about "dbt", "Airflow", "ELT", or "Analytics Engineering".
+- When building local or edge analytics using DuckDB.
+
+### Core Architectural Guidelines
+
+#### 1. ELT over ETL
+Prefer Extract-Load-Transform (ELT) over traditional ETL.
+- **Extract & Load**: Use tools like Airbyte or Fivetran to dump raw data directly into the Data Warehouse.
+- **Transform**: Perform transformations *inside* the Data Warehouse using SQL (via dbt) to leverage the warehouse's massive compute power.
+
+#### 2. Analytics Engineering with dbt
+Treat SQL like software engineering.
+- Use `dbt` (Data Build Tool) to version control your SQL transformations.
+- Implement tests (`not_null`, `unique`) on critical tables.
+- Use Jinja templating in dbt to DRY up complex SQL queries.
+
+#### 3. Data Orchestration (Airflow vs Dagster)
+- **Apache Airflow**: The industry standard for scheduling and monitoring complex DAGs (Directed Acyclic Graphs). Best for Python-heavy teams.
+- **Dagster**: A modern alternative focused on data assets rather than just tasks. Use Dagster when you want better local testing and asset-driven lineage.
+
+#### 4. OLAP Database Selection
+- **BigQuery / Snowflake**: Best for massive scale, fully managed cloud data warehousing.
+- **ClickHouse**: Best for real-time, sub-second analytical queries on massive event streams.
+- **DuckDB**: Best for local analytics, embedded analytical pipelines, or processing parquets in edge environments (Node.js/Python).
+
+## Orchestration & Integration
+- Pairs with `data-telemetry-expert` to process the raw telemetry events captured by PostHog/OpenTelemetry.
+- Complements `python-programming-expert` as Python is the lingua franca of data engineering.
+- Works with `cron-scheduler-expert` when simpler, non-DAG cron jobs are sufficient for small ETL tasks.
+
+---
+
+<a name="bahasa-indonesia"></a>
+## Bahasa Indonesia
+
+### Deskripsi
+Panduan khusus untuk membangun arsitektur data yang kuat, Analytics Engineering, dan pipeline ETL/ELT. Mencakup orkestrasi (Airflow, Dagster), alat transformasi (dbt), dan database OLAP berkinerja tinggi (BigQuery, Snowflake, ClickHouse, DuckDB).
+
+### Kondisi Pemicu
+- Saat merancang infrastruktur analitik atau dashboard pelaporan untuk SaaS.
+- Saat memindahkan data bervolume besar dari database transaksional ke Data Warehouse.
+- Saat membangun analitik lokal yang cepat menggunakan DuckDB.
+
+### Panduan Arsitektur Inti
+
+#### 1. ELT lebih disukai daripada ETL
+- **Extract & Load**: Pindahkan data mentah (raw data) langsung ke Data Warehouse (menggunakan Airbyte/Fivetran).
+- **Transform**: Lakukan transformasi data *di dalam* Data Warehouse menggunakan SQL (dbt) untuk memanfaatkan kekuatan komputasi gudang data yang masif.
+
+#### 2. Analytics Engineering dengan dbt (Data Build Tool)
+Perlakukan transformasi data (SQL) layaknya rekayasa perangkat lunak. Gunakan dbt untuk version control, pengujian otomatis (`not_null`, `unique`), dan dokumentasi skema data Anda.
+
+#### 3. Orkestrasi Data (DAG)
+Gunakan Apache Airflow atau Dagster untuk menjadwalkan dan memonitor alur kerja data yang kompleks (DAG). Dagster sangat direkomendasikan untuk pendekatan modern yang berpusat pada aset data (asset-driven orchestration).
+
+#### 4. Pemilihan Database OLAP
+- **BigQuery / Snowflake**: Gudang data cloud fully-managed untuk analitik skala masif.
+- **ClickHouse**: Sangat cepat untuk kueri analitik real-time. Cocok untuk data event stream/log.
+- **DuckDB**: SQLite untuk analitik. Sangat cepat untuk memproses file Parquet atau CSV secara lokal maupun di lingkungan edge/serverless (via Python/Node.js).
+
+## Integrasi Orkestrasi
+- Bekerja sama dengan `data-telemetry-expert` untuk memproses data mentah yang dikumpulkan.
+- Melengkapi `python-programming-expert` dalam menulis skrip orkestrasi Airflow/Dagster.