@mars-stack/cli 0.2.0 → 1.0.2

package/template/.cursor/skills/configure-jobs/SKILL.md

@@ -0,0 +1,500 @@
+# Skill: Configure Background Jobs
+
+Set up background job processing in a MARS application using Inngest, Trigger.dev, or a zero-dependency DB queue.
+
+## When to Use
+
+Use this skill when the user asks to add background jobs, async processing, task queues, scheduled tasks, cron jobs, or deferred work (e.g., "add email digests", "process webhooks in background", "schedule a report").
+
+## Prerequisites
+
+- `appConfig.services.jobs.provider` set to `'inngest'`, `'trigger'`, or `'db-queue'`
+- For Inngest/Trigger.dev: an account with the respective provider
+- For DB Queue: Prisma and PostgreSQL already configured
+
+## Provider Interface
+
+All providers implement a common interface so you can swap later:
+
+```typescript
+// src/features/jobs/types.ts
+export interface EnqueueOptions {
+  runAt?: Date;
+  maxAttempts?: number;
+  idempotencyKey?: string;
+}
+
+export interface JobProvider {
+  enqueue(type: string, payload: Record<string, unknown>, options?: EnqueueOptions): Promise<string>;
+  process(type: string, handler: (payload: Record<string, unknown>) => Promise<void>): void;
+}
+```
+
+---
+
+## Option A: Inngest
+
+### Step 1: Install Inngest
+
+```bash
+yarn add inngest
+```
+
+### Step 2: Environment Variables
+
+```bash
+INNGEST_EVENT_KEY="your-event-key"
+INNGEST_SIGNING_KEY="your-signing-key"
+```
+
+### Step 3: Create the Inngest Client
+
+```typescript
+// src/lib/core/inngest/client.ts
+import 'server-only';
+
+import { Inngest } from 'inngest';
+
+export const inngest = new Inngest({
+  id: 'mars-app',
+  eventKey: process.env.INNGEST_EVENT_KEY,
+});
+```
+
+### Step 4: Define a Function
+
+```typescript
+// src/features/jobs/server/functions/send-email-digest.ts
+import 'server-only';
+
+import { inngest } from '@/lib/core/inngest/client';
+import { prisma } from '@/lib/prisma';
+
+export const sendEmailDigest = inngest.createFunction(
+  {
+    id: 'send-email-digest',
+    retries: 3,
+  },
+  { event: 'jobs/email-digest.requested' },
+  async ({ event, step }) => {
+    const { userId } = event.data;
+
+    const user = await step.run('fetch-user', async () => {
+      return prisma.user.findUniqueOrThrow({ where: { id: userId } });
+    });
+
+    const activities = await step.run('fetch-activities', async () => {
+      return prisma.activity.findMany({
+        where: { userId, createdAt: { gte: new Date(Date.now() - 7 * 86400000) } },
+        orderBy: { createdAt: 'desc' },
+      });
+    });
+
+    await step.run('send-email', async () => {
+      // Use your email service to send the digest
+    });
+
+    return { sent: true, activityCount: activities.length };
+  },
+);
+```
+
+### Step 5: Serve via API Route
+
+```typescript
+// src/app/api/inngest/route.ts
+import { serve } from 'inngest/next';
+import { inngest } from '@/lib/core/inngest/client';
+import { sendEmailDigest } from '@/features/jobs/server/functions/send-email-digest';
+
+export const { GET, POST, PUT } = serve({
+  client: inngest,
+  functions: [sendEmailDigest],
+});
+```
+
+### Step 6: Trigger a Job
+
+```typescript
+import { inngest } from '@/lib/core/inngest/client';
+
+await inngest.send({
+  name: 'jobs/email-digest.requested',
+  data: { userId: 'user_123' },
+});
+```
+
+### Step 7: Scheduled / Cron Functions
+
+```typescript
+export const weeklyReport = inngest.createFunction(
+  { id: 'weekly-report' },
+  { cron: '0 9 * * 1' }, // Every Monday at 9am
+  async ({ step }) => {
+    // Generate and send weekly report
+  },
+);
+```
+
+### Local Development
+
+Run the Inngest dev server alongside your app:
+
+```bash
+npx inngest-cli@latest dev
+```
+
+---
+
+## Option B: Trigger.dev
+
+### Step 1: Install Trigger.dev
+
+```bash
+yarn add @trigger.dev/sdk
+```
+
+### Step 2: Environment Variables
+
+```bash
+TRIGGER_SECRET_KEY="tr_dev_..."
+TRIGGER_API_URL="https://api.trigger.dev"
+```
+
+### Step 3: Create the Trigger Client
+
+```typescript
+// src/lib/core/trigger/client.ts
+import 'server-only';
+
+import { TriggerClient } from '@trigger.dev/sdk';
+
+export const triggerClient = new TriggerClient({
+  id: 'mars-app',
+  apiKey: process.env.TRIGGER_SECRET_KEY,
+  apiUrl: process.env.TRIGGER_API_URL,
+});
+```
+
+### Step 4: Define a Task
+
+```typescript
+// src/features/jobs/server/tasks/export-data.ts
+import 'server-only';
+
+import { task } from '@trigger.dev/sdk/v3';
+import { prisma } from '@/lib/prisma';
+
+export const exportDataTask = task({
+  id: 'export-data',
+  retry: { maxAttempts: 3 },
+  run: async (payload: { userId: string; format: 'csv' | 'json' }) => {
+    const records = await prisma.record.findMany({
+      where: { userId: payload.userId },
+    });
+
+    // Transform and upload to storage
+    return { recordCount: records.length };
+  },
+});
+```
+
+### Step 5: Trigger a Task
+
+```typescript
+import { tasks } from '@trigger.dev/sdk/v3';
+
+await tasks.trigger('export-data', {
+  userId: 'user_123',
+  format: 'csv',
+});
+```
+
+### Step 6: Scheduled Tasks
+
+```typescript
+import { schedules } from '@trigger.dev/sdk/v3';
+
+export const dailyCleanup = schedules.task({
+  id: 'daily-cleanup',
+  cron: '0 2 * * *', // 2am daily
+  run: async () => {
+    // Cleanup expired records
+  },
+});
+```
+
+---
+
+## Option C: DB Queue (Zero Dependencies)
+
+A serverless-compatible job queue backed by Prisma and PostgreSQL. No external services required.
+
+### Step 1: Prisma Schema
+
+```prisma
+// prisma/schema/job.prisma
+model Job {
+  id           String   @id @default(cuid())
+  type         String
+  payload      Json
+  status       String   @default("pending") // pending | processing | completed | failed
+  attempts     Int      @default(0)
+  maxAttempts  Int      @default(3)
+  runAt        DateTime @default(now())
+  startedAt    DateTime?
+  completedAt  DateTime?
+  error        String?
+  idempotencyKey String? @unique
+  createdAt    DateTime @default(now())
+  updatedAt    DateTime @updatedAt
+
+  @@index([status, runAt])
+  @@index([type])
+}
+```
+
+Run `yarn db:push` to sync the schema.
+
+### Step 2: Job Service
+
+```typescript
+// src/features/jobs/server/index.ts
+import 'server-only';
+
+import { prisma } from '@/lib/prisma';
+import type { EnqueueOptions } from '@/features/jobs/types';
+
+export async function enqueueJob(
+  type: string,
+  payload: Record<string, unknown>,
+  options?: EnqueueOptions,
+): Promise<string> {
+  const job = await prisma.job.create({
+    data: {
+      type,
+      payload,
+      runAt: options?.runAt ?? new Date(),
+      maxAttempts: options?.maxAttempts ?? 3,
+      idempotencyKey: options?.idempotencyKey,
+    },
+  });
+  return job.id;
+}
+
+export async function claimNextJob(type: string) {
+  // Atomic claim using updateMany to prevent double-processing
+  const now = new Date();
+
+  const jobs = await prisma.$queryRaw<Array<{ id: string }>>`
+    UPDATE "Job"
+    SET status = 'processing', "startedAt" = ${now}, attempts = attempts + 1
+    WHERE id = (
+      SELECT id FROM "Job"
+      WHERE type = ${type}
+        AND status = 'pending'
+        AND "runAt" <= ${now}
+        AND attempts < "maxAttempts"
+      ORDER BY "runAt" ASC
+      LIMIT 1
+      FOR UPDATE SKIP LOCKED
+    )
+    RETURNING id
+  `;
+
+  if (jobs.length === 0) return null;
+
+  return prisma.job.findUnique({ where: { id: jobs[0].id } });
+}
+
+export async function completeJob(jobId: string) {
+  await prisma.job.update({
+    where: { id: jobId },
+    data: { status: 'completed', completedAt: new Date() },
+  });
+}
+
+export async function failJob(jobId: string, error: string) {
+  const job = await prisma.job.findUnique({ where: { id: jobId } });
+  if (!job) return;
+
+  const newStatus = job.attempts >= job.maxAttempts ? 'failed' : 'pending';
+
+  await prisma.job.update({
+    where: { id: jobId },
+    data: { status: newStatus, error, startedAt: null },
+  });
+}
+
+export async function cleanupCompletedJobs(olderThanDays: number = 30) {
+  const cutoff = new Date(Date.now() - olderThanDays * 86400000);
+
+  const { count } = await prisma.job.deleteMany({
+    where: {
+      status: { in: ['completed', 'failed'] },
+      updatedAt: { lt: cutoff },
+    },
+  });
+
+  return count;
+}
+```
+
+### Step 3: Worker API Route
+
+This route is called by a cron trigger (Vercel Cron, external cron service, etc.) to process pending jobs. No `setInterval` -- fully serverless compatible.
+
+```typescript
+// src/app/api/protected/jobs/process/route.ts
+import { handleApiError, withRole } from '@/lib/mars';
+import { claimNextJob, completeJob, failJob } from '@/features/jobs/server';
+import { jobHandlers } from '@/features/jobs/server/handlers';
+import { NextResponse } from 'next/server';
+
+export const maxDuration = 60;
+
+export const POST = withRole(['admin'], async (request) => {
+  try {
+    const { type } = await request.json();
+    const handler = jobHandlers[type];
+
+    if (!handler) {
+      return NextResponse.json({ error: `No handler for job type: ${type}` }, { status: 400 });
+    }
+
+    let processed = 0;
+    const maxBatch = 10;
+
+    for (let i = 0; i < maxBatch; i++) {
+      const job = await claimNextJob(type);
+      if (!job) break;
+
+      try {
+        await handler(job.payload as Record<string, unknown>);
+        await completeJob(job.id);
+        processed++;
+      } catch (error) {
+        await failJob(job.id, error instanceof Error ? error.message : 'Unknown error');
+      }
+    }
+
+    return NextResponse.json({ processed });
+  } catch (error) {
+    return handleApiError(error, { endpoint: '/api/protected/jobs/process' });
+  }
+});
+```
+
+### Step 4: Job Handlers Registry
+
+```typescript
+// src/features/jobs/server/handlers.ts
+import 'server-only';
+
+type JobHandler = (payload: Record<string, unknown>) => Promise<void>;
+
+export const jobHandlers: Record<string, JobHandler> = {};
+
+export function registerJobHandler(type: string, handler: JobHandler) {
+  jobHandlers[type] = handler;
+}
+```
+
+Register handlers at module load:
+
+```typescript
+// src/features/jobs/server/handlers/email-digest.ts
+import 'server-only';
+
+import { registerJobHandler } from '@/features/jobs/server/handlers';
+
+registerJobHandler('email-digest', async (payload) => {
+  const { userId } = payload as { userId: string };
+  // Build and send the email digest
+});
+```
+
+### Step 5: Cron Configuration
+
+For Vercel, add to `vercel.json`:
+
+```json
+{
+  "crons": [
+    {
+      "path": "/api/protected/jobs/process",
+      "schedule": "*/5 * * * *"
+    }
+  ]
+}
+```
+
+For cleanup, add a separate cron or call `cleanupCompletedJobs()` as part of the processing route.
+
+---
+
+## Common Use Cases
+
+| Use Case | Job Type | Recommended Provider |
+|---|---|---|
+| Email digests | `email-digest` | Any |
+| Data export (CSV/JSON) | `data-export` | Inngest / Trigger.dev |
+| Webhook delivery with retry | `webhook-delivery` | Any |
+| Scheduled reports | `scheduled-report` | Inngest (cron) / DB Queue (Vercel cron) |
+| Image processing | `image-process` | Trigger.dev (long-running) |
+| Cleanup expired data | `data-cleanup` | DB Queue (Vercel cron) |
+
+## Config Integration
+
+Add to `src/config/app.config.ts`:
+
+```typescript
+export const appConfig = {
+  // ... existing config
+  services: {
+    // ... existing services
+    jobs: {
+      provider: 'inngest' as const, // 'inngest' | 'trigger' | 'db-queue'
+    },
+  },
+};
+```
+
+## Testing
+
+Mock the job provider in tests:
+
+```typescript
+import { describe, it, expect, vi } from 'vitest';
+
+vi.mock('@/features/jobs/server', () => ({
+  enqueueJob: vi.fn().mockResolvedValue('job_123'),
+  claimNextJob: vi.fn(),
+  completeJob: vi.fn(),
+  failJob: vi.fn(),
+}));
+```
+
+For Inngest, use their test helper:
+
+```typescript
+import { inngest } from '@/lib/core/inngest/client';
+
+// In tests, assert events were sent
+const events = inngest.createFunction.mock.calls;
+```
+
+## Checklist
+
+- [ ] Provider chosen and configured in `app.config.ts`
+- [ ] Environment variables set for chosen provider
+- [ ] Client/service module created with `import 'server-only'`
+- [ ] Job functions/tasks defined
+- [ ] API route for serving (Inngest) or processing (DB Queue) created
+- [ ] Job handler registry set up (DB Queue)
+- [ ] Cron schedule configured (if using scheduled jobs)
+- [ ] Idempotency keys used where needed to prevent duplicate processing
+- [ ] Retry logic configured with sensible `maxAttempts`
+- [ ] Cleanup strategy for completed/failed jobs
+- [ ] No `setInterval` or `setTimeout` -- serverless compatible
+- [ ] Tests written for job handlers