@igniter-js/jobs 0.1.0

package/AGENTS.md

@@ -0,0 +1,557 @@
+# @igniter-js/jobs - Agent Manual
+
+> **Last Updated:** 2025-01-01
+> **Version:** 0.1.0
+
+## 1. Package Overview
+
+### Purpose
+
+`@igniter-js/jobs` provides type-safe background job processing for TypeScript applications, powered by BullMQ and Redis. It follows the Igniter Builder API pattern for maximum type safety and developer experience.
+
+### Core Concepts
+
+| Concept | Description |
+|---------|-------------|
+| **Queue** | Named container for jobs (e.g., `email`, `notifications`) |
+| **Job** | Unit of work with typed payload and handler |
+| **Cron Job** | Scheduled job with pattern or interval |
+| **Worker** | Process that executes jobs from queues |
+| **Scope** | Multi-tenant context (e.g., organization, workspace) |
+| **Actor** | Entity that triggered the job (e.g., user, system) |
+| **Adapter** | Implementation for queue backend (BullMQ, Memory) |
+
+### Package Exports
+
+```typescript
+// Main entry point
+import { IgniterJobs, IgniterQueue, IgniterJobsError } from '@igniter-js/jobs'
+
+// Adapters
+import { BullMQAdapter, MemoryAdapter } from '@igniter-js/jobs/adapters'
+```
+
+---
+
+## 2. Architecture
+
+### File Structure
+
+```
+packages/jobs/
+├── src/
+│   ├── index.ts                     # Main exports
+│   ├── types/
+│   │   ├── index.ts                 # Type exports
+│   │   ├── scope.ts                 # Scope/Actor types
+│   │   ├── job.ts                   # Job definition types
+│   │   ├── queue.ts                 # Queue config types
+│   │   ├── events.ts                # Event types
+│   │   ├── adapter.ts               # Adapter interface
+│   │   ├── config.ts                # Builder state types
+│   │   └── worker.ts                # Worker types
+│   ├── errors/
+│   │   ├── index.ts                 # Error exports
+│   │   └── igniter-jobs.error.ts    # Error class and codes
+│   ├── builders/
+│   │   ├── index.ts                 # Builder exports
+│   │   ├── igniter-queue.builder.ts # Queue builder
+│   │   ├── igniter-jobs.builder.ts  # Jobs builder
+│   │   └── igniter-worker.builder.ts # Worker builder
+│   ├── core/
+│   │   ├── index.ts                 # Core exports
+│   │   └── igniter-jobs.ts          # Runtime class
+│   └── adapters/
+│       ├── index.ts                 # Adapter exports
+│       ├── bullmq.adapter.ts        # BullMQ implementation
+│       └── memory.adapter.ts        # Memory implementation
+├── tests/
+│   ├── igniter-queue.builder.test.ts
+│   ├── igniter-jobs.builder.test.ts
+│   ├── igniter-jobs.runtime.test.ts
+│   └── memory.adapter.test.ts
+├── package.json
+├── tsconfig.json
+├── tsup.config.ts
+├── vitest.config.ts
+├── README.md
+└── AGENTS.md                        # This file
+```
+
+### Builder Pattern Flow
+
+```
+1. IgniterQueue.create('name')
+   └─► addJob() / addCron()
+       └─► build() → IgniterQueueBuiltConfig
+
+2. IgniterJobs.create()
+   └─► withAdapter()
+       └─► addQueue()
+           └─► addScope() / addActor()
+               └─► withDefaults()
+                   └─► build() → IgniterJobsRuntime (with Proxy)
+
+3. jobs.queueName.jobName.dispatch(data)
+   └─► Proxy intercepts access
+       └─► Returns typed dispatch function
+```
+
+### Proxy-Based API
+
+The runtime uses JavaScript Proxy to provide type-safe access:
+
+```typescript
+// How it works internally:
+jobs.email.sendWelcome.dispatch(data)
+     │     │          │
+     │     │          └── Job proxy with dispatch/schedule methods
+     │     └── Queue proxy returning job proxies
+     └── Root proxy returning queue proxies
+```
+
+---
+
+## 3. Key Components
+
+### 3.1 IgniterQueue Builder
+
+**Location:** `src/builders/igniter-queue.builder.ts`
+
+**Purpose:** Define queue configuration with type-safe job definitions.
+
+**Key Methods:**
+- `create(name, options?)` - Initialize builder
+- `addJob(name, config)` - Add job with schema and handler
+- `addCron(name, config)` - Add cron job
+- `build()` - Return frozen config
+
+**Type Flow:**
+```typescript
+IgniterQueue.create('email')
+  .addJob('send', { schema, handler })
+  // Jobs type accumulates: { send: {...} }
+  .build()
+  // Returns: IgniterQueueBuiltConfig<'email', Jobs>
+```
+
+### 3.2 IgniterJobs Builder
+
+**Location:** `src/builders/igniter-jobs.builder.ts`
+
+**Purpose:** Create jobs runtime with adapter, context, and queues.
+
+**Key Methods:**
+- `create<Context>()` - Initialize with context type
+- `withAdapter(adapter)` - Set queue adapter (required)
+- `withContext(factory)` - Set context factory for handlers
+- `addQueue(queue)` - Add queue configuration
+- `addScope(name, config)` - Define scope type
+- `addActor(name, config)` - Define actor type
+- `withTelemetry(adapter)` - Enable tracing
+- `withDefaults(options)` - Default job options
+- `build()` - Return runtime instance
+
+### 3.3 IgniterJobs Runtime
+
+**Location:** `src/core/igniter-jobs.ts`
+
+**Purpose:** Main runtime class with proxy-based access.
+
+**Key Properties:**
+- `[queueName]` - Queue proxy (via Proxy)
+- `job` - Job management methods
+- `queue` - Queue management methods
+- `subscribe` - Event subscription
+- `search` - Search methods
+- `worker` - Worker builder access
+- `shutdown()` - Graceful shutdown
+
+### 3.4 Adapters
+
+**BullMQ Adapter:** `src/adapters/bullmq.adapter.ts`
+- Production-ready with Redis
+- Supports all BullMQ features
+- Lazy loads BullMQ module
+
+**Memory Adapter:** `src/adapters/memory.adapter.ts`
+- For testing/development
+- No external dependencies
+- Simulates job processing
+
+---
+
+## 4. Type System
+
+### Core Type Hierarchy
+
+```typescript
+// Job Definition (queue level)
+interface IgniterJobDefinition<TData, TResult, TContext> {
+  schema: ZodSchema<TData>
+  handler: IgniterJobHandler<TData, TResult, TContext>
+  options?: IgniterJobOptions
+}
+
+// Handler Context (passed to handler)
+interface IgniterJobHandlerContext<TData, TContext> {
+  data: TData
+  context: TContext
+  job: { id: string; name: string; queue: string; attempt: number }
+  log: (level, message) => Promise<void>
+  updateProgress: (progress) => Promise<void>
+  scope?: IgniterJobScopeEntry
+  actor?: IgniterJobActorEntry
+}
+
+// Dispatch Options
+interface DispatchOptions {
+  delay?: number
+  priority?: number
+  attempts?: number
+  backoff?: IgniterJobBackoff
+  jobId?: string
+  scope?: { type: string; id: string }
+  actor?: { type: string; id: string }
+}
+```
+
+### Type Inference Chain
+
+```typescript
+// 1. Define schema
+const schema = z.object({ email: z.string() })
+// Type: z.ZodObject<{ email: z.ZodString }>
+
+// 2. Schema infers handler data type
+handler: async ({ data }) => { ... }
+// data type: { email: string }
+
+// 3. Queue builder accumulates job types
+.addJob('sendWelcome', { schema, handler })
+// Jobs type: { sendWelcome: { data: { email: string } } }
+
+// 4. Runtime proxy provides typed dispatch
+jobs.email.sendWelcome.dispatch({ email: 'x' })
+// Argument type: { email: string }
+```
+
+---
+
+## 5. Error Handling
+
+### Error Codes
+
+**Configuration Errors:**
+- `JOBS_MISSING_ADAPTER` - No adapter provided
+- `JOBS_INVALID_QUEUE_NAME` - Invalid queue name
+- `JOBS_INVALID_JOB_NAME` - Invalid job name
+- `JOBS_DUPLICATE_QUEUE` - Queue already registered
+- `JOBS_DUPLICATE_JOB` - Job already registered
+
+**Queue Errors:**
+- `JOBS_QUEUE_NOT_FOUND` - Queue doesn't exist
+- `JOBS_QUEUE_PAUSED` - Queue is paused
+- `JOBS_QUEUE_FAILED_TO_PAUSE` - Pause operation failed
+
+**Job Errors:**
+- `JOBS_JOB_NOT_FOUND` - Job doesn't exist
+- `JOBS_JOB_VALIDATION_FAILED` - Payload validation failed
+- `JOBS_JOB_HANDLER_FAILED` - Handler threw error
+- `JOBS_JOB_TIMEOUT` - Job exceeded timeout
+
+**Dispatch Errors:**
+- `JOBS_DISPATCH_FAILED` - Dispatch operation failed
+- `JOBS_SCHEDULE_FAILED` - Schedule operation failed
+- `JOBS_INVALID_DISPATCH_PAYLOAD` - Invalid payload
+
+### Error Class
+
+```typescript
+throw new IgniterJobsError({
+  code: 'JOBS_JOB_NOT_FOUND',
+  message: `Job "${jobId}" not found in queue "${queue}"`,
+  statusCode: 404,
+  details: { jobId, queue },
+})
+```
+
+---
+
+## 6. Development Guidelines
+
+### Adding New Features
+
+1. **Add Types First**
+   - Define in appropriate `src/types/*.ts` file
+   - Export from `src/types/index.ts`
+   - Export from `src/index.ts` if public
+
+2. **Update Builders**
+   - Add method to builder class
+   - Update state type if needed
+   - Maintain fluent return type
+
+3. **Update Runtime**
+   - Implement in `src/core/igniter-jobs.ts`
+   - Add proxy handling if needed
+
+4. **Update Adapter Interface**
+   - Add method to `IgniterJobsAdapter`
+   - Implement in BullMQ adapter
+   - Implement in Memory adapter
+
+5. **Add Tests**
+   - Unit test for builder method
+   - Integration test for runtime
+   - Adapter test for implementation
+
+### Testing Strategy
+
+```bash
+# Run all tests
+npm test
+
+# Watch mode
+npm run test:watch
+
+# Coverage
+npm run test:coverage
+```
+
+**Test Files:**
+- `tests/igniter-queue.builder.test.ts` - Queue builder
+- `tests/igniter-jobs.builder.test.ts` - Jobs builder
+- `tests/igniter-jobs.runtime.test.ts` - Integration
+- `tests/memory.adapter.test.ts` - Memory adapter
+
+### Code Style
+
+- Use JSDoc for all public APIs
+- Follow existing patterns exactly
+- No `any` types in public API
+- Export types explicitly
+- Use `readonly` where appropriate
+
+---
+
+## 7. Adapter Implementation Guide
+
+### Required Methods
+
+When implementing a new adapter, implement ALL methods from `IgniterJobsAdapter`:
+
+```typescript
+interface IgniterJobsAdapter {
+  // Connection
+  readonly client: unknown
+
+  // Job Operations
+  dispatch(params: AdapterDispatchParams): Promise<string>
+  schedule(params: AdapterScheduleParams): Promise<string>
+  getJob(queue: string, jobId: string): Promise<IgniterJobInfo | null>
+  getJobState(queue: string, jobId: string): Promise<IgniterJobStatus | null>
+  getJobProgress(queue: string, jobId: string): Promise<number>
+  getJobLogs(queue: string, jobId: string): Promise<IgniterJobLog[]>
+  retryJob(queue: string, jobId: string): Promise<void>
+  removeJob(queue: string, jobId: string): Promise<void>
+  promoteJob(queue: string, jobId: string): Promise<void>
+  moveJob(queue: string, jobId: string, state: string, reason?: string): Promise<void>
+  retryJobs(queue: string, jobIds: string[]): Promise<void>
+  removeJobs(queue: string, jobIds: string[]): Promise<void>
+
+  // Queue Operations
+  getQueue(queue: string): Promise<IgniterQueueInfo>
+  pauseQueue(queue: string): Promise<void>
+  resumeQueue(queue: string): Promise<void>
+  drainQueue(queue: string): Promise<number>
+  cleanQueue(queue: string, options: IgniterQueueCleanOptions): Promise<number>
+  obliterateQueue(queue: string, options?: { force?: boolean }): Promise<void>
+  retryAllFailed(queue: string): Promise<number>
+  getJobCounts(queue: string): Promise<IgniterJobCounts>
+  listJobs(queue: string, options?: {...}): Promise<IgniterJobSearchResult[]>
+  pauseJobType(queue: string, jobName: string): Promise<void>
+  resumeJobType(queue: string, jobName: string): Promise<void>
+
+  // Events
+  subscribe(pattern: string, handler: IgniterJobEventHandler): Promise<IgniterJobUnsubscribeFn>
+
+  // Workers
+  createWorker(config: AdapterWorkerConfig, handler: AdapterJobHandler): Promise<AdapterWorkerHandle>
+
+  // Search
+  searchJobs(filter: {...}): Promise<IgniterJobSearchResult[]>
+  searchQueues(filter: {...}): Promise<IgniterQueueInfo[]>
+
+  // Lifecycle
+  shutdown(): Promise<void>
+}
+```
+
+---
+
+## 8. Common Patterns
+
+### Multi-tenant Job Dispatch
+
+```typescript
+const jobs = IgniterJobs.create()
+  .withAdapter(adapter)
+  .addQueue(emailQueue)
+  .addScope('organization', {
+    resolver: (id) => db.org.findUnique({ where: { id } }),
+  })
+  .addActor('user', {
+    resolver: (id) => db.user.findUnique({ where: { id } }),
+  })
+  .build()
+
+// Dispatch with scope/actor
+await jobs.email.sendWelcome.dispatch(
+  { email: 'test@example.com' },
+  {
+    scope: { type: 'organization', id: 'org-123' },
+    actor: { type: 'user', id: 'user-456' },
+  }
+)
+```
+
+### Job With Progress Updates
+
+```typescript
+const queue = IgniterQueue.create('exports')
+  .addJob('generateReport', {
+    schema: z.object({ reportId: z.string() }),
+    handler: async ({ data, updateProgress, log }) => {
+      await updateProgress(10)
+      await log('info', 'Starting report generation')
+
+      // ... process
+
+      await updateProgress(50)
+      await log('info', 'Halfway done')
+
+      // ... finish
+
+      await updateProgress(100)
+      return { success: true }
+    },
+  })
+  .build()
+```
+
+### Scheduled Jobs
+
+```typescript
+// Schedule at specific time
+await jobs.email.sendWelcome.schedule({
+  data: { email: 'test@example.com' },
+  at: new Date('2025-12-25T00:00:00Z'),
+})
+
+// Schedule with delay
+await jobs.email.sendWelcome.schedule({
+  data: { email: 'test@example.com' },
+  delay: 60000, // 1 minute
+})
+```
+
+### Worker with Rate Limiting
+
+```typescript
+const worker = await jobs.worker
+  .forQueues(['email'])
+  .withConcurrency(10)
+  .withLimiter({
+    max: 100,      // max jobs
+    duration: 60000, // per minute
+  })
+  .withLockDuration(30000)
+  .onIdle(() => console.log('No jobs to process'))
+  .start()
+```
+
+---
+
+## 9. Version History
+
+### 0.1.0 (Initial Release)
+
+- ✅ IgniterQueue builder with addJob/addCron
+- ✅ IgniterJobs builder with full configuration
+- ✅ IgniterWorker builder with fluent API
+- ✅ Proxy-based runtime for type-safe access
+- ✅ BullMQ adapter with full feature support
+- ✅ Memory adapter for testing
+- ✅ Scope and actor support
+- ✅ Event subscription system
+- ✅ Job and queue search
+- ✅ Error codes and classes
+
+---
+
+## 10. Troubleshooting
+
+### Common Issues
+
+**"JOBS_MISSING_ADAPTER"**
+- Ensure `withAdapter()` is called before `build()`
+- Check adapter is properly instantiated
+
+**"JOBS_QUEUE_NOT_FOUND"**
+- Queue must be registered with `addQueue()`
+- Check queue name spelling
+
+**"JOBS_JOB_VALIDATION_FAILED"**
+- Payload doesn't match Zod schema
+- Check `details.issues` for validation errors
+
+**TypeScript errors on dispatch**
+- Ensure queue is added before build
+- Check job name exists in queue
+- Verify payload matches schema
+
+### Debug Tips
+
+```typescript
+// Enable BullMQ debug logging
+process.env.DEBUG = 'bullmq:*'
+
+// Check job state
+const state = await jobs.job.state('queue', 'job-id')
+console.log('Job state:', state)
+
+// Get job logs
+const logs = await jobs.job.logs('queue', 'job-id')
+console.log('Job logs:', logs)
+
+// Subscribe to all events
+await jobs.subscribe('*', (event) => {
+  console.log('Event:', event.type, event.data)
+})
+```
+
+---
+
+## 11. Dependencies
+
+### Peer Dependencies
+
+| Package | Version | Purpose |
+|---------|---------|---------|
+| `bullmq` | `^5.0.0` | Job queue implementation |
+| `ioredis` | `^5.0.0` | Redis client |
+| `zod` | `^3.0.0` | Schema validation |
+| `@igniter-js/telemetry` | `^0.1.0` | Optional tracing |
+
+### Dev Dependencies
+
+| Package | Version | Purpose |
+|---------|---------|---------|
+| `typescript` | `^5.0.0` | TypeScript compiler |
+| `vitest` | `^1.0.0` | Testing framework |
+| `tsup` | `^8.0.0` | Build tool |
+
+---
+
+**Remember:** Always follow existing patterns. When in doubt, check how similar features are implemented in `@igniter-js/store` or `@igniter-js/core`.