@boopkit/cap-background-jobs 0.2.6 → 0.3.0

package/README.md

@@ -0,0 +1,186 @@
+# @boopkit/cap-background-jobs
+
+**Part of [Boop](https://www.npmjs.com/package/@boopkit/cli)** — a bootstrap platform for AI-first builders. `boop init` scaffolds a running Node/TypeScript web app from composable **capabilities**, each one bundling the code, dependencies, middleware wiring, and agent skills needed for a feature. You pick the capabilities; `boop add` handles the rest.
+
+`cap-background-jobs` adds async job processing via RabbitMQ (CloudAMQP). You get a publish/listen API, a dedicated worker process, a scheduler CLI for cron-triggered jobs, and automatic retry/dead-letter topology per queue. Use it for anything that shouldn't block an HTTP response: image processing, outbound email, nightly reconciles, third-party webhooks.
+
+## What You Get
+
+### Scaffold files
+
+Copied into your project on install:
+
+- `src/worker.ts` — worker process entry point. Boots services, registers consumers, handles graceful shutdown
+- `src/scheduler.ts` — CLI for publishing scheduled triggers, filtered by `--interval` and `--job`
+- `src/controllers/example-job.controller.ts` — copyable template for message consumers (auto-ACK pattern)
+- `tests/controllers/example-job.controller.test.ts` — example consumer tests
+- `reference/background-jobs.ref.md` — infrastructure topology, ACK modes, Heroku setup
+
+### Scripts added to `package.json`
+
+| Script | Command | Purpose |
+|--------|---------|---------|
+| `worker` | `tsx src/worker.ts` | Start the background job worker process |
+| `scheduler` | `tsx src/scheduler.ts` | Publish trigger messages to job queues (`run-jobs`, `publish`, `list` subcommands) |
+
+### Skills installed
+
+- `design.background-jobs.skill.md` — teaches agents how to add a consumer, choose between auto- and manual-ACK, add scheduled jobs, and handle retries.
+
+### Post-install
+
+1. Provision CloudAMQP (or any RabbitMQ instance) and add `CLOUDAMQP_URL` to `.env`
+2. Set `QUEUE_PREFIX=dev/<yourname>` to isolate your queues from other developers sharing the instance
+3. Add `worker: npm run worker` to your `Procfile`
+4. Configure your external scheduler (e.g. Heroku Scheduler) to run `npm run scheduler -- run-jobs --interval <tag>` on the desired cadence
+
+`boop status` reminds you of these steps after install.
+
+## Requirements
+
+**Other capabilities:**
+
+- `cap-services` — the message queue is registered as `services.message-queue` via the service map
+
+**npm dependencies added to your project:**
+
+- `amqplib` — RabbitMQ client
+- `minimist` — scheduler CLI argument parsing
+
+**External services:**
+
+- **RabbitMQ** — CloudAMQP is the easiest path on Heroku (`heroku addons:create cloudamqp`, which sets `CLOUDAMQP_URL` automatically). Any RabbitMQ broker works — local Docker, self-hosted, or AWS MQ — point `CLOUDAMQP_URL` at it.
+
+**Environment variables:**
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `CLOUDAMQP_URL` | Yes (at boot) | RabbitMQ connection URL (`amqp://...` or `amqps://...`) |
+| `QUEUE_PREFIX` | No | Queue name prefix for environment isolation (e.g. `dev/mike`). Leave empty in production |
+| `MAX_MESSAGE_RETRIES` | No | Max retry attempts before dead-lettering (default: `5`) |
+
+## Usage
+
+Three moving parts — web publishes, scheduler publishes on a cron, worker consumes. All share one `MessageQueueService` instance registered in the service map.
+
+### Publish a job
+
+From a route handler, service, or anywhere in the web process:
+
+```ts
+import { services } from '@services';
+
+app.post('/upload', async (req, res) => {
+  const upload = await services.uploads.create(req.body);
+
+  // Enqueue async processing — returns immediately
+  await services.messageQueue.publish('process-upload', {
+    task: 'process-upload',
+    data: { uploadId: upload.id, userId: req.user.id },
+  });
+
+  res.status(202).json({ id: upload.id });
+});
+```
+
+The publish connection is lazy — it only opens when you actually publish. Your web dynos stay cheap.
+
+### Consume a job
+
+**1.** Copy `src/controllers/example-job.controller.ts` to `src/controllers/process-upload.controller.ts` and replace the handler body:
+
+```ts
+export const QUEUE_NAME = 'process-upload';
+
+export async function onMessage(msg: unknown): Promise<boolean> {
+  const payload = validatePayload(msg);
+
+  try {
+    await services.uploads.process(payload.data.uploadId);
+    return true;   // ACK — success
+  } catch (err) {
+    console.error('[process-upload] failed:', err);
+    return false;  // retry with exponential backoff
+  }
+}
+```
+
+**2.** Register the consumer in `src/worker.ts`:
+
+```ts
+import * as processUploadController from './controllers/process-upload.controller';
+
+const consumers = [
+  { queue: exampleJobController.QUEUE_NAME, handler: exampleJobController.onMessage },
+  { queue: processUploadController.QUEUE_NAME, handler: processUploadController.onMessage },
+];
+```
+
+**3.** Run the worker locally:
+
+```bash
+npm run worker
+```
+
+On first listen, the service asserts the queue topology: main queue, `.retry` queue (TTL-backed redelivery), and `.failed` dead-letter queue. Failed handlers retry with exponential backoff (4s → 60s, capped) up to `MAX_MESSAGE_RETRIES`, then land in the DLQ for inspection.
+
+### Error classification
+
+Return value from the handler decides routing:
+
+| Return | Result |
+|--------|--------|
+| `return true` | ACK — success, or permanent failure that retrying won't fix |
+| `return false` | Retry with backoff |
+| `throw` | Retry, then DLQ after max retries |
+
+### Scheduled jobs
+
+Add an entry to the `jobs` array in `src/scheduler.ts`:
+
+```ts
+const jobs: JobConfig[] = [
+  { name: 'nightly-sync', queue: 'sync-accounts', interval: '60-minutes',
+    payload: { task: 'nightly-sync' } },
+];
+```
+
+Then configure your external scheduler:
+
+```bash
+# Run all jobs tagged 60-minutes
+npm run scheduler -- run-jobs --interval 60-minutes
+
+# Run a specific job on demand
+npm run scheduler -- run-jobs --job nightly-sync
+
+# Publish an ad-hoc message
+npm run scheduler -- publish --queue sync-accounts --task manual-run
+```
+
+### Manual ACK
+
+For batch commits or transaction coordination, opt into manual ACK:
+
+```ts
+await services.messageQueue.listen('batch-job', async (msg, ack, nack) => {
+  try {
+    await processBatch(msg);
+    ack();
+  } catch {
+    nack();
+  }
+}, { manualAck: true });
+```
+
+First call to `ack()` or `nack()` wins — subsequent calls are silent no-ops.
+
+See `reference/background-jobs.ref.md` in your scaffolded project for the full topology diagram, connection lifecycle, and Heroku deployment guide.
+
+## Added by
+
+Not included by default. Add to an existing project with:
+
+```bash
+boop add background-jobs
+```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@boopkit/cap-background-jobs",
-  "version": "0.2.6",
+  "version": "0.3.0",
   "description": "Background job processing with RabbitMQ for boopkit-scaffolded projects",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",