@12-apps/jobs 4.1.0 → 4.3.0

package/ADOPTING.md

@@ -19,6 +19,59 @@ arrive as config. What the package owns is the RUNNING of them.
 | **Drivers** | `@12-apps/jobs/bullmq`, `@12-apps/jobs/inline` | Nothing, usually — `createApiJobs` resolves them. Import directly only to hand a pre-built instance in (`driver: createInlineJobDriver({ await: true })` in a test). |
 | **Prisma** | `prisma/jobs.prisma` + `prisma/migrations/*` | Sync BOTH into the host's schema/migrations folders as **copies** (see below). One table: `sweep_leases`. |
 
+## `defineJobModule` — how another PACKAGE ships background work
+
+A package that owns a domain owns the deferred half of it too: the sweep that
+finishes what a request could not, the watcher that waits for a callback, the
+expiry that stops waiting. `defineJob` cannot express that — it registers at
+module scope, so the handler would have to close over a database client, a
+gateway and a logger the package cannot know at import time.
+
+So a package declares **blueprints** (everything about a job except the deps)
+and the host supplies the deps in one line — the worker counterpart of
+`mountPayments(config) → { routes }`:
+
+```ts
+// in the package, beside the domain it belongs to
+export const paymentsJobs = defineJobModule<PaymentsJobDeps>()({
+  namespace: 'payments',
+  jobs: {
+    watchSettlement: {
+      name: 'watch-settlement',            // → `payments.watch-settlement`
+      attempts: 3,
+      backoff: { type: 'exponential', delayMs: 5_000 },
+      handle: async ({ chargeId }, deps) => deps.gateway.refreshCharge(chargeId),
+    },
+  },
+});
+
+// in the host, beside the endpoint mount it already writes
+const { jobs } = paymentsJobs.mount({ gateway, store, logger });
+createApiJobs({ jobs: [...jobs, ...ownJobs] });
+```
+
+Three properties this buys, each of which a host could otherwise satisfy by
+accident and lose silently:
+
+- **Importing the package registers nothing.** Work reaches a registry only
+  when a host mounts it.
+- **The host is never asked for the policy.** Retries, backoff, queue,
+  schedule and concurrency are the package's decisions, exactly as the HTTP
+  methods of a settings route are. A host that had to restate them is a host
+  that can get them wrong.
+- **The package can enqueue its own work.**
+  `paymentsJobs.enqueue.watchSettlement(…)` resolves through the registry by
+  NAME at call time, so it works from a module that holds no deps and was
+  imported long before the mount. Enqueueing before a mount reports
+  `unregistered` and logs — it never throws, because the emit site is usually a
+  request path (a charge being raised) and a wiring mistake in the deferred
+  half must not fail the money path in front of it.
+
+Names are namespaced here, once: a blueprint states `watch-settlement` and the
+module prepends `payments.`. Two packages therefore cannot collide by both
+calling something `drain`, and a queue dashboard says whose work it is. A blank
+or dotted namespace is refused at declaration.
+
 ## Why there is no `createWebJobs`
 
 The porting contract asks for both halves — `createApiFoo` and `createWebFoo`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@12-apps/jobs",
-  "version": "4.1.0",
+  "version": "4.3.0",
   "type": "module",
   "description": "Generic background-job library: a typed job registry with retries, exponential backoff and cron schedules, behind a swappable driver port (BullMQ/Redis in production, inline execution in tests and zero-config dev). The runtime half (./server) is one factory — createApiJobs: driver resolution, worker bootstrap, graceful drain, the single-writer sweep lease and a health endpoint — with a Hono adapter (./hono) and the package-owned SweepLease Prisma partial + migrations. Knows nothing about the host app's domain, ORM or transport.",
   "exports": {
@@ -9,6 +9,7 @@
     "./hono": "./src/hono/index.ts",
     "./bullmq": "./src/drivers/bullmq.ts",
     "./inline": "./src/drivers/inline.ts",
+    "./manifest": "./src/manifest/index.ts",
     "./package.json": "./package.json"
   },
   "scripts": {
@@ -23,7 +24,7 @@
     "bullmq": "^5.81.2"
   },
   "devDependencies": {
-    "@12-apps/eslint-config": "^1.20.0",
+    "@12-apps/eslint-config": "^1.21.0",
     "@12-apps/typescript-config": "^1.20.0",
     "eslint": "^9.39.1",
     "eslint-plugin-test-flakiness": "^1.4.0",
@@ -68,5 +69,26 @@
     "!**/*.stories.*",
     "!**/*.test-story.*",
     "!**/test-helpers.*"
-  ]
+  ],
+  "wiring": {
+    "env": [
+      {
+        "name": "REDIS_URL",
+        "description": "Set it and the queue turns itself on; unset selects the inline/off driver."
+      },
+      {
+        "name": "JOBS_DRIVER",
+        "description": "bullmq | inline | off; unset picks by REDIS_URL and NODE_ENV."
+      },
+      {
+        "name": "JOBS_WORKER",
+        "scope": "worker",
+        "description": "\"1\" on the ONE process that consumes the queue and runs schedules."
+      },
+      {
+        "name": "JOBS_QUEUE_PREFIX",
+        "description": "Optional key prefix, to share one Redis across environments."
+      }
+    ]
+  }
 }

package/src/core/module.ts

@@ -0,0 +1,236 @@
+/**
+ * `defineJobModule` — how a PACKAGE exposes background work.
+ *
+ * This is the worker counterpart of an endpoint mount, and it exists for the
+ * same reason that one does. A package that owns a domain owns the DEFERRED
+ * half of it too: the sweep that finishes what a request could not, the watcher
+ * that waits for a callback, the expiry that stops waiting. Left to each host,
+ * that half gets rewritten per adopter — and every rewrite is a place the
+ * behaviour can differ, which is exactly what happened to the payments
+ * reconciliation: one host had it, the others silently did not.
+ *
+ * ## The problem this solves
+ *
+ * `defineJob` registers at MODULE SCOPE. That is right for a host's own jobs,
+ * where the handler can close over whatever it needs, and impossible for a
+ * package's: a package cannot know the host's database client, gateway or
+ * logger at import time, and must not reach for a global to find them.
+ *
+ * So a package declares BLUEPRINTS — everything about a job except the deps —
+ * and the host supplies the deps at mount:
+ *
+ *     // in the package
+ *     export const paymentsJobs = defineJobModule<PaymentsJobDeps>()({
+ *       namespace: "payments",
+ *       jobs: { watchSettlement, expireWatches },
+ *     });
+ *
+ *     // in the host, beside the endpoint mount it already writes
+ *     const { jobs } = paymentsJobs.mount({ gateway, store, logger });
+ *     createApiJobs({ jobs: [...jobs, ...ownJobs] });
+ *
+ * The host's line is the whole integration. It cannot get the retry policy,
+ * the schedule, the queue or the concurrency wrong, because it is not asked
+ * for them — the same reason `mountPayments` does not ask a host which HTTP
+ * methods a settings route answers.
+ *
+ * ## Enqueueing without the deps
+ *
+ * A package's own code has to be able to enqueue its own jobs — a charge is
+ * raised, and something must start watching it — long after `mount` ran and
+ * from a module that holds no deps. `enqueue` is therefore a LAZY handle: it
+ * resolves through the registry by name at call time, exactly as an emit site
+ * does, so it works from anywhere and needs nothing threaded through.
+ *
+ *     await paymentsJobs.enqueue.watchSettlement({ chargeId }, { dedupeKey });
+ *
+ * Enqueueing before `mount` reports `unregistered` rather than throwing — the
+ * same answer `enqueueJob` gives for any unknown name, and the same reason: a
+ * deferred side effect must never take down the request that scheduled it. A
+ * host that forgot to mount sees it in the log and in the skip reason, while
+ * the durable row the job was paired with is still there for the sweep.
+ *
+ * ## Names are namespaced here, once
+ *
+ * A blueprint states its own short name (`watch-settlement`) and this prepends
+ * the namespace (`payments.watch-settlement`). The wire key is then derived in
+ * one place rather than spelled in each declaration, so two packages cannot
+ * collide by both calling something `drain`, and a host reading its queue
+ * dashboard can tell whose work it is looking at.
+ */
+
+import { defineJob, resolveRegisteredJob, type RegisteredJob } from "./registry";
+import { enqueueJob, getJobLogger } from "./runtime";
+import type {
+  EnqueueOptions,
+  EnqueueResult,
+  JobBackoff,
+  JobContext,
+  JobSchedule,
+} from "./types";
+
+/**
+ * One job a package declares, with its deps left open.
+ *
+ * Everything `JobDefinition` carries except `name` (namespaced at mount) and
+ * `handle`, whose signature gains the host-supplied deps. A blueprint is inert
+ * data: declaring one registers nothing and starts nothing.
+ */
+export interface JobBlueprint<TPayload, TDeps> {
+  /**
+   * Short name within the module — `watch-settlement`, not
+   * `payments.watch-settlement`. The namespace is prepended at mount.
+   */
+  name: string;
+  queue?: string;
+  attempts?: number;
+  backoff?: JobBackoff;
+  schedule?: JobSchedule;
+  concurrency?: number;
+  /**
+   * The work. Takes the host's deps as its second argument, which is the only
+   * difference from a plain {@link JobDefinition} handler — the payload rule
+   * and the idempotency contract are unchanged and still apply.
+   */
+  handle: (payload: TPayload, deps: TDeps, context: JobContext) => Promise<void>;
+}
+
+/** A record of blueprints, keyed by however the package refers to them. */
+export type JobBlueprints<TDeps> = Readonly<
+  Record<string, JobBlueprint<never, TDeps>>
+>;
+
+/** The payload a blueprint carries, recovered for the enqueue signature. */
+type PayloadOf<TBlueprint> =
+  TBlueprint extends JobBlueprint<infer TPayload, never> ? TPayload : never;
+
+/**
+ * What `mount` answers: the registered jobs, ready to hand to `createApiJobs`.
+ *
+ * An object rather than a bare array, and for the same reason
+ * `MountedPaymentsRoutes` is one: a mount grows things a host needs alongside
+ * the primary artefact, and widening an object is not a breaking change.
+ */
+export interface MountedJobs {
+  /** Pass straight to `createApiJobs({ jobs })`, spread beside the host's own. */
+  readonly jobs: readonly RegisteredJob<never>[];
+  /** The wire names this mount registered — for a host's own health output. */
+  readonly names: readonly string[];
+}
+
+/** The lazy enqueuers, one per blueprint, keyed as the blueprints were. */
+export type JobModuleEnqueue<TBlueprints> = {
+  readonly [K in keyof TBlueprints]: (
+    payload: PayloadOf<TBlueprints[K]>,
+    options?: EnqueueOptions,
+  ) => Promise<EnqueueResult>;
+};
+
+/** A package's deferred surface: mount it once, enqueue from anywhere. */
+export interface JobModule<TDeps, TBlueprints extends JobBlueprints<TDeps>> {
+  /** `payments`, `notifications`, … — the prefix on every name this owns. */
+  readonly namespace: string;
+  /** Register every job with the host's deps bound in. Call once per process. */
+  mount(deps: TDeps): MountedJobs;
+  /** Defer one run, resolved through the registry at call time. */
+  readonly enqueue: JobModuleEnqueue<TBlueprints>;
+  /** The wire name of a blueprint, for logs and dedupe keys. */
+  nameOf(key: keyof TBlueprints): string;
+}
+
+/**
+ * Declare a package's deferred surface.
+ *
+ * Curried so the deps type is stated once and every blueprint is checked
+ * against it, while the blueprint record's own keys and payload types stay
+ * inferred — TypeScript cannot do both in one call.
+ */
+export function defineJobModule<TDeps>() {
+  return function build<TBlueprints extends JobBlueprints<TDeps>>(options: {
+    namespace: string;
+    jobs: TBlueprints;
+  }): JobModule<TDeps, TBlueprints> {
+    const { namespace, jobs } = options;
+    assertNamespace(namespace);
+    const keys = Object.keys(jobs) as (keyof typeof jobs)[];
+    const wireName = (key: keyof typeof jobs): string =>
+      `${namespace}.${jobs[key]?.name ?? String(key)}`;
+
+    return {
+      namespace,
+      nameOf: wireName,
+
+      mount(deps: TDeps): MountedJobs {
+        const registered = keys.map((key) => {
+          const blueprint = jobs[key] as JobBlueprint<never, TDeps>;
+          return defineJob<never>({
+            name: wireName(key),
+            ...(blueprint.queue === undefined ? {} : { queue: blueprint.queue }),
+            ...(blueprint.attempts === undefined ? {} : { attempts: blueprint.attempts }),
+            ...(blueprint.backoff === undefined ? {} : { backoff: blueprint.backoff }),
+            ...(blueprint.schedule === undefined ? {} : { schedule: blueprint.schedule }),
+            ...(blueprint.concurrency === undefined
+              ? {}
+              : { concurrency: blueprint.concurrency }),
+            handle: (payload, context) => blueprint.handle(payload, deps, context),
+          });
+        });
+        return {
+          jobs: registered,
+          names: registered.map((job) => job.name),
+        };
+      },
+
+      enqueue: Object.fromEntries(
+        keys.map((key) => [
+          key,
+          // Resolved at CALL time, not at mount: this handle is what the
+          // package's own emit sites hold, and they are imported long before
+          // (and often without) the mount that binds the deps.
+          async (payload: never, enqueueOptions?: EnqueueOptions) => {
+            const name = wireName(key);
+            const definition = resolveRegisteredJob(name);
+            if (!definition) return unmounted(name);
+            return enqueueJob(definition, payload, enqueueOptions);
+          },
+        ]),
+      ) as JobModuleEnqueue<TBlueprints>,
+    };
+  };
+}
+
+/**
+ * A namespace that cannot silently produce a bad wire name.
+ *
+ * A blank one yields `.watch-settlement`, which registers, schedules and runs
+ * — and is indistinguishable in a queue dashboard from another package's. A
+ * dot inside one yields a name with two, which no host can parse back.
+ */
+function assertNamespace(namespace: string): void {
+  if (typeof namespace !== "string" || namespace.trim() === "") {
+    throw new TypeError("defineJobModule: `namespace` must be a non-empty string.");
+  }
+  if (namespace.includes(".")) {
+    throw new TypeError(
+      `defineJobModule: \`namespace\` must not contain a dot, got "${namespace}".`,
+    );
+  }
+}
+
+/**
+ * The answer for an enqueue that arrived before `mount` — or without one.
+ *
+ * A skip rather than a throw, because this is reached from wherever the
+ * package's domain happens to enqueue: a charge being raised, a webhook being
+ * accepted. Those are request paths, and a wiring mistake in the deferred half
+ * must not fail the money path in front of it. The log line is what makes the
+ * mistake findable; `unregistered` is what `enqueueJob` itself answers for the
+ * same condition, so a caller has one reason code to handle, not two.
+ */
+function unmounted(name: string): EnqueueResult {
+  getJobLogger().error(
+    `"${name}" was not enqueued: its job module is not mounted in this process. ` +
+      "Call the module's mount() and pass its jobs to createApiJobs({ jobs }).",
+  );
+  return { enqueued: false, reason: "unregistered" };
+}

package/src/index.ts

@@ -55,6 +55,21 @@ export {
 } from "./core/registry";
 export type { RegisteredJob } from "./core/registry";
 
+/**
+ * The PACKAGE-side seam — the worker counterpart of an endpoint mount. A
+ * package declares blueprints whose handlers take host deps; the host binds
+ * them in one line and spreads the result into `createApiJobs({ jobs })`. See
+ * `core/module.ts` for why a package cannot reach for `defineJob` directly.
+ */
+export { defineJobModule } from "./core/module";
+export type {
+  JobBlueprint,
+  JobBlueprints,
+  JobModule,
+  JobModuleEnqueue,
+  MountedJobs,
+} from "./core/module";
+
 export {
   configureJobs,
   enqueueJob,

package/src/manifest/index.ts

@@ -0,0 +1,36 @@
+/**
+ * `@12-apps/jobs/manifest` — the SHARED wiring manifest.
+ *
+ * A pure-data manifest: identity plus the env surface, nothing else. The
+ * runtime capabilities deliberately stay UNDECLARED for now — hosts mount
+ * this package directly (`createJobs`, the Hono adapter) rather than through
+ * the wiring consumer, and inventorying `server` here would oblige every one
+ * of them to adopt or decline it. The env declaration carries its own weight
+ * today: the exact `process.env` keys the queue reads, mirrored into
+ * `package.json` for tooling that cannot execute TypeScript, so a host's
+ * `.env.example` can stop rotting by hand.
+ *
+ * Every key is OPTIONAL by this package's own design: absence selects the
+ * inline/off driver and logs the decision — it never throws. Config always
+ * wins over env (`config.x ?? process.env.Y`). `NODE_ENV` is read too, but
+ * it is platform vocabulary, not a contribution.
+ *
+ * This package depends on `@12-apps/wiring` in NEITHER direction — wiring
+ * already devDepends on jobs for its blueprint-compat suite, so an edge back
+ * would close a turbo build cycle. The manifest is therefore untyped pure
+ * data here, and the producer assertions (defineManifest, both mirrors, the
+ * exports tripwire) run in WIRING's own suite, which imports this subpath —
+ * the same "fails in a test run before any host sees it" guarantee, one
+ * package over, along the dependency edge that already exists.
+ */
+
+export const jobsManifest = {
+  name: "@12-apps/jobs",
+  contract: 1,
+  env: [
+    { name: "REDIS_URL", description: "Set it and the queue turns itself on; unset selects the inline/off driver." },
+    { name: "JOBS_DRIVER", description: "bullmq | inline | off; unset picks by REDIS_URL and NODE_ENV." },
+    { name: "JOBS_WORKER", scope: "worker", description: "\"1\" on the ONE process that consumes the queue and runs schedules." },
+    { name: "JOBS_QUEUE_PREFIX", description: "Optional key prefix, to share one Redis across environments." },
+  ],
+} as const;