@pattern-stack/codegen 0.9.2 → 0.10.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (61) hide show
  1. package/CHANGELOG.md +73 -0
  2. package/README.md +5 -0
  3. package/consumer-skills/bridge/SKILL.md +265 -0
  4. package/consumer-skills/codegen/SKILL.md +115 -0
  5. package/consumer-skills/entities/SKILL.md +111 -0
  6. package/consumer-skills/entities/families-and-queries.md +82 -0
  7. package/consumer-skills/entities/yaml-reference.md +118 -0
  8. package/consumer-skills/events/SKILL.md +71 -0
  9. package/consumer-skills/events/authoring-events.md +164 -0
  10. package/consumer-skills/events/typed-bus-and-outbox.md +163 -0
  11. package/consumer-skills/jobs/SKILL.md +66 -0
  12. package/consumer-skills/jobs/handler-authoring.md +236 -0
  13. package/consumer-skills/jobs/pools-and-ordering.md +161 -0
  14. package/consumer-skills/subsystems/SKILL.md +161 -0
  15. package/consumer-skills/subsystems/wiring-and-order.md +120 -0
  16. package/consumer-skills/sync/SKILL.md +134 -0
  17. package/consumer-skills/sync/audit-and-detection.md +302 -0
  18. package/consumer-skills/sync/change-sources-and-sinks.md +442 -0
  19. package/dist/runtime/subsystems/bridge/bridge.module.d.ts +0 -1
  20. package/dist/runtime/subsystems/bridge/bridge.module.js +294 -710
  21. package/dist/runtime/subsystems/bridge/bridge.module.js.map +1 -1
  22. package/dist/runtime/subsystems/bridge/index.d.ts +0 -1
  23. package/dist/runtime/subsystems/bridge/index.js +248 -664
  24. package/dist/runtime/subsystems/bridge/index.js.map +1 -1
  25. package/dist/runtime/subsystems/events/event-bus.drizzle-backend.js +18 -10
  26. package/dist/runtime/subsystems/events/event-bus.drizzle-backend.js.map +1 -1
  27. package/dist/runtime/subsystems/events/events.module.js +43 -244
  28. package/dist/runtime/subsystems/events/events.module.js.map +1 -1
  29. package/dist/runtime/subsystems/events/index.d.ts +0 -1
  30. package/dist/runtime/subsystems/events/index.js +39 -241
  31. package/dist/runtime/subsystems/events/index.js.map +1 -1
  32. package/dist/runtime/subsystems/index.js +174 -791
  33. package/dist/runtime/subsystems/index.js.map +1 -1
  34. package/dist/runtime/subsystems/jobs/bullmq.config.d.ts +22 -3
  35. package/dist/runtime/subsystems/jobs/bullmq.config.js.map +1 -1
  36. package/dist/runtime/subsystems/jobs/index.d.ts +1 -4
  37. package/dist/runtime/subsystems/jobs/index.js +87 -506
  38. package/dist/runtime/subsystems/jobs/index.js.map +1 -1
  39. package/dist/runtime/subsystems/jobs/job-orchestrator.bullmq-backend.js.map +1 -1
  40. package/dist/runtime/subsystems/jobs/job-worker.bullmq-backend.js +3 -0
  41. package/dist/runtime/subsystems/jobs/job-worker.bullmq-backend.js.map +1 -1
  42. package/dist/runtime/subsystems/jobs/job-worker.module.d.ts +11 -4
  43. package/dist/runtime/subsystems/jobs/job-worker.module.js +248 -664
  44. package/dist/runtime/subsystems/jobs/job-worker.module.js.map +1 -1
  45. package/dist/runtime/subsystems/jobs/jobs-domain.module.d.ts +0 -1
  46. package/dist/runtime/subsystems/jobs/jobs-domain.module.js +89 -391
  47. package/dist/runtime/subsystems/jobs/jobs-domain.module.js.map +1 -1
  48. package/dist/src/cli/index.js +1065 -440
  49. package/dist/src/cli/index.js.map +1 -1
  50. package/dist/src/index.js +26 -4
  51. package/dist/src/index.js.map +1 -1
  52. package/package.json +2 -1
  53. package/runtime/subsystems/events/event-bus.drizzle-backend.ts +32 -10
  54. package/runtime/subsystems/events/events.module.ts +38 -6
  55. package/runtime/subsystems/events/index.ts +7 -1
  56. package/runtime/subsystems/jobs/bullmq.config.ts +23 -3
  57. package/runtime/subsystems/jobs/index.ts +13 -8
  58. package/runtime/subsystems/jobs/job-worker.bullmq-backend.ts +5 -2
  59. package/runtime/subsystems/jobs/job-worker.module.ts +27 -7
  60. package/runtime/subsystems/jobs/jobs-domain.module.ts +27 -2
  61. package/templates/subsystem/events/domain-events.schema.ejs.t +43 -2
@@ -0,0 +1,118 @@
1
+ <!-- managed by @pattern-stack/codegen — re-run `codegen skills install` to refresh. Edit the package source, not this vendored copy. -->
2
+
3
+ # Entity YAML reference
4
+
5
+ Every block of an `entities/<name>.yaml` file and what it generates.
6
+
7
+ ## `entity:` — identity
8
+
9
+ ```yaml
10
+ entity:
11
+ name: contact # REQUIRED. singular snake_case. Drives class names + table.
12
+ plural: contacts # optional; inferred (pluralized) if omitted
13
+ table: contacts # optional; defaults to the plural
14
+ pattern: Synced # the family — Base | Synced | Activity | Metadata | Knowledge
15
+ # (or an app-defined pattern). See families-and-queries.md.
16
+ ```
17
+
18
+ ## `fields:` — columns
19
+
20
+ Each key is a `snake_case` column; the generated TS property is `camelCase`.
21
+
22
+ ```yaml
23
+ fields:
24
+ email:
25
+ type: string # string | integer | decimal | boolean | uuid | date | datetime | json | enum
26
+ required: true # NOT NULL + required in Create DTO
27
+ max_length: 255 # string length constraint (DB + Zod)
28
+ index: true # single-column index
29
+ status:
30
+ type: enum
31
+ choices: [active, inactive, archived] # enum members
32
+ metadata:
33
+ type: json # jsonb column
34
+ score:
35
+ type: decimal
36
+ nullable: true
37
+ ```
38
+
39
+ Type notes:
40
+ - `uuid` — typically the PK and foreign keys.
41
+ - `enum` — requires `choices:`; generates a TS union + DB check/enum.
42
+ - `json` — `jsonb`; typed as `Record<string, unknown>` unless you narrow it in
43
+ your own code.
44
+ - `datetime` vs `date` — timestamp vs date-only column.
45
+
46
+ ## `behaviors:` — cross-cutting columns + logic
47
+
48
+ ```yaml
49
+ behaviors:
50
+ - timestamps # createdAt, updatedAt (auto-managed)
51
+ - soft_delete # deletedAt; queries auto-filter deleted rows; GET :id 404s on deleted
52
+ - user_tracking # createdBy, updatedBy
53
+ ```
54
+
55
+ Behaviors compose — list any subset.
56
+
57
+ ## `relationships:` — foreign keys + typed accessors
58
+
59
+ ```yaml
60
+ relationships:
61
+ account:
62
+ type: belongs_to # belongs_to | has_many | has_one
63
+ target: account # the referenced entity (must have its own YAML)
64
+ foreign_key: account_id
65
+ ```
66
+
67
+ - `belongs_to` adds the FK column on this entity.
68
+ - `has_many` / `has_one` are the inverse side (no column here; drives the typed
69
+ relation accessor + Drizzle `relations()`).
70
+ - Cross-entity targets must resolve at generation time — regenerate the set with
71
+ `codegen entity new --all`.
72
+
73
+ ## `generate:` — output toggles
74
+
75
+ ```yaml
76
+ generate:
77
+ writes: true # default true — emit POST/PATCH/DELETE + create/update/delete use cases
78
+ ```
79
+
80
+ Set `writes: false` for a read-only resource (only GET routes + read use cases).
81
+
82
+ ## EAV (custom fields)
83
+
84
+ Two independent flags:
85
+
86
+ ```yaml
87
+ # On a normal entity (e.g. opportunity): opt into custom fields
88
+ eav: true
89
+ ```
90
+
91
+ When `eav: true`:
92
+ - Service gains **paired reads**: `findById` (typed entity) and
93
+ `findByIdWithFields` (entity + merged `fields` bag); same for `list`.
94
+ - `Create*` / `Update*` use cases accept `{ ...core, fields?: Record<string,
95
+ unknown> }` and run a transactional dual-write.
96
+ - Controller adds `GET /:id/with-fields`, `GET /with-fields`, and accepts the
97
+ `fields` bag on POST/PATCH.
98
+
99
+ ```yaml
100
+ # On the value-table entity itself (e.g. field_value): mark it AS the EAV store
101
+ eav_value_table: true
102
+ eav_definition_table: field_definition # where keys → definition ids resolve
103
+ ```
104
+
105
+ When `eav_value_table: true`:
106
+ - Repository gets `upsertCurrentValues(rows, tx)` (composite conflict target).
107
+ - Service gets `upsertFieldsTransactional(...)` and `findMergedByEntity(...)`
108
+ with internal definition-id resolution.
109
+ - The module auto-imports the definition-table module so DI resolves without
110
+ consumer wiring.
111
+
112
+ The EAV dual-write is coordinated by the **use case** inside `db.transaction` —
113
+ services stay single-domain (see the layer rules in the `entities` L0 skill).
114
+
115
+ ## `queries:` — declarative finders
116
+
117
+ See `families-and-queries.md` for the full `queries:` block (column finders,
118
+ unique, ordered, filtered+paginated search).
@@ -0,0 +1,71 @@
1
+ ---
2
+ name: events
3
+ description: Load when authoring a domain event, publishing one from a use case, or subscribing to one in a project that ran `codegen subsystem install events`. Triggers include `events/*.yaml` files, the generated `TypedEventBus` facade, injecting `TYPED_EVENT_BUS` / `EVENT_BUS`, `publish(...)` inside a Drizzle transaction (the outbox), `subscribe(...)`, the `domain_events` table, and event directions / pools.
4
+ allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5
+ user-invocable: false
6
+ ---
7
+
8
+ <!-- managed by @pattern-stack/codegen — re-run `codegen skills install` to refresh. Edit the package source, not this vendored copy. -->
9
+
10
+ # Events
11
+
12
+ The events subsystem is the transactional event backbone vendored into your app by `codegen subsystem install events`. You declare each event type as a YAML file; codegen generates a typed `TypedEventBus` facade, a discriminated union of every event, Zod payload schemas, and a runtime registry. You publish events inside the same database transaction as your domain write (the outbox pattern); a background loop drains them and delivers to subscribers.
13
+
14
+ The vendored code lives under `<paths.subsystems>/events/` (default `src/shared/subsystems/events/`), imported as `@shared/subsystems/events`. The generated files live under `<paths.subsystems>/events/generated/` and are reproduced from `events/*.yaml` on every `codegen` run — do not hand-edit them.
15
+
16
+ ## Mental model
17
+
18
+ **An event is an immutable fact** — "contact was created", "Stripe webhook arrived", "opportunity moved to `won`". It has no lifecycle beyond being delivered. **A job is the stateful work** that reacts to a fact. If you are tempted to put `status`, `attempts`, `retry_count`, or `scope` on an event, stop — you want a job (see the `jobs` skill). The event is the trigger; the job is the work.
19
+
20
+ ### The outbox
21
+
22
+ Publishing an event is an `INSERT` into the `domain_events` table, performed **inside the same transaction as your domain write**. Either both commit or neither does — no phantom events, no domain drift if the process crashes between commit and publish. A separate polling loop drains pending rows and dispatches them to subscribers. This is strictly stronger than `await commit(); await publish()`.
23
+
24
+ The single most important rule: **always pass the transaction (`tx`) when publishing from a use case that also writes domain state.** Dropping it silently detaches the event from the transaction.
25
+
26
+ ### Directions and pools
27
+
28
+ Every domain event declares a **direction**, which determines the lane it drains through:
29
+
30
+ | direction | carries | example |
31
+ |---|---|---|
32
+ | `inbound` | external → us (webhooks, pub/sub, inbound email) | `stripe_payment_received` |
33
+ | `change` | internal domain mutations (drive projections/reactions) | `contact_created` |
34
+ | `outbound` | us → external (webhooks fired, sync pushes) | `webhook_outbound_contact_sync` |
35
+
36
+ Direction is a **routing** concern, not a payload concern — two events with identical payloads can have different directions. Each direction drains through its own reserved lane so a slow outbound handler can't stall internal change-event propagation.
37
+
38
+ There is also an `audit` **tier** (orthogonal to direction) for observational facts that should live in the outbox but must never spawn work — see `authoring-events.md`.
39
+
40
+ ### Typed facade vs. raw port
41
+
42
+ - `IEventBus` (token `EVENT_BUS`) is the narrow underlying port: `publish`, `publishMany`, `subscribe`. It knows nothing about your specific event types.
43
+ - `TypedEventBus` (token `TYPED_EVENT_BUS`) is the generated, injectable wrapper. Its `publish<T>()` enforces the typed payload for `T` and stamps `metadata.pool` / `metadata.direction` / `metadata.version` from the registry. **Use `TypedEventBus` in new code.** The raw port stays available for forwarders that publish types not in the registry.
44
+
45
+ ## Routing table
46
+
47
+ | For this task | Read |
48
+ |---|---|
49
+ | Declaring an `events/*.yaml`, payload types, directions, `tier: audit`, entity `emits:` | `authoring-events.md` |
50
+ | Publishing inside a transaction, the outbox, idempotency, subscribing, wiring `EventsModule` | `typed-bus-and-outbox.md` |
51
+
52
+ To run a durable background job *when an event fires*, that is the Event-to-Job Bridge — see the `bridge` skill. To do the work directly (not via an event), see the `jobs` skill.
53
+
54
+ ## Non-obvious rules
55
+
56
+ - **Always pass `tx` to `publish` inside a use-case transaction.** It is the entire outbox guarantee. There is no type-level enforcement of this yet — it is a discipline.
57
+ - **Events have no lifecycle; jobs do.** The `status` column on `domain_events` (`pending | processed | failed`) is delivery state for the drain loop, not a domain state machine.
58
+ - **Use `TypedEventBus.publish<'type'>(...)` once the type is generated.** Misspelled event names and wrong payload shapes become compile errors.
59
+ - **Subscribers must be fast.** A subscriber that makes an HTTP call blocks the drain batch (it dispatches ~50 rows serially). Heavy reactions belong in a job — subscribe, then enqueue.
60
+ - **The event `id` is the idempotency key.** For replays/backfills, derive the id deterministically from the source event (e.g. Stripe's `evt_...`) so a re-insert is a no-op rather than a duplicate.
61
+ - **Regenerate after editing YAML.** Re-run `codegen` (e.g. `codegen entity new --all`) after touching any `events/*.yaml` to refresh the generated facade, union, schemas, and registry.
62
+
63
+ ## Do not
64
+
65
+ - Do not put job-style fields on an event (`status: running`, `attempts`, `scope`, `parent_id`). Those belong on a job.
66
+ - Do not drop the `tx` argument in `publish(...)` inside a transaction.
67
+ - Do not collapse the three directions into one pool — lane isolation is the point.
68
+ - Do not couple two services with a direct method call when the second merely reacts to a state change in the first. Publish a `change` event from the first, subscribe (or bridge a job) from the second.
69
+ - Do not do heavy I/O directly in a subscriber. Enqueue a job instead.
70
+ - Do not hand-edit anything under `<paths.subsystems>/events/generated/`. It is regenerated from `events/*.yaml`.
71
+ - Do not route events through user pools (`batch`, `interactive`) — events only ever drain through the reserved `events_*` lanes.
@@ -0,0 +1,164 @@
1
+ <!-- managed by @pattern-stack/codegen — re-run `codegen skills install` to refresh. Edit the package source, not this vendored copy. -->
2
+
3
+ # Authoring Events
4
+
5
+ How to declare an event in YAML, what each field means, what gets generated, and when to reach for the `audit` tier. Read the `events` `SKILL.md` first for the outbox/direction mental model. For publishing and subscribing, see `typed-bus-and-outbox.md`.
6
+
7
+ ## One YAML file per event
8
+
9
+ Events live in `events/` at your repo root, sibling to `entities/`. One file per type; the filename matches the `type` field (snake_case).
10
+
11
+ ```yaml
12
+ # events/contact_created.yaml
13
+ type: contact_created
14
+ direction: change
15
+ aggregate: contact # must match an entity name
16
+ version: 1
17
+ payload:
18
+ contact_id: { type: uuid }
19
+ account_id: { type: uuid, nullable: true }
20
+ created_by: { type: uuid }
21
+ ```
22
+
23
+ ```yaml
24
+ # events/stripe_payment_received.yaml
25
+ type: stripe_payment_received
26
+ direction: inbound
27
+ source: stripe # inbound: logical origin
28
+ version: 1
29
+ description: Stripe charge.succeeded webhook, post-signature-verification.
30
+ payload:
31
+ event_id: { type: string, description: "Stripe event id (evt_...)" }
32
+ customer_id: { type: string }
33
+ amount_cents: { type: number }
34
+ currency: { type: string }
35
+ received_at: { type: date }
36
+ retry:
37
+ attempts: 5
38
+ backoff: exponential
39
+ ```
40
+
41
+ ```yaml
42
+ # events/webhook_outbound_contact_sync.yaml
43
+ type: webhook_outbound_contact_sync
44
+ direction: outbound
45
+ destination: crm # outbound: logical target
46
+ aggregate: contact
47
+ payload:
48
+ contact_id: { type: uuid }
49
+ operation: { type: string } # "create" | "update" | "delete"
50
+ occurred_at: { type: date }
51
+ ```
52
+
53
+ After editing any `events/*.yaml`, re-run codegen (e.g. `codegen entity new --all`) to regenerate the typed artifacts.
54
+
55
+ ## Field reference
56
+
57
+ | Field | Required | Notes |
58
+ |---|---|---|
59
+ | `type` | yes | Unique snake_case business key; matches the filename. Becomes the discriminator and the `publish<T>()` key. |
60
+ | `direction` | for `tier: domain` | `inbound \| change \| outbound`. Drives the default pool. **Omit for `tier: audit`.** |
61
+ | `tier` | no | `domain` (default) or `audit`. Audit events are outbox-only and never spawn jobs — see below. |
62
+ | `aggregate` | for `change` | The owning entity name. Required for `change` events; optional elsewhere. Lets you query/replay events for a given entity id. |
63
+ | `source` | inbound only | Logical origin ("stripe", "email"). |
64
+ | `destination` | outbound only | Logical target ("crm", "slack"). |
65
+ | `payload` | yes | Map of snake_case keys → field specs. Keys become camelCase TS props. |
66
+ | `pool` | no | Override the direction's default pool. Only valid within the same category (a `change` event can't opt into `events_inbound`). User pools are never valid. Rarely needed — if you reach for it, revisit the direction. **Must be omitted for `tier: audit`.** |
67
+ | `retry` | no | `{ attempts, backoff }`; hints surfaced to the bus. |
68
+ | `version` | no | Integer, defaults to 1. Stamped on every publish; multi-version coexistence is not exercised yet. |
69
+
70
+ ### Payload field types
71
+
72
+ `uuid | string | number | boolean | date | json | array`.
73
+
74
+ - `nullable: true` makes the TS prop `T | null`.
75
+ - `array` requires an `items:` scalar type (`items: uuid | string | number | boolean | date`) and emits `T[]` + `z.array(T)`.
76
+ - `json` means an arbitrary JSON object (`Record<string, unknown>`). **Do not use `json` for array-shaped data** — Zod will reject it at the publish boundary.
77
+
78
+ ### Default pool from direction (domain tier)
79
+
80
+ | direction | default pool |
81
+ |---|---|
82
+ | `inbound` | `events_inbound` |
83
+ | `change` | `events_change` |
84
+ | `outbound` | `events_outbound` |
85
+
86
+ ## Tier: domain vs. audit
87
+
88
+ `tier` classifies the *kind* of fact and controls whether a job can be bound to the event.
89
+
90
+ | tier | direction | pool | can a job trigger on it? |
91
+ |---|---|---|---|
92
+ | `domain` (default) | required | from direction | yes |
93
+ | `audit` | omitted | null | no — codegen rejects it |
94
+
95
+ Use `audit` for high-volume **observational** events that should exist in the outbox (so you can query/replay them) but must never spawn downstream work. The motivating case: a polling CRM sync emitting one "I scanned this row" event per record would otherwise queue thousands of inert bridge jobs per run.
96
+
97
+ ```yaml
98
+ # events/crm_sync_completed.yaml
99
+ type: crm_sync_completed
100
+ tier: audit # outbox-only; not bridge-eligible
101
+ aggregate: integration # still scoped for query-by-id
102
+ version: 1
103
+ description: A CRM sync run finished. Observational — no domain state changed.
104
+ payload:
105
+ integration_id: { type: uuid }
106
+ provider: { type: string }
107
+ counts: { type: json }
108
+ duration_ms: { type: number }
109
+ ```
110
+
111
+ For an `audit` event the generated registry entry has `pool: null` and `direction: null`; both columns land NULL. In-process subscribers (`subscribe(...)`) may still listen to audit events — only the bridge refuses to bind jobs to them.
112
+
113
+ **The discipline:** if a well-behaved consumer would write another row or enqueue work, it is a `domain` event; if it would only bump a counter or update a dashboard, it is `audit`.
114
+
115
+ Codegen hard-errors on misuse:
116
+ - `tier: audit` with a `pool` → error naming the event.
117
+ - `tier: audit` with a `direction` → error naming the event.
118
+ - A job whose `@JobHandler.triggers` points at an audit event → `AuditEventTriggerError`. Use a domain event or remove the trigger.
119
+
120
+ ## Entity `events:` block is sugar for change events
121
+
122
+ An entity's YAML may keep an `events:` block. At parse time each entry desugars into an equivalent `events/<name>.yaml` with `direction: change` and `aggregate: <entity>`. Both paths produce the same registry entry — inline blocks are convenience, not a second source of truth.
123
+
124
+ ## Entity `emits:` for typed auto-emission
125
+
126
+ An entity can declare which change events its generated use cases emit:
127
+
128
+ ```yaml
129
+ # entities/contact.yaml
130
+ emits:
131
+ - contact_created
132
+ - contact_updated
133
+ - contact_deleted
134
+ ```
135
+
136
+ Each entry must resolve to an `events/<type>.yaml` with `direction: change` and `aggregate: contact`, or codegen hard-errors. With `emits:` present, generated use cases call the typed facade inside the domain transaction:
137
+
138
+ ```ts
139
+ async execute(input: CreateContactInput): Promise<Contact> {
140
+ return this.db.transaction(async (tx) => {
141
+ const contact = await this.contacts.create(input, tx);
142
+ await this.events.publish('contact_created', contact.id, {
143
+ contactId: contact.id,
144
+ accountId: contact.accountId,
145
+ createdBy: input.actorId,
146
+ }, { tx });
147
+ return contact;
148
+ });
149
+ }
150
+ ```
151
+
152
+ ## What gets generated
153
+
154
+ Five files under `<paths.subsystems>/events/generated/`, all reproduced from your YAML — never hand-edited:
155
+
156
+ | File | Contents |
157
+ |---|---|
158
+ | `types.ts` | Per-event interfaces, the `AppDomainEvent` discriminated union, and helpers `EventTypeName`, `EventOfType<T>`, `PayloadOfType<T>`. |
159
+ | `schemas.ts` | A Zod schema per payload, plus a keyed map for boundary validation. |
160
+ | `registry.ts` | `eventRegistry` — the runtime metadata map (direction, pool, aggregate, version, retry) read by the facade, the bridge, and startup validation. |
161
+ | `bus.ts` | The `TypedEventBus` facade with typed `publish<T>()` / `subscribe<T>()`. |
162
+ | `index.ts` | Re-export surface. |
163
+
164
+ The typed payload flows end to end: declaring `payload.contact_id: { type: uuid }` makes `PayloadOfType<'contact_created'>` require `contactId: string`, and `publish('contact_created', id, { ... })` typechecks the object against it.
@@ -0,0 +1,163 @@
1
+ <!-- managed by @pattern-stack/codegen — re-run `codegen skills install` to refresh. Edit the package source, not this vendored copy. -->
2
+
3
+ # Publishing, the Outbox, and Subscribing
4
+
5
+ How to publish events transactionally, how the outbox drain works, how to subscribe, idempotency, and how to wire `EventsModule` into your app. Read the `events` `SKILL.md` first for the mental model and `authoring-events.md` for the YAML shape.
6
+
7
+ ## Publishing inside a transaction
8
+
9
+ Every use case that mutates domain state and emits an event **must pass the transaction** so the event row is part of the same atomic write. Inject the typed facade:
10
+
11
+ ```ts
12
+ import { Inject, Injectable } from '@nestjs/common';
13
+ import { TypedEventBus, TYPED_EVENT_BUS } from '@shared/subsystems/events';
14
+
15
+ @Injectable()
16
+ export class CreateContactUseCase {
17
+ constructor(
18
+ @Inject(TYPED_EVENT_BUS) private readonly events: TypedEventBus,
19
+ private readonly db: DrizzleClient,
20
+ private readonly contacts: ContactRepository,
21
+ ) {}
22
+
23
+ async execute(input: CreateContactInput): Promise<Contact> {
24
+ return this.db.transaction(async (tx) => {
25
+ const contact = await this.contacts.create(input, tx);
26
+ await this.events.publish(
27
+ 'contact_created',
28
+ contact.id,
29
+ {
30
+ contactId: contact.id,
31
+ accountId: contact.accountId,
32
+ createdBy: input.actorId,
33
+ },
34
+ { tx }, // ← the outbox guarantee
35
+ );
36
+ return contact;
37
+ });
38
+ }
39
+ }
40
+ ```
41
+
42
+ `TypedEventBus.publish(type, aggregateId, payload, opts?)`:
43
+ - Validates the payload against the generated Zod schema at the boundary (skippable with `CODEGEN_EVENT_VALIDATE=off`).
44
+ - Stamps `id`, `occurredAt`, and `metadata.pool` / `metadata.direction` / `metadata.version` from the registry.
45
+ - Inserts one `domain_events` row using `opts.tx` if provided, else the top-level connection.
46
+
47
+ **Dropping `tx` silently detaches the event.** The backend uses `tx ?? db`, so without `tx` the insert runs outside your domain transaction — the domain write can commit while the event insert fails independently. There is no compile-time guard; treat passing `tx` as non-negotiable.
48
+
49
+ ### Raw `EVENT_BUS` for untyped publishes
50
+
51
+ The raw port stays available for forwarders that publish types not in the registry (e.g. proxying an external source):
52
+
53
+ ```ts
54
+ import { EVENT_BUS, type IEventBus } from '@shared/subsystems/events';
55
+
56
+ @Inject(EVENT_BUS) private readonly bus: IEventBus;
57
+
58
+ await this.bus.publish(
59
+ { id, type, aggregateId, aggregateType, payload, occurredAt },
60
+ tx,
61
+ );
62
+ ```
63
+
64
+ New code should prefer `TypedEventBus`.
65
+
66
+ ## The outbox table and drain loop
67
+
68
+ `domain_events` is the outbox. Key columns: `id` (UUID, the idempotency key), `type`, `aggregate_id`, `aggregate_type`, `payload` (jsonb), `occurred_at`, `processed_at`, `status` (`pending | processed | failed`), `error`, `metadata`, and first-class `pool` / `direction` / `tier` columns (populated from `metadata` at insert so the drain can filter by lane without unpacking JSON). `tier` (`'domain'` | `'audit'`, default `'domain'`) is always emitted; the `domain_events_tier_routing_check` constraint enforces that `tier='audit'` rows have null `pool`/`direction`. `tenant_id` is the only conditional column — emitted only under `events.multi_tenant: true`.
69
+
70
+ The drain loop (Drizzle backend):
71
+ 1. Claims a batch (default 50) of `pending` rows with `FOR UPDATE SKIP LOCKED`, optionally filtered by pool, ordered by `occurred_at ASC`. `SKIP LOCKED` lets multiple worker processes drain concurrently without double-dispatching.
72
+ 2. For each row, invokes every registered handler for `event.type`.
73
+ 3. On success: `status='processed'`, `processed_at=now()` — in the same transaction that locked the row, so nothing can strand half-claimed.
74
+ 4. On failure: retries up to 3 attempts; if still failing, `status='failed'` with the error. No automatic retry after that — a permanently broken handler does not burn a worker forever.
75
+
76
+ Default poll interval is 1s, so there is a ~1s latency floor. There is no stale-event sweeper — it is unnecessary because the lock lifetime equals the dispatch transaction.
77
+
78
+ ## Subscribing
79
+
80
+ ```ts
81
+ import { TypedEventBus, TYPED_EVENT_BUS } from '@shared/subsystems/events';
82
+
83
+ @Inject(TYPED_EVENT_BUS) private readonly events: TypedEventBus;
84
+
85
+ this.events.subscribe('contact_created', async (event) => {
86
+ // event is narrowed to ContactCreatedEvent — event.payload.contactId is typed
87
+ await this.readModel.upsert(event.payload.contactId);
88
+ });
89
+ ```
90
+
91
+ `subscribe<T>()` returns an unsubscribe function. Subscriptions are **per-process** in the Drizzle backend — a subscriber registered in process A does not receive events drained in process B. In a multi-process deployment, each process runs its own subscribers.
92
+
93
+ **Keep subscribers fast.** The drain dispatches a batch serially; a subscriber doing HTTP or other slow I/O stalls the whole batch. For anything slow, durable, or that needs retry, do not work inline — enqueue a job. For durable async fanout from an event to a job, use the `bridge` skill; for fire-and-forget cheap reactions (metrics, cache busts), an in-process subscriber is fine.
94
+
95
+ ## Idempotency
96
+
97
+ The event `id` is the dedup token. Handlers can redeliver (a process can crash between dispatch and the `processed` update), so:
98
+
99
+ - **Make handlers idempotent** — safe to call twice with the same `event.id`. A common pattern is a small "seen ids" table the handler checks first.
100
+ - **For inbound webhooks, reuse the source system's id.** If Stripe sends `evt_123` twice, deriving the outbox `id` from `evt_123` means the second insert hits `ON CONFLICT (id) DO NOTHING` instead of creating a duplicate.
101
+ - **Never regenerate `id` on replay/backfill** — compute it deterministically from the source event.
102
+
103
+ ## Ordering
104
+
105
+ - Same aggregate, single worker: FIFO (`ORDER BY occurred_at ASC`). With multiple workers, same-aggregate ordering holds only probabilistically.
106
+ - Across aggregates: not strictly ordered — treat it as roughly wall-clock.
107
+
108
+ If a handler depends on strict cross-aggregate ordering, reshape it to tolerate independent timelines.
109
+
110
+ ## Wiring `EventsModule`
111
+
112
+ Register once in `AppModule`; it is `global: true`, so entity modules need not import it individually:
113
+
114
+ ```ts
115
+ import { EventsModule } from '@shared/subsystems/events';
116
+
117
+ @Module({
118
+ imports: [
119
+ DatabaseModule,
120
+ EventsModule.forRoot({ backend: 'drizzle' }),
121
+ // ...
122
+ ],
123
+ })
124
+ export class AppModule {}
125
+ ```
126
+
127
+ `forRoot` options:
128
+ - `backend: 'drizzle' | 'memory' | 'redis'` — match `events.backend` in your config. Tests override to `'memory'`.
129
+ - `multiTenant: true` — opt-in multi-tenancy (see below).
130
+ - `pools: ['events_change']` — restrict this process's drain loop to specific lanes. A common split is one process per direction so a slow outbound handler cannot stall change-event propagation. Undefined drains all lanes.
131
+
132
+ ### Backend notes
133
+
134
+ - **`drizzle`** (default) — Postgres outbox; transactional, crash-safe; ~1s polling floor.
135
+ - **`memory`** — synchronous, in-process, for tests. `publish` dispatches immediately (no `tx` semantics); exposes `publishedEvents[]`, `publishedEventsForPool()`, `publishedEventsForDirection()`, and `clear()` for assertions. Because dispatch is synchronous, tests need no timers.
136
+ - **`redis`** — present but not a scaffold default. **It does not participate in the Drizzle transaction** — passing `tx` is a silent no-op, so you lose the outbox guarantee. Only reach for it after measuring an actual throughput bottleneck (the Drizzle outbox tops out around ~1000 events/s).
137
+
138
+ ### Multi-tenancy
139
+
140
+ Set `events.multi_tenant: true` in config, re-run `codegen subsystem install events --force` to re-emit the schema with a `tenant_id` column (then cut a migration), and pass `multiTenant: true` to `EventsModule.forRoot(...)`:
141
+
142
+ ```ts
143
+ EventsModule.forRoot({ backend: 'drizzle', multiTenant: true });
144
+ ```
145
+
146
+ When on, `publish` throws `MissingTenantIdError` (naming the event type) if `opts.metadata.tenantId` is absent. Explicit `null` is permitted for tenant-less background events. Keep the config flag and the module option in agreement.
147
+
148
+ ## Testing with events
149
+
150
+ Swap the backend to memory and assert on what was published:
151
+
152
+ ```ts
153
+ Test.createTestingModule({
154
+ imports: [EventsModule.forRoot({ backend: 'memory' })],
155
+ });
156
+
157
+ // in a test:
158
+ expect(bus.publishedEvents).toContainEqual(
159
+ expect.objectContaining({ type: 'contact_created' }),
160
+ );
161
+ ```
162
+
163
+ Call `bus.clear()` in `beforeEach`. For Drizzle integration tests, `DrizzleEventBus.drainOnce()` runs exactly one drain cycle so you do not have to sleep past the poll interval.
@@ -0,0 +1,66 @@
1
+ ---
2
+ name: jobs
3
+ description: Load when authoring a `@JobHandler` class, kicking off a background job from a use case, or configuring job pools in a project that ran `codegen subsystem install jobs`. Triggers include `@JobHandler`, `JobContext`, `ctx.step` / `ctx.spawnChild`, injecting `JOB_ORCHESTRATOR` / `JOB_RUN_SERVICE`, registering `JobsDomainModule` / `JobWorkerModule`, and editing the `jobs:` block in `codegen.config.yaml`.
4
+ allowed-tools: Read, Write, Edit, Glob, Grep, Bash
5
+ user-invocable: false
6
+ ---
7
+
8
+ <!-- managed by @pattern-stack/codegen — re-run `codegen skills install` to refresh. Edit the package source, not this vendored copy. -->
9
+
10
+ # Jobs
11
+
12
+ The jobs subsystem is the durable background-work engine vendored into your app by `codegen subsystem install jobs`. It gives you a way to run work asynchronously, retry it, scope it to a domain entity, cancel it as a tree, and resume it after a crash without redoing finished steps. You author jobs as plain TypeScript classes decorated with `@JobHandler`; the runtime handles claiming, retry, memoization, and lifecycle.
13
+
14
+ The vendored code lives under `<paths.subsystems>/jobs/` (default `src/shared/subsystems/jobs/`) and is imported as `@shared/subsystems/jobs`. Do not hand-edit it — it is managed by the package.
15
+
16
+ ## Mental model
17
+
18
+ A job is **stateful work**. Each execution is a row in the `job_run` table — a durable state machine you can query, cancel, and replay. This is the sharp line versus events: an *event* is an immutable fact ("contact was created"); a *job* is the work that reacts to it. If you catch yourself wanting `status`, `attempts`, or a retry policy on something, you want a job. See the sibling `events` skill for that distinction.
19
+
20
+ Three tables back the system, one concept each:
21
+
22
+ | Table | Meaning |
23
+ |---|---|
24
+ | `job` | One row per registered handler type. Materialized from your `@JobHandler` metadata at app boot. |
25
+ | `job_run` | One row per execution. The durable state machine: queryable by scope, cancellable as a tree. |
26
+ | `job_step` | One row per checkpoint inside a run. Powers memoization (skip already-done work on retry). |
27
+
28
+ There is **one** claim mechanism: a worker polls `job_run` directly with `SELECT ... FOR UPDATE SKIP LOCKED`. There is no separate queue table and no separate "enqueue" port. You start work with `IJobOrchestrator.start(...)`; the worker discovers the row on its next poll.
29
+
30
+ Run hierarchy: a child run carries `parent_run_id` and `root_run_id`. Cancelling a run cascades to its tree via `root_run_id` according to each child's close policy.
31
+
32
+ ### State machine
33
+
34
+ ```
35
+ pending → running → { completed | failed | timed_out | canceled }
36
+ ```
37
+
38
+ "Scheduled for later" is not a separate state — it is `status='pending'` with `run_at` in the future; the claim query simply skips it until `run_at` passes.
39
+
40
+ ## Routing table
41
+
42
+ | For this task | Read |
43
+ |---|---|
44
+ | Writing a `@JobHandler`, using `JobContext`, `ctx.step`, spawning children, scope, plugging into a use case | `handler-authoring.md` |
45
+ | Choosing/configuring pools, concurrency, ordering guarantees, the `jobs:` config block, embedded vs. standalone workers | `pools-and-ordering.md` |
46
+
47
+ For running a job *in response to a domain event*, that is the Event-to-Job Bridge — see the `bridge` skill. You declare triggers on the same `@JobHandler` decorator; this skill covers everything else about the handler.
48
+
49
+ ## Non-obvious rules
50
+
51
+ - **Jobs are TypeScript classes, not YAML.** There is no jobs-as-YAML codegen. You write a `@JobHandler` class; the package ships the orchestration around it.
52
+ - **A `@JobHandler` class must also be a registered NestJS provider.** The decorator registers the class with the job registry for orchestration; it does NOT register it in Nest's DI container. Add it to the `providers` of its owning module, or the worker throws an unresolvable-provider error when it tries to run it.
53
+ - **`step_id` must be stable across replays.** Hardcode it (`'pull_emails'`) or derive it deterministically from input. Never `Date.now()`, never random. Memoization is keyed on `(job_run_id, step_id)`; an unstable id defeats it.
54
+ - **The default pool is `batch`.** `interactive` exists but must be opted into explicitly.
55
+ - **The `events_inbound` / `events_change` / `events_outbound` pools are reserved.** A `@JobHandler({ pool: 'events_*' })` throws `ReservedPoolViolationError` at app boot. Those lanes belong to the event/bridge machinery. To run a job when an event fires, use `@JobHandler.triggers` (the `bridge` skill), not a reserved pool.
56
+ - **Concurrency and dedupe are different things.** Dedupe collapses duplicate enqueues (returns the existing run id, no new row). Concurrency lets the new row exist but gates when it is claimed. Both are set once on the decorator, not per call site.
57
+ - **`ctx.waitFor` / `ctx.signal` / `ctx.sleep` do not exist.** If you need a delay, use `ctx.spawnChild(type, input, { runAt })`. If you need to pause for external input, split the work across parent + child runs.
58
+
59
+ ## Do not
60
+
61
+ - Do not look for an `IJobQueue` or a `job_queue` table — they do not exist. Start work with `IJobOrchestrator.start(type, input, opts)`.
62
+ - Do not target a reserved `events_*` pool from a `@JobHandler`. It fails at boot.
63
+ - Do not use `Date.now()` or randomness for a `step_id`.
64
+ - Do not wrap `ctx.spawnChild` inside a `ctx.step` — a child run is its own memoization root.
65
+ - Do not hand-edit anything under `<paths.subsystems>/jobs/`. It is vendored from the package.
66
+ - Do not reach for a job when the caller is waiting synchronously on the result — that is a use case, not a job. Jobs are durable because they are asynchronous.