@nestarc/inbox 0.0.0-stage → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +72 -0
- package/LICENSE +21 -0
- package/README.md +309 -2
- package/dist/contracts/errors.d.ts +34 -0
- package/dist/contracts/errors.js +73 -0
- package/dist/contracts/errors.js.map +1 -0
- package/dist/contracts/types.d.ts +174 -0
- package/dist/contracts/types.js +3 -0
- package/dist/contracts/types.js.map +1 -0
- package/dist/inbox-admin.service.d.ts +44 -0
- package/dist/inbox-admin.service.js +102 -0
- package/dist/inbox-admin.service.js.map +1 -0
- package/dist/inbox.module.d.ts +13 -0
- package/dist/inbox.module.js +59 -0
- package/dist/inbox.module.js.map +1 -0
- package/dist/inbox.service.d.ts +22 -0
- package/dist/inbox.service.js +74 -0
- package/dist/inbox.service.js.map +1 -0
- package/dist/index.d.ts +9 -0
- package/dist/index.js +28 -0
- package/dist/index.js.map +1 -0
- package/dist/integrations/tenancy.d.ts +25 -0
- package/dist/integrations/tenancy.js +56 -0
- package/dist/integrations/tenancy.js.map +1 -0
- package/dist/internal/handler-context.d.ts +3 -0
- package/dist/internal/handler-context.js +39 -0
- package/dist/internal/handler-context.js.map +1 -0
- package/dist/internal/options.d.ts +5 -0
- package/dist/internal/options.js +86 -0
- package/dist/internal/options.js.map +1 -0
- package/dist/internal/reliability-core/abort.d.ts +2 -0
- package/dist/internal/reliability-core/abort.js +21 -0
- package/dist/internal/reliability-core/abort.js.map +1 -0
- package/dist/internal/reliability-core/index.d.ts +3 -0
- package/dist/internal/reliability-core/index.js +14 -0
- package/dist/internal/reliability-core/index.js.map +1 -0
- package/dist/internal/reliability-core/json.d.ts +13 -0
- package/dist/internal/reliability-core/json.js +125 -0
- package/dist/internal/reliability-core/json.js.map +1 -0
- package/dist/internal/reliability-core/retry.d.ts +8 -0
- package/dist/internal/reliability-core/retry.js +26 -0
- package/dist/internal/reliability-core/retry.js.map +1 -0
- package/dist/internal/validation.d.ts +6 -0
- package/dist/internal/validation.js +123 -0
- package/dist/internal/validation.js.map +1 -0
- package/dist/storage/prisma/prisma-inbox.storage.d.ts +30 -0
- package/dist/storage/prisma/prisma-inbox.storage.js +369 -0
- package/dist/storage/prisma/prisma-inbox.storage.js.map +1 -0
- package/dist/testing/index.d.ts +40 -0
- package/dist/testing/index.js +120 -0
- package/dist/testing/index.js.map +1 -0
- package/dist/worker/inbox-worker.d.ts +31 -0
- package/dist/worker/inbox-worker.js +343 -0
- package/dist/worker/inbox-worker.js.map +1 -0
- package/docs/adr/0001-durable-inbox.md +40 -0
- package/docs/benchmarks/2026-10-27-local.md +31 -0
- package/docs/benchmarks/2026-10-28-preview1-local.md +87 -0
- package/docs/benchmarks/2026-10-28-preview1-run1.json +120 -0
- package/docs/benchmarks/2026-10-28-preview1-run2.json +120 -0
- package/docs/benchmarks/2026-10-28-preview1-run3.json +120 -0
- package/docs/benchmarks/2026-10-28-sustained-local.md +104 -0
- package/docs/benchmarks/2026-10-28-sustained-run1.json +2553 -0
- package/docs/benchmarks/2026-10-28-sustained-run2.json +2553 -0
- package/docs/broker-recovery-results.json +52 -0
- package/docs/broker-recovery.md +30 -0
- package/docs/db-operations.md +41 -0
- package/docs/implementation-roadmap.md +129 -0
- package/docs/operations/2026-10-28-database.json +34 -0
- package/docs/package-plan.md +250 -0
- package/docs/releasing.md +91 -0
- package/docs/testing.md +86 -0
- package/docs/usage.md +266 -0
- package/docs/validation.md +174 -0
- package/examples/README.md +124 -0
- package/examples/compose.yaml +26 -0
- package/examples/package.json +42 -0
- package/examples/prisma/schema.prisma +29 -0
- package/examples/prisma.config.ts +10 -0
- package/examples/scripts/prepare-db.cjs +45 -0
- package/examples/scripts/smoke.cjs +37 -0
- package/examples/scripts/with-brokers.cjs +249 -0
- package/examples/scripts/with-rabbitmq.cjs +131 -0
- package/examples/sql/application.sql +14 -0
- package/examples/sql/tenancy.sql +26 -0
- package/examples/src/broker-common.ts +79 -0
- package/examples/src/bullmq-ingress/main.ts +165 -0
- package/examples/src/common.ts +112 -0
- package/examples/src/idempotency-ingress/main.ts +211 -0
- package/examples/src/kafka-ingress/main.ts +209 -0
- package/examples/src/outbox-to-inbox/main.ts +198 -0
- package/examples/src/rabbitmq-ingress/main.ts +203 -0
- package/examples/src/sqs-ingress/main.ts +273 -0
- package/examples/src/tenancy/main.ts +210 -0
- package/examples/src/webhook-ingress/main.ts +172 -0
- package/examples/tsconfig.json +16 -0
- package/package.json +99 -4
- package/src/sql/create-inbox-table.sql +100 -0
- package/src/sql/migrate-v1-to-v2.sql +53 -0
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
# GitHub Release에서 npm 배포
|
|
2
|
+
|
|
3
|
+
`release.yml`은 GitHub Release의 **Publish release** 이벤트로 실행된다. `package.json` 버전이 `0.1.0`이면 태그는 `v0.1.0`, GitHub의 **Set as a pre-release**는 꺼져 있어야 한다. `-preview.1` 같은 prerelease 버전을 사용할 때만 이 옵션을 켠다. 태그만 push하거나 초안을 저장하면 배포하지 않는다.
|
|
4
|
+
|
|
5
|
+
인증 방식은 다른 nestarc 패키지와 동일한 **npm Trusted Publishing(GitHub OIDC)** 이다. `NPM_TOKEN`이나 `NODE_AUTH_TOKEN` secret을 추가하지 않는다. 배포 job만 `npm` GitHub Environment와 `id-token: write`를 사용하며, 빌드와 테스트 job은 저장소 읽기 권한만 가진다.
|
|
6
|
+
|
|
7
|
+
## 최초 연결
|
|
8
|
+
|
|
9
|
+
1. 이 구현과 lockfile을 `nestarc/inbox`의 `main`에 반영한다.
|
|
10
|
+
2. GitHub 저장소 **Settings → Environments → npm**을 준비한다. Deployment branches and tags는 Selected branches and tags로 설정하고 tag 패턴 `v*.*.*`를 허용한다. 팀 정책에 따라 reviewer를 추가할 수 있다.
|
|
11
|
+
3. npm에 `@nestarc/inbox`가 아직 없으면 권한 있는 관리자가 최초 버전을 등록한다. npm의 Trusted Publisher 설정은 이미 존재하는 패키지가 필요하다. 최초 등록에도 아래 검증한 tarball을 사용하고, `0.1.0`은 `--tag latest`, preview는 `--tag next`로 게시한다. 이 작업은 계정 로그인·2FA가 필요한 일회성 절차이며 워크플로우에 장기 토큰을 넣는 대체 경로는 없다.
|
|
12
|
+
4. npm 패키지 **Settings → Trusted Publisher → GitHub Actions**를 아래 값으로 연결한다. 현재 npm 설정에서 **Allow this trusted publisher to publish packages**를 활성화한다. stage만 허용된 설정으로는 이 워크플로우의 직접 publish가 성공하지 않는다.
|
|
13
|
+
|
|
14
|
+
| 항목 | 값 |
|
|
15
|
+
| -------------------- | ------------- |
|
|
16
|
+
| Organization or user | `nestarc` |
|
|
17
|
+
| Repository | `inbox` |
|
|
18
|
+
| Workflow filename | `release.yml` |
|
|
19
|
+
| Environment | `npm` |
|
|
20
|
+
|
|
21
|
+
최신 npm CLI에서 동일한 설정을 만드는 명령은 다음과 같다. 패키지 쓰기 권한과 2FA가 있는 관리자 계정에서 실행한다.
|
|
22
|
+
|
|
23
|
+
```sh
|
|
24
|
+
npm trust github @nestarc/inbox \
|
|
25
|
+
--file release.yml \
|
|
26
|
+
--repo nestarc/inbox \
|
|
27
|
+
--environment npm \
|
|
28
|
+
--allow-publish
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
공식 [Trusted Publishers 안내](https://docs.npmjs.com/trusted-publishers/)와 [npm trust 명령](https://docs.npmjs.com/cli/v11/commands/npm-trust/)을 참고한다. 새 연결은 npm이 정한 활성화 기한 안에 첫 게시를 성공시켜야 한다. 워크플로우는 GitHub hosted runner, Node 24, npm 11.5.1 이상을 사용하고 npm 버전이 부족하면 실패한다. 로컬 검증이나 dry-run으로 원격 OIDC 연결까지 확인할 수는 없다.
|
|
32
|
+
|
|
33
|
+
최초 수동 등록 이후에는 버전을 올려 첫 OIDC 배포를 확인한다. 이미 게시된 동일 버전은 무결성 확인 후 건너뛰므로 Trusted Publisher의 실제 인증을 검증하지 않는다.
|
|
34
|
+
|
|
35
|
+
## 릴리즈 절차
|
|
36
|
+
|
|
37
|
+
1. 버전과 변경 기록을 갱신하고 `package.json`과 `package-lock.json`을 함께 커밋한다. 해당 커밋을 `main`에 병합한다.
|
|
38
|
+
2. 필요하면 Actions의 **Release → Run workflow**로 검증한다. 이 경로는 전체 검증과 `npm publish --dry-run`까지만 실행하며 게시 권한을 요청하지 않는다.
|
|
39
|
+
3. `main`에 포함된 커밋에 `v<package.json 버전>` 태그로 GitHub Release를 게시한다. 버전에 `-preview.1`, `-rc.1` 같은 prerelease 식별자가 있으면 GitHub prerelease 체크도 켠다.
|
|
40
|
+
4. Release 실행에서 `publish` job 성공과 npm 버전·integrity 확인 결과를 확인한다.
|
|
41
|
+
|
|
42
|
+
| 버전 예 | GitHub prerelease | npm dist-tag |
|
|
43
|
+
| ----------------- | ----------------- | ------------ |
|
|
44
|
+
| `0.1.0-preview.1` | 켬 | `next` |
|
|
45
|
+
| `0.1.0-rc.1` | 켬 | `next` |
|
|
46
|
+
| `0.1.0` | 끔 | `latest` |
|
|
47
|
+
|
|
48
|
+
현재 `publishConfig.tag`는 `latest`이며, 배포 명령은 버전에 따라 `latest` 또는 `next`를 명시해 기본 설정을 덮어쓴다. 태그·버전·prerelease 표시가 불일치하거나 태그 커밋이 `main` 이력에 없으면 테스트와 게시 전에 거절한다. 빌드 메타데이터가 있는 버전(`1.0.0+build`)은 받지 않는다.
|
|
49
|
+
|
|
50
|
+
## 검증과 배포물
|
|
51
|
+
|
|
52
|
+
CI와 Release는 `release-validation.yml`을 공유한다. Node 22에서 한 번 빌드한 후 `package.tgz`와 `metadata.json`을 보관한다. metadata는 저장소·commit·ref·버전, 파일 목록, SHA-256과 npm SHA-512 integrity를 기록한다. SQL 설치/upgrade 파일, JavaScript 및 타입 진입점이 필요하며 환경 파일·의존성 디렉터리·생성 client·private workspace 패키지와 archive link는 거절한다.
|
|
53
|
+
|
|
54
|
+
Node 22/24 × PostgreSQL 16/18의 네 job이 같은 tarball을 내려받아 실제 DB 통합/복구 테스트와 독립 consumer 설치를 실행한다. Node 24/PostgreSQL 16에서는 tarball 내부 예제 8종을 실제 임시 브로커와 함께 검사한다. 각 consumer는 복사본을 읽기 전용으로 사용하며 실행 전후 원본·복사본·metadata 변경을 검사한다.
|
|
55
|
+
|
|
56
|
+
네 조합과 별도 Node 24/PostgreSQL 16 운영 복구 job이 모두 성공해야 배포 job이 실행된다. 운영 job은 같은 커밋을 빌드하여 TCP 단절·blackhole, 연결 풀 포화, SIGTERM 종료, 실제 standby 승격, 제한된 tablespace 공간 고갈, RabbitMQ 재시작·retry·DLQ를 확인한다. 지속 부하 benchmark는 별도 수동 측정이며 CI의 성능 합격 기준으로 사용하지 않는다. 배포 job에서는 의존성 설치·빌드·재패킹을 하지 않고 해당 tarball을 `--ignore-scripts --provenance`로 게시한다. 게시 후 레지스트리의 integrity가 검증한 tarball과 같은지 확인한다.
|
|
57
|
+
|
|
58
|
+
npm은 게시 요청을 받은 뒤 설치 가능 상태가 되기 전에 패키지를 검사한다. 공식 [게시 시점 검사 안내](https://github.blog/changelog/2026-07-28-npm-publish-time-malware-scanning-and-dual-use-metadata/)에 따르면 보통 약 5분, 혼잡하거나 추가 검토가 필요하면 15분 이상 걸릴 수 있다. 게시 성공 직후의 404만으로 실패나 미게시를 판단하지 않는다. 워크플로우는 30초 간격으로 진행 상황을 출력하며 최대 20분 동안 공개 여부와 integrity를 확인하고, 배포 job에는 총 30분을 허용한다. 내용 불일치와 404 이외의 조회 오류는 즉시 실패한다.
|
|
59
|
+
|
|
60
|
+
확인 시간이 초과되어도 같은 버전을 다시 게시하거나 전체 publish job을 바로 재실행하지 않는다. npm의 검사 상태·알림을 확인하고, 원래 검증한 `package.tgz`와 `metadata.json`을 보존한 채 확인 명령만 다시 실행한다. 검사에 더 오래 걸리거나 수동 검토가 필요한 경우 이 제한 안에 공개되지 않을 수 있다.
|
|
61
|
+
|
|
62
|
+
```sh
|
|
63
|
+
node scripts/release-artifact.mjs verify-published "$INBOX_RELEASE_TARBALL" "$INBOX_RELEASE_METADATA"
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
같은 버전이 이미 존재하면 bytes가 같은 경우만 게시를 건너뛴다. 다른 bytes이거나 레지스트리 조회가 404 이외의 오류이면 실패한다. 기존 버전을 덮어쓰지 말고 원인을 확인한 다음 새 버전을 사용한다. 같은 버전의 재실행은 기존 dist-tag를 이동시키지 않는다.
|
|
67
|
+
|
|
68
|
+
## 로컬 재현
|
|
69
|
+
|
|
70
|
+
Node 22.14 이상과 Docker가 필요하다. 임시 디렉터리는 저장소 밖에 둔다.
|
|
71
|
+
|
|
72
|
+
```sh
|
|
73
|
+
npm ci
|
|
74
|
+
npm run format:check
|
|
75
|
+
npm run test:release
|
|
76
|
+
npm run build
|
|
77
|
+
npm run typecheck
|
|
78
|
+
npm test
|
|
79
|
+
npm run test:integration
|
|
80
|
+
|
|
81
|
+
INBOX_CANDIDATE_DIR=$(mktemp -d)
|
|
82
|
+
node scripts/release-artifact.mjs pack "$INBOX_CANDIDATE_DIR"
|
|
83
|
+
export INBOX_RELEASE_TARBALL="$INBOX_CANDIDATE_DIR/package.tgz"
|
|
84
|
+
export INBOX_RELEASE_METADATA="$INBOX_CANDIDATE_DIR/metadata.json"
|
|
85
|
+
npm run test:package
|
|
86
|
+
npm run test:examples
|
|
87
|
+
node scripts/release-artifact.mjs verify "$INBOX_RELEASE_TARBALL" "$INBOX_RELEASE_METADATA"
|
|
88
|
+
npm publish "$INBOX_RELEASE_TARBALL" --ignore-scripts --access public --tag latest --dry-run
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
`INBOX_RELEASE_TARBALL`과 `INBOX_RELEASE_METADATA`는 둘 다 절대 경로로 지정한다. 두 변수를 생략한 일반 `test:package` / `test:examples` 실행은 기존처럼 로컬에서 빌드와 pack을 수행한다.
|
package/docs/testing.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# Testing application handlers
|
|
2
|
+
|
|
3
|
+
`@nestarc/inbox/testing` provides bounded helpers for integration tests using a real `InboxService` and the application's PostgreSQL test database. It has no additional runtime dependencies and works with any test runner. Apply the Inbox SQL schema through your test fixture before creating the service.
|
|
4
|
+
|
|
5
|
+
## Process specific receipts
|
|
6
|
+
|
|
7
|
+
Disable automatic polling to control when processing starts. Register your application's real handler and pass the admin view for its trusted tenant scope:
|
|
8
|
+
|
|
9
|
+
```ts
|
|
10
|
+
import { InboxService } from "@nestarc/inbox";
|
|
11
|
+
import { drainInbox } from "@nestarc/inbox/testing";
|
|
12
|
+
|
|
13
|
+
const inbox = new InboxService({
|
|
14
|
+
prisma,
|
|
15
|
+
handlers: [orderProjectionHandler],
|
|
16
|
+
worker: {
|
|
17
|
+
enabled: false,
|
|
18
|
+
retryBaseMs: 10,
|
|
19
|
+
retryMaxMs: 20,
|
|
20
|
+
},
|
|
21
|
+
});
|
|
22
|
+
|
|
23
|
+
try {
|
|
24
|
+
const receipt = await inbox.accept({
|
|
25
|
+
scope: { kind: "tenant", id: "tenant-a" },
|
|
26
|
+
source: "orders",
|
|
27
|
+
consumer: orderProjectionHandler.consumer,
|
|
28
|
+
messageId: "order-event-1",
|
|
29
|
+
eventType: orderProjectionHandler.eventType,
|
|
30
|
+
schemaVersion: orderProjectionHandler.schemaVersion,
|
|
31
|
+
payload: { orderId: "order-1" },
|
|
32
|
+
});
|
|
33
|
+
|
|
34
|
+
const [completed] = await drainInbox(
|
|
35
|
+
inbox,
|
|
36
|
+
inbox.admin.forTenant("tenant-a"),
|
|
37
|
+
[receipt.id],
|
|
38
|
+
{ timeoutMs: 5_000 },
|
|
39
|
+
);
|
|
40
|
+
|
|
41
|
+
// completed.status is COMPLETED. Assert your actual business-table result too.
|
|
42
|
+
// A repeat delivery should return the same receipt and leave that result unchanged.
|
|
43
|
+
} finally {
|
|
44
|
+
await inbox.close();
|
|
45
|
+
// Disconnect the application's Prisma client after the worker has stopped.
|
|
46
|
+
}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
`drainInbox(inbox, admin, receiptIds, options)` checks each requested receipt and calls the real `runOnce()` between state checks until every receipt is `COMPLETED`. A zero-claim poll does not finish the helper: the message may be waiting for a retry or lease expiry. Retries use the worker's real schedule and do not advance a fake database clock. Configure short retry delays explicitly in tests when appropriate.
|
|
50
|
+
|
|
51
|
+
Receipt IDs select the completion assertions, **not the worker's claims**. `runOnce()` may process other receipts for its registered consumers, including other tenants. The scoped admin view restricts reads; it does not change worker ownership. Use an isolated database or dedicated test consumers, and avoid other workers when testing deterministic ordering. Receipt IDs are deduplicated, results preserve their first occurrence order, and an empty list returns immediately. A call accepts at most 1,000 IDs.
|
|
52
|
+
|
|
53
|
+
## Observe an existing worker
|
|
54
|
+
|
|
55
|
+
`waitForInboxRecord()` only reads a receipt; it never starts processing, requeues a failure or changes the database. Use it with automatic polling or a separately managed worker:
|
|
56
|
+
|
|
57
|
+
```ts
|
|
58
|
+
import { waitForInboxRecord } from "@nestarc/inbox/testing";
|
|
59
|
+
|
|
60
|
+
const completed = await waitForInboxRecord(admin, receipt.id);
|
|
61
|
+
|
|
62
|
+
// For a test deliberately exhausting retries:
|
|
63
|
+
const dead = await waitForInboxRecord(admin, failedReceipt.id, {
|
|
64
|
+
status: "DEAD",
|
|
65
|
+
timeoutMs: 10_000,
|
|
66
|
+
});
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The expected status is `COMPLETED` by default, or `DEAD` when specified. Observing the other terminal state fails immediately. A missing or out-of-scope record remains pending until the timeout or pass limit; this helper cannot distinguish those two cases.
|
|
70
|
+
|
|
71
|
+
## Limits and failures
|
|
72
|
+
|
|
73
|
+
Both helpers accept these options:
|
|
74
|
+
|
|
75
|
+
| Option | Default | Meaning |
|
|
76
|
+
| ---------------- | ------- | --------------------------------------------------------------------------- |
|
|
77
|
+
| `timeoutMs` | `5000` | Overall budget including database reads and worker calls |
|
|
78
|
+
| `pollIntervalMs` | `10` | Delay between state-check passes |
|
|
79
|
+
| `maxPasses` | `1000` | Maximum state-check passes; no new worker call starts after the final check |
|
|
80
|
+
| `signal` | None | An `AbortSignal` that stops waiting and starting new polls |
|
|
81
|
+
|
|
82
|
+
Numeric options must be positive 32-bit integers. A stalled `get()` or `runOnce()` is also covered by the overall timeout. Timeout or cancellation stops the helper's wait; an already running database operation or handler may still finish. Always close the service before disconnecting Prisma or tearing down the fixture. These helpers do not create or reset databases and do not replace the runtime's transaction timeouts.
|
|
83
|
+
|
|
84
|
+
`InboxTestError` exposes `code` and a `records` snapshot containing only receipt IDs, last observed states and attempt counts. Codes are `TIMEOUT`, `ABORTED`, `PASS_LIMIT`, `UNEXPECTED_TERMINAL_STATE` and `INVALID_OPTIONS`. The helper's diagnostics omit payloads, tenant values and raw handler errors. Underlying database errors propagate unchanged so a broken database cannot be mistaken for an unfinished receipt.
|
|
85
|
+
|
|
86
|
+
Use real timers for PostgreSQL integration tests: advancing a test runner's fake timers does not advance PostgreSQL retry or lease timestamps. A successful wait proves that the requested receipts reached the expected state; assert application rows and outbox results separately to verify business effects.
|
package/docs/usage.md
ADDED
|
@@ -0,0 +1,266 @@
|
|
|
1
|
+
# @nestarc/inbox usage
|
|
2
|
+
|
|
3
|
+
This document describes the 0.2.0 API. Inbox durably receives messages and processes them with Prisma and PostgreSQL. The application owns authentication, broker acknowledgments, its Prisma client, and migration deployment. APIs and schema may change before 1.0; review the [changelog](../CHANGELOG.md) before upgrading. For application integration tests, see the [bounded testing helpers](testing.md).
|
|
4
|
+
|
|
5
|
+
Install with `npm install @nestarc/inbox@0.2.0`. To evaluate a source build, run `npm ci` and `npm pack` in the repository, then install the resulting `nestarc-inbox-0.2.0.tgz` in your application.
|
|
6
|
+
|
|
7
|
+
## Initialize the schema and Prisma client
|
|
8
|
+
|
|
9
|
+
Apply the SQL exported as `@nestarc/inbox/sql/create-inbox-table.sql` through your application's migration process before starting Inbox. Runtime initialization checks the schema; it does not create or upgrade tables. Keep the inbox SQL tables in your migration history and do not let Prisma schema synchronization remove unmanaged tables.
|
|
10
|
+
|
|
11
|
+
Versions 0.1.0 and 0.2.0 require schema version 2; upgrading from 0.1.0 needs no SQL migration. If you used schema v1 in a local development build, stop those processes, apply the exported `@nestarc/inbox/sql/migrate-v1-to-v2.sql`, then start 0.2.0 workers. The transactional migration preserves receipt identity, payloads, state, attempts, and tokens and may be applied again safely. It takes a table lock, so schedule the upgrade appropriately. Fresh installs use the current creation SQL. Old workers must not be restarted against v2. Local Preview.1 builds already use schema v2; the 0.1.0 promotion adds no schema change.
|
|
12
|
+
|
|
13
|
+
Prisma 7 requires a driver adapter. Construct one application-owned client and pass it to Inbox:
|
|
14
|
+
|
|
15
|
+
```ts
|
|
16
|
+
import { PrismaPg } from "@prisma/adapter-pg";
|
|
17
|
+
import { PrismaClient } from "./generated/prisma/client";
|
|
18
|
+
|
|
19
|
+
const prisma = new PrismaClient({
|
|
20
|
+
adapter: new PrismaPg({ connectionString: process.env.DATABASE_URL }),
|
|
21
|
+
});
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Use the API for your installed Prisma major. The [runnable examples](../examples/README.md) pin Prisma 7 and support the npm package or a local tarball; automated package checks use a tarball built from the checkout.
|
|
25
|
+
|
|
26
|
+
## Register a consumer
|
|
27
|
+
|
|
28
|
+
Register stable logical consumer names and explicitly enable the worker:
|
|
29
|
+
|
|
30
|
+
```ts
|
|
31
|
+
import { Module } from "@nestjs/common";
|
|
32
|
+
import { InboxModule, InboxPermanentError } from "@nestarc/inbox";
|
|
33
|
+
import type { Prisma } from "./generated/prisma/client";
|
|
34
|
+
import type { InboxHandler } from "@nestarc/inbox";
|
|
35
|
+
|
|
36
|
+
const handler: InboxHandler<{ orderId: string }, Prisma.TransactionClient> = {
|
|
37
|
+
consumer: "orders.projection.v1",
|
|
38
|
+
eventType: "order.created",
|
|
39
|
+
schemaVersion: 1,
|
|
40
|
+
validate(payload) {
|
|
41
|
+
return (
|
|
42
|
+
!!payload &&
|
|
43
|
+
typeof payload === "object" &&
|
|
44
|
+
!Array.isArray(payload) &&
|
|
45
|
+
typeof payload.orderId === "string"
|
|
46
|
+
);
|
|
47
|
+
},
|
|
48
|
+
async handle(payload, context) {
|
|
49
|
+
if (context.scope.kind !== "tenant") throw new InboxPermanentError();
|
|
50
|
+
await context.tx.orderProjection.create({
|
|
51
|
+
data: { tenantId: context.scope.id, orderId: payload.orderId },
|
|
52
|
+
});
|
|
53
|
+
},
|
|
54
|
+
};
|
|
55
|
+
|
|
56
|
+
@Module({
|
|
57
|
+
imports: [
|
|
58
|
+
InboxModule.forRoot({
|
|
59
|
+
prisma,
|
|
60
|
+
handlers: [handler],
|
|
61
|
+
worker: { enabled: true },
|
|
62
|
+
}),
|
|
63
|
+
],
|
|
64
|
+
})
|
|
65
|
+
export class AppModule {}
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The application defines its Prisma models and supplies `prisma` above. For dependency injection, use `InboxModule.forRootAsync({ imports, inject, useFactory })`; the factory returns the same options. Create handlers inside the factory when they depend on other Nest services.
|
|
69
|
+
|
|
70
|
+
Worker processing is disabled unless `worker.enabled` is true. An ingress-only process supplies `consumers: ['orders.projection.v1']` and no handlers. A worker process registers handlers and may restrict processing through `worker.consumers`. Claiming, lease recovery, and exhausted-attempt handling stay inside that list. A missing handler for a configured worker consumer or a duplicate handler registration fails startup.
|
|
71
|
+
|
|
72
|
+
Removing every worker for a consumer leaves its backlog pending. Monitor backlog age before removing a deployment. Unsupported event types or schema versions for a registered consumer are retained as DEAD; they are never silently acknowledged as completed business work.
|
|
73
|
+
|
|
74
|
+
## Worker configuration
|
|
75
|
+
|
|
76
|
+
| Option | Default | Meaning |
|
|
77
|
+
| ----------------------------- | ------------------------ | ----------------------------------------------------------------------- |
|
|
78
|
+
| `worker.enabled` | `false` | Start polling during application initialization |
|
|
79
|
+
| `worker.consumers` | Registered handler names | The worker's explicit ownership list |
|
|
80
|
+
| `worker.concurrency` | `4` | Maximum claimed, active records; upper bound 100 |
|
|
81
|
+
| `worker.pollIntervalMs` | `1000` | Delay between polling opportunities |
|
|
82
|
+
| `worker.leaseMs` | `30000` | Database lease duration |
|
|
83
|
+
| `worker.heartbeatIntervalMs` | `5000` | Renewal interval; must be less than half the lease duration |
|
|
84
|
+
| `worker.transactionTimeoutMs` | `10000` | Maximum handler transaction duration |
|
|
85
|
+
| `worker.transactionMaxWaitMs` | `5000` | Maximum wait to acquire a Prisma transaction connection |
|
|
86
|
+
| `worker.lockTimeoutMs` | `1000` | Database lock-wait bound for the handler transaction |
|
|
87
|
+
| `worker.shutdownTimeoutMs` | `15000` | Bounded drain interval on shutdown |
|
|
88
|
+
| `worker.maxAttempts` | `8` | Total claims per retry cycle, including the initial attempt and crashes |
|
|
89
|
+
| `worker.retryBaseMs` | `1000` | Base for exponential retry delay |
|
|
90
|
+
| `worker.retryMaxMs` | `300000` | Maximum retry delay before jitter |
|
|
91
|
+
| `maxPayloadBytes` | `262144` | Accepted JSON payload size in UTF-8 bytes |
|
|
92
|
+
| `maxPendingMessages` | Unset | Optional approximate backlog admission bound |
|
|
93
|
+
|
|
94
|
+
Keep enough pool connections available for admission, heartbeat, and state changes in addition to handler transactions. Larger concurrency does not automatically improve throughput. Leave external network work out of the transaction and tune limits with measurements from your own database.
|
|
95
|
+
|
|
96
|
+
For explicit execution without a continuously polling process, `InboxService.initialize()` checks the schema, `runOnce()` processes a bounded batch, and `close()` stops the worker. A Nest module invokes initialization and shutdown automatically. `close()` does not disconnect the application's Prisma client.
|
|
97
|
+
|
|
98
|
+
## Accept a message and acknowledge it
|
|
99
|
+
|
|
100
|
+
```ts
|
|
101
|
+
const receipt = await inbox.accept({
|
|
102
|
+
scope: { kind: "tenant", id: verifiedTenantId },
|
|
103
|
+
source: "orders-service",
|
|
104
|
+
consumer: "orders.projection.v1",
|
|
105
|
+
messageId: event.id,
|
|
106
|
+
eventType: "order.created",
|
|
107
|
+
schemaVersion: 1,
|
|
108
|
+
payload: { orderId: event.orderId },
|
|
109
|
+
});
|
|
110
|
+
channel.ack(message);
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`accept()` resolves only after its own transaction commits. It returns `{ kind: 'accepted' | 'duplicate', id, status }`. Both kinds permit transport acknowledgment. A duplicate can refer to a PENDING, PROCESSING, RETRYABLE, COMPLETED, or DEAD record; it never resets processing state or retry budgets. DEAD records require operator intervention.
|
|
114
|
+
|
|
115
|
+
A storage exception or uncertain acceptance commit result must not produce a success ACK. Let the transport redeliver the same stable identity. Authentication failures, invalid input, and identity conflicts require an explicit rejection or transport dead-letter policy rather than a hot retry loop.
|
|
116
|
+
|
|
117
|
+
Inbox does not acknowledge messages, return HTTP responses, or connect to brokers itself. [Runnable RabbitMQ and webhook examples](../examples/README.md) show those application boundaries.
|
|
118
|
+
|
|
119
|
+
## Identity and payload
|
|
120
|
+
|
|
121
|
+
The unique identity combines scope, source, consumer, and message ID. Scope is either `{ kind: 'tenant', id }` or `{ kind: 'global' }`. Tenant IDs are application-authenticated values; scope selection alone does not implement PostgreSQL RLS.
|
|
122
|
+
|
|
123
|
+
Use a producer's stable event ID. Do not use a delivery tag, attempt count, or a new random ID for every redelivery. Renaming a consumer creates a different logical effect and permits processing again.
|
|
124
|
+
|
|
125
|
+
Fingerprinting covers event type, schema version, and canonical JSON payload. Object key order does not change identity content; array order does. Trace headers, correlation ID, and causation ID are metadata and do not participate. Keep values that affect business decisions inside the payload.
|
|
126
|
+
|
|
127
|
+
The same identity with different fingerprint throws `InboxConflictError` with code `IDENTITY_CONFLICT` and leaves the existing record unchanged. JSON must contain only supported JSON values. Non-finite numbers, undefined, arbitrary class instances, circular values, excessive depth, and values exceeding configured limits are rejected. No value is silently truncated to fit.
|
|
128
|
+
|
|
129
|
+
## Handler transaction and retries
|
|
130
|
+
|
|
131
|
+
Use only `context.tx` for handler writes. Inbox adds the fenced completion write and commits the whole transaction. Let transaction errors escape. Calling another Prisma client, nesting a separate transaction, or making an external API call breaks this atomic boundary.
|
|
132
|
+
|
|
133
|
+
An outbox event can join the same transaction:
|
|
134
|
+
|
|
135
|
+
```ts
|
|
136
|
+
await outbox.emit(context.tx, new OrderProjectedEvent(payload.orderId), {
|
|
137
|
+
tenantId: context.scope.kind === "tenant" ? context.scope.id : undefined,
|
|
138
|
+
...(context.scope.kind === "global"
|
|
139
|
+
? { tenantScope: "global" as const }
|
|
140
|
+
: {}),
|
|
141
|
+
causationId: context.messageId,
|
|
142
|
+
});
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Handler invocations can repeat. Successfully committed same-database effects are deduplicated while the completion receipt remains. Different event IDs for the same business action still require an application unique constraint. Ordering is not guaranteed.
|
|
146
|
+
|
|
147
|
+
Claims consume an attempt before handler execution, so crashes consume retry budget too. A normal error schedules a durable retry using capped exponential backoff and full jitter. `InboxPermanentError`, invalid handler payloads, unsupported schemas, and exhausted attempts lead to DEAD. Error details are redacted before persistence and observation.
|
|
148
|
+
|
|
149
|
+
Heartbeats extend only an owned, unexpired lease. Stale completion rolls back handler writes. `context.signal` is an advisory cancellation signal; handlers should stop cooperative work when it aborts. Database and callback timeouts bound the transaction even when a handler ignores that signal. No cancellation mechanism can undo an external API call.
|
|
150
|
+
|
|
151
|
+
When a completion commit response is lost, Inbox reads the current record before deciding on a failure transition. If that read is unavailable, it leaves recovery to the durable lease rather than overwriting a potentially completed record.
|
|
152
|
+
|
|
153
|
+
## Configuration
|
|
154
|
+
|
|
155
|
+
The default settings favor short database work. All durations are milliseconds.
|
|
156
|
+
|
|
157
|
+
| Option | Default |
|
|
158
|
+
| ------------------------------------------------------ | ----------------------------------- |
|
|
159
|
+
| `worker.enabled` | `false` |
|
|
160
|
+
| `worker.concurrency` / `pollIntervalMs` | `4` / `1000` |
|
|
161
|
+
| `worker.leaseMs` / `heartbeatIntervalMs` | `30000` / `5000` |
|
|
162
|
+
| `worker.transactionTimeoutMs` / `transactionMaxWaitMs` | `10000` / `5000` |
|
|
163
|
+
| `worker.lockTimeoutMs` / `shutdownTimeoutMs` | `1000` / `15000` |
|
|
164
|
+
| `worker.maxAttempts` | `8`, including the initial attempt |
|
|
165
|
+
| `worker.retryBaseMs` / `retryMaxMs` | `1000` / `300000`, with full jitter |
|
|
166
|
+
| `maxPayloadBytes` | `262144` (256 KiB) |
|
|
167
|
+
| `maxPendingMessages` | No configured admission limit |
|
|
168
|
+
|
|
169
|
+
Handler deadlines bound heartbeats. Failure reconciliation has a separate deadline of at most five seconds so an unavailable status query cannot hold a worker slot forever. Configure the application's Prisma driver pool with enough connections for business transactions and control queries; Inbox does not create a second pool.
|
|
170
|
+
|
|
171
|
+
## Administration
|
|
172
|
+
|
|
173
|
+
Bind the tenant scope before reading or changing records:
|
|
174
|
+
|
|
175
|
+
```ts
|
|
176
|
+
const tenant = admin.forTenant(authenticatedTenantId);
|
|
177
|
+
const page = await tenant.list({ status: "DEAD", limit: 50 });
|
|
178
|
+
const item = await tenant.get(recordId);
|
|
179
|
+
if (item?.status === "DEAD") {
|
|
180
|
+
await tenant.requeue({
|
|
181
|
+
id: item.id,
|
|
182
|
+
expectedVersion: item.rowVersion,
|
|
183
|
+
reason: "Handler corrected and deployed",
|
|
184
|
+
actor: operatorId,
|
|
185
|
+
});
|
|
186
|
+
}
|
|
187
|
+
const stats = await tenant.stats();
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The application authorizes the operator and derives the tenant scope. `forGlobal()` manages intentionally global records, not all tenants. `InboxPrivilegedAdminService` is an explicitly enabled, cross-tenant operator surface; configure `privilegedAdmin: true` only in an authorized control-plane application and never expose it directly as a tenant-facing controller. For `forRootAsync`, that opt-in belongs at the top level beside `imports`, `inject`, and `useFactory`, not in the returned factory options.
|
|
191
|
+
|
|
192
|
+
Requeue only accepts DEAD with the observed version. Concurrent or repeated stale operations fail with `InboxStateConflictError`. It keeps the original payload, identity, claim token, lifetime attempts, and requeue history counters, and resets the current retry-cycle budget. COMPLETED receipts cannot be reset through requeue; use the separate explicit replay API below.
|
|
193
|
+
|
|
194
|
+
### Explicit replay of a completed message
|
|
195
|
+
|
|
196
|
+
Replay deliberately permits another logical processing operation. It creates a new PENDING receipt and leaves the original completion and deduplication identity unchanged. Choose a handler whose business semantics allow reprocessing, for example a projection rebuild consumer. Business uniqueness and external side effects still need their own contracts.
|
|
197
|
+
|
|
198
|
+
```ts
|
|
199
|
+
const original = await admin.forTenant(authenticatedTenantId).get(originalId);
|
|
200
|
+
if (original?.status === "COMPLETED") {
|
|
201
|
+
const replay = await admin.forTenant(authenticatedTenantId).replay({
|
|
202
|
+
id: original.id,
|
|
203
|
+
expectedVersion: original.rowVersion,
|
|
204
|
+
requestId: persistedOperatorRequestUuid,
|
|
205
|
+
targetConsumer: "orders.projection.rebuild.v1",
|
|
206
|
+
actor: operatorId,
|
|
207
|
+
reason: "Rebuild the projection after deploying the corrected handler",
|
|
208
|
+
});
|
|
209
|
+
// replay.kind is accepted or duplicate; replay.record is the new receipt.
|
|
210
|
+
}
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Generate and persist one UUID per intended operator request, then reuse it unchanged after timeouts. The UUID is unique within the original tenant/global scope. Reusing it with different arguments throws `InboxReplayConflictError`. Retrying the same request returns its existing receipt even after payload cleanup. A new request needs a retained COMPLETED payload and the observed version; otherwise it fails. Payload mutation is not supported. The destination consumer must have a deployed matching handler; replay does not create one. `record.replay` and `context.replay` expose original inbox ID, source version, request ID, actor, reason, and request time. Payload cleanup preserves this lineage.
|
|
214
|
+
|
|
215
|
+
### Tenant context in the handler transaction
|
|
216
|
+
|
|
217
|
+
Install the optional `@nestarc/tenancy` peer and inject its context and service:
|
|
218
|
+
|
|
219
|
+
```ts
|
|
220
|
+
InboxModule.forRootAsync({
|
|
221
|
+
imports: [TenancyModule],
|
|
222
|
+
inject: [PrismaService, TenancyService],
|
|
223
|
+
useFactory: (prisma, tenancy) => ({
|
|
224
|
+
prisma,
|
|
225
|
+
handlers,
|
|
226
|
+
handlerContext: createInboxTenancyContext({
|
|
227
|
+
context: new TenancyContext(),
|
|
228
|
+
tenancy,
|
|
229
|
+
}),
|
|
230
|
+
worker: { enabled: true },
|
|
231
|
+
}),
|
|
232
|
+
});
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Use the application's raw Prisma client, not the tenancy extension that opens transactions of its own. The adapter binds async tenant context and sets the service's canonical PostgreSQL tenant setting with transaction-local `set_config` before invoking the handler on the same `context.tx`. The setting clears on commit or rollback. The application authenticates the scope, installs policies using that setting, and runs with a non-owner, non-superuser, non-BYPASSRLS database role. Inbox control tables are worker-owned; tenant isolation policies apply to application data. Scope selection alone does not authorize access.
|
|
236
|
+
|
|
237
|
+
`TenancyModule` 0.16.2 exports `TenancyService`, not the `TenancyContext` provider. Its public `new TenancyContext()` shares the package's async-local storage with the injected service. Import the configured tenancy module and the application's Prisma provider module into the factory module; the runnable example shows the full wiring.
|
|
238
|
+
|
|
239
|
+
Global messages default to a permanent rejection. `global: "shared"` explicitly permits shared-table work with an empty tenant setting; it does not bypass RLS or expose tenant rows. Generic `handlerContext(context, next)` wrappers must invoke and await `next()` exactly once; skipping it, invoking it twice, or swallowing a handler failure cannot produce a successful completion. Canceled work cannot start the handler after delayed context setup.
|
|
240
|
+
|
|
241
|
+
The tested tenancy version is 0.16.2, whose Node engine range is stricter than Inbox's: use Node 22.23.4+ on the 22 branch or 24.22.1+ on the 24 branch for that optional integration.
|
|
242
|
+
|
|
243
|
+
List results contain `records` and `nextCursor`. Pass the cursor back with the same filters to continue. Pagination order does not imply event execution order. Statistics count retained records and report the oldest pending work, so large histories have a query cost.
|
|
244
|
+
|
|
245
|
+
## Retention and observability
|
|
246
|
+
|
|
247
|
+
Completed payload cleanup is explicit:
|
|
248
|
+
|
|
249
|
+
```ts
|
|
250
|
+
await admin.forTenant(authenticatedTenantId).purgeCompletedPayload({
|
|
251
|
+
before: new Date("2026-01-01T00:00:00Z"),
|
|
252
|
+
limit: 100,
|
|
253
|
+
});
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
This removes eligible COMPLETED payloads while preserving identity and fingerprint. It does not delete PENDING, PROCESSING, RETRYABLE, or DEAD payloads. Retained receipts keep old redeliveries deduplicated. Manually deleting a receipt ends that guarantee; restoring an older database backup also restores an older deduplication history.
|
|
257
|
+
|
|
258
|
+
`observer(event)` receives bounded, redacted observations for accepted, duplicate, completed, retry, dead, lease-lost, and infrastructure-error outcomes. Observers are best-effort and must not be used as a durable audit trail. Avoid high-cardinality metrics labels and monitor backlog age, DEAD count, pool pressure, and processing delay. Unknown or inactive consumers can otherwise accumulate pending work without a running owner.
|
|
259
|
+
|
|
260
|
+
The optional `maxPendingMessages` admission check is approximate under concurrency; it is not a strict quota. Duplicate acceptance remains valid when that limit is reached. Database exhaustion or admission rejection must remain a failed acceptance, never a success ACK.
|
|
261
|
+
|
|
262
|
+
## Shutdown and compatibility
|
|
263
|
+
|
|
264
|
+
Enable Nest shutdown hooks for operating-system signals, and close the Nest application before disconnecting its Prisma pool. Inbox stops new claims, drains active work within its shutdown limit, and aborts timed-out work so leases can recover. The process manager's grace period should accommodate application cleanup.
|
|
265
|
+
|
|
266
|
+
Support declarations must match the executed compatibility suite. See repository validation results for tested Node, NestJS, Prisma, and PostgreSQL versions; the planning matrix is not itself evidence of support. Release migrations, actual SQL assets, and packed-consumer tests are part of that contract.
|