@drarzter/kafka-client 0.8.0 → 0.9.3

package/README.md

@@ -961,6 +961,8 @@ Options for `sendMessage()` — the third argument:
 | `batch` | `false` | (decorator only) Use `startBatchConsumer` instead of `startConsumer` |
 | `partitionAssigner` | `'cooperative-sticky'` | Partition assignment strategy: `'cooperative-sticky'` (minimal movement on rebalance, best for horizontal scaling), `'roundrobin'` (even distribution), `'range'` (contiguous partition ranges) |
 | `onTtlExpired` | — | Per-consumer override of the client-level `onTtlExpired` callback; takes precedence when set. Receives `TtlExpiredContext` — same shape as the client-level hook |
+| `onMessageLost` | — | Per-consumer override of the client-level `onMessageLost` callback; takes precedence when set. Use for consumer-specific dead-message alerting or structured logging |
+| `onRetry` | — | Per-consumer retry callback; fires **in addition to** the built-in metrics hook (does not replace it). Same signature as `KafkaInstrumentation.onRetry` |
 | `subscribeRetry.retries` | `5` | Max attempts for `consumer.subscribe()` when topic doesn't exist yet |
 | `subscribeRetry.backoffMs` | `5000` | Delay between subscribe retry attempts (ms) |
 
@@ -1422,6 +1424,7 @@ Options:
 | `targetTopic` | `x-dlq-original-topic` header | Override the destination topic |
 | `dryRun` | `false` | Count messages without sending |
 | `filter` | — | `(headers) => boolean` — skip messages where the callback returns `false` |
+| `fromBeginning` | `true` | `true` = full replay every call; `false` = incremental (only new messages since the last call) |
 
 ```typescript
 // Dry run — see how many messages would be replayed
@@ -1434,9 +1437,12 @@ const result = await kafka.replayDlq('orders', { targetTopic: 'orders.v2' });
 const filtered = await kafka.replayDlq('orders', {
   filter: (headers) => headers['x-correlation-id'] === 'corr-123',
 });
+
+// Incremental replay — only messages added since the last call
+const incremental = await kafka.replayDlq('orders', { fromBeginning: false });
 ```
 
-`replayDlq` creates a temporary consumer group that reads the DLQ topic up to the high-watermark at the time of the call — messages published after replay starts are not included. DLQ metadata headers (`x-dlq-original-topic`, `x-dlq-error-message`, `x-dlq-error-stack`, `x-dlq-failed-at`, `x-dlq-attempt-count`) are stripped from the replayed messages; all other headers (e.g. `x-correlation-id`) are preserved.
+`replayDlq` reads the DLQ topic up to the high-watermark at the time of the call — messages published after replay starts are not included. By default (`fromBeginning: true`) an ephemeral group ID is used on each call so all messages are always replayed; the group is deleted from the broker after use. With `fromBeginning: false` a stable group ID persists committed offsets between calls, enabling incremental replay. DLQ metadata headers (`x-dlq-original-topic`, `x-dlq-error-message`, `x-dlq-error-stack`, `x-dlq-failed-at`, `x-dlq-attempt-count`) are stripped from the replayed messages; all other headers (e.g. `x-correlation-id`) are preserved.
 
 ## Read snapshot
 
@@ -1761,19 +1767,25 @@ await kafka.disconnect(10_000); // wait up to 10 s, then force disconnect
 
 ## Consumer handles
 
-`startConsumer()` and `startBatchConsumer()` return a `ConsumerHandle` instead of `void`. Use it to stop a specific consumer without needing to remember the group ID:
+`startConsumer()` and `startBatchConsumer()` return a `ConsumerHandle` instead of `void`. Use it to stop a specific consumer or wait for it to be ready:
 
 ```typescript
-const handle = await kafka.startConsumer(['orders'], handler);
+const handle = await kafka.startConsumer(['orders'], handler, { fromBeginning: false });
 
 console.log(handle.groupId); // e.g. "my-group"
 
+// Wait until Kafka has assigned at least one partition to this consumer.
+// Safe to call before sending test messages — eliminates fixed setTimeout delays.
+await handle.ready();
+
 // Later — stop only this consumer, producer stays connected
 await handle.stop();
 ```
 
 `handle.stop()` is equivalent to `kafka.stopConsumer(handle.groupId)`. Useful in lifecycle methods or when you need to conditionally stop one consumer while others keep running.
 
+`handle.ready()` resolves once the broker fires the first partition-assignment event. For `fromBeginning: false` consumers it adds a 500 ms settle window so librdkafka can complete the async `latest` offset fetch before you send; for `fromBeginning: true` consumers it resolves immediately on assignment. In unit tests with `FakeTransport`, `subscribe()` fires the assignment synchronously, so `handle.ready()` resolves in the same tick.
+
 ## onMessageLost
 
 By default, if a consumer handler throws and `dlq` is not enabled, the message is logged and dropped. Use `onMessageLost` to catch these silent losses:
@@ -2117,6 +2129,31 @@ The integration suite spins up a single-node KRaft Kafka container and tests sen
 
 Both suites run in CI on every push to `main` and on pull requests.
 
+## File naming conventions
+
+Hyphens within a multi-word name; dot separates the name from its role suffix.
+
+| Suffix | Role |
+| --- | --- |
+| `.types` | Data/option shapes — configs, options, result objects |
+| `.interface` | Contract interfaces — capability boundaries, polymorphism points |
+| `.manager` | Stateful manager class |
+| `.tracker` | Tracking/counting utility |
+| `.module` | NestJS module |
+| `.explorer` | NestJS metadata scanner |
+| `.decorator` | NestJS decorator definitions |
+| `.health` | Health indicator |
+| `.constants` | Constant values and DI tokens |
+| `.mock` | Jest/Vitest spy double |
+| `.fake` | Standalone fake implementation |
+| `.container` | Test infrastructure wrapper |
+| `.spec` | Unit test |
+| `.integration.spec` | Integration test |
+
+A suffix is added only when it carries information the name alone does not. When the filename already expresses the role — `handler.ts` is a handler, `pipeline.ts` is a pipeline, `queue.ts` is a queue — no suffix is needed.
+
+---
+
 ## Project structure
 
 ```text
@@ -2176,11 +2213,11 @@ src/
 │   │   │                            #   backpressure via queueHighWaterMark (pause/resume)
 │   │   │
 │   │   └── infra/
-│   │       ├── circuit-breaker.ts   # CircuitBreakerManager — per groupId:topic:partition state
+│   │       ├── circuit-breaker.manager.ts  # CircuitBreakerManager — per groupId:topic:partition state
 │   │       │                        #   machine (CLOSED → OPEN → HALF_OPEN); sliding failure window
-│   │       ├── metrics-manager.ts   # MetricsManager — in-process counters (processed / retry /
+│   │       ├── metrics.manager.ts          # MetricsManager — in-process counters (processed / retry /
 │   │       │                        #   dlq / dedup) per topic; getMetrics() / resetMetrics()
-│   │       └── inflight-tracker.ts  # InFlightTracker — tracks running handlers for graceful
+│   │       └── inflight.tracker.ts         # InFlightTracker — tracks running handlers for graceful
 │   │                                #   shutdown drain (disconnect waits for all to settle)
 │   │
 │   └── __tests__/                   # Unit tests — mocked @confluentinc/kafka-javascript
@@ -2240,13 +2277,13 @@ src/
 │
 ├── testing/                         # Testing utilities — no runtime Kafka deps
 │   ├── index.ts                     # Re-exports createMockKafkaClient, KafkaTestContainer
-│   ├── mock-client.ts               # createMockKafkaClient<T>() — jest.fn() on every
+│   ├── client.mock.ts               # createMockKafkaClient<T>() — jest.fn() on every
 │   │                                #   IKafkaClient method with sensible defaults
-│   ├── test-container.ts            # KafkaTestContainer — thin @testcontainers/kafka wrapper;
+│   ├── test.container.ts            # KafkaTestContainer — thin @testcontainers/kafka wrapper;
 │   │                                #   transaction coordinator warmup, topic pre-creation
 │   └── __tests__/
-│       ├── mock-client.spec.ts      # Mock client method stubs and overrides
-│       └── test-container.spec.ts   # Container start/stop lifecycle
+│       ├── client.mock.spec.ts      # Mock client method stubs and overrides
+│       └── test.container.spec.ts   # Container start/stop lifecycle
 │
 ├── integration/                     # Integration tests — require Docker (testcontainers)
 │   ├── global-setup.ts              # Start shared Kafka container before all suites