sentinel-kafka-manager 1.0.0 → 1.0.1

package/README.md

@@ -2,56 +2,168 @@
 
 Reusable Kafka manager for Node.js microservices built on top of `kafkajs`.
 
-This package helps services:
+This package gives you one shared way to:
 
-- connect to Kafka with a small shared API
-- create and reuse producers, consumers, and admin clients safely
+- connect to Kafka
+- reuse a single producer safely
+- reuse consumers by `groupId`
+- provision topics if they do not exist
 - publish JSON messages
-- provision topics if they do not already exist
-- bootstrap from environment variables for both Docker-internal and host-external Kafka access
+- build broker lists from environment variables
+- attach logging and metrics hooks
+- publish standard event envelopes
+- run consumers with DLQ support
+- inspect connection health
+
+## What This Package Solves
+
+In most services, Kafka setup gets repeated:
+
+- build broker arrays
+- create producers
+- create consumers
+- manage connect and disconnect
+- handle Docker-internal vs host-external broker addresses
+
+`sentinel-kafka-manager` wraps those common tasks in one small class: `KafkaManager`.
 
 ## Installation
 
+Install from npm:
+
 ```bash
 npm install sentinel-kafka-manager
 ```
 
-For local development in this package:
+For local package development:
 
 ```bash
 npm install
 npm run build
 ```
 
-## Features
+## Requirements
+
+- Node.js service
+- Kafka cluster reachable from the service
+- `kafkajs` is included automatically as a dependency of this package
+
+## Enterprise Features
+
+This version now includes a stronger production baseline:
+
+- config validation for client id, brokers, and broker discovery inputs
+- shared producer, admin, and consumer lifecycle
+- logger and metrics hooks
+- standard message envelope publishing
+- managed consumer runner with dead-letter topic support
+- health snapshot reporting
+- safer startup behavior for concurrent connections
+- stable error codes and structured exception details
+
+## Recommended Usage Level
 
-- built on `kafkajs`
-- supports direct broker lists or env-driven broker discovery
-- supports internal Docker-network broker routing
-- supports external localhost or host-based broker routing
-- supports optional SSL, SASL, retry, and timeout settings
-- avoids duplicate producer, consumer, and admin connections during concurrent startup
+This package is now suitable as a shared internal Kafka module for SaaS microservices.
 
-## Exports
+Recommended use:
+
+- internal platform package for Node.js services
+- event publishing with standard envelopes
+- managed consumers with DLQ handling
+- consistent service-layer success and error responses
+
+Still recommended outside the package:
+
+- automated unit and integration tests in the consuming service or platform repo
+- schema validation for business payloads when required by your organization
+- organization-specific tracing, alerting, and compliance rules
+
+## Basic Usage
+
+### 1. Import the package
 
 ```ts
 import { KafkaManager } from 'sentinel-kafka-manager'
 ```
 
-## Quick Start
-
-### 1. Create a manager with explicit brokers
+If you want to catch package-specific errors explicitly:
 
 ```ts
-import { KafkaManager } from 'sentinel-kafka-manager'
+import {
+  KafkaErrorCode,
+  KafkaManager,
+  KafkaManagerError,
+  OperationResponse,
+} from 'sentinel-kafka-manager'
+```
+
+### 2. Create a manager
+
+You can create it in two ways:
 
+1. pass brokers directly
+2. build brokers from environment variables
+
+### Direct broker example
+
+```ts
 const kafkaManager = new KafkaManager({
   clientId: 'claims-service',
   brokers: ['localhost:29092', 'localhost:39092', 'localhost:49092'],
 })
 ```
 
-### 2. Publish a message
+### Environment-based example
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+})
+```
+
+## Recommended Project Pattern
+
+In a real service, create one shared manager instance and reuse it.
+
+Example:
+
+```ts
+import { KafkaManager } from 'sentinel-kafka-manager'
+
+export const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+})
+```
+
+Then import that shared instance anywhere you need Kafka access.
+
+You can also attach enterprise hooks:
+
+```ts
+import { KafkaManager } from 'sentinel-kafka-manager'
+
+export const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+  logger: {
+    debug: (message, meta) => console.debug(message, meta),
+    info: (message, meta) => console.info(message, meta),
+    warn: (message, meta) => console.warn(message, meta),
+    error: (message, meta) => console.error(message, meta),
+  },
+  metrics: {
+    emit: (event) => {
+      console.log('METRIC', event)
+    },
+  },
+})
+```
+
+## How To Use In Your Service
+
+### Publish a message
 
 ```ts
 await kafkaManager.publish({
@@ -60,11 +172,206 @@ await kafkaManager.publish({
   payload: {
     claimId: 'claim-1001',
     status: 'created',
+    source: 'claims-service',
   },
 })
 ```
 
-### 3. Provision topics
+`publish()` automatically:
+
+- gets a shared producer
+- connects it once
+- serializes `payload` with `JSON.stringify()`
+
+### Publish an enterprise event envelope
+
+Use this when you want traceable event metadata such as `eventType`, `version`, `traceId`, or `tenantId`.
+
+```ts
+await kafkaManager.publishEnvelope({
+  topic: 'claims.created',
+  key: 'claim-1001',
+  envelope: {
+    eventId: 'evt-claim-1001',
+    eventType: 'claims.created',
+    version: '1.0.0',
+    timestamp: new Date().toISOString(),
+    source: 'claims-service',
+    traceId: 'trace-123',
+    tenantId: 'tenant-abc',
+    correlationId: 'corr-001',
+    payload: {
+      claimId: 'claim-1001',
+      status: 'created',
+    },
+  },
+})
+```
+
+### Create or reuse a producer manually
+
+If you need direct producer access:
+
+```ts
+const producer = await kafkaManager.getProducer()
+
+await producer.send({
+  topic: 'claims.created',
+  messages: [
+    {
+      key: 'claim-1001',
+      value: JSON.stringify({ claimId: 'claim-1001' }),
+    },
+  ],
+})
+```
+
+### Create or reuse a consumer
+
+```ts
+const consumer = await kafkaManager.getConsumer('claims-service-group')
+
+await consumer.subscribe({
+  topic: 'claims.created',
+  fromBeginning: false,
+})
+
+await consumer.run({
+  eachMessage: async ({ topic, partition, message }) => {
+    const rawValue = message.value?.toString() ?? '{}'
+    const payload = JSON.parse(rawValue)
+
+    console.log({
+      topic,
+      partition,
+      key: message.key?.toString(),
+      payload,
+    })
+  },
+})
+```
+
+`getConsumer(groupId)` returns one shared connected consumer per group id.
+
+### Run a managed consumer with DLQ support
+
+For SaaS-style services, this is the recommended consumer pattern.
+
+```ts
+await kafkaManager.runConsumer({
+  groupId: 'claims-service-group',
+  topic: 'claims.created',
+  dlqTopic: 'claims.created.dlq',
+  onMessage: async ({ message, headers, key }) => {
+    console.log('Processing:', {
+      key,
+      traceId: headers['x-trace-id'],
+      payload: message,
+    })
+  },
+  onError: async (error, context) => {
+    console.error('Consumer error:', {
+      error: error.message,
+      topic: context.topic,
+      offset: context.offset,
+    })
+  },
+})
+```
+
+If processing fails:
+
+- the error is logged
+- a metric hook can receive the failure event
+- the message can be published to a dead-letter topic when `dlqTopic` is set
+
+### Catch structured errors
+
+All package-thrown operational errors now use `KafkaManagerError`.
+
+```ts
+import {
+  KafkaErrorCode,
+  KafkaManagerError,
+} from 'sentinel-kafka-manager'
+
+try {
+  await kafkaManager.publish({
+    topic: 'claims.created',
+    payload: { claimId: 'claim-1001' },
+  })
+} catch (error) {
+  if (error instanceof KafkaManagerError) {
+    console.error('Kafka error', {
+      code: error.code,
+      message: error.message,
+      details: error.details,
+    })
+
+    if (error.code === KafkaErrorCode.MESSAGE_PUBLISH_FAILED) {
+      // retry, alert, or degrade gracefully
+    }
+  }
+
+  throw error
+}
+```
+
+### Use common success and error response types
+
+If your service wraps Kafka operations in API or service-layer responses, you can use the shared response contracts from this package.
+
+```ts
+import {
+  ErrorResponse,
+  OperationResponse,
+  SuccessResponse,
+} from 'sentinel-kafka-manager'
+
+type PublishClaimCreatedResponse = OperationResponse<{
+  topic: string
+  key: string
+}>
+```
+
+Success example:
+
+```ts
+const response: SuccessResponse<{ topic: string; key: string }> = {
+  success: true,
+  message: 'Kafka message published successfully.',
+  data: {
+    topic: 'claims.created',
+    key: 'claim-1001',
+  },
+  meta: {
+    timestamp: new Date().toISOString(),
+    traceId: 'trace-123',
+  },
+}
+```
+
+Error example:
+
+```ts
+const response: ErrorResponse = {
+  success: false,
+  message: 'Failed to publish Kafka message.',
+  error: {
+    code: KafkaErrorCode.MESSAGE_PUBLISH_FAILED,
+    details: {
+      topic: 'claims.created',
+      key: 'claim-1001',
+    },
+  },
+  meta: {
+    timestamp: new Date().toISOString(),
+    retryable: true,
+  },
+}
+```
+
+### Provision topics
 
 ```ts
 await kafkaManager.provisionTopics([
@@ -73,54 +380,74 @@ await kafkaManager.provisionTopics([
     numPartitions: 6,
     replicationFactor: 3,
   },
+  {
+    topic: 'claims.updated',
+    numPartitions: 6,
+    replicationFactor: 3,
+  },
 ])
 ```
 
-### 4. Reuse a connected consumer
+This is useful during service startup when your topics should exist before producers or consumers begin work.
+
+### Disconnect on shutdown
+
+Always close Kafka clients during application shutdown.
 
 ```ts
-const consumer = await kafkaManager.getConsumer('claims-service-group')
+process.on('SIGINT', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
+})
 
-await consumer.subscribe({
-  topic: 'claims.created',
-  fromBeginning: false,
+process.on('SIGTERM', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
 })
 ```
 
-### 5. Disconnect on shutdown
+### Inspect health
 
 ```ts
-await kafkaManager.disconnect()
+const health = kafkaManager.getHealthSnapshot()
+
+console.log(health)
 ```
 
-## Environment-Based Setup
+## Environment Setup
 
-`KafkaManager.fromEnv()` supports two styles:
+This package supports two environment styles:
 
-1. explicit broker list with `KAFKA_BROKERS`
-2. derived broker list from your Kafka runtime env naming
+1. one explicit broker list with `KAFKA_BROKERS`
+2. derived brokers from your existing Kafka runtime variables
 
-### Option A: Explicit broker list
+## Option 1: Use `KAFKA_BROKERS`
+
+This is the simplest option.
+
+### `.env`
 
 ```env
 KAFKA_BROKERS=localhost:29092,localhost:39092,localhost:49092
 ```
 
-```ts
-import { KafkaManager } from 'sentinel-kafka-manager'
+### Code
 
+```ts
 const kafkaManager = KafkaManager.fromEnv({
   clientId: 'claims-service',
 })
 ```
 
-### Option B: Derived from your current Kafka cluster env
+## Option 2: Use your current Kafka cluster variables
 
-This package supports the env structure you shared for your KRaft cluster.
+This matches the Kafka server setup you shared.
 
-#### External mode
+### For host machine apps
 
-Use this when your app runs on the host machine or outside the Kafka Docker network.
+Use this when your Node.js service runs on the host machine.
+
+### `.env`
 
 ```env
 KAFKA_EXTERNAL_HOST=localhost
@@ -130,6 +457,8 @@ KAFKA_BROKER_3_EXTERNAL_PORT=49092
 KAFKA_BROKER_INTERNAL_PORT=9092
 ```
 
+### Code
+
 ```ts
 const kafkaManager = KafkaManager.fromEnv({
   clientId: 'claims-service',
@@ -145,14 +474,18 @@ localhost:39092
 localhost:49092
 ```
 
-#### Internal mode
+### For Dockerized apps on the Kafka network
+
+Use this when your Node.js service runs inside Docker on the same Kafka network.
 
-Use this when your app runs inside Docker on the same Kafka network.
+### `.env`
 
 ```env
 KAFKA_BROKER_INTERNAL_PORT=9092
 ```
 
+### Code
+
 ```ts
 const kafkaManager = KafkaManager.fromEnv({
   clientId: 'claims-service',
@@ -168,97 +501,144 @@ broker-2:9092
 broker-3:9092
 ```
 
-## Supported Environment Variables
+## Which Mode Should You Use?
+
+- Use `mode: 'external'` when your service uses `localhost` broker ports like `29092`, `39092`, and `49092`
+- Use `mode: 'internal'` when your service is inside Docker and should reach brokers like `broker-1:9092`
+- Use `KAFKA_BROKERS` when you want the simplest and most explicit configuration
 
-### Minimal variables
+## Your Current Running Kafka Cluster
 
-- `KAFKA_BROKERS`
-- `KAFKA_EXTERNAL_HOST`
-- `KAFKA_BROKER_1_EXTERNAL_PORT`
-- `KAFKA_BROKER_2_EXTERNAL_PORT`
-- `KAFKA_BROKER_3_EXTERNAL_PORT`
-- `KAFKA_BROKER_INTERNAL_PORT`
+Based on your running Docker containers, your Kafka cluster is exposed like this:
 
-### `fromEnv()` options
+- `broker-1` -> host port `29092`
+- `broker-2` -> host port `39092`
+- `broker-3` -> host port `49092`
+- `controller-1`, `controller-2`, and `controller-3` are controller-only nodes and should not be used by application clients
 
-You can customize the env lookup behavior:
+If your Node.js service runs on the host machine, use:
+
+```env
+KAFKA_EXTERNAL_HOST=localhost
+KAFKA_BROKER_1_EXTERNAL_PORT=29092
+KAFKA_BROKER_2_EXTERNAL_PORT=39092
+KAFKA_BROKER_3_EXTERNAL_PORT=49092
+KAFKA_BROKER_INTERNAL_PORT=9092
+```
 
 ```ts
 const kafkaManager = KafkaManager.fromEnv({
   clientId: 'claims-service',
   mode: 'external',
-  brokerCount: 3,
-  envKey: 'KAFKA_BROKERS',
-  externalHostEnvKey: 'KAFKA_EXTERNAL_HOST',
-  internalPortEnvKey: 'KAFKA_BROKER_INTERNAL_PORT',
-  externalPortEnvKeyPrefix: 'KAFKA_BROKER_',
-  externalPortEnvKeySuffix: '_EXTERNAL_PORT',
-  internalBrokerHostPrefix: 'broker-',
 })
 ```
 
-## Advanced Connection Options
-
-### SSL
+If your Node.js service runs inside Docker on the same Kafka network, use:
 
 ```ts
-const kafkaManager = new KafkaManager({
+const kafkaManager = KafkaManager.fromEnv({
   clientId: 'claims-service',
-  brokers: ['localhost:29092'],
-  ssl: {
-    rejectUnauthorized: true,
-  },
+  mode: 'internal',
 })
 ```
 
-### SASL
+This package is designed to work with that exact broker layout.
+
+## Full Service Example
 
 ```ts
-const kafkaManager = new KafkaManager({
-  clientId: 'claims-service',
-  brokers: ['localhost:29092'],
-  sasl: {
-    mechanism: 'plain',
-    username: 'kafka-user',
-    password: 'secret',
-  },
+import { KafkaManager } from 'sentinel-kafka-manager'
+
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-api',
+  mode: 'external',
+})
+
+async function bootstrap(): Promise<void> {
+  await kafkaManager.provisionTopics([
+    {
+      topic: 'claims.created',
+      numPartitions: 6,
+      replicationFactor: 3,
+    },
+  ])
+
+  await kafkaManager.publish({
+    topic: 'claims.created',
+    key: 'claim-1001',
+    payload: {
+      claimId: 'claim-1001',
+      source: 'claims-api',
+    },
+  })
+
+  await kafkaManager.runConsumer({
+    groupId: 'claims-api-group',
+    topic: 'claims.created',
+    dlqTopic: 'claims.created.dlq',
+    onMessage: async ({ message, headers }) => {
+      console.log('Received event:', {
+        traceId: headers['x-trace-id'],
+        payload: message,
+      })
+    },
+  })
+}
+
+bootstrap().catch(async (error) => {
+  console.error('Kafka bootstrap failed:', error)
+  await kafkaManager.disconnect()
+  process.exit(1)
+})
+
+process.on('SIGINT', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
+})
+
+process.on('SIGTERM', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
 })
 ```
 
-### Retry and timeout tuning
+## API Summary
+
+### `new KafkaManager(options)`
+
+Use this when you already know your brokers.
 
 ```ts
 const kafkaManager = new KafkaManager({
   clientId: 'claims-service',
   brokers: ['localhost:29092'],
-  connectionTimeout: 5000,
-  requestTimeout: 30000,
-  retry: {
-    initialRetryTime: 300,
-    retries: 10,
-  },
 })
 ```
 
-## API
-
-### `new KafkaManager(options)`
-
-Creates a manager from an explicit broker list.
-
 Options:
 
 - `clientId: string`
 - `brokers: string[]`
 - `ssl?: boolean | { rejectUnauthorized?: boolean }`
-- `sasl?: { mechanism, username, password }`
+- `sasl?: SASLOptions`
 - `connectionTimeout?: number`
 - `requestTimeout?: number`
 - `retry?: { initialRetryTime?: number; retries?: number }`
+- `logger?: KafkaLogger`
+- `metrics?: KafkaMetrics`
+- `producerConfig?: ProducerConfig`
+- `consumerDefaults?: Omit<ConsumerConfig, 'groupId'>`
 
 ### `KafkaManager.fromEnv(options)`
 
-Creates a manager by reading broker information from environment variables.
+Use this when you want brokers to come from environment variables.
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+})
+```
 
 Options:
 
@@ -272,10 +652,14 @@ Options:
 - `externalPortEnvKeySuffix?: string`
 - `internalBrokerHostPrefix?: string`
 - `ssl?: boolean | { rejectUnauthorized?: boolean }`
-- `sasl?: { mechanism, username, password }`
+- `sasl?: SASLOptions`
 - `connectionTimeout?: number`
 - `requestTimeout?: number`
 - `retry?: { initialRetryTime?: number; retries?: number }`
+- `logger?: KafkaLogger`
+- `metrics?: KafkaMetrics`
+- `producerConfig?: ProducerConfig`
+- `consumerDefaults?: Omit<ConsumerConfig, 'groupId'>`
 
 ### `getProducer()`
 
@@ -283,109 +667,252 @@ Returns a shared connected Kafka producer.
 
 ### `getConsumer(groupId)`
 
-Returns a shared connected Kafka consumer for the provided group id.
-
-### `provisionTopics(topics)`
+Returns a shared connected Kafka consumer for that `groupId`.
 
-Creates missing topics using the Kafka admin client.
+### `publish(options)`
 
-Topic shape:
+Publishes one JSON message.
 
 ```ts
-{
-  topic: string
-  numPartitions?: number
-  replicationFactor?: number
-}
+await kafkaManager.publish({
+  topic: 'claims.created',
+  key: 'claim-1001',
+  payload: { claimId: 'claim-1001' },
+})
 ```
 
-### `publish(options)`
+### `publishEnvelope(options)`
 
-Publishes one JSON-encoded message.
+Publishes a standard metadata envelope for traceable event-driven systems.
 
-Message shape:
+### `provisionTopics(topics)`
 
-```ts
-{
-  topic: string
-  payload: T
-  key?: string
-  headers?: IHeaders
-}
-```
+Creates missing topics.
+
+### `runConsumer(options)`
+
+Runs a managed consumer with:
+
+- a typed message parser
+- application callback handling
+- optional DLQ publishing
+- optional error callback
+
+### `getHealthSnapshot()`
+
+Returns a lightweight connection-health snapshot.
 
 ### `disconnect()`
 
-Disconnects the managed producer, admin client, and any connected consumers.
+Disconnects the producer, admin client, and any connected consumers.
 
-## Example With Your Kafka Cluster
+## Error Handling
 
-### Host-based app
+This package now throws structured errors with stable codes for production handling.
+
+### Error class
+
+- `KafkaManagerError`
+
+### Common response types
+
+- `SuccessResponse<T>`
+- `ErrorResponse`
+- `OperationResponse<T>`
+
+### Error fields
+
+- `code`: stable machine-readable error code
+- `message`: human-readable explanation
+- `details`: structured metadata for logs or monitoring
+- `cause`: original thrown error when available
+
+### Common error codes
+
+- `INVALID_CLIENT_ID`
+- `INVALID_BROKERS`
+- `INVALID_GROUP_ID`
+- `INVALID_TOPIC`
+- `INVALID_ENVELOPE`
+- `INVALID_BROKER_COUNT`
+- `MISSING_ENVIRONMENT_VARIABLE`
+- `PRODUCER_CONNECTION_FAILED`
+- `CONSUMER_CONNECTION_FAILED`
+- `ADMIN_CONNECTION_FAILED`
+- `MESSAGE_PUBLISH_FAILED`
+- `TOPIC_PROVISION_FAILED`
+- `CONSUMER_RUN_FAILED`
+- `CONSUMER_HANDLER_FAILED`
+- `DLQ_PUBLISH_FAILED`
+- `MESSAGE_PARSE_FAILED`
+- `DISCONNECT_FAILED`
+
+### Example
 
 ```ts
-import { KafkaManager } from 'sentinel-kafka-manager'
+import { KafkaManagerError } from 'sentinel-kafka-manager'
 
+try {
+  await kafkaManager.runConsumer({
+    groupId: 'claims-service-group',
+    topic: 'claims.created',
+    onMessage: async ({ message }) => {
+      console.log(message)
+    },
+  })
+} catch (error) {
+  if (error instanceof KafkaManagerError) {
+    console.error(error.code, error.message, error.details)
+  }
+
+  throw error
+}
+```
+
+## Advanced Examples
+
+### Logger and metrics example
+
+```ts
 const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-api',
+  clientId: 'claims-service',
   mode: 'external',
+  logger: {
+    debug: (message, meta) => console.debug(message, meta),
+    info: (message, meta) => console.info(message, meta),
+    warn: (message, meta) => console.warn(message, meta),
+    error: (message, meta) => console.error(message, meta),
+  },
+  metrics: {
+    emit: (event) => {
+      console.log('METRIC', event.name, event.tags, event.meta)
+    },
+  },
 })
+```
 
-await kafkaManager.publish({
+### SSL example
+
+```ts
+const kafkaManager = new KafkaManager({
+  clientId: 'claims-service',
+  brokers: ['localhost:29092'],
+  ssl: {
+    rejectUnauthorized: true,
+  },
+})
+```
+
+### Managed consumer with custom parser
+
+```ts
+await kafkaManager.runConsumer({
+  groupId: 'claims-service-group',
   topic: 'claims.created',
-  key: 'claim-1001',
-  payload: {
-    claimId: 'claim-1001',
-    source: 'claims-api',
+  dlqTopic: 'claims.created.dlq',
+  parser: (payload) => {
+    const rawValue = payload.message.value?.toString() ?? '{}'
+    return JSON.parse(rawValue) as {
+      claimId: string
+      status: string
+    }
+  },
+  onMessage: async ({ message }) => {
+    console.log(message.claimId, message.status)
   },
 })
 ```
 
-### Dockerized app on the same network
+### SASL example
 
 ```ts
-import { KafkaManager } from 'sentinel-kafka-manager'
+const kafkaManager = new KafkaManager({
+  clientId: 'claims-service',
+  brokers: ['localhost:29092'],
+  sasl: {
+    mechanism: 'plain',
+    username: 'kafka-user',
+    password: 'secret',
+  },
+})
+```
 
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-worker',
-  mode: 'internal',
+### Retry and timeout example
+
+```ts
+const kafkaManager = new KafkaManager({
+  clientId: 'claims-service',
+  brokers: ['localhost:29092'],
+  connectionTimeout: 5000,
+  requestTimeout: 30000,
+  retry: {
+    initialRetryTime: 300,
+    retries: 10,
+  },
 })
 ```
 
-## Publishing
+## Common Mistakes
+
+- Do not connect application clients to KRaft controller nodes. Connect only to brokers.
+- Do not use `mode: 'external'` inside Docker unless you truly want host-published ports.
+- Do not create a new `KafkaManager` for every request. Reuse one shared instance.
+- Do not forget `disconnect()` during shutdown.
+- Do not assume topics will exist unless your platform creates them or your service provisions them.
+- Do not ignore failed consumer messages in production. Use `runConsumer()` with a `dlqTopic`.
+- Do not publish business-critical events without metadata. Prefer `publishEnvelope()` for traceability.
+- Do not depend on raw driver error strings in application logic. Use `KafkaManagerError.code`.
+
+## Recommended SaaS Pattern
+
+For most SaaS services, the cleanest pattern is:
+
+1. Create one shared `KafkaManager` during service bootstrap.
+2. Use `publishEnvelope()` for business events.
+3. Use `runConsumer()` instead of raw `consumer.run()` for managed handling.
+4. Configure a `dlqTopic` for important consumers.
+5. Catch `KafkaManagerError` and branch on `error.code`.
+6. Wrap service outcomes in `OperationResponse<T>` where useful.
 
-Build before publishing:
+## Publishing This Package
+
+Build:
 
 ```bash
 npm run build
 ```
 
-Publish the package:
+Publish:
 
 ```bash
 npm publish
 ```
 
-This package already sets `"publishConfig": { "access": "public" }`, so you do not need to pass `--access public` every time.
-
-### Publish Troubleshooting
-
-#### `403` with 2FA or token message
+If npm rejects the publish:
 
-If npm says two-factor authentication or a granular token with bypass 2FA is required, authenticate with an account that can publish or use a publish-capable npm token.
+- `403` usually means authentication, 2FA, or token permission issues
+- `404` usually means the package name is unavailable or not publishable by your account
 
-Useful checks:
+Useful check:
 
 ```bash
 npm whoami
 ```
 
-#### `404` or package name unavailable
+## Notes
 
-If npm says the package could not be published, confirm that the package name is available to your npm account and not already owned by someone else.
+- Your current Kafka cluster works fine with PLAINTEXT, so SSL and SASL are optional unless your environment changes
+- This package is designed for both host-based services and Dockerized services
+- If you want the least surprise, use `KAFKA_BROKERS`
 
-## Notes
+## Links
+
+- LinkedIn: https://www.linkedin.com/in/dilip-shaw-2740769/
+
+## About Me
+
+I'm a full stack developer. Experienced developer with over 20 years of expertise in crafting scalable web applications. Proficient in frontend technologies such as Angular and React, alongside extensive experience in WordPress, Drupal, and backend frameworks like Node.js. With a proven track record of delivering high-quality solutions to meet client needs and drive business objectives, I bring a versatile skill set and a commitment to excellence to every project.
+
+## Feedback
 
-- connect clients only to brokers, not KRaft controller nodes
-- for your current cluster, PLAINTEXT is enough and SSL/SASL is optional
-- if your services use `localhost`, use `mode: 'external'`
-- if your services run in Docker on the Kafka network, use `mode: 'internal'`
+If you have any feedback, please reach out to us at `dilipabc@gmail.com`