sentinel-kafka-manager 1.0.2 → 1.0.4

package/README.md

@@ -27,6 +27,36 @@ In most services, Kafka setup gets repeated:
 
 `sentinel-kafka-manager` wraps those common tasks in one small class: `KafkaManager`.
 
+## Enterprise Features
+
+This version now includes a stronger production baseline:
+
+- config validation for client id, brokers, and broker discovery inputs
+- shared producer, admin, and consumer lifecycle
+- logger and metrics hooks
+- standard message envelope publishing
+- managed consumer runner with dead-letter topic support
+- health snapshot reporting
+- safer startup behavior for concurrent connections
+- stable error codes and structured exception details
+
+## Recommended Usage Level
+
+This package is now suitable as a shared internal Kafka module for SaaS microservices.
+
+Recommended use:
+
+- internal platform package for Node.js services
+- event publishing with standard envelopes
+- managed consumers with DLQ handling
+- consistent service-layer success and error responses
+
+Still recommended outside the package:
+
+- automated unit and integration tests in the consuming service or platform repo
+- schema validation for business payloads when required by your organization
+- organization-specific tracing, alerting, and compliance rules
+
 ## Installation
 
 Install from npm:
@@ -48,321 +78,63 @@ npm run build
 - Kafka cluster reachable from the service
 - `kafkajs` is included automatically as a dependency of this package
 
-## Create A Kafka Server With This Repo
-
-This repository already includes a full local Kafka server setup in [docker-compose.yml](/var/www/html/matrix-project/sentinel-kafka-manager/docker-compose.yml) and sample environment values in [.env.local.example](/var/www/html/matrix-project/sentinel-kafka-manager/.env.local.example).
-
-The stack includes:
-
-- 3 KRaft controllers
-- 3 Kafka brokers
-- 1 Kafka UI instance
-
-### 1. Create your `.env`
-
-Copy the example file:
-
-```bash
-cp .env.local.example .env
-```
-
-If you want a standard localhost setup, the example values are already suitable.
-
-### 2. Start the Kafka cluster
-
-```bash
-docker compose --env-file .env up -d
-```
-
-This creates:
-
-- controller quorum on port `9093` inside Docker
-- internal broker traffic on port `9092` inside Docker
-- host-accessible broker ports `29092`, `39092`, and `49092`
-- Kafka UI on `127.0.0.1:5001`
-
-### 3. Stop the Kafka cluster
-
-```bash
-docker compose --env-file .env down
-```
-
-To also remove persisted Kafka data volumes:
-
-```bash
-docker compose --env-file .env down -v
-```
-
-### 4. Check container status
-
-```bash
-docker compose --env-file .env ps
-```
-
-### 5. Current broker endpoints
-
-For applications running on your host machine:
-
-```text
-localhost:29092
-localhost:39092
-localhost:49092
-```
-
-For applications running inside Docker on the same Compose network:
-
-```text
-broker-1:9092
-broker-2:9092
-broker-3:9092
-```
-
-Important:
-
-- application clients must connect to brokers, not controllers
-- controllers are internal cluster metadata nodes only
-- `mode: 'external'` is for host-based apps
-- `mode: 'internal'` is for Dockerized apps on the same network
-
-## Kafka UI Usage
-
-The Docker stack also starts Kafka UI for cluster inspection.
-
-### Open the UI
-
-Visit:
-
-```text
-http://127.0.0.1:5001
-```
-
-### Login credentials
-
-By default, the UI uses the credentials from `.env`:
-
-```env
-KAFKA_UI_USERNAME=admin
-KAFKA_UI_PASSWORD=Admin@007
-```
-
-### What the UI connects to
-
-Kafka UI is preconfigured in `docker-compose.yml` to use all three brokers:
-
-```text
-broker-1:9092,broker-2:9092,broker-3:9092
-```
-
-The cluster name shown in the UI comes from:
-
-```env
-KAFKA_UI_CLUSTER_NAME=Project Omni Enterprise Kafka
-```
-
-### What you can do in the UI
-
-- view brokers and cluster health
-- inspect topics and partitions
-- browse messages
-- inspect consumer groups
-- check offsets and lag
-
-### Read-only mode
-
-The example `.env` sets:
-
-```env
-KAFKA_UI_READONLY=true
-```
-
-That means the UI is intended for safe inspection only.
-
-If you want to create topics or make changes from the UI, change it to:
-
-```env
-KAFKA_UI_READONLY=false
-```
-
-Then restart the stack:
-
-```bash
-docker compose --env-file .env up -d
-```
-
-### Typical UI flow
-
-1. Start the Docker stack.
-2. Open `http://127.0.0.1:5001`.
-3. Log in with `KAFKA_UI_USERNAME` and `KAFKA_UI_PASSWORD`.
-4. Open the configured cluster.
-5. Go to Topics, Consumer Groups, or Brokers as needed.
-
-## Docker Compose And `.env` Reference
-
-This project does not use a separate `Dockerfile` for Kafka. The Kafka server is created from `docker-compose.yml` and environment values from `.env`.
-
-### Core Kafka image and cluster identity
-
-```env
-KAFKA_IMAGE=confluentinc/cp-kafka:7.6.6
-KAFKA_CLUSTER_ID=MkU3OEVBNTcwNTJENDM2Qk
-KAFKA_RESTART_POLICY=unless-stopped
-KAFKA_STOP_GRACE_PERIOD=60s
-KAFKA_NETWORK_NAME=kafka-network
-```
-
-- `KAFKA_IMAGE`: Kafka container image used by controllers and brokers
-- `KAFKA_CLUSTER_ID`: shared KRaft cluster id for all nodes
-- `KAFKA_NETWORK_NAME`: Docker network name used by the full stack
-
-### Controller quorum settings
-
-```env
-KAFKA_CONTROLLER_PORT=9093
-KAFKA_CONTROLLER_LISTENER_NAMES=CONTROLLER
-KAFKA_CONTROLLER_QUORUM_VOTERS=1@controller-1:9093,2@controller-2:9093,3@controller-3:9093
-```
-
-- controllers run only inside Docker
-- apps should never use these controller addresses as Kafka client brokers
-
-### Broker listener settings
-
-```env
-KAFKA_BROKER_BIND_ADDRESS=0.0.0.0
-KAFKA_EXTERNAL_HOST=localhost
-KAFKA_BROKER_INTERNAL_PORT=9092
-KAFKA_BROKER_EXTERNAL_CONTAINER_PORT=19092
-KAFKA_BROKER_1_EXTERNAL_PORT=29092
-KAFKA_BROKER_2_EXTERNAL_PORT=39092
-KAFKA_BROKER_3_EXTERNAL_PORT=49092
-KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
-KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
-```
-
-- `KAFKA_EXTERNAL_HOST=localhost` exposes brokers to your host machine
-- `KAFKA_BROKER_1_EXTERNAL_PORT`, `KAFKA_BROKER_2_EXTERNAL_PORT`, and `KAFKA_BROKER_3_EXTERNAL_PORT` are the ports your local apps should use
-- `KAFKA_BROKER_INTERNAL_PORT=9092` is used by containers on the Docker network
-
-### Replication and durability defaults
-
-```env
-KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=3
-KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=3
-KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=2
-KAFKA_DEFAULT_REPLICATION_FACTOR=3
-KAFKA_MIN_INSYNC_REPLICAS=2
-KAFKA_NUM_PARTITIONS=6
-```
-
-These values are designed for the included three-broker cluster.
-
-### Broker behavior
-
-```env
-KAFKA_AUTO_CREATE_TOPICS_ENABLE=false
-KAFKA_DELETE_TOPIC_ENABLE=true
-KAFKA_LOG_RETENTION_HOURS=168
-KAFKA_LOG_SEGMENT_BYTES=1073741824
-KAFKA_MESSAGE_MAX_BYTES=10485880
-KAFKA_REPLICA_FETCH_MAX_BYTES=10485880
-KAFKA_SOCKET_REQUEST_MAX_BYTES=104857600
-```
-
-- topics are not auto-created by default
-- deleting topics is allowed
-- retention is 7 days by default
-
-### Storage and safety limits
-
-```env
-KAFKA_LOG_DIRS=/var/lib/kafka/data
-KAFKA_ULIMIT_NOFILE_SOFT=65536
-KAFKA_ULIMIT_NOFILE_HARD=65536
-KAFKA_LOG_MAX_SIZE=100m
-KAFKA_LOG_MAX_FILE=5
-```
-
-### Kafka UI settings
-
-```env
-KAFKA_UI_IMAGE=provectuslabs/kafka-ui:latest
-KAFKA_UI_CONTAINER_NAME=kafka-ui
-KAFKA_UI_HOSTNAME=kafka-ui
-KAFKA_UI_RESTART_POLICY=unless-stopped
-KAFKA_UI_BIND_ADDRESS=127.0.0.1
-KAFKA_UI_PORT=5001
-KAFKA_UI_DYNAMIC_CONFIG_ENABLED=false
-KAFKA_UI_CLUSTER_NAME=Project Omni Enterprise Kafka
-KAFKA_UI_READONLY=true
-KAFKA_UI_AUTH_TYPE=LOGIN_FORM
-KAFKA_UI_USERNAME=admin
-KAFKA_UI_PASSWORD=Admin@007
-```
-
-- `KAFKA_UI_BIND_ADDRESS=127.0.0.1` keeps the UI local-only
-- `KAFKA_UI_PORT=5001` publishes the UI on your machine
-- `KAFKA_UI_READONLY=true` prevents write actions from the UI
-
-### Healthcheck settings
-
-```env
-KAFKA_HEALTHCHECK_INTERVAL=10s
-KAFKA_HEALTHCHECK_TIMEOUT=5s
-KAFKA_HEALTHCHECK_RETRIES=15
-KAFKA_HEALTHCHECK_START_PERIOD=30s
-```
-
-## Creating Topics
-
-Because `KAFKA_AUTO_CREATE_TOPICS_ENABLE=false`, topics should be created deliberately.
-
-You have two good options:
-
-- create topics from your service with `kafkaManager.provisionTopics()`
-- allow UI-based creation by setting `KAFKA_UI_READONLY=false`
-
-Example with this package:
+## Full Service Example
 
 ```ts
-await kafkaManager.provisionTopics([
-  {
-    topic: 'claims.created',
-    numPartitions: 6,
-    replicationFactor: 3,
-  },
-])
-```
-
-## Enterprise Features
-
-This version now includes a stronger production baseline:
+import { KafkaManager } from 'sentinel-kafka-manager'
 
-- config validation for client id, brokers, and broker discovery inputs
-- shared producer, admin, and consumer lifecycle
-- logger and metrics hooks
-- standard message envelope publishing
-- managed consumer runner with dead-letter topic support
-- health snapshot reporting
-- safer startup behavior for concurrent connections
-- stable error codes and structured exception details
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-api',
+  mode: 'external',
+})
 
-## Recommended Usage Level
+async function bootstrap(): Promise<void> {
+  await kafkaManager.provisionTopics([
+    {
+      topic: 'claims.created',
+      numPartitions: 6,
+      replicationFactor: 3,
+    },
+  ])
 
-This package is now suitable as a shared internal Kafka module for SaaS microservices.
+  await kafkaManager.publish({
+    topic: 'claims.created',
+    key: 'claim-1001',
+    payload: {
+      claimId: 'claim-1001',
+      source: 'claims-api',
+    },
+  })
 
-Recommended use:
+  await kafkaManager.runConsumer({
+    groupId: 'claims-api-group',
+    topic: 'claims.created',
+    dlqTopic: 'claims.created.dlq',
+    onMessage: async ({ message, headers }) => {
+      console.log('Received event:', {
+        traceId: headers['x-trace-id'],
+        payload: message,
+      })
+    },
+  })
+}
 
-- internal platform package for Node.js services
-- event publishing with standard envelopes
-- managed consumers with DLQ handling
-- consistent service-layer success and error responses
+bootstrap().catch(async (error) => {
+  console.error('Kafka bootstrap failed:', error)
+  await kafkaManager.disconnect()
+  process.exit(1)
+})
 
-Still recommended outside the package:
+process.on('SIGINT', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
+})
 
-- automated unit and integration tests in the consuming service or platform repo
-- schema validation for business payloads when required by your organization
-- organization-specific tracing, alerting, and compliance rules
+process.on('SIGTERM', async () => {
+  await kafkaManager.disconnect()
+  process.exit(0)
+})
+```
 
 ## Basic Usage
 
@@ -702,190 +474,420 @@ console.log(health)
 
 ## Environment Setup
 
-This package supports two environment styles:
+This package supports two environment styles:
+
+1. one explicit broker list with `KAFKA_BROKERS`
+2. derived brokers from your existing Kafka runtime variables
+
+### Option 1: Use `KAFKA_BROKERS`
+
+This is the simplest option.
+
+### `.env`
+
+```env
+KAFKA_BROKERS=localhost:29092,localhost:39092,localhost:49092
+```
+
+### Code
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+})
+```
+
+### Option 2: Use your current Kafka cluster variables
+
+This matches the Kafka server setup you shared.
+
+### For host machine apps
+
+Use this when your Node.js service runs on the host machine.
+
+### `.env`
+
+```env
+KAFKA_EXTERNAL_HOST=localhost
+KAFKA_BROKER_1_EXTERNAL_PORT=29092
+KAFKA_BROKER_2_EXTERNAL_PORT=39092
+KAFKA_BROKER_3_EXTERNAL_PORT=49092
+KAFKA_BROKER_INTERNAL_PORT=9092
+```
+
+### Code
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+})
+```
+
+This resolves to:
+
+```text
+localhost:29092
+localhost:39092
+localhost:49092
+```
+
+### For Dockerized apps on the Kafka network
+
+Use this when your Node.js service runs inside Docker on the same Kafka network.
+
+### `.env`
+
+```env
+KAFKA_BROKER_INTERNAL_PORT=9092
+```
+
+### Code
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'internal',
+})
+```
+
+This resolves to:
+
+```text
+broker-1:9092
+broker-2:9092
+broker-3:9092
+```
+
+### Which Mode Should You Use?
+
+- Use `mode: 'external'` when your service uses `localhost` broker ports like `29092`, `39092`, and `49092`
+- Use `mode: 'internal'` when your service is inside Docker and should reach brokers like `broker-1:9092`
+- Use `KAFKA_BROKERS` when you want the simplest and most explicit configuration
+
+### Your Current Running Kafka Cluster
+
+Based on your running Docker containers, your Kafka cluster is exposed like this:
+
+- `broker-1` -> host port `29092`
+- `broker-2` -> host port `39092`
+- `broker-3` -> host port `49092`
+- `controller-1`, `controller-2`, and `controller-3` are controller-only nodes and should not be used by application clients
+
+If your Node.js service runs on the host machine, use:
+
+```env
+KAFKA_EXTERNAL_HOST=localhost
+KAFKA_BROKER_1_EXTERNAL_PORT=29092
+KAFKA_BROKER_2_EXTERNAL_PORT=39092
+KAFKA_BROKER_3_EXTERNAL_PORT=49092
+KAFKA_BROKER_INTERNAL_PORT=9092
+```
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'external',
+})
+```
+
+If your Node.js service runs inside Docker on the same Kafka network, use:
+
+```ts
+const kafkaManager = KafkaManager.fromEnv({
+  clientId: 'claims-service',
+  mode: 'internal',
+})
+```
+
+This package is designed to work with that exact broker layout.
+
+## Local Infrastructure And Operations
+
+### Create A Kafka Server With This Repo
+
+This repository already includes a full local Kafka server setup in [docker-compose.yml](/var/www/html/matrix-project/sentinel-kafka-manager/docker-compose.yml) and sample environment values in [.env.local.example](/var/www/html/matrix-project/sentinel-kafka-manager/.env.local.example).
+
+The stack includes:
+
+- 3 KRaft controllers
+- 3 Kafka brokers
+- 1 Kafka UI instance
+
+### 1. Create your `.env`
+
+Copy the example file:
+
+```bash
+cp .env.local.example .env
+```
+
+If you want a standard localhost setup, the example values are already suitable.
+
+### 2. Start the Kafka cluster
+
+```bash
+docker compose --env-file .env up -d
+```
+
+This creates:
+
+- controller quorum on port `9093` inside Docker
+- internal broker traffic on port `9092` inside Docker
+- host-accessible broker ports `29092`, `39092`, and `49092`
+- Kafka UI on `127.0.0.1:5001`
+
+### 3. Stop the Kafka cluster
+
+```bash
+docker compose --env-file .env down
+```
+
+To also remove persisted Kafka data volumes:
+
+```bash
+docker compose --env-file .env down -v
+```
+
+### 4. Check container status
+
+```bash
+docker compose --env-file .env ps
+```
+
+### 5. Current broker endpoints
+
+For applications running on your host machine:
+
+```text
+localhost:29092
+localhost:39092
+localhost:49092
+```
+
+For applications running inside Docker on the same Compose network:
+
+```text
+broker-1:9092
+broker-2:9092
+broker-3:9092
+```
+
+Important:
+
+- application clients must connect to brokers, not controllers
+- controllers are internal cluster metadata nodes only
+- `mode: 'external'` is for host-based apps
+- `mode: 'internal'` is for Dockerized apps on the same network
+
+### Kafka UI Usage
+
+The Docker stack also starts Kafka UI for cluster inspection.
+
+### Open the UI
+
+Visit:
+
+```text
+http://127.0.0.1:5001
+```
+
+### Login credentials
+
+By default, the UI uses the credentials from `.env`:
+
+```env
+KAFKA_UI_USERNAME=admin
+KAFKA_UI_PASSWORD=Admin@007
+```
+
+### What the UI connects to
+
+Kafka UI is preconfigured in `docker-compose.yml` to use all three brokers:
+
+```text
+broker-1:9092,broker-2:9092,broker-3:9092
+```
+
+The cluster name shown in the UI comes from:
+
+```env
+KAFKA_UI_CLUSTER_NAME=Project Omni Enterprise Kafka
+```
+
+### What you can do in the UI
+
+- view brokers and cluster health
+- inspect topics and partitions
+- browse messages
+- inspect consumer groups
+- check offsets and lag
+
+### Read-only mode
 
-1. one explicit broker list with `KAFKA_BROKERS`
-2. derived brokers from your existing Kafka runtime variables
+The example `.env` sets:
 
-## Option 1: Use `KAFKA_BROKERS`
+```env
+KAFKA_UI_READONLY=true
+```
 
-This is the simplest option.
+That means the UI is intended for safe inspection only.
 
-### `.env`
+If you want to create topics or make changes from the UI, change it to:
 
 ```env
-KAFKA_BROKERS=localhost:29092,localhost:39092,localhost:49092
+KAFKA_UI_READONLY=false
 ```
 
-### Code
+Then restart the stack:
 
-```ts
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-service',
-})
+```bash
+docker compose --env-file .env up -d
 ```
 
-## Option 2: Use your current Kafka cluster variables
+### Typical UI flow
 
-This matches the Kafka server setup you shared.
+1. Start the Docker stack.
+2. Open `http://127.0.0.1:5001`.
+3. Log in with `KAFKA_UI_USERNAME` and `KAFKA_UI_PASSWORD`.
+4. Open the configured cluster.
+5. Go to Topics, Consumer Groups, or Brokers as needed.
 
-### For host machine apps
+### Docker Compose And `.env` Reference
 
-Use this when your Node.js service runs on the host machine.
+This project does not use a separate `Dockerfile` for Kafka. The Kafka server is created from `docker-compose.yml` and environment values from `.env`.
 
-### `.env`
+### Core Kafka image and cluster identity
 
 ```env
-KAFKA_EXTERNAL_HOST=localhost
-KAFKA_BROKER_1_EXTERNAL_PORT=29092
-KAFKA_BROKER_2_EXTERNAL_PORT=39092
-KAFKA_BROKER_3_EXTERNAL_PORT=49092
-KAFKA_BROKER_INTERNAL_PORT=9092
+KAFKA_IMAGE=confluentinc/cp-kafka:7.6.6
+KAFKA_CLUSTER_ID=MkU3OEVBNTcwNTJENDM2Qk
+KAFKA_RESTART_POLICY=unless-stopped
+KAFKA_STOP_GRACE_PERIOD=60s
+KAFKA_NETWORK_NAME=kafka-network
 ```
 
-### Code
-
-```ts
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-service',
-  mode: 'external',
-})
-```
+- `KAFKA_IMAGE`: Kafka container image used by controllers and brokers
+- `KAFKA_CLUSTER_ID`: shared KRaft cluster id for all nodes
+- `KAFKA_NETWORK_NAME`: Docker network name used by the full stack
 
-This resolves to:
+### Controller quorum settings
 
-```text
-localhost:29092
-localhost:39092
-localhost:49092
+```env
+KAFKA_CONTROLLER_PORT=9093
+KAFKA_CONTROLLER_LISTENER_NAMES=CONTROLLER
+KAFKA_CONTROLLER_QUORUM_VOTERS=1@controller-1:9093,2@controller-2:9093,3@controller-3:9093
 ```
 
-### For Dockerized apps on the Kafka network
-
-Use this when your Node.js service runs inside Docker on the same Kafka network.
+- controllers run only inside Docker
+- apps should never use these controller addresses as Kafka client brokers
 
-### `.env`
+### Broker listener settings
 
 ```env
+KAFKA_BROKER_BIND_ADDRESS=0.0.0.0
+KAFKA_EXTERNAL_HOST=localhost
 KAFKA_BROKER_INTERNAL_PORT=9092
+KAFKA_BROKER_EXTERNAL_CONTAINER_PORT=19092
+KAFKA_BROKER_1_EXTERNAL_PORT=29092
+KAFKA_BROKER_2_EXTERNAL_PORT=39092
+KAFKA_BROKER_3_EXTERNAL_PORT=49092
+KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL
+KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT
 ```
 
-### Code
-
-```ts
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-service',
-  mode: 'internal',
-})
-```
+- `KAFKA_EXTERNAL_HOST=localhost` exposes brokers to your host machine
+- `KAFKA_BROKER_1_EXTERNAL_PORT`, `KAFKA_BROKER_2_EXTERNAL_PORT`, and `KAFKA_BROKER_3_EXTERNAL_PORT` are the ports your local apps should use
+- `KAFKA_BROKER_INTERNAL_PORT=9092` is used by containers on the Docker network
 
-This resolves to:
+### Replication and durability defaults
 
-```text
-broker-1:9092
-broker-2:9092
-broker-3:9092
+```env
+KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=3
+KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR=3
+KAFKA_TRANSACTION_STATE_LOG_MIN_ISR=2
+KAFKA_DEFAULT_REPLICATION_FACTOR=3
+KAFKA_MIN_INSYNC_REPLICAS=2
+KAFKA_NUM_PARTITIONS=6
 ```
 
-## Which Mode Should You Use?
-
-- Use `mode: 'external'` when your service uses `localhost` broker ports like `29092`, `39092`, and `49092`
-- Use `mode: 'internal'` when your service is inside Docker and should reach brokers like `broker-1:9092`
-- Use `KAFKA_BROKERS` when you want the simplest and most explicit configuration
+These values are designed for the included three-broker cluster.
 
-## Your Current Running Kafka Cluster
+### Broker behavior
 
-Based on your running Docker containers, your Kafka cluster is exposed like this:
+```env
+KAFKA_AUTO_CREATE_TOPICS_ENABLE=false
+KAFKA_DELETE_TOPIC_ENABLE=true
+KAFKA_LOG_RETENTION_HOURS=168
+KAFKA_LOG_SEGMENT_BYTES=1073741824
+KAFKA_MESSAGE_MAX_BYTES=10485880
+KAFKA_REPLICA_FETCH_MAX_BYTES=10485880
+KAFKA_SOCKET_REQUEST_MAX_BYTES=104857600
+```
 
-- `broker-1` -> host port `29092`
-- `broker-2` -> host port `39092`
-- `broker-3` -> host port `49092`
-- `controller-1`, `controller-2`, and `controller-3` are controller-only nodes and should not be used by application clients
+- topics are not auto-created by default
+- deleting topics is allowed
+- retention is 7 days by default
 
-If your Node.js service runs on the host machine, use:
+### Storage and safety limits
 
 ```env
-KAFKA_EXTERNAL_HOST=localhost
-KAFKA_BROKER_1_EXTERNAL_PORT=29092
-KAFKA_BROKER_2_EXTERNAL_PORT=39092
-KAFKA_BROKER_3_EXTERNAL_PORT=49092
-KAFKA_BROKER_INTERNAL_PORT=9092
-```
-
-```ts
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-service',
-  mode: 'external',
-})
+KAFKA_LOG_DIRS=/var/lib/kafka/data
+KAFKA_ULIMIT_NOFILE_SOFT=65536
+KAFKA_ULIMIT_NOFILE_HARD=65536
+KAFKA_LOG_MAX_SIZE=100m
+KAFKA_LOG_MAX_FILE=5
 ```
 
-If your Node.js service runs inside Docker on the same Kafka network, use:
+### Kafka UI settings
 
-```ts
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-service',
-  mode: 'internal',
-})
+```env
+KAFKA_UI_IMAGE=provectuslabs/kafka-ui:latest
+KAFKA_UI_CONTAINER_NAME=kafka-ui
+KAFKA_UI_HOSTNAME=kafka-ui
+KAFKA_UI_RESTART_POLICY=unless-stopped
+KAFKA_UI_BIND_ADDRESS=127.0.0.1
+KAFKA_UI_PORT=5001
+KAFKA_UI_DYNAMIC_CONFIG_ENABLED=false
+KAFKA_UI_CLUSTER_NAME=Project Omni Enterprise Kafka
+KAFKA_UI_READONLY=true
+KAFKA_UI_AUTH_TYPE=LOGIN_FORM
+KAFKA_UI_USERNAME=admin
+KAFKA_UI_PASSWORD=Admin@007
 ```
 
-This package is designed to work with that exact broker layout.
-
-## Full Service Example
+- `KAFKA_UI_BIND_ADDRESS=127.0.0.1` keeps the UI local-only
+- `KAFKA_UI_PORT=5001` publishes the UI on your machine
+- `KAFKA_UI_READONLY=true` prevents write actions from the UI
 
-```ts
-import { KafkaManager } from 'sentinel-kafka-manager'
+### Healthcheck settings
 
-const kafkaManager = KafkaManager.fromEnv({
-  clientId: 'claims-api',
-  mode: 'external',
-})
+```env
+KAFKA_HEALTHCHECK_INTERVAL=10s
+KAFKA_HEALTHCHECK_TIMEOUT=5s
+KAFKA_HEALTHCHECK_RETRIES=15
+KAFKA_HEALTHCHECK_START_PERIOD=30s
+```
 
-async function bootstrap(): Promise<void> {
-  await kafkaManager.provisionTopics([
-    {
-      topic: 'claims.created',
-      numPartitions: 6,
-      replicationFactor: 3,
-    },
-  ])
+### Creating Topics
 
-  await kafkaManager.publish({
-    topic: 'claims.created',
-    key: 'claim-1001',
-    payload: {
-      claimId: 'claim-1001',
-      source: 'claims-api',
-    },
-  })
+Because `KAFKA_AUTO_CREATE_TOPICS_ENABLE=false`, topics should be created deliberately.
 
-  await kafkaManager.runConsumer({
-    groupId: 'claims-api-group',
-    topic: 'claims.created',
-    dlqTopic: 'claims.created.dlq',
-    onMessage: async ({ message, headers }) => {
-      console.log('Received event:', {
-        traceId: headers['x-trace-id'],
-        payload: message,
-      })
-    },
-  })
-}
+You have two good options:
 
-bootstrap().catch(async (error) => {
-  console.error('Kafka bootstrap failed:', error)
-  await kafkaManager.disconnect()
-  process.exit(1)
-})
+- create topics from your service with `kafkaManager.provisionTopics()`
+- allow UI-based creation by setting `KAFKA_UI_READONLY=false`
 
-process.on('SIGINT', async () => {
-  await kafkaManager.disconnect()
-  process.exit(0)
-})
+Example with this package:
 
-process.on('SIGTERM', async () => {
-  await kafkaManager.disconnect()
-  process.exit(0)
-})
+```ts
+await kafkaManager.provisionTopics([
+  {
+    topic: 'claims.created',
+    numPartitions: 6,
+    replicationFactor: 3,
+  },
+])
 ```
 
 ## API Summary
@@ -1138,7 +1140,9 @@ const kafkaManager = new KafkaManager({
 })
 ```
 
-## Common Mistakes
+## Operational Guidance
+
+### Common Mistakes
 
 - Do not connect application clients to KRaft controller nodes. Connect only to brokers.
 - Do not use `mode: 'external'` inside Docker unless you truly want host-published ports.
@@ -1149,7 +1153,7 @@ const kafkaManager = new KafkaManager({
 - Do not publish business-critical events without metadata. Prefer `publishEnvelope()` for traceability.
 - Do not depend on raw driver error strings in application logic. Use `KafkaManagerError.code`.
 
-## Recommended SaaS Pattern
+### Recommended SaaS Pattern
 
 For most SaaS services, the cleanest pattern is:
 
@@ -1160,38 +1164,13 @@ For most SaaS services, the cleanest pattern is:
 5. Catch `KafkaManagerError` and branch on `error.code`.
 6. Wrap service outcomes in `OperationResponse<T>` where useful.
 
-## Publishing This Package
-
-Build:
-
-```bash
-npm run build
-```
-
-Publish:
-
-```bash
-npm publish
-```
-
-If npm rejects the publish:
-
-- `403` usually means authentication, 2FA, or token permission issues
-- `404` usually means the package name is unavailable or not publishable by your account
-
-Useful check:
-
-```bash
-npm whoami
-```
-
-## Notes
+### Notes
 
 - Your current Kafka cluster works fine with PLAINTEXT, so SSL and SASL are optional unless your environment changes
 - This package is designed for both host-based services and Dockerized services
 - If you want the least surprise, use `KAFKA_BROKERS`
 
-## Links
+## Additional Documentation
 
 - npm deployment guide: [NPM_DEPLOYMENT.md](/var/www/html/matrix-project/sentinel-kafka-manager/NPM_DEPLOYMENT.md)
 - Example folder: [examples/service-module-usage/README.md](/var/www/html/matrix-project/sentinel-kafka-manager/examples/service-module-usage/README.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "sentinel-kafka-manager",
-  "version": "1.0.2",
+  "version": "1.0.4",
   "description": "Reusable Kafka manager for Node.js microservices",
   "main": "dist/index.js",
   "module": "dist/index.mjs",