kafka-crab-js 3.1.0 → 4.0.0-beta.3

package/README.md

@@ -1,838 +1,809 @@
-<div align="center">
+# Kafka Crab JS
 
-# 🦀 Kafka Crab JS 🦀
-
-A lightweight, flexible, and reliable Kafka client for JavaScript/TypeScript. It is built using Rust and librdkafka, providing a high-performance and feature-rich Kafka client.
+A high-performance Kafka client for Node.js and TypeScript, implemented with Rust, NAPI-RS, librdkafka, and a small
+JavaScript API layer.
 
 [![npm version](https://img.shields.io/npm/v/kafka-crab-js.svg)](https://www.npmjs.com/package/kafka-crab-js)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-</div>
 
----
+## Highlights
+
+- Native librdkafka client exposed through a TypeScript-friendly API.
+- Producer, direct consumer, Node.js `Readable` stream, and native Web Stream consumer APIs.
+- High-throughput batch receive APIs for workloads that can process more than one message at a time.
+- Manual commit helpers that commit `message.offset + 1` correctly.
+- Optional diagnostics-channel instrumentation for OpenTelemetry through `kafka-crab-js-otel`.
+- Advanced librdkafka settings are forwarded through `configuration` without forcing a custom allowlist.
+
+## Requirements
 
-## What's New in Version 3.0.0
+- Node.js `>= 22`.
+- A Kafka broker reachable from the Node.js process.
+- No separate librdkafka install is required for the published binaries.
 
-### BREAKING CHANGES ⚠️
+## Table Of Contents
+
+1. [Installation](#installation)
+2. [Quick Start](#quick-start)
+3. [Choosing A Consumer API](#choosing-a-consumer-api)
+4. [Producer API](#producer-api)
+5. [Consumer API](#consumer-api)
+6. [Stream Consumers](#stream-consumers)
+7. [Configuration](#configuration)
+8. [OpenTelemetry](#opentelemetry)
+9. [Performance Benchmarks](#performance-benchmarks)
+10. [Migration Notes](#migration-notes)
+11. [Troubleshooting](#troubleshooting)
+12. [Development](#development)
+13. [License](#license)
 
-**OpenTelemetry instrumentation has been moved to a separate package: `kafka-crab-js-otel`**
+## Installation
 
-This change reduces the core package size and makes OTEL an opt-in dependency.
+```bash
+npm install kafka-crab-js
+```
 
-#### Migration from v2.x
+```bash
+pnpm add kafka-crab-js
+```
+
+```bash
+yarn add kafka-crab-js
+```
+
+## Quick Start
 
-**Before (v2.x):**
-```javascript
+### Produce Messages
+
+```ts
 import { KafkaClient } from 'kafka-crab-js'
 
 const client = new KafkaClient({
   brokers: 'localhost:9092',
-  clientId: 'my-app',
-  otel: {
-    serviceName: 'my-service',
-    metrics: { enabled: true },
-  },
+  clientId: 'orders-api',
+  securityProtocol: 'Plaintext',
 })
-```
 
-**After (v3.x):**
-```javascript
-import { KafkaClient } from 'kafka-crab-js'
-import { enableOtelInstrumentation, endSpan } from 'kafka-crab-js-otel'
+const producer = client.createProducer()
 
-// 1. Enable OTEL instrumentation BEFORE creating client
-// Note: serviceName is set via OTEL SDK Resource, not here
-enableOtelInstrumentation({
-  metrics: { enabled: true },
+const metadata = await producer.send({
+  topic: 'orders',
+  messages: [
+    {
+      key: Buffer.from('order-123'),
+      payload: Buffer.from(JSON.stringify({ id: 'order-123', status: 'created' })),
+      headers: {
+        'content-type': Buffer.from('application/json'),
+      },
+    },
+  ],
 })
 
-// 2. Create client with diagnostics enabled
+console.log(metadata)
+```
+
+### Consume One Message At A Time
+
+```ts
+import { KafkaClient } from 'kafka-crab-js'
+
 const client = new KafkaClient({
   brokers: 'localhost:9092',
-  clientId: 'my-app',
-  diagnostics: true, // Required for OTEL to receive events
+  clientId: 'orders-worker',
+  securityProtocol: 'Plaintext',
 })
 
-// 3. For consumers, call endSpan() when processing is complete
-const message = await consumer.recv()
-// ... process message ...
-endSpan(message)
+const consumer = client.createConsumer({
+  groupId: 'orders-worker',
+  enableAutoCommit: false,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+  },
+})
+
+await consumer.subscribe('orders')
+
+try {
+  while (true) {
+    const message = await consumer.recv()
+    if (!message) {
+      break
+    }
+
+    const value = JSON.parse(message.payload.toString('utf8'))
+    console.log({ value, topic: message.topic, partition: message.partition, offset: message.offset })
+
+    await consumer.commitMessage(message, 'Sync')
+  }
+} finally {
+  await consumer.disconnect()
+}
 ```
 
-#### Key Changes
+### Consume In Batches
 
-| v2.x | v3.x |
-|------|------|
-| `otel` config in KafkaClient | `enableOtelInstrumentation()` from `kafka-crab-js-otel` |
-| Automatic span ending | Call `endSpan(message)` manually for consumers |
-| N/A | `diagnostics: true` required in KafkaClient config |
-| OTEL bundled | Install `kafka-crab-js-otel` separately |
+```ts
+const consumer = client.createConsumer({
+  groupId: 'orders-batch-worker',
+  enableAutoCommit: false,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+  },
+})
+
+await consumer.subscribe('orders')
 
-#### New Package Structure
+try {
+  while (true) {
+    const batch = await consumer.recvBatch(500, 50)
+    if (batch.length === 0) {
+      continue
+    }
 
-| Package | Description |
-|---------|-------------|
-| `kafka-crab-js` | Core Kafka client (producer, consumer, streams) |
-| `kafka-crab-js-otel` | OpenTelemetry instrumentation (separate install) |
+    for (const message of batch) {
+      await processOrder(message.payload)
+      await consumer.commitMessage(message, 'Async')
+    }
+  }
+} finally {
+  await consumer.disconnect()
+}
+```
 
-### Internal Improvements
-- Uses Node.js `diagnostics_channel` for observability (zero overhead when not subscribed)
-- Cleaner separation of concerns between core and observability
-- Smaller bundle size for users who don't need OTEL
+## Choosing A Consumer API
 
----
+| API                                | Emits                                                        | Best For                                               |
+| ---------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------ |
+| `consumer.recv()`                  | `Message \| null`                                            | Simple sequential workers and explicit control.        |
+| `consumer.recvBatch()`             | `Message[]`                                                  | High-throughput workers that can process chunks.       |
+| `consumer.recvStream()`            | Web `ReadableStream<Message>`                                | Pull-based Web Stream workflows.                       |
+| `consumer.recvBatchStream()`       | Web `ReadableStream<Message[]>`                              | Native batch streaming with explicit consumer control. |
+| `client.createStreamConsumer()`    | Node.js `Readable` emitting `Message`                        | Existing Node stream pipelines.                        |
+| `client.createWebStreamConsumer()` | Web `ReadableStream<Message>` or `ReadableStream<Message[]>` | v4 high-throughput stream consumers.                   |
 
-## Previous Versions
+Two details matter for choosing correctly:
 
-<details>
-<summary>What's New in Version 2.1.0</summary>
+- `createStreamConsumer()` is a Node.js `Readable` compatibility API. Even when `batchSize > 1`, it emits individual
+  `Message` objects and uses batching internally to reduce native boundary crossings.
+- `createWebStreamConsumer()` returns a discriminated object. In serial mode it emits `Message`; in batch mode it emits
+  `Message[]`. Use this API when your code can process whole batches directly.
 
-### New Features
+## Producer API
 
-1. **Simplified Message Commits with `commitMessage()`**:
-   - New convenience method that accepts a message and commit mode directly
-   - Automatically handles `offset + 1` increment internally - no more manual offset arithmetic
-   - Available on both `KafkaConsumer` and stream consumers
-   - **Before** (v2.0.0):
-     ```javascript
-     const message = await consumer.recv();
-     await consumer.commit(message.topic, message.partition, message.offset + 1, 'Sync');
-     ```
-   - **After** (v2.1.0):
-     ```javascript
-     const message = await consumer.recv();
-     await consumer.commitMessage(message, 'Sync');
-     ```
+Create a producer from a `KafkaClient`:
 
-2. **Enhanced OpenTelemetry Support**:
-   - Improved OTEL context propagation for better distributed tracing
-   - Safe handling when OTEL SDK is not installed (no-op behavior)
-   - Better span context management across producer and consumer operations
-   - Seamless integration with standard OTEL SDK setup
+```ts
+const producer = client.createProducer({
+  autoFlush: true,
+  queueTimeout: 5000,
+  configuration: {
+    'compression.type': 'lz4',
+    acks: 'all',
+  },
+})
+```
 
-3. **CI/CD Improvements**:
-   - Updated to Node.js 24 support
-   - GitHub Actions updated to v6
-   - Improved caching with actions/cache v4
+Send one or more messages:
 
-</details>
+```ts
+const records = await producer.send({
+  topic: 'orders',
+  messages: [
+    { key: Buffer.from('1'), payload: Buffer.from('first') },
+    { key: Buffer.from('2'), payload: Buffer.from('second') },
+  ],
+})
+```
 
-<details>
-<summary>What's New in Version 2.0.0</summary>
+When `autoFlush` is enabled, `send()` waits for delivery confirmation and returns `RecordMetadata[]`. When `autoFlush`
+is disabled, `send()` buffers messages and `flush()` sends pending messages:
 
-### BREAKING CHANGES ⚠️
+```ts
+const producer = client.createProducer({ autoFlush: false })
 
-1. **Consumer Configuration API Changes**:
-   - **REMOVED**: `createTopic` field from `ConsumerConfiguration`
-   - **Migration**: Use `createTopic` field in `TopicPartitionConfig` instead when subscribing to topics
+await producer.send({
+  topic: 'orders',
+  messages: [{ payload: Buffer.from('buffered') }],
+})
 
-2. **Stream Lifecycle Management**:
-   - Stream consumers now properly implement Node.js stream lifecycle methods (`_destroy()`)
-   - **Memory leak prevention**: Streams now automatically disconnect Kafka consumers during destruction
+const records = await producer.flush()
+console.log(records)
+```
 
-3. **Async Consumer Commit**:
-   - **BREAKING**: The `consumer.commit()` method is now async and must be awaited
+`producer.inFlightCount()` returns the number of messages sent but not yet acknowledged.
 
-</details>
+## Consumer API
 
----
+### Subscribe
 
-## Features
+The simplest subscription accepts a topic string:
 
-- 🦀 Simple and intuitive API
-- 🚀 High-performance message processing
-- 🔄 Automatic reconnection handling
-- 🎯 Type-safe interfaces (TypeScript support)
-- ⚡ Async/await support
-- 🛠️ Configurable consumer and producer options
-- 📊 Stream processing support with configurable stream options
-- 📦 Message batching capabilities
-- 🔍 Comprehensive error handling
-- 📈 Performance benchmarking suite
-- 🔧 Flexible configuration system supporting complex data types
+```ts
+await consumer.subscribe('orders')
+```
 
-## Table of Contents
+Use `TopicPartitionConfig[]` when you need offsets, partition-specific offsets, or topic creation:
 
-1. [Installation](#installation)
-2. [Quick Start](#quick-start)
-3. [Consumer Examples](#basic-consumer-setup)
-4. [Producer Examples](#producer-examples)
-5. [Stream Processing](#stream-processing)
-6. [Configuration](#configuration)
-7. [Performance Benchmarks](#performance-benchmarks)
-8. [Best Practices](#best-practices)
-9. [Contributing](#contributing)
-10. [License](#license)
-11. [OpenTelemetry Instrumentation](#opentelemetry-instrumentation)
+```ts
+await consumer.subscribe([
+  {
+    topic: 'orders',
+    createTopic: true,
+    numPartitions: 3,
+    replicas: 1,
+    allOffsets: { position: 'Beginning' },
+  },
+])
+```
 
-## Installation
+For partition-specific offsets:
 
-```bash
-npm install kafka-crab-js
-# or
-yarn add kafka-crab-js
-# or
-pnpm add kafka-crab-js
+```ts
+await consumer.subscribe([
+  {
+    topic: 'orders',
+    partitionOffset: [
+      {
+        partition: 0,
+        offset: { offset: 42 },
+      },
+    ],
+  },
+])
 ```
 
-## Quick Start
+### Receive
 
-### Basic Consumer Setup
-
-```javascript
-import { KafkaClient } from 'kafka-crab-js';
-async function run() {
-  const kafkaClient = new KafkaClient({
-    brokers: 'localhost:29092',
-    clientId: 'foo-client',
-    logLevel: 'debug',
-    brokerAddressFamily: 'v4',
-  });
-
-  // Create consumer with topic creation control
-  const consumer = kafkaClient.createConsumer({
-    groupId: 'foo-group',
-    configuration: {
-      'auto.offset.reset': 'earliest',
-      'enable.auto.commit': false, // Use manual commit for better control
-    }
-  });
+`recv()` waits for one message and returns `null` when the consumer is disconnected:
 
-  // Subscribe with topic creation options
-  await consumer.subscribe([{
-    topic: 'foo',
-    createTopic: true,
-    numPartitions: 3
-  }]);
-
-  const message = await consumer.recv();
-  const { payload, partition, offset, topic } = message;
-  console.log({
-    topic,
-    partition,
-    offset,
-    value: payload.toString()
-  });
-
-  // Manual commit - two options:
-  // Option 1 (v2.1.0+): Simplified with commitMessage
-  await consumer.commitMessage(message, 'Sync');
-  
-  // Option 2: Traditional commit with manual offset increment
-  // await consumer.commit(topic, partition, offset + 1, 'Sync');
-
-  consumer.unsubscribe();
+```ts
+const message = await consumer.recv()
+if (message) {
+  await handleMessage(message)
 }
+```
 
-await run();
+`recvBatch(size, timeoutMs)` returns up to `size` messages:
+
+```ts
+const messages = await consumer.recvBatch(1000, 100)
 ```
 
-### Basic Producer Setup
+An empty array means the timeout elapsed without a full batch. It does not mean the consumer is closed.
 
-```javascript
-import { KafkaClient } from 'kafka-crab-js';
+### Commit
 
-const kafkaClient = new KafkaClient({
-  brokers: 'localhost:29092',
-  clientId: 'my-client-id',
-  logLevel: 'info',
-  brokerAddressFamily: 'v4',
-});
+Use `commitMessage()` for normal message processing. It commits `message.offset + 1`, which is the offset Kafka expects:
 
-// Producer configuration is now optional with sensible defaults
-const producer = kafkaClient.createProducer({
-  configuration: {
-    'message.timeout.ms': 5000,  // Now supports number values
-    'batch.size': 16384,
-    'compression.type': 'snappy'
-  }
-});
-
-const message = {
-  id: 1,
-  name: "Sample Message",
-  timestamp: new Date().toISOString()
-};
-
-const result = await producer.send({
-  topic: 'my-topic',
-  messages: [{
-    payload: Buffer.from(JSON.stringify(message))
-  }]
-});
-
-const errors = result.map(r => r.error).filter(Boolean);
-if (errors.length > 0) {
-  console.error('Error sending message:', errors);
-} else {
-  console.log('Message sent. Offset:', result);
-}
+```ts
+await consumer.commitMessage(message, 'Sync')
 ```
 
-## Stream Processing
-
-### Enhanced Stream Consumer Example
+Use `commit()` when you already calculated the offset:
 
-```javascript
-import { KafkaClient } from 'kafka-crab-js';
+```ts
+await consumer.commit(message.topic, message.partition, message.offset + 1, 'Async')
+```
 
-const kafkaClient = new KafkaClient({
-  brokers: 'localhost:29092',
-  clientId: 'my-client-id',
-  logLevel: 'info',
-  brokerAddressFamily: 'v4',
-});
-
-// Stream consumer with custom ReadableOptions (v2.0.0+)
-const kafkaStream = kafkaClient.createStreamConsumer({
-  groupId: `my-group-id`,
-  enableAutoCommit: true,
-}, {
-  objectMode: true,  // Default in v2.0.0+
-  highWaterMark: 1024,
-  encoding: null
-});
-
-await kafkaStream.subscribe([
-  { topic: 'foo', createTopic: true },
-  { topic: 'bar', createTopic: true }
-]);
-
-kafkaStream.on('data', (message) => {
-  console.log('>>> Message received:', {
-    payload: message.payload.toString(),
-    offset: message.offset,
-    partition: message.partition,
-    topic: message.topic
-  });
-
-  if (message.offset > 10) {
-    kafkaStream.destroy();
-  }
-});
+Commit modes are string literal values:
 
-kafkaStream.on('close', () => {
-  kafkaStream.unsubscribe();
-  console.log('Stream ended');
-});
+```ts
+type CommitMode = 'Sync' | 'Async'
 ```
 
-## Producer Examples
+### Pause, Resume, Seek, And Assignment
 
-### Batch Message Production
+```ts
+consumer.pause()
+consumer.resume()
 
-```javascript
-const kafkaClient = new KafkaClient({
-  brokers: 'localhost:29092',
-  clientId: 'my-client-id',
-  brokerAddressFamily: 'v4',
-});
+consumer.seek('orders', 0, { position: 'Beginning' })
+consumer.seek('orders', 0, { offset: 42 })
 
-// Enhanced producer with flexible configuration
-const producer = kafkaClient.createProducer({
-  configuration: {
-    'batch.size': 50000,      // Number value supported
-    'linger.ms': 10,          // Number value supported
-    'compression.type': 'lz4',
-    'enable.idempotence': true  // Boolean value supported
+const assignment = consumer.assignment()
+const subscription = consumer.getSubscription()
+```
+
+Consumer events are available through `onEvents()`:
+
+```ts
+consumer.onEvents((error, event) => {
+  if (error) {
+    console.error(error)
+    return
   }
-});
 
-const messages = Array.from({ length: 100 }, (_, i) => ({
-  payload: Buffer.from(JSON.stringify({
-    _id: i,
-    name: `Batch Message ${i}`,
-    timestamp: new Date().toISOString()
-  }))
-}));
+  console.log(event.name, event.payload)
+})
+```
+
+### Cleanup
 
+Always disconnect direct consumers:
+
+```ts
 try {
-  const result = await producer.send({
-    topic: 'my-topic',
-    messages
-  });
-  console.log('Batch sent. Offset:', result);
-  console.assert(result.length === 100);
-} catch (error) {
-  console.error('Batch error:', error);
+  await consumer.subscribe('orders')
+  // consume...
+} finally {
+  await consumer.disconnect()
 }
 ```
 
-### Producer with Keys and Headers
+## Stream Consumers
 
-```javascript
-async function produceWithMetadata() {
-  const producer = kafkaClient.createProducer({
-    configuration: {
-      'acks': 'all',
-      'retries': 5,
-      'max.in.flight.requests.per.connection': 1
-    }
-  });
+### Node.js Readable Stream
+
+Use `createStreamConsumer()` when integrating with Node.js stream tooling. It emits `Message` objects.
+
+```ts
+const stream = client.createStreamConsumer({
+  groupId: 'orders-stream',
+  enableAutoCommit: false,
+  batchSize: 256,
+  batchTimeout: 50,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+  },
+})
 
+await stream.subscribe('orders')
+
+stream.on('data', async (message) => {
   try {
-    await producer.send({
-      topic: 'user-events',
-      messages: [{
-        key: 'user-123',
-        payload: Buffer.from(JSON.stringify({
-          userId: 123,
-          action: 'update'
-        })),
-        headers: {
-          'correlation-id': 'txn-123',
-          'source': 'user-service'
-        }
-      }]
-    });
+    await processOrder(message.payload)
+    await stream.commitMessage(message, 'Async')
   } catch (error) {
-    console.error('Error:', error);
+    stream.destroy(error instanceof Error ? error : new Error(String(error)))
   }
-}
+})
+
+stream.on('error', (error) => {
+  console.error('stream error', error)
+})
+
+stream.on('close', () => {
+  console.log('stream closed')
+})
 ```
 
-### Reconnecting Kafka Consumer
+Destroy stream consumers instead of calling `disconnect()` directly. The stream cleanup path cancels the source reader,
+unsubscribes, and disconnects the native consumer:
 
-```javascript
-import { KafkaClient } from 'kafka-crab-js'
+```ts
+stream.destroy()
+```
 
-const kafkaClient = new KafkaClient({
-  brokers: 'localhost:29092',
-  clientId: 'reconnect-test',
-  logLevel: 'debug',
-  brokerAddressFamily: 'v4',
+### Native Web Stream
+
+Use `createWebStreamConsumer()` when you want a Web Stream and clear serial/batch typing.
+
+Serial mode is selected when `batchSize` is omitted, `0`, or `1`:
+
+```ts
+const webConsumer = client.createWebStreamConsumer({
+  groupId: 'orders-web-serial',
+  serialPrefetchSize: 64,
+  serialPrefetchTimeout: 1,
   configuration: {
     'auto.offset.reset': 'earliest',
-    'session.timeout.ms': 30000,
-    'heartbeat.interval.ms': 10000
   },
 })
 
-/**
- * Creates and configures a new Kafka stream consumer
- */
-async function createConsumer() {
-  const kafkaStream = kafkaClient.createStreamConsumer({
-    groupId: 'reconnect-test',
-    enableAutoCommit: true,
-  }, {
-    highWaterMark: 100,
-    objectMode: true
-  })
-
-  await kafkaStream.subscribe([
-    { topic: 'foo', createTopic: true },
-    { topic: 'bar', createTopic: true },
-  ])
-  return kafkaStream
-}
+await webConsumer.consumer.subscribe('orders')
 
-/**
- * Starts a Kafka consumer with auto-restart capability
- */
-async function startConsumer() {
-  let counter = 0
-  let retryCount = 0
-  const MAX_RETRIES = 5
-  const RETRY_DELAY = 5000 // 5 seconds
-
-  async function handleRetry() {
-    if (retryCount < MAX_RETRIES) {
-      retryCount++
-      console.log(
-        `Attempting to restart consumer (attempt ${retryCount}/${MAX_RETRIES}) in ${RETRY_DELAY / 1000} seconds...`,
-      )
-      setTimeout(setupConsumerWithRetry, RETRY_DELAY)
-    } else {
-      console.error(`Maximum retry attempts (${MAX_RETRIES}) reached. Stopping consumer.`)
-      process.exit(1)
-    }
-  }
+const reader = webConsumer.stream.getReader()
 
-  async function setupConsumerWithRetry() {
-    try {
-      const kafkaStream = await createConsumer()
-      retryCount = 0 // Reset retry count on successful connection
-
-      console.log('Starting consumer')
-
-      kafkaStream.on('data', (message) => {
-        counter++
-        console.log('>>> Message received:', {
-          counter,
-          payload: message.payload.toString(),
-          offset: message.offset,
-          partition: message.partition,
-          topic: message.topic,
-        })
-      })
-
-      kafkaStream.on('error', async (error) => {
-        console.error('Stream error:', error)
-        handleRetry()
-      })
-
-      kafkaStream.on('close', () => {
-        console.log('Stream ended')
-        try {
-          kafkaStream.unsubscribe()
-        } catch (unsubError) {
-          console.error('Error unsubscribing:', unsubError)
-        }
-      })
-    } catch (error) {
-      console.error('Error setting up consumer:', error)
-      handleRetry()
-    }
+try {
+  const { value: message, done } = await reader.read()
+  if (!done && message) {
+    await processOrder(message.payload)
+    await webConsumer.consumer.commitMessage(message, 'Async')
   }
-
-  await setupConsumerWithRetry()
+} finally {
+  await reader.cancel()
+  await webConsumer.consumer.disconnect()
 }
-
-await startConsumer()
 ```
 
-### Examples
+Batch mode is selected with `batchSize > 1` and emits `Message[]` chunks:
+
+```ts
+const webConsumer = client.createWebStreamConsumer({
+  groupId: 'orders-web-batch',
+  batchSize: 1024,
+  batchTimeout: 10,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+  },
+})
 
-You can find some examples on the [example](https://github.com/flash-tecnologia/kafka-crab-js/tree/main/example) folder of this project.
+if (webConsumer.mode === 'batch') {
+  await webConsumer.consumer.subscribe('orders')
+  const reader = webConsumer.stream.getReader()
+
+  try {
+    const { value: batch, done } = await reader.read()
+    if (!done && batch) {
+      for (const message of batch) {
+        await processOrder(message.payload)
+      }
+    }
+  } finally {
+    await reader.cancel()
+    await webConsumer.consumer.disconnect()
+  }
+}
+```
 
 ## Configuration
 
-### KafkaConfiguration
+### KafkaClient
+
+```ts
+const client = new KafkaClient({
+  brokers: 'localhost:9092',
+  clientId: 'orders-service',
+  securityProtocol: 'Plaintext',
+  logLevel: 'info',
+  brokerAddressFamily: 'v4',
+  diagnostics: true,
+  configuration: {
+    'socket.keepalive.enable': true,
+  },
+})
+```
 
-| Property | Type | Default | Description |
-| --- | --- | --- | --- |
-| `brokers` | `string` || List of brokers to connect to |
-| `clientId` | `string` | `"rdkafka"` | Client id to use for the connection |
-| `securityProtocol` | `SecurityProtocol` || Security protocol to use (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL) |
-| `logLevel` | `string` | `info`  | Log level for the client |
-| `brokerAddressFamily` | `string` | `"v4"` | Address family to use for the connection (v4, v6) |
-| `configuration` | `Record<string, any>` | `{}` | Additional configuration options for the client |
-| `diagnostics` | `boolean` | `false` | **v3.0.0+**: Enable diagnostics channel for OTEL instrumentation |
+| Option                | Type                                                   | Default   | Description                                                               |
+| --------------------- | ------------------------------------------------------ | --------- | ------------------------------------------------------------------------- |
+| `brokers`             | `string`                                               | required  | Comma-separated broker list, for example `localhost:9092,localhost:9093`. |
+| `clientId`            | `string`                                               | `rdkafka` | Client identifier sent to Kafka.                                          |
+| `securityProtocol`    | `'Plaintext' \| 'Ssl' \| 'SaslPlaintext' \| 'SaslSsl'` |           | Security protocol.                                                        |
+| `logLevel`            | `string`                                               |           | librdkafka log level.                                                     |
+| `brokerAddressFamily` | `string`                                               |           | Address family hint such as `v4`.                                         |
+| `diagnostics`         | `boolean`                                              | `true`    | Enables diagnostic-channel events used by `kafka-crab-js-otel`.           |
+| `configuration`       | `Record<string, any>`                                  |           | Additional librdkafka client settings.                                    |
 
 ### ConsumerConfiguration
 
-| Property | Type | Default | Description |
-| --- | --- | --- | --- |
-| `groupId` | `string` || Consumer group ID |
-| `enableAutoCommit` | `boolean` | `true` | Enable automatic offset commits |
-| `configuration` | `Record<string, any>` | `{}` | Additional consumer configuration options |
-| `fetchMetadataTimeout` | `number` | `60000` | Timeout for fetching metadata (ms) |
-| `maxBatchMessages` | `number` | `1000` | Maximum messages in a batch operation |
+```ts
+const consumer = client.createConsumer({
+  groupId: 'orders-worker',
+  enableAutoCommit: false,
+  fetchMetadataTimeout: 5000,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+    'enable.auto.commit': false,
+    'fetch.min.bytes': 1,
+  },
+})
+```
 
-### Consumer Commit Methods
+| Option                 | Type                  | Description                                |
+| ---------------------- | --------------------- | ------------------------------------------ |
+| `groupId`              | `string`              | Kafka consumer group id.                   |
+| `enableAutoCommit`     | `boolean`             | Convenience flag for auto commit behavior. |
+| `fetchMetadataTimeout` | `number`              | Metadata fetch timeout in milliseconds.    |
+| `configuration`        | `Record<string, any>` | Additional librdkafka consumer settings.   |
 
-kafka-crab-js provides two methods for committing offsets:
+### ProducerConfiguration
 
-| Method | Signature | Description |
-| --- | --- | --- |
-| `commit` | `commit(topic, partition, offset, mode)` | Traditional commit - you must calculate `offset + 1` |
-| `commitMessage` | `commitMessage(message, mode)` | **v2.1.0+**: Simplified commit - automatically handles offset increment |
+```ts
+const producer = client.createProducer({
+  autoFlush: true,
+  queueTimeout: 5000,
+  configuration: {
+    'compression.type': 'lz4',
+  },
+})
+```
 
-```javascript
-// Using commitMessage (recommended for v2.1.0+)
-const message = await consumer.recv();
-await consumer.commitMessage(message, 'Sync');
+| Option          | Type                  | Description                                             |
+| --------------- | --------------------- | ------------------------------------------------------- |
+| `autoFlush`     | `boolean`             | When enabled, `send()` waits for delivery confirmation. |
+| `queueTimeout`  | `number`              | Queue timeout in milliseconds.                          |
+| `configuration` | `Record<string, any>` | Additional librdkafka producer settings.                |
 
-// Using commit (traditional)
-const message = await consumer.recv();
-await consumer.commit(message.topic, message.partition, message.offset + 1, 'Sync');
-```
+### TopicPartitionConfig
 
-Both methods support `'Sync'` and `'Async'` commit modes.
+| Option            | Type                | Description                                      |
+| ----------------- | ------------------- | ------------------------------------------------ |
+| `topic`           | `string`            | Topic name.                                      |
+| `allOffsets`      | `OffsetModel`       | Offset model applied to all assigned partitions. |
+| `partitionOffset` | `PartitionOffset[]` | Partition-specific offset model.                 |
+| `createTopic`     | `boolean`           | Create the topic before subscribing.             |
+| `numPartitions`   | `number`            | Partition count when creating a topic.           |
+| `replicas`        | `number`            | Replica count when creating a topic.             |
 
-### ProducerConfiguration
+Offset models:
 
-| Property | Type | Default | Description |
-| --- | --- | --- | --- |
-| `queueTimeout` | `number` | `5000` | Queue timeout in milliseconds |
-| `autoFlush` | `boolean` | `true` | Enable automatic message flushing |
-| `configuration` | `Record<string, any>` | `{}` | Additional producer configuration options |
+```ts
+{
+  position: 'Beginning'
+}
+{
+  position: 'End'
+}
+{
+  position: 'Stored'
+}
+{
+  position: 'Invalid'
+}
+{
+  offset: 42
+}
+```
 
-### TopicPartitionConfig
+### Type-Only Literals
+
+In v4, these names are TypeScript-only exports:
+
+```ts
+import type { CommitMode, KafkaEventName, PartitionPosition, SecurityProtocol } from 'kafka-crab-js'
+```
 
-| Property | Type | Default | Description |
-| --- | --- | --- | --- |
-| `topic` | `string` || Topic name |
-| `allOffsets` | `OffsetModel` || Offset configuration for all partitions |
-| `partitionOffset` | `Array<PartitionOffset>` || Per-partition offset configuration |
-| `createTopic` | `boolean` | `false` | **v2.0.0+**: Create topic if it doesn't exist |
-| `numPartitions` | `number` | `1` | **v2.0.0+**: Number of partitions when creating topic |
-| `replicas` | `number` | `1` | **v2.0.0+**: Number of replicas when creating topic |
+Use literal values at runtime:
 
-You can see the available options here: [librdkafka](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_CONFIGURATION.html).
+```ts
+const commitMode: CommitMode = 'Sync'
+const securityProtocol: SecurityProtocol = 'Plaintext'
+```
 
-### OpenTelemetry Instrumentation
+## OpenTelemetry
 
-> **Note:** Starting with v3.0.0, OpenTelemetry instrumentation has been moved to a separate package: [`kafka-crab-js-otel`](https://www.npmjs.com/package/kafka-crab-js-otel)
+OpenTelemetry support lives in the separate `kafka-crab-js-otel` package. The core package emits diagnostic-channel
+events when `diagnostics !== false`; the OTEL package subscribes to those events.
 
-#### Installation
+Install the instrumentation package and OpenTelemetry dependencies:
 
 ```bash
-npm install kafka-crab-js-otel @opentelemetry/api
+npm install kafka-crab-js-otel @opentelemetry/api @opentelemetry/sdk-node
 ```
 
-#### Usage
+Enable instrumentation before creating `KafkaClient`:
 
-```javascript
+```ts
 import { KafkaClient } from 'kafka-crab-js'
 import { enableOtelInstrumentation, endSpan } from 'kafka-crab-js-otel'
 
-// Enable instrumentation before creating client
-// Note: serviceName is configured via OTEL SDK Resource
 enableOtelInstrumentation({
   metrics: { enabled: true },
 })
 
-// Create client with diagnostics enabled
 const client = new KafkaClient({
   brokers: 'localhost:9092',
-  clientId: 'my-app',
+  clientId: 'orders-worker',
   diagnostics: true,
 })
 
-// For consumers, call endSpan() when done processing
+const consumer = client.createConsumer({ groupId: 'orders-worker' })
+await consumer.subscribe('orders')
+
 const message = await consumer.recv()
-// ... process message ...
-endSpan(message)
+if (message) {
+  try {
+    await processOrder(message.payload)
+  } finally {
+    endSpan(message)
+  }
+}
 ```
 
-For complete configuration options and examples, see the [kafka-crab-js-otel README](../kafka-crab-js-otel/README.md).
-
-#### Examples
-
-See comprehensive examples in the `example/` directory:
-- `example/otel-tracing-example.mjs` - Complete tracing setup with Jaeger
-- `example/otel-metrics-example.mjs` - Metrics collection with Prometheus
-- `example/README.md` - Full documentation for all examples
+See the [OpenTelemetry package README](../otel/README.md) for full tracing and metrics setup.
 
 ## Performance Benchmarks
 
-### Running Benchmarks
+The repository includes a benchmark suite that compares `kafka-crab-js` with KafkaJS and `@platformatic/kafka`.
 
-kafka-crab-js v2.0.0+ includes a comprehensive benchmark suite to compare performance against other popular Kafka clients:
+From the repository root:
 
 ```bash
-# Set up benchmark environment (requires Kafka running locally)
-pnpm add -D tsx  # For running TypeScript files directly
-npx tsx benchmark/utils/setup.ts
+cd packages/benchmark
+
+# Set up benchmark data. Requires Kafka running locally.
+vp install
+vp run setup:consumer
 
-# Run consumer performance benchmarks
-node benchmark/consumer.ts
+# Run the default isolated-process memory benchmark.
+vp run benchmark
 ```
 
-### Benchmark Results
+The default benchmark runs one child Node.js process per selected scenario. This avoids carrying V8 heap pages, native
+allocator arenas, sockets, librdkafka state, and Kafka metadata from one client into the next client's measurement.
 
-*Benchmarks run on macOS with Apple M1 chip processing 50,000 messages (December 2024)*
+### Consumer Benchmark Snapshot
 
-```
-╔════════════════════════╤═════════╤══════════════════╤═══════════╤══════════════════════════╗
-║ Slower tests           │ Samples │           Result │ Tolerance │ Difference with previous ║
-╟────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
-║ kafkajs                │   50000 │    834.12 op/sec │ ±  0.22 % │                          ║
-║ node-rdkafka (evented) │   84115 │  24922.67 op/sec │ ± 74.82 % │ + 2887.91 %              ║
-║ kafka-crab-js (serial) │   50000 │  43213.86 op/sec │ ±  3.46 % │ + 73.39 %                ║
-║ node-rdkafka (stream)  │   50000 │  49805.32 op/sec │ ± 27.10 % │ + 15.25 %                ║
-╟────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
-║ Fastest test           │ Samples │           Result │ Tolerance │ Difference with previous ║
-╟────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
-║ kafka-crab-js (batch)  │   50000 │ 205985.31 op/sec │ ± 16.53 % │ + 313.58 %               ║
-╚════════════════════════╧═════════╧══════════════════╧═══════════╧══════════════════════════╝
+The captured run below is equivalent to:
+
+```bash
+cd packages/benchmark
+BENCHMARK_BATCH_SIZE=300000 BENCHMARK_BATCH_TIMEOUT_MS=2 vp run benchmark
 ```
 
-The benchmark suite compares:
-- **kafka-crab-js (serial)**: Single message processing - **43,214 ops/sec**
-- **kafka-crab-js (batch)**: Batch message processing - **205,985 ops/sec** (fastest)
-- **node-rdkafka (evented)**: Event-based processing - **24,923 ops/sec**
-- **node-rdkafka (stream)**: Stream-based processing - **49,805 ops/sec**
-- **kafkajs**: Official KafkaJS client - **834 ops/sec**
+Batch scenarios were normalized to the comparable effective batch size of `16384`.
 
-Performance characteristics:
-- **52x faster than kafkajs** in serial mode, **247x faster in batch mode**
-- **High throughput**: Batch processing provides 4.8x performance improvement over serial mode
-- **Low latency**: Optimized for both single and batch message processing
-- **Memory efficient**: Lock-free data structures minimize memory overhead
-- **Concurrent processing**: Zero-contention concurrent operations
+![Consumer benchmark snapshot](./assets/consumer-benchmark-snapshot.svg)
 
-### Benchmark Configuration
+#### Throughput
 
-You can customize benchmark parameters in `benchmark/utils/definitions.ts`:
+| Rank | Scenario                            |                Result | Relative |
+| ---: | ----------------------------------- | --------------------: | -------: |
+|    1 | `kafka-crab-js v4 (stream, batch)`  | `1,089,284.97 op/sec` | `100.0%` |
+|    2 | `@platformatic/kafka`               |   `732,207.27 op/sec` |  `67.2%` |
+|    3 | `KafkaJS (eachBatch)`               |   `676,576.98 op/sec` |  `62.1%` |
+|    4 | `kafka-crab-js v4 (stream, serial)` |   `544,658.33 op/sec` |  `50.0%` |
+|    5 | `KafkaJS (eachMessage, concurrent)` |   `509,082.43 op/sec` |  `46.7%` |
+|    6 | `KafkaJS (eachMessage)`             |   `505,815.57 op/sec` |  `46.4%` |
 
-```typescript
-export const topic = 'benchmarks'
-export const brokers = ['localhost:9092', 'localhost:9093', 'localhost:9094']
+`kafka-crab-js v4 (stream, batch)` is the fastest scenario in this run. It is about `49%` faster than
+`@platformatic/kafka` and about `61%` faster than `KafkaJS (eachBatch)`. Its tolerance is higher than the other top
+rows (`+/- 9.15%`), so the exact gap should be treated as a benchmark snapshot rather than a universal constant.
 
-// Benchmark parameters can be adjusted in consumer.ts:
-const iterations = 10_000  // Number of messages to process
-const maxBytes = 200       // Maximum message size
-```
+For message-by-message consumption, `kafka-crab-js v4 (stream, serial)` is close to KafkaJS throughput while using much
+less lifecycle memory. In this run it is about `7.7%` faster than `KafkaJS (eachMessage)` and about `7.0%` faster than
+the concurrent KafkaJS `eachMessage` scenario.
 
-## Best Practices
+#### Memory And GC
 
-### Error Handling
-- Always wrap async operations in try-catch blocks
-- Implement proper error logging and monitoring
-- Handle both operational and programming errors separately
+| Scenario                            |   RSS delta |  Peak heap |    GC time | GC share | Notes                                           |
+| ----------------------------------- | ----------: | ---------: | ---------: | -------: | ----------------------------------------------- |
+| `kafka-crab-js v4 (stream, serial)` |  `72.3 MiB` | `11.7 MiB` | `36.75 ms` |  `4.00%` | Best memory efficiency.                         |
+| `kafka-crab-js v4 (stream, batch)`  | `167.0 MiB` | `35.9 MiB` | `30.60 ms` |  `6.67%` | Best throughput and lowest GC time.             |
+| `KafkaJS (eachBatch)`               | `193.1 MiB` | `70.0 MiB` | `63.03 ms` |  `8.53%` | Strong batch baseline.                          |
+| `@platformatic/kafka`               | `217.6 MiB` | `93.7 MiB` | `73.62 ms` | `10.78%` | Fastest non-crab competitor.                    |
+| `KafkaJS (eachMessage)`             | `181.5 MiB` | `77.0 MiB` | `79.11 ms` |  `8.00%` | Similar throughput to v4 serial, higher memory. |
+| `KafkaJS (eachMessage, concurrent)` | `188.2 MiB` | `75.6 MiB` | `77.17 ms` |  `7.86%` | Concurrency did not improve this workload.      |
 
-### Performance
-- Use batch operations for high-throughput scenarios
-- Configure appropriate batch sizes and compression settings
-- Monitor and tune consumer group performance
-- Leverage the benchmark suite to optimize your specific use case
+Lifecycle memory includes module loading, client creation, subscription, consumption, cleanup, and retained RSS after
+cleanup. GC metrics use the narrower first-to-last-message window used for throughput.
 
-### Configuration (v2.0.0+)
-- Use the flexible configuration system with proper data types:
-  ```javascript
-  const config = {
-    'batch.size': 16384,           // number
-    'compression.type': 'snappy',  // string
-    'enable.idempotence': true,    // boolean
-    'retries': 5                   // number
-  }
-  ```
-
-### Message Processing
-- Validate message formats before processing
-- Implement proper serialization/deserialization
-- Handle message ordering when required
-- Use topic creation options for better topic management
+The most important memory result is efficiency, not just raw RSS. `kafka-crab-js v4 (stream, serial)` delivered
+`7529 op/sec/MiB`, the best score in the run. `kafka-crab-js v4 (stream, batch)` delivered `6521 op/sec/MiB`, ahead of
+`KafkaJS (eachBatch)` at `3504 op/sec/MiB` and `@platformatic/kafka` at `3365 op/sec/MiB`.
 
-### Stream Processing (v2.0.0+)
-- Configure appropriate `ReadableOptions` for your use case
-- Use `objectMode: true` for structured message processing
-- Set appropriate `highWaterMark` based on memory constraints
+### Benchmark Interpretation
 
-## Contributing
+- Use `kafka-crab-js v4 (stream, batch)` when raw throughput is the priority.
+- Use `kafka-crab-js v4 (stream, serial)` when memory efficiency and low heap pressure matter most.
+- Compare batch scenarios with batch scenarios and message scenarios with message scenarios.
+- `@platformatic/kafka` is the strongest non-crab throughput competitor in this run, but it used more RSS, heap, and GC
+  time than the v4 batch scenario.
+- KafkaJS `eachMessage` concurrency did not help this workload.
 
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/amazing-feature`)
-3. Run the benchmark suite to ensure performance isn't degraded
-4. Commit your changes (`git commit -m 'Add some amazing feature'`)
-5. Push to the branch (`git push origin feature/amazing-feature`)
-6. Open a Pull Request
+### Benchmark Configuration
 
-### Development Commands
+Common knobs:
 
 ```bash
-# Build the project
-pnpm build
+BENCHMARK_ITERATIONS=100000
+BENCHMARK_RUNS=5
+BENCHMARK_BATCH_SIZE=4096
+BENCHMARK_BATCH_TIMEOUT_MS=2
+BENCHMARK_MAX_BYTES=2048
+BENCHMARK_MEMORY=1
+```
+
+See the [benchmark README](../benchmark/README.md) for the full benchmark methodology and environment variables.
 
-# Run type checking
-pnpm typecheck
+## Migration Notes
 
-# Run linting
-pnpm lint
+### v4 Type-Only Runtime Exports
 
-# Format code
-pnpm fmt
+`CommitMode`, `KafkaEventName`, `PartitionPosition`, and `SecurityProtocol` are no longer runtime exports from
+`kafka-crab-js`.
 
-# Run benchmarks
-npx tsx benchmark/utils/setup.ts
-npx tsx benchmark/consumer.ts
+Before:
+
+```ts
+import { CommitMode, SecurityProtocol } from 'kafka-crab-js'
 ```
 
-## OpenTelemetry Instrumentation
+After:
 
-> **Note:** Starting with v3.0.0, OpenTelemetry instrumentation has been moved to a separate package: [`kafka-crab-js-otel`](https://www.npmjs.com/package/kafka-crab-js-otel)
+```ts
+import type { CommitMode, SecurityProtocol } from 'kafka-crab-js'
 
-Kafka Crab JS offers turnkey tracing for Kafka workloads:
+const commitMode: CommitMode = 'Sync'
+const securityProtocol: SecurityProtocol = 'Plaintext'
+```
 
-- **Seamless propagation** – Producer instrumentation injects `traceparent`/`tracestate` into Kafka headers while retaining any existing headers (including `Buffer` values) so downstream systems continue to see custom metadata.
-- **Consumer & stream coverage** – Standard consumers, batch consumers, and `createStreamConsumer` streams emit spans that include consumer group, topic, partition, offset, and batch size semantics.
-- **Hook-friendly spans** – Both `messageHook` and `producerHook` callbacks run inside the active span context, simplifying attribute decoration or error handling.
-- **Zero overhead when disabled** – Uses Node.js `diagnostics_channel` for near-zero cost when OTEL is not active.
+### v3 OpenTelemetry Split
 
-### Consumer span lifecycle (important)
+OpenTelemetry instrumentation moved from the core package to `kafka-crab-js-otel` in v3. This keeps the core package
+small and makes tracing/metrics opt-in.
 
-Kafka Crab JS creates consumer `process <topic>` spans, but only your application knows when processing is complete.
+## Troubleshooting
 
-- For single-message consumers, call `endSpan(message)` when you're done processing the message.
-- For batch consumers, call `endSpan(batch)` when you're done processing the batch.
-- The `endSpan()` helper is exported from `kafka-crab-js-otel`.
+### The Consumer Does Not Receive Existing Messages
 
-This closes the span(s) and (when metrics are enabled) records `messaging.process.duration`.
+Use an offset reset policy and a fresh consumer group:
 
-### Global instrumentation (singleton)
+```ts
+const consumer = client.createConsumer({
+  groupId: `orders-worker-${Date.now()}`,
+  configuration: {
+    'auto.offset.reset': 'earliest',
+  },
+})
+```
 
-`kafka-crab-js-otel` uses a process-wide OpenTelemetry instrumentation singleton.
+Or subscribe with an explicit starting position:
 
-- Call `enableOtelInstrumentation()` **before** creating any `KafkaClient` instances.
-- Creating multiple `KafkaClient` instances shares the same instrumentation.
-- For tests, use `resetKafkaInstrumentation()` to clear the singleton between runs.
+```ts
+await consumer.subscribe([{ topic: 'orders', allOffsets: { position: 'Beginning' } }])
+```
 
-### Minimal Setup Example
+### The Process Hangs On Shutdown
 
-When using stream consumers, the easiest pattern is calling `endSpan()` in a `finally` block:
+Direct consumers should call `disconnect()`:
 
 ```ts
-import { KafkaClient } from 'kafka-crab-js'
-import { enableOtelInstrumentation, endSpan } from 'kafka-crab-js-otel'
-import { NodeTracerProvider } from '@opentelemetry/sdk-trace-node'
-import { SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base'
-import { AsyncHooksContextManager } from '@opentelemetry/context-async-hooks'
-import { context } from '@opentelemetry/api'
-
-// 1. Set up OpenTelemetry SDK
-const provider = new NodeTracerProvider()
-provider.addSpanProcessor(new SimpleSpanProcessor(exporter))
-provider.register()
-context.setGlobalContextManager(new AsyncHooksContextManager().enable())
-
-// 2. Enable kafka-crab-js instrumentation BEFORE creating client
-enableOtelInstrumentation({
-  captureMessagePayload: true,
-  captureMessageHeaders: true,
-  producerHook: span => span.setAttribute('messaging.client.kind', 'producer'),
-})
+await consumer.disconnect()
+```
 
-// 3. Create client with diagnostics enabled
-const client = new KafkaClient({
-  brokers: 'localhost:29092',
-  clientId: 'orders-api',
-  diagnostics: true, // Required for OTEL to receive events
-})
+Node stream consumers should call `destroy()`:
 
-// 4. Use producer - spans are created automatically
-const producer = client.createProducer()
-await producer.send({
-  topic: 'orders',
-  messages: [{
-    payload: Buffer.from(JSON.stringify({ orderId: '123' })),
-    headers: { 'custom-header': 'foo' },
-  }],
-})
+```ts
+stream.destroy()
+```
 
-// 5. Use stream consumer with proper cleanup
-const consumer = client.createStreamConsumer({
-  groupId: 'orders-consumer',
-  enableAutoCommit: false,
-})
+Web stream consumers should cancel the reader and disconnect the underlying consumer:
 
-await consumer.subscribe('orders')
+```ts
+await reader.cancel()
+await webConsumer.consumer.disconnect()
+```
 
-consumer.on('data', message => {
-  try {
-    console.log(message.headers?.['custom-header']?.toString())
-  } finally {
-    // IMPORTANT: close the "process <topic>" span when your app finishes handling the message
-    endSpan(message)
-  }
+### Localhost Resolves To IPv6 But Kafka Listens On IPv4
+
+Set `brokerAddressFamily: 'v4'`:
+
+```ts
+const client = new KafkaClient({
+  brokers: 'localhost:9092',
+  brokerAddressFamily: 'v4',
 })
+```
+
+### Need Advanced Kafka Settings
 
-// Proper cleanup for stream consumers - use destroy() not disconnect()
-consumer.on('close', () => console.log('Consumer closed'))
-// consumer.destroy()
+Pass librdkafka settings through `configuration`:
+
+```ts
+const client = new KafkaClient({
+  brokers: 'localhost:9092',
+  configuration: {
+    'socket.keepalive.enable': true,
+    'metadata.max.age.ms': 300000,
+  },
+})
 ```
 
-To disable OTEL, simply don't call `enableOtelInstrumentation()` or set `diagnostics: false` in the client config.
+Consumer and producer configuration objects also accept their own `configuration` maps.
 
-## License
+## Development
 
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+From the repository root:
 
----
+```bash
+vp install
+vp run build
+vp run test
+vp run test:integration
+vp check
+```
 
-<div align="center">
+Useful package-local commands:
 
-[![Built with Rust](https://img.shields.io/badge/Built%20with-Rust-orange)](https://www.rust-lang.org/)
+```bash
+vp run --filter kafka-crab-js build
+vp run --filter kafka-crab-js test
+vp run --filter kafka-crab-js test:integration
+```
+
+Integration tests require Kafka at `localhost:9092` unless `KAFKA_BROKERS` is set.
+
+## License
 
-</div>
+MIT