@autofleet/kafka 0.1.1 → 0.2.0

package/README.md

@@ -7,6 +7,11 @@ Internal wrapper for Apache Kafka producer using [@platformatic/kafka](https://w
 - [Installation](#installation)
 - [Features](#features)
 - [Quick Start](#quick-start)
+- [Initialization & Bootstrap](#initialization--bootstrap)
+  - [Lifecycle Overview](#lifecycle-overview)
+  - [Production Bootstrap Pattern](#production-bootstrap-pattern)
+  - [Development Bootstrap Pattern](#development-bootstrap-pattern)
+  - [Bootstrap Options](#bootstrap-options)
 - [Usage](#usage)
   - [Setup with Multiple Producers](#setup-with-multiple-producers)
   - [Publishing Messages](#publishing-messages)
@@ -20,6 +25,7 @@ Internal wrapper for Apache Kafka producer using [@platformatic/kafka](https://w
   - [Partition Control](#partition-control)
 - [API Reference](#api-reference)
 - [Best Practices](#best-practices)
+- [Troubleshooting](#troubleshooting)
 - [Testing](#testing)
 
 ## Installation
@@ -91,6 +97,177 @@ app.get('/health/ready', async (req, res) => {
 });
 ```
 
+## Initialization & Bootstrap
+
+### Lifecycle Overview
+
+The `@autofleet/kafka` package follows a clear, predictable lifecycle:
+
+```
+┌─────────────┐     ┌──────────────┐     ┌──────────────┐     ┌────────────┐
+│   create()  │────▶│ bootstrap()  │────▶│ publish/use  │────▶│disconnect()│
+│ (sync)      │     │ (async)      │     │ (async)      │     │ (async)    │
+└─────────────┘     └──────────────┘     └──────────────┘     └────────────┘
+    Instant              Validates            Runtime           Cleanup
+  construction         connectivity           operations
+```
+
+**1. `create()` - Synchronous Construction**
+- Creates the KafkaManager instance
+- Sets up configuration
+- Does NOT connect to Kafka yet
+- Returns immediately
+
+**2. `bootstrap()` - Async Initialization**
+- Explicitly connects all producers to Kafka
+- Validates broker connectivity
+- Fetches cluster metadata
+- Fails fast with detailed diagnostics
+- **Recommended before serving traffic**
+
+**3. Runtime Operations**
+- `publish()`, `publishBatch()` work normally
+- Lazy initialization also supported (backwards compatible)
+- Health checks available
+
+**4. `disconnect()` - Cleanup**
+- Gracefully closes all connections
+- Automatic on SIGTERM/SIGINT
+
+### Production Bootstrap Pattern
+
+**Recommended production service initialization:**
+
+```typescript
+import { KafkaManager } from '@autofleet/kafka';
+import logger from './logger';
+
+// 1. Synchronous construction
+export const kafka = KafkaManager.create({
+  enabled: process.env.ENABLE_KAFKA === 'true',
+  logger,
+  bootstrapTimeoutMs: 15000,  // 15s bootstrap timeout
+  healthCheckCacheMs: 2000,   // Cache health for 2s
+  strictBootstrap: true,      // Fail if ANY producer fails
+  producers: {
+    main: {
+      brokers: process.env.KAFKA_BROKERS?.split(',') || [],
+      clientId: 'my-service-main',
+    },
+    analytics: {
+      brokers: process.env.ANALYTICS_BROKERS?.split(',') || [],
+      clientId: 'my-service-analytics',
+    },
+  },
+});
+
+// 2. Bootstrap before serving traffic
+async function startServer() {
+  logger.info('Bootstrapping Kafka...');
+
+  try {
+    const result = await kafka.bootstrap();
+
+    logger.info('Kafka bootstrap successful', {
+      duration: result.duration,
+      producers: Object.keys(result.results),
+    });
+  } catch (error) {
+    logger.error('Fatal: Kafka bootstrap failed', { error });
+    process.exit(1); // Fail fast in production
+  }
+
+  // 3. Start HTTP server only after Kafka is ready
+  app.listen(3000, () => {
+    logger.info('Server ready - Kafka connected');
+  });
+}
+
+startServer();
+```
+
+### Development Bootstrap Pattern
+
+**For development/local environments with optional Kafka:**
+
+```typescript
+async function startServer() {
+  // Try to bootstrap, but don't fail if Kafka unavailable
+  try {
+    await kafka.bootstrap({
+      timeoutMs: 5000,  // Shorter timeout for dev
+      strict: false,    // Allow partial failures
+    });
+    logger.info('Kafka connected');
+  } catch (error) {
+    logger.warn('Kafka unavailable - using lazy initialization', { error });
+    // Server still starts, producers will retry on first publish
+  }
+
+  app.listen(3000, () => {
+    logger.info('Server ready');
+  });
+}
+```
+
+### Bootstrap Options
+
+#### Strict Mode (Default)
+
+Bootstrap fails if **any** producer fails to connect:
+
+```typescript
+await kafka.bootstrap(); // strict: true by default
+
+// Throws if any producer fails
+// Perfect for production where all clusters must be available
+```
+
+#### Non-Strict Mode
+
+Bootstrap succeeds if **at least one** producer connects:
+
+```typescript
+const result = await kafka.bootstrap({
+  strict: false,
+});
+
+if (!result.success) {
+  logger.warn('Some producers failed:', result.results);
+}
+
+// Check which ones failed
+const failed = Object.entries(result.results)
+  .filter(([_, r]) => !r.success)
+  .map(([name]) => name);
+
+logger.warn('Failed producers:', failed);
+// Continue - failed producers use lazy initialization
+```
+
+#### Selective Bootstrap
+
+Bootstrap specific producers only:
+
+```typescript
+// Only bootstrap critical producer
+await kafka.bootstrap({
+  producers: ['main'],
+  timeoutMs: 10000,
+});
+
+// Analytics producer will use lazy initialization
+```
+
+#### Custom Timeouts
+
+```typescript
+await kafka.bootstrap({
+  timeoutMs: 20000, // 20 second timeout
+  strict: true,
+});
+```
+
 ## Usage
 
 ### Setup with Multiple Producers
@@ -202,30 +379,148 @@ await kafka.publish('main', 'topic', { data: 'test' });
 
 ### Health Checks & Readiness Probes
 
-Use the built-in health check for Kubernetes readiness probes:
+The package provides comprehensive health check APIs suitable for Kubernetes probes:
+
+#### Readiness Probe
+
+Checks if Kafka is ready to handle traffic (connects to brokers):
 
 ```typescript
 import { kafka } from './kafka';
 
-// Express/Fastify/etc
 app.get('/health/ready', async (req, res) => {
   const ready = await kafka.isReady();
-  res.status(ready ? 200 : 503).json({
-    ready,
-    kafka: kafka.getConnectionStatus(),
-  });
+
+  if (!ready) {
+    const health = kafka.getHealth();
+    return res.status(503).json({
+      ready: false,
+      kafka: health,
+    });
+  }
+
+  res.json({ ready: true });
 });
+```
+
+**Readiness check features:**
+- Revalidates connectivity if cache is stale
+- Configurable cache duration (default: 1 second)
+- Force revalidation with `isReady({ force: true })`
+- Respects `healthCheckTimeoutMs` configuration
+
+#### Liveness Probe
+
+Lightweight check for process health (does NOT connect to Kafka):
 
-// Or manually check each producer
-app.get('/health/detailed', async (req, res) => {
-  const status = kafka.getConnectionStatus();
-  const allConnected = Object.values(status).every(s => s);
+```typescript
+app.get('/health/live', (req, res) => {
+  const live = kafka.isLive();
+  res.status(live ? 200 : 503).json({ live });
+});
+```
 
-  res.status(allConnected ? 200 : 503).json({
-    producers: status,
+**Liveness check:**
+- Returns `false` if manager is in a fatal state
+- Returns `false` if graceful shutdown has started
+- Does NOT perform Kafka operations (very fast)
+- Perfect for Kubernetes liveness probes
+
+#### Detailed Health Snapshot
+
+Get comprehensive health information for all producers:
+
+```typescript
+app.get('/status/kafka', (req, res) => {
+  const health = kafka.getHealth();
+
+  res.json({
+    producers: health,
     enabled: kafka.isEnabled,
+    live: kafka.isLive(),
   });
 });
+
+// Example response:
+{
+  "producers": {
+    "main": {
+      "name": "main",
+      "enabled": true,
+      "isConnected": true,
+      "lastPingAt": 1701234567890,
+      "lastPingSucceededAt": 1701234567890,
+      "lastError": null,
+      "clusterId": "prod-kafka-main-01",
+      "brokerCount": 3,
+      "brokers": ["kafka1:9092", "kafka2:9092", "kafka3:9092"]
+    },
+    "analytics": {
+      "name": "analytics",
+      "enabled": true,
+      "isConnected": false,
+      "lastPingAt": 1701234560000,
+      "lastPingSucceededAt": 1701234500000,
+      "lastError": {
+        "message": "Connection refused",
+        "timestamp": 1701234560000
+      },
+      "clusterId": null,
+      "brokerCount": 0,
+      "brokers": ["analytics-kafka:9092"]
+    }
+  },
+  "enabled": true,
+  "live": true
+}
+```
+
+#### Connection Status
+
+Get quick connection status summary:
+
+```typescript
+const status = kafka.getConnectionStatus();
+
+// Returns:
+{
+  "main": {
+    "connected": true,
+    "lastSuccessAt": 1701234567890,
+    "lastError": null
+  },
+  "analytics": {
+    "connected": false,
+    "lastSuccessAt": 1701234500000,
+    "lastError": "Connection refused"
+  }
+}
+```
+
+#### Kubernetes Example
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  name: my-service
+spec:
+  containers:
+  - name: app
+    image: my-service:latest
+    livenessProbe:
+      httpGet:
+        path: /health/live
+        port: 3000
+      initialDelaySeconds: 10
+      periodSeconds: 10
+    readinessProbe:
+      httpGet:
+        path: /health/ready
+        port: 3000
+      initialDelaySeconds: 15
+      periodSeconds: 5
+      timeoutSeconds: 3
 ```
 
 ### Migration from getKafka() Pattern
@@ -303,16 +598,41 @@ async function publishEvent() {
 ```typescript
 interface KafkaManagerOptions {
   // Enable/disable Kafka - when false, returns mock implementations
+  // @default true
   enabled?: boolean;
 
   // Custom logger instance
   logger?: LoggerInstanceManager;
 
-  // Skip automatic graceful shutdown (default: false)
+  // Skip automatic graceful shutdown
+  // @default false
   dontGracefulShutdown?: boolean;
 
-  // Named producers configuration
+  // Named producers configuration (required)
   producers: Record<string, ProducerConfig>;
+
+  // ===== NEW: Health & Bootstrap Options =====
+
+  // Global timeout for health checks in ms
+  // Used by isReady() and ping operations
+  // @default 5000 (5 seconds)
+  healthCheckTimeoutMs?: number;
+
+  // How long to cache health check results in ms
+  // Set to 0 to always revalidate
+  // @default 1000 (1 second)
+  healthCheckCacheMs?: number;
+
+  // Default timeout for bootstrap operations in ms
+  // Can be overridden per bootstrap() call
+  // @default 30000 (30 seconds)
+  bootstrapTimeoutMs?: number;
+
+  // Default strict mode for bootstrap
+  // If true, all producers must succeed
+  // If false, at least one producer must succeed
+  // @default true
+  strictBootstrap?: boolean;
 }
 ```
 
@@ -402,9 +722,11 @@ await kafka.publish('main', 'events', data, {
 
 ## API Reference
 
-### `KafkaManager.create(options): KafkaManager<ProducerNames>`
+### Core Lifecycle Methods
 
-Creates a new KafkaManager instance with multiple named producers. Producers are initialized lazily on first use. **Producer names are type-safe** - TypeScript will autocomplete and validate them based on your configuration.
+#### `KafkaManager.create(options): KafkaManager<ProducerNames>`
+
+Creates a new KafkaManager instance with multiple named producers. **Producer names are type-safe** - TypeScript will autocomplete and validate them based on your configuration.
 
 **Parameters:**
 - `options` - Configuration options (see [Configuration](#configuration))
@@ -437,7 +759,123 @@ await kafka.publish('analytics', 'metrics', { value: 1 }); // ✅ Valid
 // await kafka.publish('wrong', 'topic', {}); // ❌ TypeScript error!
 ```
 
-### `publish<T>(producerName, topic, value, options?): Promise<RecordMetadata[]>`
+#### `bootstrap(options?): Promise<BootstrapResult>`
+
+**NEW:** Explicitly bootstrap all (or specific) producers. Connects to brokers, validates connectivity, and returns detailed results. **Recommended before serving traffic in production.**
+
+**Parameters:**
+- `options.timeoutMs` (optional) - Maximum time to wait for all producers (default: `bootstrapTimeoutMs` config)
+- `options.strict` (optional) - Fail if ANY producer fails (default: `strictBootstrap` config)
+- `options.producers` (optional) - Array of specific producer names to bootstrap (default: all)
+
+**Returns:** `BootstrapResult` with success status, duration, and per-producer results
+
+**Examples:**
+```typescript
+// Bootstrap all producers with defaults
+await kafka.bootstrap();
+
+// Non-strict mode - continue if some fail
+const result = await kafka.bootstrap({
+  timeoutMs: 10000,
+  strict: false,
+});
+if (!result.success) {
+  console.error('Some producers failed:', result.results);
+}
+
+// Bootstrap specific producers only
+await kafka.bootstrap({ producers: ['main'] });
+```
+
+### Health & Monitoring Methods
+
+#### `getHealth(): Record<ProducerNames, ProducerHealth>`
+
+Get detailed health snapshot for all producers. Returns comprehensive metadata including connection state, timestamps, errors, and cluster info.
+
+**Returns:** Object mapping producer names to `ProducerHealth` objects
+
+**Example:**
+```typescript
+const health = kafka.getHealth();
+console.log(health.main.clusterId);          // 'prod-kafka-01'
+console.log(health.main.lastPingSucceededAt);// 1701234567890
+console.log(health.main.lastError);          // null or error object
+```
+
+#### `getProducerHealth(name): ProducerHealth`
+
+Get detailed health for a specific producer.
+
+**Parameters:**
+- `name` - Producer name (type-safe)
+
+**Returns:** `ProducerHealth` object
+
+**Example:**
+```typescript
+const health = kafka.getProducerHealth('main');
+console.log('Cluster:', health.clusterId);
+console.log('Brokers:', health.brokerCount);
+```
+
+#### `isReady(options?): Promise<boolean>`
+
+Check if Kafka is ready to handle traffic. Revalidates connectivity if cache is stale. **Use for Kubernetes readiness probes.**
+
+**Parameters:**
+- `options.timeout` (optional) - Override default health check timeout
+- `options.force` (optional) - Force revalidation, ignore cache
+
+**Returns:** `true` if all producers are connected, `false` otherwise
+
+**Examples:**
+```typescript
+// Use cached result if fresh
+const ready = await kafka.isReady();
+
+// Force immediate revalidation
+const ready = await kafka.isReady({ force: true });
+
+// Custom timeout
+const ready = await kafka.isReady({ timeout: 10000 });
+```
+
+#### `isLive(): boolean`
+
+Lightweight liveness check - does NOT perform Kafka operations. Only checks internal state for fatal errors. **Perfect for Kubernetes liveness probes.**
+
+**Returns:** `false` if manager is in fatal state or graceful shutdown has started
+
+**Example:**
+```typescript
+app.get('/health/live', (req, res) => {
+  res.status(kafka.isLive() ? 200 : 503).json({ live: kafka.isLive() });
+});
+```
+
+#### `getConnectionStatus(): Record<ProducerNames, ConnectionStatus>`
+
+Get connection status for all producers with timestamps and errors.
+
+**Returns:** Object mapping producer names to connection status
+
+**Example:**
+```typescript
+const status = kafka.getConnectionStatus();
+// {
+//   main: {
+//     connected: true,
+//     lastSuccessAt: 1701234567890,
+//     lastError: null
+//   }
+// }
+```
+
+### Publishing Methods
+
+#### `publish<T>(producerName, topic, value, options?): Promise<RecordMetadata[]>`
 
 Publishes a single message to a topic using a named producer. Producer name is **type-safe** - must match one of the configured producers.
 
@@ -657,6 +1095,200 @@ await kafka.publish('main', 'orders', orderData);
 await kafka.publish('analytics', 'metrics', metricsData);
 ```
 
+## Troubleshooting
+
+### Common Issues and Solutions
+
+#### Bootstrap Fails with Connection Timeout
+
+**Problem:**
+```
+[main] Failed to connect to Kafka brokers: Connection timeout after 5000ms
+
+Possible causes:
+  1. Brokers are unreachable: kafka:9092
+  2. Network connectivity issues
+  3. Firewall blocking ports
+```
+
+**Solutions:**
+1. **Verify brokers are running:**
+   ```bash
+   kubectl get pods -l app=kafka
+   ```
+
+2. **Test connectivity:**
+   ```bash
+   nc -zv kafka-broker 9092
+   ```
+
+3. **Check network policies:**
+   - Ensure service can reach Kafka namespace
+   - Verify firewall rules allow traffic on port 9092
+
+4. **Increase timeout:**
+   ```typescript
+   await kafka.bootstrap({ timeoutMs: 30000 }); // 30 seconds
+   ```
+
+#### SASL Authentication Failed
+
+**Problem:**
+```
+[main] Failed to connect to Kafka brokers: SASL authentication failed
+```
+
+**Solutions:**
+1. **Verify credentials:**
+   ```typescript
+   producers: {
+     main: {
+       brokers: ['kafka:9092'],
+       sasl: {
+         mechanism: 'scram-sha-256',
+         username: process.env.KAFKA_USERNAME,  // Check this
+         password: process.env.KAFKA_PASSWORD,  // And this
+       },
+     },
+   }
+   ```
+
+2. **Check mechanism:**
+   - Verify broker supports the SASL mechanism
+   - Common mechanisms: `'plain'`, `'scram-sha-256'`, `'scram-sha-512'`
+
+3. **Inspect secrets:**
+   ```bash
+   kubectl get secret kafka-credentials -o yaml
+   ```
+
+#### Producer Not Found Error
+
+**Problem:**
+```
+Producer 'analytics' not found. Available producers: main
+```
+
+**Solution:**
+You're trying to use a producer that wasn't configured:
+
+```typescript
+// Add the missing producer
+const kafka = KafkaManager.create({
+  producers: {
+    main: { brokers: ['kafka:9092'] },
+    analytics: { brokers: ['kafka-analytics:9092'] }, // Add this
+  },
+});
+```
+
+#### Readiness Check Always Fails
+
+**Problem:**
+Health checks keep failing even though Kafka is up.
+
+**Solutions:**
+1. **Check timeout is sufficient:**
+   ```typescript
+   const kafka = KafkaManager.create({
+     healthCheckTimeoutMs: 5000, // Increase if needed
+   });
+   ```
+
+2. **Force revalidation:**
+   ```typescript
+   const ready = await kafka.isReady({ force: true });
+   ```
+
+3. **Inspect detailed health:**
+   ```typescript
+   const health = kafka.getHealth();
+   console.log(health); // Check lastError for details
+   ```
+
+#### Bootstrap Succeeds but Publish Fails
+
+**Problem:**
+Bootstrap passes, but publish throws "topic auto-create disabled".
+
+**Solution:**
+Enable topic auto-creation or create topics manually:
+
+```typescript
+producers: {
+  main: {
+    brokers: ['kafka:9092'],
+    autoCreateTopics: true, // Enable for dev/test
+  },
+}
+
+// Or create topics manually:
+// kafka-topics --create --topic my-topic --bootstrap-server kafka:9092
+```
+
+#### Graceful Shutdown Not Working
+
+**Problem:**
+Service doesn't close Kafka connections on shutdown.
+
+**Solutions:**
+1. **Ensure graceful shutdown is enabled:**
+   ```typescript
+   const kafka = KafkaManager.create({
+     dontGracefulShutdown: false, // Default
+     producers: { /* ... */ },
+   });
+   ```
+
+2. **Manual disconnect if needed:**
+   ```typescript
+   process.on('SIGTERM', async () => {
+     await kafka.disconnect();
+     process.exit(0);
+   });
+   ```
+
+### Debugging Tips
+
+#### Enable Debug Logging
+
+```typescript
+import { KafkaManager } from '@autofleet/kafka';
+import logger from './logger';
+
+const kafka = KafkaManager.create({
+  logger, // Ensure logger has debug level enabled
+  producers: { /* ... */ },
+});
+
+// Check health frequently
+setInterval(() => {
+  const health = kafka.getHealth();
+  logger.debug('Kafka health', { health });
+}, 10000);
+```
+
+#### Inspect Cluster Metadata
+
+```typescript
+const health = kafka.getProducerHealth('main');
+console.log('Cluster ID:', health.clusterId);
+console.log('Brokers:', health.brokerCount);
+console.log('Last success:', new Date(health.lastPingSucceededAt));
+console.log('Last error:', health.lastError);
+```
+
+#### Test Connectivity Manually
+
+```typescript
+try {
+  await kafka.pingProducer('main');
+  console.log('✓ Connection successful');
+} catch (error) {
+  console.error('✗ Connection failed:', error.message);
+}
+```
+
 ## Testing
 
 Run the test suite: