pulse-sdk 0.0.4 → 0.0.6

package/README.md

@@ -1,177 +1,116 @@
-# Pulse SDK — TypeScript
+# Pulse TypeScript SDK
 
-TypeScript client for Pulse Broker (gRPC). This README is written for end users and explains installation, configuration, and basic usage with practical examples.
+Official TypeScript client for [Pulse Broker](https://github.com/marcosrosa/pulse).
 
 ## Installation
 
-Install from npm (published package):
-
 ```bash
 npm install pulse-sdk
 ```
 
-Or install locally from the repository (for development):
-
-```bash
-npm install
-```
-
-To compile the TypeScript sources (optional for development/CI):
-
-```bash
-npm run build
-```
-
-Run the lightweight integration tests (a test gRPC server is started automatically):
-
-```bash
-npm test
-```
-
 ## Configuration
 
-The SDK supports configuration via (in priority order):
+The SDK looks for a `pulse.yaml` (or `pulse.yml`) file in your project root. If not found, it defaults to `localhost:5555` (HTTP) and `localhost:5556` (gRPC).
 
-1. Programmatic: pass a `PulseConfig` object when creating `Producer` or `Consumer`.
-2. Environment variables: `PULSE_GRPC_URL`, `PULSE_EVENT_TYPES` (comma-separated), `PULSE_CONSUMER_NAME`.
-3. File: a `pulse.yml` or `pulse.yaml` file placed at the process working directory or in `~/.pulse/pulse.yml`.
-
-The SDK exposes two helpers:
-
-- `loadConfig(configPath?: string): PulseConfig` — reads `pulse.yml` / env and returns a merged config object.
-- `initFromConfig(cfg: PulseConfig): Promise<void>` — calls the broker `CreateTopic` RPC for every topic declared in `cfg.topics`. This lets the SDK create any required topics automatically at startup (useful for tests or first-run).
-
-Example `pulse.yml`:
+### Example `pulse.yaml`
 
 ```yaml
-grpcUrl: localhost:5556
-eventTypes:
-	- events
-	- transactions
-consumerName: my-consumer
+# Connection Settings
+broker:
+  host: "localhost"
+  http_port: 5555
+  grpc_port: 5556
+  timeout_ms: 5000
+
+# Client Defaults
+client:
+  id: "my-typescript-app"
+  auto_commit: true       # Automatically commit offsets after successful processing
+  max_retries: 3
+
+# Topic Configuration
 topics:
-	- name: events
-		fifo: false
-	- name: transactions
-		fifo: true
-```
-
-Usage example (auto-create topics then start):
-
-```ts
-import { loadConfig, initFromConfig, Producer, Consumer } from 'pulse-sdk';
-
-async function main() {
-	const cfg = loadConfig(); // reads pulse.yml or environment
-	await initFromConfig(cfg); // create topics listed in cfg.topics (no-op if none)
-
-	const producer = new Producer(cfg);
-	await producer.send('events', { type: 'user.created', id: 1 });
-
-	const consumer = new Consumer(cfg);
-	consumer.on('events', (msg) => console.log('received', msg.payload));
-	await consumer.start('events', cfg.consumerName || 'default');
-}
-
-main().catch(console.error);
+  - name: "events"
+    create_if_missing: true
+    config:
+      fifo: false
+      retention_bytes: 1073741824  # 1GB
+    consume:
+      auto_commit: true
+
+  - name: "transactions"
+    create_if_missing: true
+    config:
+      fifo: true
+    consume:
+      auto_commit: false  # Manual commit required
+
+  - name: "logs"
+    create_if_missing: true
+    config:
+      fifo: false
+    consume:
+      auto_commit: true
 ```
 
-Notes:
-- `initFromConfig` calls the gRPC `CreateTopic` RPC. If the broker responds with an error for a topic that already exists, the SDK logs a warning and continues.
-- Ensure the broker is running and reachable at `cfg.grpcUrl` before calling `initFromConfig`.
-- The SDK bundles `pulse.proto` inside the package so consumers do not need to copy proto files into their projects.
+## Usage
 
-## Quick Start
+### Producer
 
-Import from the published package and pass configuration programmatically:
+You can send objects (automatically serialized to JSON), strings, or raw buffers.
 
-```ts
-import { Producer, Consumer } from 'pulse-sdk';
+```typescript
+import { Producer } from 'pulse-sdk';
 
-const cfg = { grpcUrl: 'localhost:50052', eventTypes: ['events'] };
+// Initialize (uses pulse.yaml or defaults)
+// You can override settings: new Producer("10.0.0.1", 9090)
+const producer = new Producer();
 
-// Send a JSON-serializable event
-const producer = new Producer(cfg);
-await producer.send('events', { type: 'user.created', id: 123 });
+async function main() {
+  // Send JSON
+  await producer.send("events", { type: "user_created", id: 123 });
 
-// Consume events and register a handler
-const consumer = new Consumer(cfg);
-consumer.on('events', (msg) => {
-	console.log('received', msg.payload);
-});
-await consumer.start('events', 'my-consumer');
-```
+  // Send String
+  await producer.send("logs", "raw log line");
 
-### Programmatic override
+  // Send Buffer
+  await producer.send("logs", Buffer.from("binary data"));
 
-You can create and pass `PulseConfig` directly in code:
+  producer.close();
+}
 
-```ts
-const cfg = { grpcUrl: '10.0.0.5:50052', eventTypes: ['events'] };
-const producer = new Producer(cfg);
+main();
 ```
 
-## Technical Details
-
-- The SDK dynamically loads `pulse.proto` at runtime using `@grpc/proto-loader` and `@grpc/grpc-js`.
-- The protobuf field `bytes payload` is used to carry the message body; the SDK serializes JSON payloads into that field by default.
-- `Consumer.start()` opens a server stream (`Consume`) and dispatches incoming messages to handlers registered via `consumer.on(topic, handler)`.
-
-## Examples
+### Consumer
 
-Producer (send multiple events):
+Use the `consumer` function to register message handlers, and `run` to start the loop.
 
-```ts
-const producer = new Producer(loadConfig());
-await producer.send('events', { type: 'login', userId: 1 });
-await producer.send('events', { type: 'logout', userId: 1 });
-```
-
-Consumer (simple processing):
-
-```ts
-const cfg = loadConfig();
-const consumer = new Consumer(cfg);
+```typescript
+import { consumer, run, commit, Message } from 'pulse-sdk';
 
-consumer.on('events', (msg) => {
-	console.log('event payload', msg.payload);
+// Simple Consumer (uses auto_commit from config)
+consumer("events", async (msg: Message) => {
+  console.log(`Received event:`, msg.payload);
+  // msg.payload is an object if JSON, string if String, else Buffer
 });
 
-await consumer.start('events', cfg.consumerName || 'default');
-```
-
-## Local Tests
-
-The tests in `tests/` start a lightweight gRPC test server automatically; run them with `npm test`.
-
-## FAQ (short)
-
-- Can I use decorators for handlers? Yes — TypeScript supports decorators, but this SDK does not use them by default. We can add decorator helpers later if desired.
-- How do I enable TLS / secure credentials? The client factory currently uses `createInsecure()` by default. We can expose a `credentials` option on `PulseConfig` to accept `grpc.credentials.createSsl(...)` or other credential objects.
-
-**Grouped consumption**
-
-- **What it does:** When `grouped` is `true` (the default) the SDK coalesces consumers inside the same process that use the same `consumerName` into a single streaming connection. Messages are distributed (round-robin) among the registered handlers so each message is delivered to only one handler in the group (1x per client id).
-- **Why:** This mirrors the Python SDK behavior where handlers registered with `grouped=True` share a single consumer stream and avoid duplicate processing inside the same client.
-- **How to configure:**
-	- `pulse.yml` (recommended): set `grouped: true` or `grouped: false` under top-level config.
-	- Environment variable: `PULSE_GROUPED=true|false`.
-	- Programmatically: pass `grouped` in the `PulseConfig` passed to `Producer`/`Consumer`.
-- **Default:** `grouped` defaults to `true`.
-- **Grouped=false behavior:** When `grouped` is `false`, the SDK will ensure consumers use unique consumer IDs (if you pass the default client name), so multiple consumers in the same process each receive all messages independently (useful for testing or when you want duplicate consumption).
-- **Example `pulse.yml` entry:**
-
-```yaml
-grpcUrl: localhost:5556
-grouped: true
+// Manual Commit Consumer
+// Override config params directly in the options object
+consumer("transactions", async (msg: Message) => {
+  try {
+    await processPayment(msg.payload);
+    await commit();  // Manually commit offset
+    console.log(`Processed transaction ${msg.offset}`);
+  } catch (e) {
+    console.error(`Failed to process: ${e}`);
+  }
+}, { autoCommit: false });
+
+// Start all consumers
+run();
+
+async function processPayment(data: any) {
+  // ... implementation
+}
 ```
-
-The test-suite includes integration tests that validate both `grouped=true` and `grouped=false` behaviour.
-
-## Contributing
-
-Please open a PR with tests. The existing tests validate basic Producer/Consumer behaviour.
-
----
-
-If you want this README expanded to mirror the Python SDK more closely (topic-level config, manual offset commits, longer examples), tell me which sections to add and I will update it.

package/dist/config.d.ts

@@ -1,15 +1,26 @@
+export interface BrokerConfig {
+    host: string;
+    http_port: number;
+    grpc_port: number;
+    timeout_ms: number;
+}
+export interface ClientConfig {
+    id: string;
+    auto_commit: boolean;
+    max_retries: number;
+}
 export interface TopicConfig {
     name: string;
-    fifo?: boolean;
-    retention_bytes?: number;
-    retention_time?: number;
+    create_if_missing?: boolean;
+    config?: {
+        fifo?: boolean;
+        retention_bytes?: number;
+    };
 }
 export interface PulseConfig {
-    grpcUrl: string;
-    eventTypes: string[];
-    consumerName?: string;
-    grouped?: boolean;
-    topics?: TopicConfig[];
+    broker: BrokerConfig;
+    client: ClientConfig;
+    topics: TopicConfig[];
 }
 export declare function loadConfig(configPath?: string): PulseConfig;
-export declare function initFromConfig(cfg: PulseConfig): Promise<void>;
+export declare function getConfig(): PulseConfig;

package/dist/config.js

@@ -4,73 +4,62 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
 };
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.loadConfig = loadConfig;
-exports.initFromConfig = initFromConfig;
+exports.getConfig = getConfig;
 const fs_1 = __importDefault(require("fs"));
 const path_1 = __importDefault(require("path"));
 const os_1 = __importDefault(require("os"));
 const js_yaml_1 = __importDefault(require("js-yaml"));
-const client_1 = require("./proto/client");
-const DEFAULTS = {
-    grpcUrl: 'localhost:50052',
-    eventTypes: ['events'],
-    grouped: true,
+const DEFAULT_CONFIG = {
+    broker: {
+        host: 'localhost',
+        http_port: 5555,
+        grpc_port: 5556,
+        timeout_ms: 5000,
+    },
+    client: {
+        id: 'typescript-client',
+        auto_commit: true,
+        max_retries: 3,
+    },
+    topics: [],
 };
 function loadConfig(configPath) {
     const candidates = [
         configPath,
-        path_1.default.resolve(process.cwd(), 'pulse.yml'),
         path_1.default.resolve(process.cwd(), 'pulse.yaml'),
+        path_1.default.resolve(process.cwd(), 'pulse.yml'),
         path_1.default.join(os_1.default.homedir(), '.pulse', 'pulse.yml'),
     ].filter(Boolean);
     let fileCfg = {};
     for (const p of candidates) {
         if (p && fs_1.default.existsSync(p)) {
-            const raw = fs_1.default.readFileSync(p, 'utf8');
-            const parsed = js_yaml_1.default.load(raw);
-            fileCfg = parsed || {};
-            break;
+            try {
+                const raw = fs_1.default.readFileSync(p, 'utf8');
+                fileCfg = js_yaml_1.default.load(raw) || {};
+                break;
+            }
+            catch (e) {
+                console.warn(`Failed to load config from ${p}:`, e);
+            }
         }
     }
-    const envCfg = {};
-    if (process.env.PULSE_GRPC_URL)
-        envCfg.grpcUrl = process.env.PULSE_GRPC_URL;
-    if (process.env.PULSE_EVENT_TYPES)
-        envCfg.eventTypes = process.env.PULSE_EVENT_TYPES.split(',');
-    if (process.env.PULSE_CONSUMER_NAME)
-        envCfg.consumerName = process.env.PULSE_CONSUMER_NAME;
-    if (process.env.PULSE_GROUPED)
-        envCfg.grouped = process.env.PULSE_GROUPED === 'true';
-    const merged = Object.assign({}, DEFAULTS, fileCfg, envCfg);
-    // Ensure eventTypes array exists
-    if (!merged.eventTypes)
-        merged.eventTypes = DEFAULTS.eventTypes;
-    if (merged.grouped === undefined)
-        merged.grouped = true;
-    return merged;
+    // Deep merge defaults with file config
+    const config = JSON.parse(JSON.stringify(DEFAULT_CONFIG)); // Deep copy
+    if (fileCfg.broker) {
+        config.broker = { ...config.broker, ...fileCfg.broker };
+    }
+    if (fileCfg.client) {
+        config.client = { ...config.client, ...fileCfg.client };
+    }
+    if (fileCfg.topics) {
+        config.topics = fileCfg.topics;
+    }
+    return config;
 }
-// Initialize topics from config using gRPC CreateTopic RPC.
-async function initFromConfig(cfg) {
-    if (!cfg.topics || cfg.topics.length === 0)
-        return;
-    const client = (0, client_1.createClient)(cfg.grpcUrl);
-    for (const t of cfg.topics) {
-        const req = {
-            topic: t.name || t['name'],
-            fifo: !!t.fifo,
-            retention_bytes: t.retention_bytes || 0,
-            retention_time: t.retention_time || 0,
-        };
-        await new Promise((resolve, reject) => {
-            client.CreateTopic(req, (err, res) => {
-                if (err) {
-                    // If topic exists the server may return an error; log and continue
-                    console.warn('CreateTopic error for', req.topic, err.message || err);
-                    resolve();
-                }
-                else {
-                    resolve();
-                }
-            });
-        });
+let _config = null;
+function getConfig() {
+    if (!_config) {
+        _config = loadConfig();
     }
+    return _config;
 }

package/dist/consumer.d.ts

@@ -1,12 +1,12 @@
-import { PulseConfig } from './config';
-export type EventHandler = (payload: any) => void;
-export declare class Consumer {
-    private config;
-    private handlers;
-    private client;
-    private unregisterFns;
-    constructor(config: PulseConfig);
-    on(eventType: string, handler: EventHandler): void;
-    start(topic?: string, consumerName?: string): Promise<void>;
-    close(): void;
+import { Message, commit } from './message';
+export { commit, Message };
+interface ConsumerOptions {
+    host?: string;
+    port?: number;
+    consumerGroup?: string;
+    autoCommit?: boolean;
+    grouped?: boolean;
 }
+export declare function stop(): void;
+export declare function consumer(topic: string, handler: (msg: Message) => void | Promise<void>, options?: ConsumerOptions): void;
+export declare function run(): Promise<void>;

package/dist/consumer.js

@@ -1,99 +1,173 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.Consumer = void 0;
+exports.Message = exports.commit = void 0;
+exports.stop = stop;
+exports.consumer = consumer;
+exports.run = run;
+const config_1 = require("./config");
 const client_1 = require("./proto/client");
-const consumerManager_1 = require("./consumerManager");
-const crypto_1 = require("crypto");
 const message_1 = require("./message");
-class Consumer {
-    constructor(config) {
-        this.config = config;
-        this.handlers = {};
-        this.unregisterFns = [];
-        this.client = (0, client_1.createClient)(config.grpcUrl);
+Object.defineProperty(exports, "Message", { enumerable: true, get: function () { return message_1.Message; } });
+Object.defineProperty(exports, "commit", { enumerable: true, get: function () { return message_1.commit; } });
+const crypto_1 = require("crypto");
+const _consumers = [];
+const _cleanupFunctions = [];
+function stop() {
+    _cleanupFunctions.forEach(fn => fn());
+    _cleanupFunctions.length = 0;
+    _consumers.length = 0;
+}
+function consumer(topic, handler, options = {}) {
+    const config = (0, config_1.getConfig)();
+    const host = options.host || config.broker.host;
+    const port = options.port || config.broker.grpc_port;
+    const baseGroup = options.consumerGroup || config.client.id;
+    const grouped = options.grouped !== undefined ? options.grouped : true;
+    let group = baseGroup;
+    if (!grouped) {
+        group = `${baseGroup}-${(0, crypto_1.randomUUID)().replace(/-/g, '')}`;
     }
-    on(eventType, handler) {
-        if (!this.handlers[eventType]) {
-            this.handlers[eventType] = [];
+    let autoCommit = options.autoCommit;
+    if (autoCommit === undefined) {
+        autoCommit = config.client.auto_commit;
+        // Check topic specific config
+        const topicCfg = config.topics?.find(t => t.name === topic);
+        // Note: Python SDK checks topic_config["consume"]["auto_commit"] but our config.ts 
+        // currently only has create_if_missing and config (fifo, retention).
+        // We'll stick to global client config for now unless we expand TopicConfig.
+    }
+    _consumers.push({
+        topic,
+        host,
+        port,
+        group,
+        autoCommit: autoCommit,
+        handler,
+        grouped,
+    });
+}
+async function run() {
+    // Group consumers by (topic, host, port, group)
+    const groupedMap = new Map();
+    for (const c of _consumers) {
+        const key = `${c.topic}:${c.host}:${c.port}:${c.group}`;
+        if (!groupedMap.has(key)) {
+            groupedMap.set(key, {
+                topic: c.topic,
+                host: c.host,
+                port: c.port,
+                group: c.group,
+                autoCommit: c.autoCommit,
+                handlers: [],
+            });
         }
-        this.handlers[eventType].push(handler);
+        groupedMap.get(key).handlers.push(c.handler);
     }
-    async start(topic, consumerName = 'default') {
-        const topicName = topic || this.config.eventTypes[0];
-        // If grouped is enabled (default true) use shared in-process stream,
-        // otherwise create a dedicated stream (each consumer gets all messages).
-        const grouped = this.config.grouped !== false;
-        if (grouped) {
-            // register handlers for this topic to shared stream
-            const handlers = this.handlers[topicName] || [];
-            for (const h of handlers) {
-                const unregister = (0, consumerManager_1.registerSharedHandler)(this.config.grpcUrl, topicName, consumerName, (msg, stub, offset) => {
-                    // run handler with context so commit() works
-                    // debug: console.log('consumer.wrapper.invoke', consumerName, topicName);
-                    (0, message_1.runWithContext)({ stub: stub || this.client, topic: topicName, consumerName, offset: offset ?? msg.offset }, () => {
-                        try {
-                            h(msg);
-                        }
-                        catch (e) { /* ignore */ }
-                    });
-                });
-                this.unregisterFns.push(unregister);
+    const promises = [];
+    for (const groupConfig of groupedMap.values()) {
+        promises.push(consumeLoopGroup(groupConfig));
+    }
+    // We don't await promises here to let them run in background, 
+    // but we could if we wanted to block until they all finish (which they won't).
+    // However, to keep the process alive, the user should probably await this or we return a promise that never resolves?
+    // Python's run() blocks. In Node, usually we just start things.
+    // But if the script ends, the process exits.
+    // We'll return a promise that never resolves to simulate blocking if awaited.
+    return new Promise(() => { });
+}
+async function consumeLoopGroup(groupConfig) {
+    const address = `${groupConfig.host}:${groupConfig.port}`;
+    // console.log(`Starting consumer for topic '${groupConfig.topic}' (group: ${groupConfig.group}) on ${address}`);
+    let handlerIdx = 0;
+    let currentStream = null;
+    let retryTimeout = null;
+    let isStopped = false;
+    const cleanup = () => {
+        isStopped = true;
+        if (currentStream) {
+            try {
+                currentStream.cancel();
             }
-            // Return a promise that never resolves (stream runs until process exits)
-            return new Promise(() => { });
-        }
-        // If grouped is explicitly false, and the consumerName equals the configured
-        // client name or the default literal, generate a unique consumer id so each
-        // consumer receives messages independently (mirrors Python behaviour).
-        if (!grouped) {
-            const base = this.config.consumerName || 'default';
-            if (consumerName === base || consumerName === 'default') {
-                consumerName = `${base}-${(0, crypto_1.randomUUID)().replace(/-/g, '')}`;
+            catch (e) {
+                // Ignore cancel errors
             }
+            currentStream = null;
         }
-        const req = { topic: topicName, consumer_name: consumerName, offset: 0 };
-        let stream = null;
-        try {
-            stream = this.client.Consume(req);
+        if (retryTimeout) {
+            clearTimeout(retryTimeout);
+            retryTimeout = null;
         }
-        catch (err) {
-            // If the client throws synchronously (e.g. CANCELLED), return a promise
-            // that is rejected but handled to avoid an unhandled rejection when
-            // callers don't await `start()` (tests call start() without awaiting).
-            const p = Promise.reject(err);
-            p.catch(() => { }); // swallow to avoid uncaught rejection
-            return p;
+    };
+    _cleanupFunctions.push(cleanup);
+    const startStream = async () => {
+        if (isStopped)
+            return;
+        try {
+            const client = (0, client_1.createClient)(address);
+            const req = {
+                topic: groupConfig.topic,
+                consumer_name: groupConfig.group,
+                offset: 0,
+            };
+            const stream = client.Consume(req);
+            currentStream = stream;
+            stream.on('data', async (protoMsg) => {
+                if (isStopped)
+                    return;
+                // Pause stream to process message sequentially
+                stream.pause();
+                const msg = new message_1.Message(protoMsg);
+                if (groupConfig.handlers.length === 0) {
+                    stream.resume();
+                    return;
+                }
+                const handler = groupConfig.handlers[handlerIdx % groupConfig.handlers.length];
+                handlerIdx++;
+                const ctx = {
+                    stub: client,
+                    topic: groupConfig.topic,
+                    consumerGroup: groupConfig.group,
+                    offset: msg.offset,
+                    committed: false,
+                };
+                try {
+                    await (0, message_1.runWithContext)(ctx, async () => {
+                        await handler(msg);
+                        if (groupConfig.autoCommit && !ctx.committed) {
+                            await (0, message_1.commit)();
+                        }
+                    });
+                }
+                catch (e) {
+                    console.error(`Error processing message: ${e}`);
+                }
+                finally {
+                    // Resume stream after processing
+                    stream.resume();
+                }
+            });
+            stream.on('error', (err) => {
+                if (isStopped)
+                    return;
+                // 1 = CANCELLED
+                if (err.code === 1)
+                    return;
+                console.error(`Connection lost for ${groupConfig.topic}: ${err.message}. Retrying in 5s...`);
+                retryTimeout = setTimeout(startStream, 5000);
+            });
+            stream.on('end', () => {
+                if (isStopped)
+                    return;
+                // console.warn(`Stream ended for ${groupConfig.topic}. Retrying in 5s...`);
+                retryTimeout = setTimeout(startStream, 5000);
+            });
         }
-        stream.on('data', (msg) => {
-            const message = new message_1.Message(msg);
-            const handlers = this.handlers[req.topic] || [];
-            for (const h of handlers) {
-                // run handler within AsyncLocalStorage context so commit() can access stub and offset
-                (0, message_1.runWithContext)({ stub: this.client, topic: req.topic, consumerName, offset: message.offset }, () => {
-                    try {
-                        h(message);
-                    }
-                    catch (e) { /* handler error ignored here */ }
-                });
-            }
-        });
-        const p = new Promise((resolve, reject) => {
-            stream.on('end', () => resolve());
-            stream.on('error', (e) => reject(e));
-        });
-        // prevent unhandled rejections when callers don't await the returned promise
-        p.catch(() => { });
-        return p;
-    }
-    // unregister any shared handlers when this consumer is discarded
-    close() {
-        for (const u of this.unregisterFns) {
-            try {
-                u();
-            }
-            catch (e) { /* ignore */ }
+        catch (e) {
+            if (isStopped)
+                return;
+            console.error(`Unexpected error in consumer ${groupConfig.topic}: ${e.message}`);
+            retryTimeout = setTimeout(startStream, 5000);
         }
-        this.unregisterFns = [];
-    }
+    };
+    startStream();
 }
-exports.Consumer = Consumer;

package/dist/index.d.ts

@@ -2,4 +2,3 @@ export * from './producer';
 export * from './consumer';
 export * from './config';
 export * from './message';
-export * from './consumerManager';

package/dist/index.js

@@ -18,4 +18,3 @@ __exportStar(require("./producer"), exports);
 __exportStar(require("./consumer"), exports);
 __exportStar(require("./config"), exports);
 __exportStar(require("./message"), exports);
-__exportStar(require("./consumerManager"), exports);

package/dist/message.d.ts

@@ -1,24 +1,19 @@
-export interface IMessage {
-    offset: number;
-    timestamp: number;
-    payload: any;
-    headers: Record<string, string>;
-}
-interface MessageContext {
+export interface MessageContext {
     stub: any;
     topic: string;
-    consumerName: string;
+    consumerGroup: string;
     offset: number;
-    committed?: boolean;
+    committed: boolean;
 }
-export declare function runWithContext(ctx: MessageContext, fn: () => void): void;
-export declare function getContext(): MessageContext | undefined;
+export declare function runWithContext(ctx: MessageContext, fn: () => void | Promise<void>): void | Promise<void>;
 export declare function commit(): Promise<void>;
-export declare class Message implements IMessage {
+export declare class Message {
     offset: number;
     timestamp: number;
-    payload: any;
-    headers: Record<string, string>;
+    private _rawPayload;
+    private _headers;
     constructor(protoMsg: any);
+    get payload(): any;
+    get rawPayload(): Buffer;
+    get headers(): Record<string, string>;
 }
-export {};

package/dist/message.js

@@ -2,76 +2,73 @@
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.Message = void 0;
 exports.runWithContext = runWithContext;
-exports.getContext = getContext;
 exports.commit = commit;
 const async_hooks_1 = require("async_hooks");
-const storage = new async_hooks_1.AsyncLocalStorage();
+const contextStorage = new async_hooks_1.AsyncLocalStorage();
 function runWithContext(ctx, fn) {
-    storage.run(ctx, fn);
-}
-function getContext() {
-    return storage.getStore();
+    return contextStorage.run(ctx, fn);
 }
 async function commit() {
-    const ctx = storage.getStore();
-    if (!ctx)
+    const ctx = contextStorage.getStore();
+    if (!ctx) {
         throw new Error('commit() called outside of a consumer handler');
-    if (ctx.committed)
+    }
+    if (ctx.committed) {
         return;
+    }
     return new Promise((resolve, reject) => {
-        try {
-            ctx.stub.CommitOffset({ topic: ctx.topic, consumer_name: ctx.consumerName, offset: ctx.offset + 1 }, (err, res) => {
-                if (err)
-                    return reject(err);
-                ctx.committed = true;
-                resolve();
-            });
-        }
-        catch (e) {
-            reject(e);
-        }
+        const req = {
+            topic: ctx.topic,
+            consumer_name: ctx.consumerGroup,
+            offset: ctx.offset + 1,
+        };
+        ctx.stub.CommitOffset(req, (err, res) => {
+            if (err) {
+                console.error(`Error committing offset: ${err}`);
+                return reject(err);
+            }
+            // console.log(`Committed offset ${req.offset} for ${req.consumer_name}`);
+            ctx.committed = true;
+            resolve();
+        });
     });
 }
 class Message {
     constructor(protoMsg) {
-        this.offset = protoMsg.offset;
-        this.timestamp = protoMsg.timestamp;
-        this.headers = {};
-        try {
-            this.headers = Object.assign({}, protoMsg.headers);
-        }
-        catch (e) {
-            this.headers = {};
-        }
-        const buf = protoMsg.payload;
-        const ptype = this.headers['payload-type'];
+        this.offset = typeof protoMsg.offset === 'string' ? parseInt(protoMsg.offset, 10) : protoMsg.offset;
+        this.timestamp = typeof protoMsg.timestamp === 'string' ? parseInt(protoMsg.timestamp, 10) : protoMsg.timestamp;
+        this._rawPayload = protoMsg.payload;
+        this._headers = protoMsg.headers || {};
+    }
+    get payload() {
+        const ptype = this._headers['payload-type'];
         if (ptype === 'json') {
             try {
-                this.payload = JSON.parse(buf.toString());
+                return JSON.parse(this._rawPayload.toString('utf-8'));
             }
             catch (e) {
-                this.payload = buf;
+                return this._rawPayload;
             }
         }
-        else if (ptype === 'string') {
-            try {
-                this.payload = buf.toString('utf8');
-            }
-            catch (e) {
-                this.payload = buf;
-            }
+        if (ptype === 'string') {
+            return this._rawPayload.toString('utf-8');
         }
-        else if (ptype === 'bytes') {
-            this.payload = buf;
+        if (ptype === 'bytes') {
+            return this._rawPayload;
         }
-        else {
-            try {
-                this.payload = JSON.parse(buf.toString());
-            }
-            catch (e) {
-                this.payload = buf;
-            }
+        // Fallback
+        try {
+            return JSON.parse(this._rawPayload.toString('utf-8'));
+        }
+        catch (e) {
+            return this._rawPayload;
         }
     }
+    get rawPayload() {
+        return this._rawPayload;
+    }
+    get headers() {
+        return this._headers;
+    }
 }
 exports.Message = Message;

package/dist/producer.d.ts

@@ -1,11 +1,8 @@
-import { PulseConfig } from './config';
-export interface PublishResponse {
-    id: string;
-    offset: number | string;
-}
 export declare class Producer {
-    private config;
     private client;
-    constructor(config: PulseConfig);
-    send(topic: string, payload: any, headers?: Record<string, string>): Promise<PublishResponse>;
+    private config;
+    constructor(host?: string, port?: number);
+    private setupTopics;
+    send(topic: string, payload: any): Promise<void>;
+    close(): void;
 }

package/dist/producer.js

@@ -1,25 +1,68 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.Producer = void 0;
+const config_1 = require("./config");
 const client_1 = require("./proto/client");
 class Producer {
-    constructor(config) {
-        this.config = config;
-        this.client = (0, client_1.createClient)(config.grpcUrl);
+    constructor(host, port) {
+        this.config = (0, config_1.getConfig)();
+        const h = host || this.config.broker.host;
+        const p = port || this.config.broker.grpc_port;
+        const address = `${h}:${p}`;
+        this.client = (0, client_1.createClient)(address);
+        this.setupTopics();
     }
-    async send(topic, payload, headers) {
+    setupTopics() {
+        const topics = this.config.topics || [];
+        for (const topicCfg of topics) {
+            if (topicCfg.create_if_missing) {
+                const req = {
+                    topic: topicCfg.name,
+                    fifo: topicCfg.config?.fifo || false,
+                    retention_bytes: topicCfg.config?.retention_bytes || 0,
+                };
+                this.client.CreateTopic(req, (err, res) => {
+                    // Ignore errors (e.g. topic already exists)
+                    if (err) {
+                        // console.warn(`Failed to create topic ${topicCfg.name}:`, err.message);
+                    }
+                });
+            }
+        }
+    }
+    async send(topic, payload) {
+        let data;
+        const headers = {};
+        if (Buffer.isBuffer(payload)) {
+            data = payload;
+            headers['payload-type'] = 'bytes';
+        }
+        else if (typeof payload === 'string') {
+            data = Buffer.from(payload, 'utf-8');
+            headers['payload-type'] = 'string';
+        }
+        else if (typeof payload === 'object') {
+            data = Buffer.from(JSON.stringify(payload), 'utf-8');
+            headers['payload-type'] = 'json';
+        }
+        else {
+            throw new Error('Payload must be Buffer, string, or object');
+        }
         const req = {
             topic,
-            payload: Buffer.from(JSON.stringify(payload)),
-            headers: headers || {},
+            payload: data,
+            headers,
         };
         return new Promise((resolve, reject) => {
             this.client.Publish(req, (err, res) => {
                 if (err)
                     return reject(err);
-                resolve({ id: res.id, offset: res.offset });
+                resolve();
             });
         });
     }
+    close() {
+        this.client.close();
+    }
 }
 exports.Producer = Producer;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pulse-sdk",
-  "version": "0.0.4",
+  "version": "0.0.6",
   "description": "Pulse SDK for Node.js/TypeScript",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",

package/dist/consumerManager.d.ts

@@ -1,4 +0,0 @@
-type Handler = (msg: any) => void;
-export declare function shutdownAll(): Promise<void>;
-export declare function registerSharedHandler(grpcUrl: string, topic: string, consumerName: string, handler: Handler): () => void;
-export {};

package/dist/consumerManager.js

@@ -1,191 +0,0 @@
-"use strict";
-Object.defineProperty(exports, "__esModule", { value: true });
-exports.shutdownAll = shutdownAll;
-exports.registerSharedHandler = registerSharedHandler;
-const client_1 = require("./proto/client");
-const message_1 = require("./message");
-let suppressStreamWarnings = false;
-async function shutdownAll() {
-    // When shutting down tests/teardown, suppress stream warnings so Jest doesn't
-    // complain about logs after tests are finished.
-    suppressStreamWarnings = true;
-    for (const [k, entry] of Array.from(registry.entries())) {
-        try {
-            if (entry.stream) {
-                try {
-                    entry.stream.removeAllListeners();
-                }
-                catch (_) { }
-                try {
-                    if (entry.stream.cancel)
-                        entry.stream.cancel();
-                }
-                catch (_) { }
-                entry.stream = null;
-            }
-            try {
-                entry.handlers.clear();
-            }
-            catch (_) { }
-        }
-        catch (_) {
-            // ignore individual errors
-        }
-        try {
-            if (entry.client && typeof entry.client.close === 'function')
-                entry.client.close();
-        }
-        catch (_) { }
-        try {
-            registry.delete(k);
-        }
-        catch (_) { }
-    }
-}
-const registry = new Map();
-function keyFor(grpcUrl, topic, consumerName) {
-    return `${grpcUrl}::${topic}::${consumerName}`;
-}
-function registerSharedHandler(grpcUrl, topic, consumerName, handler) {
-    const k = keyFor(grpcUrl, topic, consumerName);
-    let entry = registry.get(k);
-    if (!entry) {
-        const client = (0, client_1.createClient)(grpcUrl);
-        entry = { topic, consumerName, client, stream: null, handlers: new Set(), nextIndex: 0, grpcUrl };
-        // add handler before starting the stream to avoid losing early messages
-        entry.handlers.add(handler);
-        registry.set(k, entry);
-        startStream(entry);
-        return () => {
-            // unregister
-            const e = registry.get(k);
-            if (!e)
-                return;
-            e.handlers.delete(handler);
-            if (e.handlers.size === 0) {
-                try {
-                    if (e.stream && e.stream.cancel)
-                        e.stream.cancel();
-                }
-                catch (err) { }
-                registry.delete(k);
-            }
-        };
-    }
-    entry.handlers.add(handler);
-    return () => {
-        // unregister
-        const e = registry.get(k);
-        if (!e)
-            return;
-        e.handlers.delete(handler);
-        if (e.handlers.size === 0) {
-            // cleanup stream
-            try {
-                if (e.stream && e.stream.cancel)
-                    e.stream.cancel();
-            }
-            catch (e) {
-                // ignore
-            }
-            registry.delete(k);
-        }
-    };
-}
-function startStream(entry) {
-    const req = { topic: entry.topic, consumer_name: entry.consumerName, offset: 0 };
-    const stream = entry.client.Consume(req);
-    entry.stream = stream;
-    // debug: console.log('consumerManager.startStream', entry.grpcUrl, entry.topic, entry.consumerName);
-    stream.on('data', (msg) => {
-        const message = new message_1.Message(msg);
-        const handlers = Array.from(entry.handlers);
-        if (handlers.length === 0)
-            return;
-        if (entry.nextIndex === undefined)
-            entry.nextIndex = 0;
-        const h = handlers[entry.nextIndex % handlers.length];
-        entry.nextIndex = (entry.nextIndex + 1) % handlers.length;
-        try {
-            // run handler inside context providing the stub for commit()
-            // debug: console.log('consumerManager.dispatch', entry.topic, 'offset', message.offset, 'handlerIndex', entry.nextIndex);
-            (0, message_1.runWithContext)({ stub: entry.client, topic: entry.topic, consumerName: entry.consumerName, offset: message.offset }, () => {
-                h(message);
-            });
-        }
-        catch (e) {
-            console.warn('handler error', e);
-        }
-    });
-    stream.on('error', (e) => {
-        // Log once and clean up the registry entry to avoid reconnect storms and test leaks
-        if (!suppressStreamWarnings) {
-            // Ignore normal client-side cancellations which happen during unregister
-            // and shutdown; only log unexpected stream errors.
-            try {
-                const code = e && typeof e.code !== 'undefined' ? e.code : null;
-                if (code !== 1) {
-                    console.warn('shared consumer stream error for', entry.topic, e);
-                }
-            }
-            catch (_) {
-                // if anything goes wrong determining code, log the error
-                console.warn('shared consumer stream error for', entry.topic, e);
-            }
-        }
-        try {
-            if (entry.stream) {
-                try {
-                    entry.stream.removeAllListeners();
-                }
-                catch (_) { }
-                try {
-                    if (entry.stream.cancel)
-                        entry.stream.cancel();
-                }
-                catch (_) { }
-                entry.stream = null;
-            }
-        }
-        catch (err) {
-            // ignore
-        }
-        try {
-            try {
-                if (entry.client && typeof entry.client.close === 'function')
-                    entry.client.close();
-            }
-            catch (_) { }
-            registry.delete(keyFor(entry.grpcUrl || '', entry.topic, entry.consumerName));
-        }
-        catch (err) {
-            // ignore
-        }
-    });
-    stream.on('end', () => {
-        // stream ended; clean up entry
-        try {
-            if (entry.stream) {
-                try {
-                    entry.stream.removeAllListeners();
-                }
-                catch (_) { }
-                try {
-                    if (entry.stream.cancel)
-                        entry.stream.cancel();
-                }
-                catch (_) { }
-                entry.stream = null;
-            }
-            try {
-                if (entry.client && typeof entry.client.close === 'function')
-                    entry.client.close();
-            }
-            catch (_) { }
-            registry.delete(keyFor(entry.grpcUrl || '', entry.topic, entry.consumerName));
-        }
-        catch (err) {
-            // ignore
-        }
-    });
-}