@aforoai/mqtt-metering 1.0.0 → 1.2.2

package/README.md

@@ -2,7 +2,7 @@
 
 Meter MQTT traffic into Aforo two ways: hook an Aedes broker you operate to meter every PUBLISH/SUBSCRIBE/CONNECT/DISCONNECT, or wrap an `mqtt.js` client to meter what it publishes and receives against a third-party broker (AWS IoT, HiveMQ Cloud, EMQ X Cloud).
 
-**Version:** 1.0.0 · Apache-2.0 · [Changelog](CHANGELOG.md) · [User guide](USER_GUIDE.md)
+**Version:** 1.2.2 · Apache-2.0 · [Changelog](CHANGELOG.md) · [User guide](USER_GUIDE.md)
 
 | Mode | When to use | Entry point |
 |---|---|---|
@@ -19,7 +19,7 @@ npm i @aforoai/mqtt-metering aedes   # broker mode
 npm i @aforoai/mqtt-metering mqtt    # client mode
 ```
 
-> **Not yet on the public npm registry — install from source for now.** `aedes` and `mqtt` are **optional** peer dependencies — install only the one your mode needs.
+> **Install `1.2.2` or later. `1.0.0` on npm was built from an older copy of this code and lacks the fixes listed in the changelog.** If `1.2.2` is not on npm yet, install from source with the steps below. `aedes` and `mqtt` are **optional** peer dependencies — install only the one your mode needs.
 
 ```bash
 # from the SDKs repo root
@@ -89,7 +89,8 @@ Each event uses `metricName: "mqtt_broker.<event>"` (`mqtt_broker.publish`, `mqt
 | `emitDeliverEvents` | `boolean` | `false` | Emit a `DELIVER` event per fan-out delivery. Off → `DELIVER` events are dropped (both modes). |
 | `flushCount` | `number` | `200` | Buffered events that trigger an immediate flush. Highest default of the SDKs — MQTT is very high-volume. |
 | `flushIntervalMs` | `number` | `2000` | Max ms before a partial batch is flushed. |
-| `onError` | `(error: Error) => void` | logs to `console.error` | Called when a batch is dropped: after 3 attempts on network errors / 408 / 429 (honouring `Retry-After`) / 5xx, immediately on any other 4xx (not retried), and when a 202 reports per-event failures (`errors[].message`). |
+| `onError` | `(error: Error) => void` | logs to `console.error` | Called once per delivery problem: a batch dropped after 3 attempts (network errors / 408 / 429 honouring `Retry-After` / 5xx), a batch refused with any other 4xx (not retried; the message includes the ingestor's `errors[].message`), per-event failures reported in a 2xx response, and an unusable `executionStatus`. Exceptions it throws are swallowed. |
+| `onDrop` | `(events, reason) => void` | none | Receives events that were permanently dropped, with reason `invalid`, `rejected` or `retry_exhausted`. See [Dropped events](#dropped-events). |
 
 Mode-specific options:
 
@@ -103,7 +104,42 @@ Mode-specific options:
 
 Every event carries `mqttQos` (0/1/2) and `mqttRetained` — use them in Aforo rate-plan filter conditions to price QoS ≥ 1 or retained messages separately.
 
-Exported symbols: `AforoMqttBilling` (with `wrapAedesBroker` / `wrapMqttClient` / `shutdown`) and the `AforoMqttConfig` / `AedesBrokerOptions` / `MqttClientOptions` types.
+Exported symbols: `AforoMqttBilling` (with `wrapAedesBroker` / `wrapMqttClient` / `shutdown` / `droppedCount`), `mqttConnectTopic` / `mqttDisconnectTopic`, `DEFAULT_PRODUCT_TYPE`, and the `AforoMqttConfig` / `AedesBrokerOptions` / `MqttClientOptions` / `MqttUsageEvent` / `DropReason` types.
+
+## Execution status
+
+Outcome-based pricing bills each event at the weight set for its `executionStatus`; events without one bill at full price. The SDK trims and upper-cases the value and leaves the key out of the event when it is unset or blank. Accepted values: `SUCCESS`, `PARTIAL`, `TIMEOUT`, `ERROR`, `VALIDATION_FAILED`, `FAILED`, `FAILURE`, `CANCELLED`, `PENDING`, `BLOCKED`, `HITL_REQUIRED`. Any other value (or a Promise returned by an async resolver) is reported through `onError` and left off the event — the event itself is still sent — because the server would reject the event. Resolvers must be synchronous.
+
+The SDK never sets `executionStatus` on its own: the broker hook sees no success or failure signal, and client publishes are metered before the broker acknowledges them. Pass it yourself, as a string or as a function called for each event:
+
+```typescript
+billing.wrapAedesBroker(broker, {
+  resolveCustomerId: (clientId) => customerIdLookup(clientId),
+  // (event, clientId) => status | undefined
+  executionStatus: (event) => (event.mqttTopic.startsWith('alerts/') ? 'SUCCESS' : undefined),
+});
+
+billing.wrapMqttClient(client, { customerId: 'cust_42', executionStatus: 'SUCCESS' });
+```
+
+## Dropped events
+
+The SDK does not throw into your broker or client handlers for event content. An event that cannot be delivered is counted in `billing.droppedCount`, logged with `console.warn`, and passed to the optional `onDrop(events, reason)` hook. Events handed to the hook keep their `idempotencyKey`, so storing them and re-submitting later is dedup-safe. Exceptions thrown by the hook are swallowed.
+
+| Reason | When |
+|---|---|
+| `invalid` | The event failed a client-side check and was never buffered or sent: `customerId` longer than 64 characters, a missing or whitespace-only `mqttTopic`, or `productType` longer than 20. These are never truncated. A topic longer than 500 characters, or a client id longer than 128, is not a drop: it is cut to the limit and the event is sent (one warning per label). The warning names the field, the limit and the value (first 80 characters); it is logged for the first occurrence per field and for every 1000th after that. |
+| `rejected` | The ingestor refused the batch with a 4xx other than 408 / 429 (not retried), or named individual events in the `errors[]` of a 2xx response — then only those events are dropped. When a 2xx reports `failed > 0` without a usable `index`, the count is added to `droppedCount` and `onDrop` is not called. |
+| `retry_exhausted` | Network errors, 408, 429 or 5xx persisted through 3 attempts (1s / 2s / 4s backoff; `Retry-After` honoured up to 30 s). |
+
+A call with no resolvable customer id is not billable; it is skipped and is not counted as a drop.
+
+```ts
+const billing = new AforoMqttBilling({
+  // …
+  onDrop: (events, reason) => deadLetterQueue.push({ reason, events }),
+});
+```
 
 ## Walk me through it
 

package/dist/index.d.ts

@@ -47,12 +47,45 @@ export interface AforoMqttConfig {
     emitDeliverEvents?: boolean;
     /** Callback for terminal flush failures. */
     onError?: (error: Error) => void;
+    /**
+     * Opt-in hook receiving events that were permanently dropped (retry
+     * exhaustion, a rejection by the ingestor, or a failed client-side check —
+     * see DropReason). Events keep their idempotency keys,
+     * so persisting and re-submitting them after recovery is dedup-safe.
+     * Exceptions thrown by the hook are swallowed. Default: none (drops are
+     * still counted in droppedCount and WARN-logged).
+     */
+    onDrop?: (events: MqttUsageEvent[], reason: DropReason) => void;
 }
+/**
+ * Why events were permanently dropped. `retry_exhausted`: the ingestor stayed
+ * unreachable / kept answering 408, 429 or 5xx. `rejected`: the ingestor
+ * refused the batch (non-retryable 4xx) or individual events of an accepted
+ * batch. `invalid`: the event failed a client-side check and was never sent.
+ * The buffer is unbounded (drained at flush start), so unlike the core SDK
+ * there is no 'overflow' reason here.
+ */
+export type DropReason = 'retry_exhausted' | 'rejected' | 'invalid';
 export interface AedesBrokerOptions {
     /** Map an MQTT client ID to an Aforo customer ID. Required. */
     resolveCustomerId: (clientId: string, username?: string) => string | undefined | Promise<string | undefined>;
     /** Optional per-client metadata (tags, device type). */
     resolveMetadata?: (clientId: string) => Record<string, unknown> | undefined;
+    /**
+     * Optional outcome of the request (used by OUTCOME_BASED pricing, which bills
+     * each event at the weight set for its status). Events without a status bill
+     * at full price. Trimmed and upper-cased by the SDK; blank is treated as absent.
+     * Accepted values: SUCCESS, PARTIAL, TIMEOUT, ERROR, VALIDATION_FAILED, FAILED,
+     * FAILURE, CANCELLED, PENDING, BLOCKED, HITL_REQUIRED (max 20 chars). Any other
+     * value — or a Promise from an async resolver — is reported through `onError`
+     * and left off the event (the event is still sent), because the server would
+     * reject the event. Resolvers must be synchronous.
+     *
+     * MQTT packets carry no success/failure signal the broker hook sees, so the
+     * SDK never derives one: pass a fixed string, or a function called for each
+     * event (with the MQTT client ID) that returns the status or undefined.
+     */
+    executionStatus?: string | ((event: MqttUsageEvent, clientId: string) => string | undefined);
     /** Product type for events from this broker. Default: the client-level `productType`. */
     productType?: string;
 }
@@ -61,6 +94,21 @@ export interface MqttClientOptions {
     customerId: string;
     /** Fixed client identifier (defaults to mqtt.js connection options.clientId). */
     clientId?: string;
+    /**
+     * Optional outcome of the request (used by OUTCOME_BASED pricing, which bills
+     * each event at the weight set for its status). Events without a status bill
+     * at full price. Trimmed and upper-cased by the SDK; blank is treated as absent.
+     * Accepted values: SUCCESS, PARTIAL, TIMEOUT, ERROR, VALIDATION_FAILED, FAILED,
+     * FAILURE, CANCELLED, PENDING, BLOCKED, HITL_REQUIRED (max 20 chars). Any other
+     * value — or a Promise from an async resolver — is reported through `onError`
+     * and left off the event (the event is still sent), because the server would
+     * reject the event. Resolvers must be synchronous.
+     *
+     * Never derived by the SDK (publishes are metered before the broker
+     * acknowledges them): pass a fixed string, or a function called for each
+     * event that returns the status or undefined.
+     */
+    executionStatus?: string | ((event: MqttUsageEvent) => string | undefined);
     /** Product type for events from this client. Default: the client-level `productType`. */
     productType?: string;
 }
@@ -75,6 +123,24 @@ export declare const DEFAULT_PRODUCT_TYPE = "MQTT_BROKER";
  */
 export declare const mqttConnectTopic: (clientId: string) => string;
 export declare const mqttDisconnectTopic: (clientId: string) => string;
+type MqttEventType = 'PUBLISH' | 'DELIVER' | 'SUBSCRIBE' | 'UNSUBSCRIBE' | 'CONNECT' | 'DISCONNECT';
+export interface MqttUsageEvent {
+    customerId: string;
+    metricName: string;
+    quantity: number;
+    occurredAt: string;
+    idempotencyKey: string;
+    productType: string;
+    mqttTopic: string;
+    mqttQos: number;
+    mqttRetained: boolean;
+    mqttEventType: MqttEventType;
+    mqttClientId: string;
+    dataBytes: number;
+    metadata?: Record<string, unknown>;
+    /** Normalized (trimmed, upper-cased) execution status; omitted when not set. */
+    executionStatus?: string;
+}
 /** Minimal Aedes broker surface — matches the `aedes` package. */
 interface MinimalAedes {
     on(event: 'publish', fn: (packet: any, client: any) => void): void;
@@ -100,17 +166,62 @@ export declare class AforoMqttBilling {
     private readonly flushIntervalMs;
     private readonly emitDeliverEvents;
     private readonly onError;
+    private readonly onDrop?;
     private buffer;
     private flushTimer;
+    private dropped;
+    /** Invalid-event count per field, for WARN throttling. */
+    private readonly invalidSeen;
+    /** Request-derived labels already reported as truncated (one WARN per label). */
+    private readonly truncationWarned;
     constructor(config: AforoMqttConfig);
     wrapAedesBroker(broker: MinimalAedes, options: AedesBrokerOptions): void;
     wrapMqttClient(client: MinimalMqttClient, options: MqttClientOptions): void;
+    /**
+     * Bound an MQTT label (topic, client id) to the ingestor's limit. The event
+     * is still sent; one WARN per label name per client. Identity fields
+     * (customerId, productType) never go through here — an over-long one drops
+     * the event as 'invalid'.
+     */
+    private requestLabel;
+    /** Topic off an incoming packet, bound to the ingestor's mqttTopic limit. */
+    private topicLabel;
+    /** Client id a connecting client chose, bound to the ingestor's mqttClientId limit. */
+    private clientIdLabel;
     private push;
     private flush;
     private send;
-    /** A 202 can still carry per-event rejections: `{accepted, duplicates, failed, errors:[{index, message}]}`. */
-    private reportPartialFailures;
+    /**
+     * A 2xx can still carry per-event rejections:
+     * `{accepted, duplicates, failed, errors:[{index, message}]}`. Events the
+     * response names by index are dropped with reason 'rejected'; a `failed`
+     * count the response does not attribute to an index is counted only.
+     */
+    private handlePartialFailures;
+    /** Invoke onError; a hook bug must never break metering or flushing. */
+    private reportError;
+    /** Number of events permanently dropped since this instance was created. */
+    get droppedCount(): number;
+    /**
+     * Account for permanently lost events: bump the counter, WARN-log, and
+     * invoke the opt-in onDrop hook. The buffer is drained at flush start, so
+     * drops here are bounded by flush cadence — no log throttle needed.
+     */
+    private recordDrop;
+    /**
+     * Account for an event that failed a client-side check: never buffered,
+     * never sent. Runs on the hot path, so the WARN is throttled per field
+     * (first occurrence, then every 1000th).
+     */
+    private recordInvalid;
+    private notifyDrop;
     private startTimer;
+    /** Flush any buffered events and stop the background timer. Call before process exit. */
     shutdown(): Promise<void>;
 }
+/**
+ * Cut `value` to at most `max` UTF-16 code units — how the ingestor counts
+ * (`String.length()` in Java) — without leaving half a surrogate pair.
+ */
+export declare function truncateToLimit(value: string, max: number): string;
 export {};

package/dist/index.js

@@ -32,7 +32,9 @@
  */
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.AforoMqttBilling = exports.mqttDisconnectTopic = exports.mqttConnectTopic = exports.DEFAULT_PRODUCT_TYPE = void 0;
-const SDK_VERSION = '1.0.0';
+exports.truncateToLimit = truncateToLimit;
+const node_crypto_1 = require("node:crypto");
+const SDK_VERSION = '1.2.2';
 /** Default top-level `productType` for this SDK. */
 exports.DEFAULT_PRODUCT_TYPE = 'MQTT_BROKER';
 /** Upper bound on a server-requested Retry-After wait. */
@@ -53,6 +55,9 @@ const MAX_BATCH_EVENTS = 1000;
 /** usage-ingestor limits (IngestUsageEventRequest). */
 const MAX_CUSTOMER_ID = 64;
 const MAX_IDEMPOTENCY_KEY = 255;
+const MAX_PRODUCT_TYPE = 20;
+const MAX_MQTT_TOPIC = 500;
+const MAX_MQTT_CLIENT_ID = 128;
 class AforoMqttBilling {
     config;
     productType;
@@ -60,8 +65,14 @@ class AforoMqttBilling {
     flushIntervalMs;
     emitDeliverEvents;
     onError;
+    onDrop;
     buffer = [];
     flushTimer = null;
+    dropped = 0;
+    /** Invalid-event count per field, for WARN throttling. */
+    invalidSeen = new Map();
+    /** Request-derived labels already reported as truncated (one WARN per label). */
+    truncationWarned = new Set();
     constructor(config) {
         this.config = {
             tenantId: config.tenantId,
@@ -74,11 +85,14 @@ class AforoMqttBilling {
         this.flushIntervalMs = config.flushIntervalMs ?? 2000;
         this.emitDeliverEvents = config.emitDeliverEvents ?? false;
         this.onError = config.onError ?? ((err) => console.error('[aforo-mqtt]', err.message));
+        this.onDrop = config.onDrop;
         this.startTimer();
     }
     // ── Broker-side integration (Aedes) ─────────────────────────────
     wrapAedesBroker(broker, options) {
         const resolveCustomer = async (clientId, username) => Promise.resolve(options.resolveCustomerId(clientId, username));
+        const status = options.executionStatus;
+        const brokerStatus = (clientId) => typeof status === 'function' ? (event) => status(event, clientId) : status;
         broker.on('publish', async (packet, client) => {
             if (!client)
                 return; // broker-originated publishes skip billing
@@ -90,15 +104,15 @@ class AforoMqttBilling {
             const bytes = estimateBytes(payload);
             this.push({
                 customerId,
-                mqttTopic: packet.topic,
+                mqttTopic: this.topicLabel(packet.topic),
                 mqttQos: packet.qos ?? 0,
                 mqttRetained: !!packet.retain,
                 mqttEventType: 'PUBLISH',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: bytes,
                 metadata: options.resolveMetadata?.(clientId),
-            });
+            }, brokerStatus(clientId), clientId);
         });
         broker.on('subscribe', async (subs, client) => {
             const clientId = client?.id;
@@ -111,15 +125,15 @@ class AforoMqttBilling {
             for (const s of arr) {
                 this.push({
                     customerId,
-                    mqttTopic: s.topic,
+                    mqttTopic: this.topicLabel(s.topic),
                     mqttQos: s.qos ?? 0,
                     mqttRetained: false,
                     mqttEventType: 'SUBSCRIBE',
-                    mqttClientId: clientId,
+                    mqttClientId: this.clientIdLabel(clientId),
                     productType: options.productType,
                     dataBytes: 0,
                     metadata: options.resolveMetadata?.(clientId),
-                });
+                }, brokerStatus(clientId), clientId);
             }
         });
         broker.on('unsubscribe', async (unsubs, client) => {
@@ -133,15 +147,15 @@ class AforoMqttBilling {
             for (const topic of arr) {
                 this.push({
                     customerId,
-                    mqttTopic: typeof topic === 'string' ? topic : topic?.topic ?? '',
+                    mqttTopic: this.topicLabel(typeof topic === 'string' ? topic : topic?.topic ?? ''),
                     mqttQos: 0,
                     mqttRetained: false,
                     mqttEventType: 'UNSUBSCRIBE',
-                    mqttClientId: clientId,
+                    mqttClientId: this.clientIdLabel(clientId),
                     productType: options.productType,
                     dataBytes: 0,
                     metadata: options.resolveMetadata?.(clientId),
-                });
+                }, brokerStatus(clientId), clientId);
             }
         });
         broker.on('client', async (client) => {
@@ -151,15 +165,17 @@ class AforoMqttBilling {
                 return;
             this.push({
                 customerId,
-                mqttTopic: (0, exports.mqttConnectTopic)(clientId), // CONNECT has no topic; mqttTopic is required
+                // CONNECT has no topic; mqttTopic is required. Built from the bounded
+                // client id so the stand-in topic keeps its `/connected` suffix.
+                mqttTopic: (0, exports.mqttConnectTopic)(this.clientIdLabel(clientId)),
                 mqttQos: 0,
                 mqttRetained: false,
                 mqttEventType: 'CONNECT',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: 0,
                 metadata: options.resolveMetadata?.(clientId),
-            });
+            }, brokerStatus(clientId), clientId);
         });
         broker.on('clientDisconnect', async (client) => {
             const clientId = client.id;
@@ -168,15 +184,15 @@ class AforoMqttBilling {
                 return;
             this.push({
                 customerId,
-                mqttTopic: (0, exports.mqttDisconnectTopic)(clientId),
+                mqttTopic: (0, exports.mqttDisconnectTopic)(this.clientIdLabel(clientId)),
                 mqttQos: 0,
                 mqttRetained: false,
                 mqttEventType: 'DISCONNECT',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: 0,
                 metadata: options.resolveMetadata?.(clientId),
-            });
+            }, brokerStatus(clientId), clientId);
         });
     }
     // ── Client-side integration (mqtt.js) ───────────────────────────
@@ -185,79 +201,100 @@ class AforoMqttBilling {
         client.on('connect', () => {
             this.push({
                 customerId: options.customerId,
-                mqttTopic: (0, exports.mqttConnectTopic)(clientId),
+                mqttTopic: (0, exports.mqttConnectTopic)(this.clientIdLabel(clientId)),
                 mqttQos: 0,
                 mqttRetained: false,
                 mqttEventType: 'CONNECT',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: 0,
-            });
+            }, options.executionStatus, clientId);
         });
         client.on('close', () => {
             this.push({
                 customerId: options.customerId,
-                mqttTopic: (0, exports.mqttDisconnectTopic)(clientId),
+                mqttTopic: (0, exports.mqttDisconnectTopic)(this.clientIdLabel(clientId)),
                 mqttQos: 0,
                 mqttRetained: false,
                 mqttEventType: 'DISCONNECT',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: 0,
-            });
+            }, options.executionStatus, clientId);
         });
         client.on('message', (topic, payload, packet) => {
             this.push({
                 customerId: options.customerId,
-                mqttTopic: topic,
+                mqttTopic: this.topicLabel(topic),
                 mqttQos: packet?.qos ?? 0,
                 mqttRetained: !!packet?.retain,
                 mqttEventType: 'DELIVER',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: estimateBytes(payload),
-            });
+            }, options.executionStatus, clientId);
         });
         const origPublish = client.publish.bind(client);
         client.publish = (topic, message, opts, cb) => {
             this.push({
                 customerId: options.customerId,
-                mqttTopic: topic,
+                mqttTopic: this.topicLabel(topic),
                 mqttQos: opts?.qos ?? 0,
                 mqttRetained: !!opts?.retain,
                 mqttEventType: 'PUBLISH',
-                mqttClientId: clientId,
+                mqttClientId: this.clientIdLabel(clientId),
                 productType: options.productType,
                 dataBytes: estimateBytes(message),
-            });
+            }, options.executionStatus, clientId);
             return origPublish(topic, message, opts, cb);
         };
     }
     // ── Event pipeline ──────────────────────────────────────────────
-    push(partial) {
+    /**
+     * Bound an MQTT label (topic, client id) to the ingestor's limit. The event
+     * is still sent; one WARN per label name per client. Identity fields
+     * (customerId, productType) never go through here — an over-long one drops
+     * the event as 'invalid'.
+     */
+    requestLabel(field, value, max) {
+        if (typeof value !== 'string' || value.length <= max)
+            return value;
+        if (!this.truncationWarned.has(field)) {
+            this.truncationWarned.add(field);
+            console.warn(`[aforo-mqtt] ${field} was longer than the ingestor's limit and was ` +
+                `truncated to ${max} characters; the event is still sent. Logged once per label.`);
+        }
+        return truncateToLimit(value, max);
+    }
+    /** Topic off an incoming packet, bound to the ingestor's mqttTopic limit. */
+    topicLabel(topic) {
+        return typeof topic === 'string' ? this.requestLabel('mqttTopic', topic, MAX_MQTT_TOPIC) : topic;
+    }
+    /** Client id a connecting client chose, bound to the ingestor's mqttClientId limit. */
+    clientIdLabel(clientId) {
+        return typeof clientId === 'string' ? this.requestLabel('mqttClientId', clientId, MAX_MQTT_CLIENT_ID) : clientId;
+    }
+    push(partial, executionStatus, 
+    /** Untruncated client id for the idempotency key, when `partial.mqttClientId` was cut to the limit. */
+    keyClientId) {
         if (!this.emitDeliverEvents && partial.mqttEventType === 'DELIVER')
             return;
-        if (!partial.customerId || !partial.customerId.trim())
+        // No customer resolved — not billable, not a drop.
+        if (typeof partial.customerId !== 'string' || !partial.customerId.trim())
             return;
-        if (partial.customerId.length > MAX_CUSTOMER_ID) {
-            this.onError(new Error(`MQTT metering: customerId longer than ${MAX_CUSTOMER_ID} chars; event dropped`));
-            return;
-        }
-        const mqttTopic = partial.mqttTopic;
-        if (!mqttTopic || !mqttTopic.trim())
-            return; // mqttTopic is required on every MQTT_BROKER event
         const now = new Date();
-        const mqttClientId = partial.mqttClientId.slice(0, 128);
+        const mqttClientId = typeof partial.mqttClientId === 'string' ? partial.mqttClientId : String(partial.mqttClientId ?? '');
         const event = {
             ...partial,
-            mqttTopic: mqttTopic.slice(0, 500),
             mqttQos: partial.mqttQos === 1 || partial.mqttQos === 2 ? partial.mqttQos : 0,
             mqttClientId,
             metricName: `mqtt_broker.${partial.mqttEventType.toLowerCase()}`,
             quantity: 1,
             occurredAt: now.toISOString(),
-            // Topic left out (it can be 500 chars); tail-trimmed to keep the unique millis:random suffix.
-            idempotencyKey: `mqtt:${this.config.tenantId}:${mqttClientId}:${partial.mqttEventType}:${now.getTime()}:${randomSuffix()}`.slice(-MAX_IDEMPOTENCY_KEY),
+            // Minted once, here, from the untruncated client id; the topic is left
+            // out. When the client id makes the key longer than the ingestor allows,
+            // it is replaced by its SHA-256 digest; the key itself is never cut.
+            idempotencyKey: boundedIdempotencyKey(`mqtt:${this.config.tenantId}:`, typeof keyClientId === 'string' ? keyClientId : mqttClientId, `:${partial.mqttEventType}:${now.getTime()}:${randomSuffix()}`),
             productType: normalizeProductType(partial.productType) ?? this.productType,
             metadata: {
                 ...(partial.metadata ?? {}),
@@ -265,6 +302,24 @@ class AforoMqttBilling {
                 productId: this.config.productId,
             },
         };
+        const checked = checkExecutionStatus(resolveExecutionStatus(executionStatus, event));
+        if (checked.problem)
+            this.reportError(new Error(`[aforo-mqtt] ${checked.problem}`));
+        if (checked.status)
+            event.executionStatus = checked.status;
+        // An event the ingestor would reject is not sent (one bad event fails its
+        // whole batch): it is dropped here with reason 'invalid', never thrown
+        // into the broker / client handlers. mqttTopic is required on every
+        // MQTT_BROKER event.
+        const invalid = tooLong('customerId', event.customerId, MAX_CUSTOMER_ID) ??
+            requiredText('mqttTopic', event.mqttTopic) ??
+            tooLong('mqttTopic', event.mqttTopic, MAX_MQTT_TOPIC) ??
+            tooLong('mqttClientId', event.mqttClientId, MAX_MQTT_CLIENT_ID) ??
+            tooLong('productType', event.productType, MAX_PRODUCT_TYPE);
+        if (invalid) {
+            this.recordInvalid(event, invalid);
+            return;
+        }
         this.buffer.push(event);
         if (this.buffer.length >= this.flushCount) {
             void this.flush();
@@ -284,9 +339,11 @@ class AforoMqttBilling {
         const body = JSON.stringify({ events: batch });
         const maxRetries = 3;
         for (let attempt = 1; attempt <= maxRetries; attempt++) {
-            let delayMs = Math.pow(2, attempt - 1) * 1000;
+            let delayMs = Math.pow(2, attempt - 1) * 1000; // 1s, 2s, 4s
+            let res;
+            let networkError;
             try {
-                const res = await fetch(this.config.ingestorUrl.replace(/\/$/, '') + '/v1/ingest/batch', {
+                res = await fetch(this.config.ingestorUrl.replace(/\/$/, '') + '/v1/ingest/batch', {
                     method: 'POST',
                     headers: {
                         'Content-Type': 'application/json',
@@ -295,33 +352,101 @@ class AforoMqttBilling {
                     },
                     body,
                 });
+            }
+            catch (err) {
+                networkError = err;
+            }
+            if (res) {
                 if (res.ok) {
-                    await this.reportPartialFailures(res);
+                    await this.handlePartialFailures(res, batch);
                     return;
                 }
                 if (!isRetryableStatus(res.status)) {
-                    const { details } = await readErrorMessages(res);
-                    this.onError(new Error(`MQTT metering batch rejected with HTTP ${res.status}${details ? ` — ${details}` : ''} (dropped ${batch.length} events, not retried)`));
+                    // 4xx other than 408/429: the same body would be refused again.
+                    const { details } = await readBatchResult(res, batch.length);
+                    this.recordDrop(batch, 'rejected');
+                    this.reportError(new Error(`MQTT metering batch rejected with HTTP ${res.status}${details ? ` — ${details}` : ''} (dropped ${batch.length} events, not retried)`));
                     return;
                 }
                 delayMs = parseRetryAfter(res) ?? delayMs;
             }
-            catch (err) {
-                if (attempt === maxRetries) {
-                    this.onError(err);
-                    return;
-                }
+            else if (attempt === maxRetries) {
+                this.recordDrop(batch, 'retry_exhausted');
+                this.reportError(networkError ?? new Error('MQTT metering request failed'));
+                return;
             }
             if (attempt < maxRetries)
                 await sleep(delayMs);
         }
-        this.onError(new Error(`MQTT metering flush failed after ${maxRetries} attempts (dropped ${batch.length} events)`));
+        // Not re-queued: dropping avoids unbounded memory growth.
+        this.recordDrop(batch, 'retry_exhausted');
+        this.reportError(new Error(`MQTT metering flush failed after ${maxRetries} attempts (dropped ${batch.length} events)`));
+    }
+    /**
+     * A 2xx can still carry per-event rejections:
+     * `{accepted, duplicates, failed, errors:[{index, message}]}`. Events the
+     * response names by index are dropped with reason 'rejected'; a `failed`
+     * count the response does not attribute to an index is counted only.
+     */
+    async handlePartialFailures(res, batch) {
+        const { failed, details, indexes } = await readBatchResult(res, batch.length);
+        if (failed <= 0)
+            return;
+        if (indexes.length > 0)
+            this.recordDrop(indexes.map((i) => batch[i]), 'rejected');
+        const unattributed = Math.min(failed, batch.length) - indexes.length;
+        if (unattributed > 0) {
+            this.dropped += unattributed;
+            console.warn(`[aforo-mqtt] Dropped ${unattributed} event(s) — rejected, not identified by the ingestor (${this.dropped} total dropped).`);
+        }
+        this.reportError(new Error(`Aforo ingestor rejected ${failed} event(s)${details ? ` — ${details}` : ''}`));
+    }
+    /** Invoke onError; a hook bug must never break metering or flushing. */
+    reportError(err) {
+        try {
+            this.onError(err);
+        }
+        catch {
+            // ignore
+        }
+    }
+    /** Number of events permanently dropped since this instance was created. */
+    get droppedCount() {
+        return this.dropped;
+    }
+    /**
+     * Account for permanently lost events: bump the counter, WARN-log, and
+     * invoke the opt-in onDrop hook. The buffer is drained at flush start, so
+     * drops here are bounded by flush cadence — no log throttle needed.
+     */
+    recordDrop(events, reason) {
+        this.dropped += events.length;
+        console.warn(`[aforo-mqtt] Dropped ${events.length} event(s) — ${reason} (${this.dropped} total dropped).`);
+        this.notifyDrop(events, reason);
+    }
+    /**
+     * Account for an event that failed a client-side check: never buffered,
+     * never sent. Runs on the hot path, so the WARN is throttled per field
+     * (first occurrence, then every 1000th).
+     */
+    recordInvalid(event, invalid) {
+        this.dropped += 1;
+        const seen = (this.invalidSeen.get(invalid.field) ?? 0) + 1;
+        this.invalidSeen.set(invalid.field, seen);
+        if (seen === 1 || seen % 1000 === 0) {
+            console.warn(`[aforo-mqtt] Dropped 1 event — invalid: ${invalid.problem} ` +
+                `(${seen} for ${invalid.field}, ${this.dropped} total dropped).`);
+        }
+        this.notifyDrop([event], 'invalid');
     }
-    /** A 202 can still carry per-event rejections: `{accepted, duplicates, failed, errors:[{index, message}]}`. */
-    async reportPartialFailures(res) {
-        const { failed, details } = await readErrorMessages(res);
-        if (failed && failed > 0) {
-            this.onError(new Error(`Aforo ingestor rejected ${failed} event(s)${details ? ` — ${details}` : ''}`));
+    notifyDrop(events, reason) {
+        if (!this.onDrop)
+            return;
+        try {
+            this.onDrop(events, reason);
+        }
+        catch {
+            // A hook bug must never break metering or flushing.
         }
     }
     startTimer() {
@@ -329,12 +454,10 @@ class AforoMqttBilling {
             return;
         this.flushTimer = setInterval(() => { void this.flush(); }, this.flushIntervalMs);
         // Unref so the background timer never blocks host-process exit (final flush still needs shutdown()).
-        if (this.flushTimer && typeof this.flushTimer === 'object' && 'unref' in this.flushTimer) {
-            this.flushTimer.unref();
-        }
         if (typeof this.flushTimer.unref === 'function')
             this.flushTimer.unref();
     }
+    /** Flush any buffered events and stop the background timer. Call before process exit. */
     async shutdown() {
         if (this.flushTimer) {
             clearInterval(this.flushTimer);
@@ -355,6 +478,51 @@ function estimateBytes(data) {
         return data.byteLength;
     return 0;
 }
+/** Statuses the Aforo ingestor accepts (contract/ingest-contract.json, max 20 chars). */
+const CANONICAL_EXECUTION_STATUSES = new Set([
+    'SUCCESS', 'PARTIAL', 'TIMEOUT', 'ERROR', 'VALIDATION_FAILED', 'FAILED',
+    'FAILURE', 'CANCELLED', 'PENDING', 'BLOCKED', 'HITL_REQUIRED',
+]);
+/**
+ * Check a caller-supplied status. `{ status }` when canonical; `{ problem }`
+ * when it has to be left off the event (an unknown value would make the
+ * ingestor reject the event; a Promise from an async resolver can't be
+ * awaited on this path); `{}` when blank or absent.
+ */
+function checkExecutionStatus(value) {
+    if (value !== null && typeof value === 'object' && typeof value.then === 'function') {
+        // Swallow a later rejection so it doesn't surface as an unhandled rejection.
+        value.then(undefined, () => { });
+        return { problem: 'executionStatus resolver returned a Promise — resolvers must be synchronous; field omitted' };
+    }
+    const status = normalizeExecutionStatus(value);
+    if (!status)
+        return {};
+    if (!CANONICAL_EXECUTION_STATUSES.has(status)) {
+        return {
+            problem: `unknown executionStatus "${status.slice(0, 40)}" — field omitted. ` +
+                `Expected one of: ${[...CANONICAL_EXECUTION_STATUSES].join(', ')}`,
+        };
+    }
+    return { status };
+}
+/** Trim + upper-case; blank/absent → undefined (key omitted from the wire body). */
+function normalizeExecutionStatus(value) {
+    if (typeof value !== 'string')
+        return undefined;
+    const trimmed = value.trim();
+    return trimmed ? trimmed.toUpperCase() : undefined;
+}
+function resolveExecutionStatus(option, event) {
+    if (typeof option !== 'function')
+        return option;
+    try {
+        return option(event);
+    }
+    catch {
+        return undefined; // a resolver bug must never break metering
+    }
+}
 /** Trim + uppercase a product type; blank/non-string → undefined (unknown values pass through). */
 function normalizeProductType(value) {
     if (typeof value !== 'string')
@@ -362,6 +530,22 @@ function normalizeProductType(value) {
     const trimmed = value.trim();
     return trimmed ? trimmed.toUpperCase() : undefined;
 }
+/** The offending value for a WARN line, cut to 80 chars. */
+function preview(value) {
+    const text = typeof value === 'string' ? value : String(value);
+    return JSON.stringify(text.length > 80 ? `${text.slice(0, 80)}…` : text);
+}
+/** Limits mirror the ingestor's IngestUsageEventRequest; identity values are never truncated. */
+function tooLong(field, value, max) {
+    if (typeof value !== 'string' || value.length <= max)
+        return undefined;
+    return { field, problem: `${field} is ${value.length} chars, limit ${max}: ${preview(value)}` };
+}
+function requiredText(field, value) {
+    if (typeof value === 'string' && value.trim())
+        return undefined;
+    return { field, problem: `${field} is required and must not be blank: ${preview(value)}` };
+}
 /** Network errors, 408, 429 and 5xx are transient; every other 4xx (400/401/403/422...) is not. */
 function isRetryableStatus(status) {
     return status === 408 || status === 429 || status >= 500;
@@ -379,24 +563,78 @@ function parseRetryAfter(res) {
         return Math.min(Math.max(date - Date.now(), 0), MAX_RETRY_AFTER_MS);
     return undefined;
 }
-/** Reads the ingestor's `errors[].message` entries (first 5) from a response body, if any. */
-async function readErrorMessages(res) {
+/**
+ * Reads the ingestor's batch response body, if any: the `failed` count, the
+ * first 5 `errors[].message` entries, and the batch indexes `errors[]` names.
+ */
+async function readBatchResult(res, batchSize) {
+    const none = { failed: 0, details: '', indexes: [] };
     if (typeof res?.json !== 'function')
-        return { details: '' };
+        return none;
     try {
-        const body = await res.json();
-        const details = Array.isArray(body?.errors)
-            ? body.errors.slice(0, 5).map((e) => `#${e?.index}: ${e?.message}`).join('; ')
+        const body = unwrapEnvelope(await res.json());
+        const errors = Array.isArray(body?.errors) ? body.errors : undefined;
+        const details = errors
+            ? errors.slice(0, 5).map((e) => `#${e?.index}: ${e?.message}`).join('; ')
             : typeof body?.message === 'string' ? body.message : '';
-        return { failed: typeof body?.failed === 'number' ? body.failed : undefined, details };
+        const indexes = [
+            ...new Set((errors ?? [])
+                .map((e) => e?.index)
+                .filter((i) => Number.isInteger(i) && i >= 0 && i < batchSize)),
+        ].sort((a, b) => a - b);
+        const failed = typeof body?.failed === 'number' && body.failed > 0 ? body.failed : 0;
+        return { failed, details, indexes };
     }
     catch {
-        return { details: '' }; // Non-JSON or empty body — nothing to report.
+        return none; // Non-JSON or empty body — nothing to report.
     }
 }
+/**
+ * Cut `value` to at most `max` UTF-16 code units — how the ingestor counts
+ * (`String.length()` in Java) — without leaving half a surrogate pair.
+ */
+function truncateToLimit(value, max) {
+    if (value.length <= max)
+        return value;
+    let end = Math.max(0, max);
+    const last = end > 0 ? value.charCodeAt(end - 1) : 0;
+    // A high surrogate at the cut means its low half was cut off: drop it too.
+    if (last >= 0xd800 && last <= 0xdbff)
+        end -= 1;
+    return value.slice(0, end);
+}
+/**
+ * Build `head + component + tail`, where `component` is text taken from the
+ * incoming request and may be any length. Returned unchanged when it fits the
+ * ingestor's idempotencyKey limit; otherwise the component is replaced by the
+ * SHA-256 hex digest of its full, untruncated value (and, if the fixed parts
+ * alone are too long, `head + component` is). The key is never cut, so the
+ * unique tail always survives and two different components never share a key
+ * prefix by accident.
+ */
+function boundedIdempotencyKey(head, component, tail) {
+    const key = `${head}${component}${tail}`;
+    if (key.length <= MAX_IDEMPOTENCY_KEY)
+        return key;
+    const hashed = `${head}${sha256Hex(component)}${tail}`;
+    if (hashed.length <= MAX_IDEMPOTENCY_KEY)
+        return hashed;
+    return `${sha256Hex(head + component)}${tail}`;
+}
+function sha256Hex(text) {
+    return (0, node_crypto_1.createHash)('sha256').update(text, 'utf8').digest('hex');
+}
 function randomSuffix() {
     return Math.random().toString(36).slice(2, 10);
 }
 function sleep(ms) {
     return new Promise((r) => setTimeout(r, ms));
 }
+/**
+ * The ingestor wraps every 2xx JSON body in `{success, data, meta}`. Returns
+ * the inner `data` object when present, else the body unchanged (bare shape).
+ */
+function unwrapEnvelope(body) {
+    const data = body && typeof body === 'object' && !Array.isArray(body) ? body.data : undefined;
+    return data && typeof data === 'object' && !Array.isArray(data) ? data : body;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@aforoai/mqtt-metering",
-  "version": "1.0.0",
+  "version": "1.2.2",
   "description": "Aforo MQTT Metering SDK — aedes broker hooks + mqtt.js client wrapper that meters PUBLISH/DELIVER/SUBSCRIBE/CONNECT/DISCONNECT events to Aforo.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",