@palmetto/pubsub 1.0.0

package/src/bullmq/README.md

@@ -0,0 +1,63 @@
+# @palmetto/pubsub
+
+The BullMq transport provider for @palmetto/pubsub
+
+## Installation
+
+```sh
+yarn add @palmetto/pubsub bullmq zod
+```
+
+## Getting started
+
+1. Define your connection string
+
+   ```ts
+   const config: ConnectionOptions = {
+     host: "localhost",
+     port: 6379,
+   };
+   ```
+
+2. Create the publisher
+
+   ```ts
+   const publisher = new Publisher(console, [BullMqPublisher(config, console)]);
+   ```
+
+3. Create the subscriber
+
+   ```ts
+   const subscriber = new Subscriber(console, [
+     new BullMqSubscriber(config, console),
+   ]);
+   ```
+
+4. Create the queue configuration
+
+   ```ts
+   const queue: BullMqQueueConfiguration = {
+     name: "my-queue-name",
+     schema: MyModelSchema,
+     transport: BULLMQ_TRANSPORT,
+     job: "my-optional-job-name",
+   };
+   ```
+
+5. Publish a message
+
+   ```ts
+   const message: MyModel = {... };
+
+   await publisher.publish(config, message);
+   ```
+
+6. Subscribe to a queue
+
+   ```ts
+
+   subscriber.addSubscriber(queue, (message: MyModel, context: BullMqMessageContext) => {
+       ...
+       return MessageResult.Ok;
+   });
+   ```

package/src/rabbitmq/README.md

@@ -0,0 +1,184 @@
+# @palmetto/pubsub
+
+The RabbitMq (AMQP) transport provider for @palmetto/pubsub
+
+## Installation
+
+```sh
+yarn add @palmetto/pubsub amqp-connection-manager amqplib zod
+```
+
+## Usage
+
+1. Define your connection string
+
+   ```ts
+   const config: RabbitMqConnectionConfig = {
+     host: "amqp://guest:guest@localhost:5672/",
+   };
+   ```
+
+2. Create the connection
+
+   ```ts
+   const connection = new RabbitMqConnection(config, console);
+   ```
+
+3. Create the publisher
+
+   ```ts
+   const publisher = new Publisher(console, [
+     RabbitMqPublisher(connection, console),
+   ]);
+   ```
+
+4. Create the subscriber
+
+   ```ts
+   const subscriber = new Subscriber(console, [
+     new RabbitMqSubscriber(connection, console),
+   ]);
+   ```
+
+5. Create the queue configuration
+
+   ```ts
+   const queue: RabbitQueueExchangeConfiguration = {
+     name: "my-queue-name",
+     schema: MyModelSchema,
+     transport: RABBITMQ_TRANSPORT,
+     retries: 360,
+     retryDelay: 1_000,
+   };
+   ```
+
+6. Publish a message
+
+   ```ts
+   const message: MyModel = {... };
+
+   await publisher.publish(config, message);
+   ```
+
+7. Subscribe to a queue
+
+   ```ts
+   subscriber.addSubscriber(queue, (message: MyModel, context: RabbitMqMessageContext) => {
+       ...
+       return MessageResult.Ok;
+   });
+   ```
+
+8. Subscribe to the dead-letter queue if you want to re-process completely failed messages. Set the `queueType` to `"dead-letter"`
+
+   ```ts
+   const dlQueue = {
+     ...queue,
+     queueType: "dead-letter",
+   };
+
+   subscriber.addSubscriber(dlQueue, (message: MyModel, context: RabbitMqMessageContext) => {
+      ...
+      return MessageResult.Ok; // returning Ok will remove the message from the dlq
+      return MessageResult.Fail; // returning Fail will remove the message from the dlq
+      return MessageResult.Retry; // returning Retry will move the message to the retry queue and back into the normal queue processing
+   });
+   ```
+
+## Message failures
+
+Message failures in RabbitMq subscribers are handled using special queues and exchanges. A retry queue is used to enable retry delays. A dead-letter queue is used to store messages that fail and run out of retries.
+
+### Retrying messages
+
+These properties configure message retries:
+
+- `retries` : specifies the number of retries after which the message is discarded. When this value is undefined, the message will retry forever.
+- `retryDelay`: specifies the number of milliseconds between retries.
+
+#### The retry queue
+
+The retry queue is created using the queue name and ends with `.rlq`. The queue is configured with a dead-letter exchange that sends messages back to the original queue. A retry exchange with a default name of the queue ending with `.rlx` is bound to the retry queue. When a `retryDelay` is defined, the original message is re-published into the retry exchange with an `expiration` of the `retryDelay`. When the message is at the head of the retry queue and the expiration has elapsed, then RabbitMq removes the message from the queue and publishes it back to the original queue.
+
+It's important to set a `retryDelay` that works for all messages of the queue. The `expiration` cannot be changed after starting. Every message must have the same expiration in order for retries to work correctly. Only the message at the head of the queue is removed at the expiration time. If there are more messages in the queue with earlier expirations, they will not be removed (re-tried) until they are at the head of the queue.
+
+### Failing messages
+
+A dead-letter queue is created using the queue name and ends with `.dlq`. A dead-letter exchange is created using the queue name and ends with `.dlx`. Messages that fail are nack'ed to rabbit and are published to the main queue's dead-letter exchange.
+
+Messages are sent to the dead-letter exchange when the handler returns `MessageResult.Fail` or when the message exceeds the number of `retries` configured.
+
+## Queue and Exchange Configuration
+
+The simplest way to configure PubSub RabbitMq is to pass a single name for the queue and let the framework decide the rest.
+
+```ts
+const config: RabbitQueueExchangeConfiguration = {
+  name: "my.queue",
+  transport: RABBITMQ_TRANSPORT,
+  schema: MyModelSchema,
+};
+```
+
+This configuration creates the following queues:
+
+- `my.queue` the main queue which receives all messages published using the config
+  - dead-letter-exchange is set to `my.queue.dlx`
+- `my.queue.dlq` the main queue's dead-letter queue where failed messages end up
+- `my.queue.rtq` the main queue's retry queue where messages to be retried end up
+  - dead-letter-exchange is set to the main queue's exchange so that messages are retried
+
+And these direct exchanges:
+
+- `my.queue` the main exchange that is bound to the main queue
+- `my.queue.dlx` the dead-letter exchange that receives failed messages from the main queue
+- `my.queue.rtx` the retry exchange that receives messages to retry. it is bound to the retry queue
+
+### Topic exchanges
+
+Topic exchanges provide the ability to use a single exchange to copy messages to multiple queues simultaneously. When subscribing to a topic exchange, you must also provide a subscriber name to include in the queue name. This ensures that there is a unique queue for each subscriber of the exchange.
+
+```ts
+const config: RabbitQueueExchangeConfiguration = {
+  name: "my.queue",
+  transport: RABBITMQ_TRANSPORT,
+  schema: MyModelSchema,
+  exchangeType: "topic",
+  topicSubscriberName: "my-api-name",
+  retries: 100,
+  retryDelay: 10_000,
+};
+```
+
+This configuration would create a main queue called `my.queue.my-api-name`
+
+### Custom names
+
+You can override all the queue, exchange names and routing keys for each `queueType`
+
+```ts
+const config: RabbitQueueExchangeConfiguration = {
+  name: "my.queue",
+  transport: RABBITMQ_TRANSPORT,
+  schema: MyModelSchema,
+  overrides: {
+    names: {
+      default: {
+        queueName: "my.queue.name",
+        exchangeName: "some.exchange",
+        routingKey: "some.routing.key",
+      },
+      "dead-letter": {
+        queueName: "my.dead.letter.queue",
+        exchangeName: "my.dead.letter.exchange",
+        routingKey: "some.routing.key",
+      },
+      retry: {
+        queueName: "my.retry.queue",
+        exchangeName: "my.retry.exchange",
+        routingKey: "some.routing.key",
+      },
+    },
+  },
+};
+```