@relaybus/testing 0.1.0 → 0.2.0

package/README.md

@@ -1,5 +1,7 @@
 # @relaybus/testing
 
+@relaybus/testing is the test toolkit of relaybus, a reliable, storage-agnostic event bus for NestJS and Node.js: it provides the EventStore contract suite, a `FakeClock` and an in-memory transport for deterministic tests of retries and delivery.
+
 Test utilities for relaybus.
 
 ```sh
@@ -25,9 +27,18 @@ eventStoreContract('MyEventStore', {
 });
 ```
 
-It covers transactional visibility, idempotent appends, leasing and fencing,
-retries and dead letters, release, per-key ordering in commit order, starvation,
-purge, statistics, data fidelity and cross-instance claim races.
+It covers transactional visibility, idempotent appends (a handler that already
+completed is never delivered again), leasing and fencing, batched completion
+(`completeMany`), lost leases (redelivery, attempt counting for crashed workers,
+poison deliveries), retries and dead letters, release, relinquish, per-key
+ordering in commit order, starvation, purge, statistics, data fidelity and
+cross-instance claim races.
+
+A store may lease groups of deliveries (lanes) instead of single deliveries, so
+the contract never assumes that two claimers see each other's unclaimed work: a
+claimer that keeps claiming drains the workload. Pass `keepsCompletedRecords:
+true` when the store keeps a completion history; run the suite once per mode the
+store supports.
 
 ## FakeClock
 
@@ -43,6 +54,31 @@ await bus.drain();
 Records publishes, simulates outages with `failNextPublishes(n)` and delivers
 JSON copies to subscribers with `deliver(destination, envelope, maxRedeliveries)`.
 
+## When to use @relaybus/testing, and when not to
+
+Use it in your test suite for fast, deterministic tests, or to prove that a custom `EventStore` satisfies the contract. It is a development dependency and requires `vitest` 3 or newer.
+
+Documentation: <https://berkcansavur.github.io/relaybus/packages/testing>
+
+### relaybus vs alternatives
+
+<!-- relaybus:snippet:comparison -->
+
+|                                                                      | What it is                                                                                                                                                | Survives a restart                | Written in your database transaction                                      | Choose it when                                                                                                       |
+| -------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
+| **relaybus**                                                         | Event bus built on the transactional outbox: PostgreSQL and MongoDB stores, Kafka and RabbitMQ transports, per-handler deliveries                         | Yes, events live in your database | Yes                                                                       | Events must never be lost or phantom, and handlers need retries, dead letters and ordering keys                      |
+| [`@nestjs/event-emitter`](https://docs.nestjs.com/techniques/events) | In-process event emitter built on `eventemitter2`                                                                                                         | No persistence is documented      | No                                                                        | Simple in-process notifications where losing an event on restart is acceptable                                       |
+| [`@nestjs/cqrs`](https://docs.nestjs.com/recipes/cqrs)               | CommandBus, QueryBus, EventBus and sagas; the docs say sagas and event handlers run in memory                                                             | No built-in persistence           | No                                                                        | You want CQRS structure; it can sit on top of an outbox                                                              |
+| [`@nestjs/outbox`](https://github.com/nestjs/outbox)                 | Transactional outbox module from the NestJS organisation: PostgreSQL, MySQL and in-memory stores, relay with leases and retries, dead-letter table, inbox | Yes with the database stores      | Yes                                                                       | You need MySQL, or a NestJS-organisation module; compare features with relaybus before choosing                      |
+| [BullMQ](https://docs.bullmq.io/)                                    | Job queue on Redis with retries, delayed jobs and flows; at-least-once in the worst case                                                                  | Yes, in Redis                     | No, jobs are stored in Redis and cannot join a SQL or MongoDB transaction | Your stack is Redis-based and background jobs matter more than event atomicity                                       |
+| [pg-boss](https://github.com/timgit/pg-boss)                         | Job queue on PostgreSQL using `SKIP LOCKED`, with retries, dead letter queues, scheduling and pub/sub; jobs can be created in your own transaction        | Yes, in PostgreSQL                | Yes, through its database adapters                                        | You want a PostgreSQL job queue with scheduling; relaybus is an event bus with per-handler deliveries and transports |
+| Kafka client directly                                                | A distributed log; the idempotent producer and transactions give exactly-once within Kafka                                                                | Yes, in Kafka                     | No, producing to Kafka is a second write                                  | You consume and produce only inside Kafka; add an outbox when a database write must be announced                     |
+
+This table summarises the vendors' own documentation as of October 2026; check their docs for current details.
+Full comparison and sources: <https://berkcansavur.github.io/relaybus/comparison>.
+
+<!-- relaybus:end -->
+
 ## License
 
 MIT

package/dist/index.cjs

@@ -34,6 +34,7 @@ function createDelivery(handler, availableAt = at(0)) {
   return { id: core.createUuidV7(), handler, availableAt };
 }
 function eventStoreContract(name, options) {
+  const keepsCompleted = options.keepsCompletedRecords ?? false;
   vitest.describe(`${name} satisfies the EventStore contract`, () => {
     let store;
     vitest.beforeEach(async () => {
@@ -45,6 +46,16 @@ function eventStoreContract(name, options) {
     async function append(entries, now = at(0)) {
       return store.transaction((transaction) => store.append(transaction, entries, now));
     }
+    async function completeMany(target, leases, now) {
+      if (target.completeMany !== void 0) {
+        return target.completeMany(leases, now);
+      }
+      const outcomes = [];
+      for (const lease of leases) {
+        outcomes.push(await target.complete(lease, now));
+      }
+      return outcomes;
+    }
     async function claim(handlers, now = at(0), limit = 100, owner = "worker-1") {
       return store.claim({
         handlers,
@@ -221,12 +232,19 @@ function eventStoreContract(name, options) {
         }));
         await append(entries);
         const batches = await Promise.all(
-          Array.from(
-            { length: 8 },
-            (_, index) => claim(["handler-a"], at(0), 10, `worker-${index}`)
-          )
+          Array.from({ length: 8 }, async (_, index) => {
+            const own = [];
+            for (; ; ) {
+              const batch = await claim(["handler-a"], at(0), 10, `worker-${index}`);
+              if (batch.length === 0) {
+                return own;
+              }
+              own.push(...batch);
+            }
+          })
         );
-        const claimedIds = batches.flat().map((delivery) => delivery.deliveryId);
+        const leftovers = await claim(["handler-a"], at(0), 100, "worker-late");
+        const claimedIds = [...batches.flat(), ...leftovers].map((delivery) => delivery.deliveryId);
         vitest.expect(new Set(claimedIds).size).toBe(claimedIds.length);
         vitest.expect(claimedIds).toHaveLength(40);
       });
@@ -245,6 +263,10 @@ function eventStoreContract(name, options) {
           false
         );
         vitest.expect(await store.complete(current, at(LEASE_MS + 2))).toBe(true);
+        if (!keepsCompleted) {
+          vitest.expect(await store.listDeliveries({})).toEqual([]);
+          return;
+        }
         const [record] = await store.listDeliveries({});
         vitest.expect(record).toMatchObject({
           status: "completed",
@@ -285,7 +307,11 @@ function eventStoreContract(name, options) {
         await store.transaction(async (transaction) => {
           vitest.expect(await store.complete(delivery, at(2), transaction)).toBe(true);
         });
-        vitest.expect((await store.listDeliveries({}))[0].status).toBe("completed");
+        if (keepsCompleted) {
+          vitest.expect((await store.listDeliveries({}))[0].status).toBe("completed");
+        } else {
+          vitest.expect(await store.listDeliveries({})).toEqual([]);
+        }
       });
       vitest.it("refuses a transactional completion with a lost lease", async () => {
         await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
@@ -459,7 +485,11 @@ function eventStoreContract(name, options) {
         });
         vitest.expect((await store.statistics(at(LEASE_MS))).oldestDueAt).toEqual(at(200));
         await store.complete(processing, at(1));
-        vitest.expect(await store.statistics(at(0))).toMatchObject({ completed: 1, oldestDueAt: null });
+        vitest.expect(await store.statistics(at(0))).toMatchObject({
+          completed: keepsCompleted ? 1 : 0,
+          processing: 0,
+          oldestDueAt: null
+        });
       });
       vitest.it("counts an expired lease as due", async () => {
         await append([
@@ -471,7 +501,7 @@ function eventStoreContract(name, options) {
       });
     });
     vitest.describe("purge", () => {
-      vitest.it("deletes old completed and dead deliveries and their orphaned events", async () => {
+      async function appendDoneFailedShared() {
         const done = createEnvelope();
         const failed = createEnvelope();
         const shared = createEnvelope();
@@ -486,23 +516,52 @@ function eventStoreContract(name, options) {
         await store.bury(failedDelivery, { name: "Error", message: "x" }, at(10));
         const [sharedDelivery] = await claim(["handler-c"]);
         await store.complete(sharedDelivery, at(10));
-        vitest.expect(await store.purge({})).toEqual({ deletedDeliveries: 0, deletedEvents: 0 });
-        vitest.expect(await store.purge({ completedBefore: at(10) })).toEqual({
-          deletedDeliveries: 0,
-          deletedEvents: 0
-        });
-        vitest.expect(await store.purge({ completedBefore: at(11) })).toEqual({
-          deletedDeliveries: 2,
-          deletedEvents: 1
-        });
-        vitest.expect(await store.purge({ deadBefore: at(11) })).toEqual({
-          deletedDeliveries: 1,
-          deletedEvents: 1
-        });
-        const remaining = await store.listDeliveries({});
-        vitest.expect(remaining.map((record) => record.handler)).toEqual(["handler-d"]);
-        vitest.expect(remaining[0].event.id).toBe(shared.id);
-      });
+        return { shared };
+      }
+      vitest.it.runIf(keepsCompleted)(
+        "deletes old completed and dead deliveries and their orphaned events",
+        async () => {
+          const { shared } = await appendDoneFailedShared();
+          vitest.expect(await store.purge({})).toEqual({ deletedDeliveries: 0, deletedEvents: 0 });
+          vitest.expect(await store.purge({ completedBefore: at(10) })).toEqual({
+            deletedDeliveries: 0,
+            deletedEvents: 0
+          });
+          vitest.expect(await store.purge({ completedBefore: at(11) })).toEqual({
+            deletedDeliveries: 2,
+            deletedEvents: 1
+          });
+          vitest.expect(await store.purge({ deadBefore: at(11) })).toEqual({
+            deletedDeliveries: 1,
+            deletedEvents: 1
+          });
+          const remaining = await store.listDeliveries({});
+          vitest.expect(remaining.map((record) => record.handler)).toEqual(["handler-d"]);
+          vitest.expect(remaining[0].event.id).toBe(shared.id);
+        }
+      );
+      vitest.it.runIf(!keepsCompleted)(
+        "deletes old dead deliveries and the events no delivery references any more",
+        async () => {
+          const { shared } = await appendDoneFailedShared();
+          vitest.expect(await store.purge({})).toEqual({ deletedDeliveries: 0, deletedEvents: 0 });
+          vitest.expect(await store.purge({ completedBefore: at(0) })).toEqual({
+            deletedDeliveries: 0,
+            deletedEvents: 0
+          });
+          vitest.expect(await store.purge({ completedBefore: at(11) })).toEqual({
+            deletedDeliveries: 0,
+            deletedEvents: 1
+          });
+          vitest.expect(await store.purge({ deadBefore: at(11) })).toEqual({
+            deletedDeliveries: 1,
+            deletedEvents: 1
+          });
+          const remaining = await store.listDeliveries({});
+          vitest.expect(remaining.map((record) => record.handler)).toEqual(["handler-d"]);
+          vitest.expect(remaining[0].event.id).toBe(shared.id);
+        }
+      );
       vitest.it("lets a purged event id be appended again", async () => {
         const event = createEnvelope();
         await append([{ event, deliveries: [createDelivery("handler-a")] }]);
@@ -768,6 +827,366 @@ function eventStoreContract(name, options) {
         vitest.expect(listed.event.headers).toEqual({ a: "1" });
       });
     });
+    vitest.describe("completion", () => {
+      vitest.it("removes a completed delivery from every listing", async () => {
+        const event = createEnvelope();
+        await append([
+          { event, deliveries: [createDelivery("handler-a"), createDelivery("handler-b")] }
+        ]);
+        const [delivery] = await claim(["handler-a"]);
+        vitest.expect(await store.complete(delivery, at(5))).toBe(true);
+        const records = await store.listDeliveries({ eventId: event.id });
+        if (keepsCompleted) {
+          vitest.expect(records.map((record) => [record.handler, record.status]).sort()).toEqual([
+            ["handler-a", "completed"],
+            ["handler-b", "pending"]
+          ]);
+          const [completed] = await store.listDeliveries({ status: "completed" });
+          vitest.expect(completed).toMatchObject({ handler: "handler-a", attempts: 1 });
+          vitest.expect(completed.completedAt).toEqual(at(5));
+        } else {
+          vitest.expect(records.map((record) => [record.handler, record.status])).toEqual([
+            ["handler-b", "pending"]
+          ]);
+          vitest.expect(await store.listDeliveries({ status: "completed" })).toEqual([]);
+        }
+        vitest.expect(await claim(["handler-a"], at(LEASE_MS * 10))).toEqual([]);
+        vitest.expect(await store.statistics(at(0))).toMatchObject({
+          pending: 1,
+          processing: 0,
+          completed: keepsCompleted ? 1 : 0
+        });
+      });
+      vitest.it("does not create the delivery of a handler that already completed again", async () => {
+        const event = createEnvelope();
+        await append([{ event, deliveries: [createDelivery("handler-a")] }]);
+        const [delivery] = await claim(["handler-a"]);
+        await store.complete(delivery, at(1));
+        const result = await append([
+          { event, deliveries: [createDelivery("handler-a"), createDelivery("handler-b")] }
+        ]);
+        vitest.expect(result).toEqual({
+          appendedEventIds: [],
+          duplicateEventIds: [event.id],
+          appendedDeliveryCount: 1
+        });
+        vitest.expect(await claim(["handler-a"], at(LEASE_MS * 10))).toEqual([]);
+        vitest.expect(await claim(["handler-b"], at(LEASE_MS * 10))).toHaveLength(1);
+        const again = await append([{ event, deliveries: [createDelivery("handler-a")] }]);
+        vitest.expect(again.appendedDeliveryCount).toBe(0);
+      });
+      vitest.it("completes many leases at once and reports each one", async () => {
+        await append(
+          Array.from({ length: 5 }, () => ({
+            event: createEnvelope(),
+            deliveries: [createDelivery("handler-a")]
+          }))
+        );
+        const claimed = await claim(["handler-a"]);
+        vitest.expect(claimed).toHaveLength(5);
+        vitest.expect(await completeMany(store, [], at(1))).toEqual([]);
+        vitest.expect(await completeMany(store, claimed.slice(0, 3), at(1))).toEqual([true, true, true]);
+        vitest.expect(await store.statistics(at(2))).toMatchObject({ processing: 2, pending: 0 });
+        vitest.expect(await completeMany(store, claimed, at(2))).toEqual([
+          false,
+          false,
+          false,
+          true,
+          true
+        ]);
+        vitest.expect(await store.statistics(at(3))).toMatchObject({ processing: 0, pending: 0 });
+      });
+      vitest.it("completes a lease listed twice only once", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [delivery] = await claim(["handler-a"]);
+        vitest.expect(await completeMany(store, [delivery, delivery], at(1))).toEqual([true, false]);
+      });
+      vitest.it("fences every lease of a batch on its own", async () => {
+        await append(
+          Array.from({ length: 3 }, () => ({
+            event: createEnvelope(),
+            deliveries: [createDelivery("handler-a")]
+          }))
+        );
+        const stale = await claim(["handler-a"], at(0));
+        const current = await claim(["handler-a"], at(LEASE_MS), 100, "worker-2");
+        vitest.expect(current).toHaveLength(3);
+        await append(
+          [{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }],
+          at(1)
+        );
+        const [fresh] = await claim(["handler-a"], at(LEASE_MS + 1), 100, "worker-2");
+        vitest.expect(
+          await completeMany(store, [stale[0], fresh, stale[1], current[2]], at(LEASE_MS + 2))
+        ).toEqual([false, true, false, true]);
+        vitest.expect(await store.statistics(at(LEASE_MS + 3))).toMatchObject({ processing: 2 });
+      });
+      vitest.it("does not complete a lease of a delivery that went back to pending", async () => {
+        const error = { name: "Error", message: "x" };
+        await append([
+          { event: createEnvelope(), deliveries: [createDelivery("a"), createDelivery("b")] }
+        ]);
+        const [retrying] = await claim(["a"]);
+        const [released] = await claim(["b"]);
+        await store.reschedule(retrying, at(1e3), error, at(1));
+        await store.release(released, at(1));
+        vitest.expect(await completeMany(store, [retrying, released], at(2))).toEqual([false, false]);
+        vitest.expect(await store.statistics(at(2))).toMatchObject({ pending: 2, processing: 0 });
+      });
+    });
+    vitest.describe("stale leases of one delivery", () => {
+      const error = { name: "Error", message: "x" };
+      vitest.it("refuses the lease of an attempt that was released when the same owner claims it again", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [stale] = await claim(["handler-a"], at(0), 100, "worker-1");
+        vitest.expect(await store.release(stale, at(1))).toBe(true);
+        const [current] = await claim(["handler-a"], at(2), 100, "worker-1");
+        vitest.expect(current.deliveryId).toBe(stale.deliveryId);
+        vitest.expect(current.leaseToken).not.toBe(stale.leaseToken);
+        vitest.expect(await store.complete(stale, at(3))).toBe(false);
+        vitest.expect(await store.bury(stale, error, at(3))).toBe(false);
+        vitest.expect(await store.reschedule(stale, at(9), error, at(3))).toBe(false);
+        vitest.expect(await store.release(stale, at(3))).toBe(false);
+        vitest.expect(await store.extendLease(stale, at(LEASE_MS * 3), at(3))).toBe(false);
+        vitest.expect(await store.complete(current, at(4))).toBe(true);
+      });
+      vitest.it("refuses the lease of an attempt that was rescheduled when the same owner claims it again", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [stale] = await claim(["handler-a"], at(0), 100, "worker-1");
+        vitest.expect(await store.reschedule(stale, at(10), error, at(1))).toBe(true);
+        const [current] = await claim(["handler-a"], at(10), 100, "worker-1");
+        vitest.expect(current).toMatchObject({ deliveryId: stale.deliveryId, attempt: 2 });
+        vitest.expect(await store.complete(stale, at(11))).toBe(false);
+        vitest.expect(await store.bury(stale, error, at(11))).toBe(false);
+        vitest.expect(await completeMany(store, [stale], at(11))).toEqual([false]);
+        vitest.expect(await store.complete(current, at(12))).toBe(true);
+      });
+      vitest.it("matches the results of completeMany to the lease, not to the delivery id", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [stale] = await claim(["handler-a"], at(0), 100, "worker-1");
+        await store.release(stale, at(1));
+        const [current] = await claim(["handler-a"], at(2), 100, "worker-1");
+        vitest.expect(await completeMany(store, [stale, current], at(3))).toEqual([false, true]);
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-b")] }]);
+        const [staleB] = await claim(["handler-b"], at(4), 100, "worker-1");
+        await store.release(staleB, at(5));
+        const [currentB] = await claim(["handler-b"], at(6), 100, "worker-1");
+        vitest.expect(await completeMany(store, [currentB, staleB], at(7))).toEqual([true, false]);
+      });
+    });
+    vitest.describe("retention of the deduplication record", () => {
+      const DAY = 864e5;
+      vitest.it("keeps an event for the retention window after its last delivery completed, however old it is", async () => {
+        const event = createEnvelope();
+        await append([{ event, deliveries: [createDelivery("handler-a")] }]);
+        const [delivery] = await claim(["handler-a"], at(30 * DAY));
+        vitest.expect(await store.complete(delivery, at(30 * DAY + 1))).toBe(true);
+        vitest.expect(await store.purge({ completedBefore: at(29 * DAY) })).toMatchObject({
+          deletedEvents: 0
+        });
+        const again = await append(
+          [{ event, deliveries: [createDelivery("handler-a")] }],
+          at(30 * DAY + 2)
+        );
+        vitest.expect(again.appendedDeliveryCount).toBe(0);
+        vitest.expect(await store.purge({ completedBefore: at(31 * DAY) })).toMatchObject({
+          deletedEvents: 1
+        });
+        const afterwards = await append(
+          [{ event, deliveries: [createDelivery("handler-a")] }],
+          at(31 * DAY)
+        );
+        vitest.expect(afterwards.appendedEventIds).toEqual([event.id]);
+      });
+      vitest.it("keeps the later completion time of an event when completions are recorded out of order", async () => {
+        const event = createEnvelope();
+        await append([
+          { event, deliveries: [createDelivery("handler-a"), createDelivery("handler-b")] }
+        ]);
+        const claimed = await claim(["handler-a", "handler-b"]);
+        vitest.expect(claimed).toHaveLength(2);
+        const [first, second] = claimed.sort(
+          (left, right) => left.handler.localeCompare(right.handler)
+        );
+        vitest.expect(await store.complete(first, at(200))).toBe(true);
+        vitest.expect(await store.complete(second, at(100))).toBe(true);
+        vitest.expect(await store.purge({ completedBefore: at(150) })).toMatchObject({ deletedEvents: 0 });
+        const again = await append(
+          [{ event, deliveries: [createDelivery("handler-a"), createDelivery("handler-b")] }],
+          at(201)
+        );
+        vitest.expect(again.appendedDeliveryCount).toBe(0);
+      });
+      vitest.it("remembers every handler of an event appended twice in one transaction", async () => {
+        const event = createEnvelope();
+        const result = await store.transaction(async (transaction) => {
+          await store.append(
+            transaction,
+            [{ event, deliveries: [createDelivery("handler-a")] }],
+            at(0)
+          );
+          return store.append(
+            transaction,
+            [{ event, deliveries: [createDelivery("handler-b")] }],
+            at(0)
+          );
+        });
+        vitest.expect(result).toMatchObject({ duplicateEventIds: [event.id], appendedDeliveryCount: 1 });
+        const claimed = await claim(["handler-a", "handler-b"]);
+        vitest.expect(claimed).toHaveLength(2);
+        for (const delivery of claimed) {
+          vitest.expect(await store.complete(delivery, at(1))).toBe(true);
+        }
+        const again = await append(
+          [{ event, deliveries: [createDelivery("handler-a"), createDelivery("handler-b")] }],
+          at(2)
+        );
+        vitest.expect(again).toMatchObject({ duplicateEventIds: [event.id], appendedDeliveryCount: 0 });
+      });
+      vitest.it("does not let the dead-letter cutoff remove events whose deliveries completed", async () => {
+        const event = createEnvelope();
+        await append([{ event, deliveries: [createDelivery("handler-a")] }]);
+        const [delivery] = await claim(["handler-a"]);
+        await store.complete(delivery, at(1));
+        vitest.expect(await store.purge({ deadBefore: at(100 * DAY) })).toMatchObject({
+          deletedEvents: 0
+        });
+        const again = await append([{ event, deliveries: [createDelivery("handler-a")] }], at(2));
+        vitest.expect(again.appendedDeliveryCount).toBe(0);
+      });
+    });
+    vitest.describe("lost leases", () => {
+      vitest.it("hands a delivery of a crashed worker to the next claimer with the attempt counted", async () => {
+        await append(
+          Array.from({ length: 4 }, () => ({
+            event: createEnvelope(),
+            deliveries: [createDelivery("handler-a")]
+          }))
+        );
+        const first = await claim(["handler-a"], at(0), 100, "worker-1");
+        vitest.expect(first.map((delivery) => delivery.attempt)).toEqual([1, 1, 1, 1]);
+        vitest.expect(await claim(["handler-a"], at(LEASE_MS - 1), 100, "worker-2")).toEqual([]);
+        const second = await claim(["handler-a"], at(LEASE_MS), 100, "worker-2");
+        vitest.expect(second.map((delivery) => delivery.deliveryId).sort()).toEqual(
+          first.map((delivery) => delivery.deliveryId).sort()
+        );
+        vitest.expect(second.map((delivery) => delivery.attempt)).toEqual([2, 2, 2, 2]);
+        for (const stale of first) {
+          vitest.expect(await store.complete(stale, at(LEASE_MS + 1))).toBe(false);
+        }
+        vitest.expect(await completeMany(store, first, at(LEASE_MS + 1))).toEqual([
+          false,
+          false,
+          false,
+          false
+        ]);
+        vitest.expect(await completeMany(store, second, at(LEASE_MS + 2))).toEqual([
+          true,
+          true,
+          true,
+          true
+        ]);
+      });
+      vitest.it("keeps counting attempts of a poison delivery until it is dead-lettered", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        for (let crash = 0; crash < 4; crash += 1) {
+          const [delivery] = await claim(
+            ["handler-a"],
+            at(crash * LEASE_MS),
+            100,
+            `worker-${crash}`
+          );
+          vitest.expect(delivery.attempt).toBe(crash + 1);
+        }
+        const [last] = await claim(["handler-a"], at(4 * LEASE_MS), 100, "worker-4");
+        vitest.expect(last.attempt).toBe(5);
+        vitest.expect(
+          await store.bury(last, { name: "Error", message: "poison" }, at(4 * LEASE_MS))
+        ).toBe(true);
+        const [dead] = await store.listDeliveries({ status: "dead" });
+        vitest.expect(dead).toMatchObject({ attempts: 5, lastError: { message: "poison" } });
+        vitest.expect(await claim(["handler-a"], at(100 * LEASE_MS))).toEqual([]);
+      });
+      vitest.it("does not count an attempt that was reported as failed twice", async () => {
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [delivery] = await claim(["handler-a"], at(0), 100, "worker-1");
+        await store.reschedule(delivery, at(10), { name: "Error", message: "x" }, at(1));
+        const [retry] = await claim(["handler-a"], at(LEASE_MS), 100, "worker-2");
+        vitest.expect(retry.attempt).toBe(2);
+        const [afterCrash] = await claim(["handler-a"], at(LEASE_MS * 2), 100, "worker-3");
+        vitest.expect(afterCrash.attempt).toBe(3);
+      });
+      vitest.it("keeps the order of a key across a takeover", async () => {
+        const [head, next] = await (async () => {
+          const events = [createEnvelope({ key: "k" }), createEnvelope({ key: "k" })];
+          for (const event of events) {
+            await append([{ event, deliveries: [createDelivery("handler-a")] }]);
+          }
+          return events;
+        })();
+        const [stale] = await claim(["handler-a"], at(0), 100, "worker-1");
+        vitest.expect(stale.event.id).toBe(head.id);
+        const taken = await claim(["handler-a"], at(LEASE_MS), 100, "worker-2");
+        vitest.expect(taken.map((delivery) => delivery.event.id)).toEqual([head.id]);
+        vitest.expect(await store.complete(stale, at(LEASE_MS + 1))).toBe(false);
+        vitest.expect(await store.complete(taken[0], at(LEASE_MS + 2))).toBe(true);
+        const [following] = await claim(["handler-a"], at(LEASE_MS + 3), 100, "worker-2");
+        vitest.expect(following.event.id).toBe(next.id);
+      });
+      vitest.it("lets every worker see only deliveries nobody else holds, and none twice", async () => {
+        await append(
+          Array.from({ length: 30 }, () => ({
+            event: createEnvelope(),
+            deliveries: [createDelivery("handler-a")]
+          }))
+        );
+        const seen = [];
+        for (let round = 0; round < 40; round += 1) {
+          const owners = ["worker-1", "worker-2", "worker-3"];
+          const batches = await Promise.all(
+            owners.map((owner) => claim(["handler-a"], at(round), 4, owner))
+          );
+          for (const batch of batches) {
+            seen.push(...batch.map((delivery) => delivery.deliveryId));
+            vitest.expect(await completeMany(store, batch, at(round))).toEqual(batch.map(() => true));
+          }
+        }
+        vitest.expect(seen).toHaveLength(30);
+        vitest.expect(new Set(seen).size).toBe(30);
+      });
+    });
+    vitest.describe("relinquish", () => {
+      vitest.it("hands idle leases to other workers at once and keeps leases with running work", async (context) => {
+        if (store.relinquish === void 0) {
+          return context.skip();
+        }
+        await append([
+          { event: createEnvelope(), deliveries: [createDelivery("handler-a")] },
+          { event: createEnvelope(), deliveries: [createDelivery("handler-a")] }
+        ]);
+        const claimed = await claim(["handler-a"], at(0), 100, "worker-1");
+        vitest.expect(claimed).toHaveLength(2);
+        await store.relinquish("worker-1", at(1));
+        vitest.expect(await store.complete(claimed[0], at(2))).toBe(true);
+        vitest.expect(await store.complete(claimed[1], at(2))).toBe(true);
+        await store.relinquish("worker-1", at(3));
+        await append(
+          [{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }],
+          at(4)
+        );
+        vitest.expect(await claim(["handler-a"], at(4), 100, "worker-2")).toHaveLength(1);
+      });
+      vitest.it("does not give away a delivery that is still running", async (context) => {
+        if (store.relinquish === void 0) {
+          return context.skip();
+        }
+        await append([{ event: createEnvelope(), deliveries: [createDelivery("handler-a")] }]);
+        const [running] = await claim(["handler-a"], at(0), 100, "worker-1");
+        await store.relinquish("worker-1", at(1));
+        vitest.expect(await claim(["handler-a"], at(2), 100, "worker-2")).toEqual([]);
+        vitest.expect(await store.complete(running, at(3))).toBe(true);
+      });
+    });
     vitest.describe.skipIf(options.createPeerStore === void 0)("across store instances", () => {
       let peer;
       vitest.beforeEach(async () => {
@@ -812,6 +1231,32 @@ function eventStoreContract(name, options) {
         vitest.expect(await store.complete(stale, at(LEASE_MS + 1))).toBe(false);
         vitest.expect(await peer.complete(current, at(LEASE_MS + 1))).toBe(true);
       });
+      vitest.it("completes batches in either process and fences the one that lost its lease", async () => {
+        await append(
+          Array.from({ length: 6 }, () => ({
+            event: createEnvelope(),
+            deliveries: [createDelivery("handler-a")]
+          }))
+        );
+        const request = (owner, now) => ({
+          handlers: ["handler-a"],
+          limit: 100,
+          leaseOwner: owner,
+          leaseExpiresAt: new Date(now.getTime() + LEASE_MS),
+          now
+        });
+        const stale = await store.claim(request("primary", at(0)));
+        const current = await peer.claim(request("peer", at(LEASE_MS)));
+        vitest.expect(current).toHaveLength(6);
+        vitest.expect(await completeMany(store, stale, at(LEASE_MS + 1))).toEqual(stale.map(() => false));
+        vitest.expect(await completeMany(peer, current, at(LEASE_MS + 1))).toEqual(
+          current.map(() => true)
+        );
+        vitest.expect(await store.statistics(at(LEASE_MS + 2))).toMatchObject({
+          pending: 0,
+          processing: 0
+        });
+      });
     });
   });
 }