@hile/redis-stream-queue 3.0.1

package/README.md

@@ -0,0 +1,167 @@
+# @hile/redis-stream-queue
+
+Redis Streams backed durable job queue primitives for Hile applications.
+
+Use this package when work should not block the current request but still needs to be persisted, retried, recovered after worker crashes, and inspected when it finally fails.
+
+## Install
+
+```bash
+pnpm add @hile/redis-stream-queue @hile/ioredis
+```
+
+The package accepts any Redis client matching `RedisStreamQueueLike`. In Hile apps the usual client comes from `@hile/ioredis`.
+
+## Quick Start
+
+```typescript
+import { RedisStreamQueue, defineQueue } from '@hile/redis-stream-queue'
+
+type EmailPayload = {
+  template: 'welcome'
+  userId: string
+}
+
+const emailQueue = defineQueue<EmailPayload>('email')
+const queue = new RedisStreamQueue(redis, { prefix: 'myapp:' })
+
+await queue.add(emailQueue, {
+  template: 'welcome',
+  userId: 'user-1',
+}, {
+  jobId: 'welcome:user-1',
+  maxAttempts: 5,
+  backoff: { type: 'exponential', baseMs: 1_000 },
+})
+
+queue.worker(emailQueue, async (job) => {
+  await sendEmail(job.data)
+}, {
+  group: 'email-workers',
+  consumer: 'worker-1',
+  concurrency: 8,
+}).start()
+```
+
+## Core Concepts
+
+### defineQueue(name, schema?)
+
+`defineQueue()` declares the queue name and optional payload schema.
+
+```typescript
+const imageQueue = defineQueue('image-resize', {
+  parse(value) {
+    if (typeof value !== 'object' || value === null) throw new Error('invalid payload')
+    return value as { imageId: string; size: 'small' | 'large' }
+  },
+})
+```
+
+The schema can expose `parse(value)` or `safeParse(value)`. Payloads are validated before enqueueing and again when workers read them.
+
+### queue.add(queue, payload, options?)
+
+Adds a durable job.
+
+```typescript
+await queue.add(emailQueue, payload, {
+  jobId: 'welcome:user-1',
+  delay: 30_000,
+  maxAttempts: 5,
+  backoff: { type: 'fixed', delay: 5_000 },
+})
+```
+
+Options:
+
+| Option | Meaning |
+|---|---|
+| `jobId` | Deduplication key. A second add with the same `jobId` returns `{ duplicate: true }`. |
+| `delay` | Milliseconds before the job becomes visible to workers. |
+| `maxAttempts` | Maximum attempts before the job goes to DLQ. Defaults to `1`. |
+| `backoff` | Retry delay. Use a number, `{ type: 'fixed', delay }`, or `{ type: 'exponential', baseMs, maxMs }`. |
+
+### queue.worker(queue, handler, options?)
+
+Creates a worker around a Redis consumer group.
+
+```typescript
+const worker = queue.worker(emailQueue, async (job) => {
+  await sendEmail(job.data)
+}, {
+  group: 'email-workers',
+  consumer: process.env.HOSTNAME ?? 'local',
+  concurrency: 4,
+  claimIdle: 60_000,
+})
+
+worker.start()
+```
+
+`runOnce()` is available for tests and controlled scripts:
+
+```typescript
+await worker.runOnce()
+await worker.stop()
+```
+
+### readDeadLetters(queue)
+
+Reads failed jobs from the dead-letter stream:
+
+```typescript
+const failed = await queue.readDeadLetters(emailQueue, { count: 20 })
+```
+
+## State Machine
+
+| State | Redis structure | Meaning |
+|---|---|---|
+| `scheduled` | sorted set `{prefix}queue:{name}:delayed` | Job has a future `runAt`. |
+| `ready` | stream `{prefix}queue:{name}:stream` | Job is visible to the consumer group. |
+| `pending` | Redis Streams PEL | A worker received the job but has not acked it yet. |
+| `retrying` | delayed sorted set | A failed job is waiting for backoff before another attempt. |
+| `done` | acked stream entry | Handler completed and the pending entry was acked. |
+| `dead-lettered` | stream `{prefix}queue:{name}:dlq` | Attempts were exhausted. |
+
+## Crash Recovery
+
+Workers use Redis consumer groups. If a worker reads a job and dies before `XACK`, the job remains in the pending entries list. Another worker can claim it after `claimIdle` milliseconds and continue processing.
+
+## Context Propagation
+
+If `@hile/context` has an active context when a job is enqueued, the queue stores a snapshot and restores it while the worker handler runs.
+
+```typescript
+await runWithContext<AppContext>({ shopId: 'shop-1' }, async () => {
+  await queue.add(emailQueue, payload)
+})
+```
+
+```typescript
+queue.worker(emailQueue, async () => {
+  const context = getContext<AppContext>()
+  console.log(context.shopId)
+})
+```
+
+## Boundaries
+
+- This package is for background jobs, not request/response RPC.
+- It provides at-least-once delivery. Handlers must be idempotent.
+- It does not promise exactly-once execution.
+- Delayed jobs and retries are promoted when workers poll the queue.
+- Payload and context values must be JSON-serializable.
+- `jobId` prevents duplicate enqueue for the same key; it does not make the handler side effect exactly-once.
+
+## Testing
+
+```bash
+pnpm --filter @hile/redis-stream-queue test
+pnpm --filter @hile/redis-stream-queue build
+```
+
+## License
+
+MIT

package/SKILL.md

@@ -0,0 +1,45 @@
+---
+name: redis-stream-queue
+description: Use when implementing Redis Streams backed durable job queues, background workers, retry/backoff, delayed jobs, consumer-group pending recovery, DLQ handling, or Hile queue context propagation.
+---
+
+# Redis Stream Queue
+
+Use `@hile/redis-stream-queue` when work should be persisted and processed asynchronously by background workers.
+
+## Core Rule
+
+This package provides at-least-once background job execution. Handlers must be idempotent; do not claim exactly-once delivery.
+
+```typescript
+const emailQueue = defineQueue<EmailPayload>('email')
+
+await queue.add(emailQueue, payload, {
+  jobId: `welcome:${payload.userId}`,
+  maxAttempts: 5,
+  backoff: { type: 'exponential', baseMs: 1_000 },
+})
+
+queue.worker(emailQueue, async (job) => {
+  await sendEmail(job.data)
+}, { concurrency: 8 }).start()
+```
+
+## Design Boundaries
+
+- Use Redis Streams consumer groups for ready jobs.
+- Use the pending entries list plus `XCLAIM` for worker crash recovery.
+- Use a delayed sorted set for delayed jobs and retry backoff.
+- Move exhausted jobs to `{prefix}queue:{name}:dlq`.
+- Store job attempts, first/last failure reason, and context metadata.
+- Validate payloads through the queue schema before enqueueing and before handling.
+- `jobId` is enqueue dedupe only; side effects still need idempotency.
+
+## Testing Priorities
+
+- Pending job claimed by another worker after `claimIdle`.
+- Failed job retries with configured backoff.
+- Exhausted job appears in DLQ.
+- Duplicate `jobId` enqueues only one job.
+- Worker concurrency caps simultaneous handlers.
+- Active `@hile/context` is restored inside handlers.

package/dist/define.d.ts

@@ -0,0 +1,4 @@
+import type { InferQueueSchema, QueueDefinition, QueueSchema } from './types';
+export declare function defineQueue<const TName extends string, TSchema extends QueueSchema<any>>(name: TName, schema: TSchema): QueueDefinition<InferQueueSchema<TSchema>, TName>;
+export declare function defineQueue<TData = unknown, const TName extends string = string>(name: TName): QueueDefinition<TData, TName>;
+export declare function parsePayload<TData>(definition: QueueDefinition<TData>, payload: unknown): TData;

package/dist/define.js

@@ -0,0 +1,28 @@
+import { QueueSchemaError } from './errors.js';
+const VALID_QUEUE_NAME = /^[a-zA-Z0-9:_-]+$/;
+export function defineQueue(name, schema) {
+    if (!VALID_QUEUE_NAME.test(name)) {
+        throw new TypeError('Queue name must contain only letters, numbers, ":", "_" or "-"');
+    }
+    return schema ? { name, schema } : { name };
+}
+export function parsePayload(definition, payload) {
+    const schema = definition.schema;
+    if (!schema)
+        return payload;
+    try {
+        if ('parse' in schema && typeof schema.parse === 'function') {
+            return schema.parse(payload);
+        }
+        if (!('safeParse' in schema) || typeof schema.safeParse !== 'function') {
+            throw new TypeError('Queue schema must expose parse() or safeParse()');
+        }
+        const result = schema.safeParse(payload);
+        if (result.success)
+            return result.data;
+        throw result.error;
+    }
+    catch (err) {
+        throw new QueueSchemaError(definition.name, err);
+    }
+}

package/dist/errors.d.ts

@@ -0,0 +1,13 @@
+export declare class QueueError extends Error {
+    constructor(message: string);
+}
+export declare class QueueSchemaError extends QueueError {
+    readonly queue: string;
+    readonly cause: unknown;
+    constructor(queue: string, cause: unknown);
+}
+export declare class QueueSerializationError extends QueueError {
+    readonly queue: string;
+    readonly cause: unknown;
+    constructor(queue: string, cause: unknown);
+}

package/dist/errors.js

@@ -0,0 +1,25 @@
+export class QueueError extends Error {
+    constructor(message) {
+        super(message);
+        this.name = new.target.name;
+        Object.setPrototypeOf(this, new.target.prototype);
+    }
+}
+export class QueueSchemaError extends QueueError {
+    queue;
+    cause;
+    constructor(queue, cause) {
+        super(`Invalid payload for queue "${queue}"`);
+        this.queue = queue;
+        this.cause = cause;
+    }
+}
+export class QueueSerializationError extends QueueError {
+    queue;
+    cause;
+    constructor(queue, cause) {
+        super(`Queue job for "${queue}" must be JSON-serializable`);
+        this.queue = queue;
+        this.cause = cause;
+    }
+}

package/dist/index.d.ts

@@ -0,0 +1,4 @@
+export { QueueError, QueueSchemaError, QueueSerializationError, } from './errors.js';
+export { defineQueue, } from './define.js';
+export { RedisStreamQueue, RedisStreamQueueWorker, } from './queue.js';
+export type { InferQueueSchema, QueueAddOptions, QueueAddResult, QueueBackoff, QueueDeadLetter, QueueDefinition, QueueJob, QueueSafeParseResult, QueueSchema, QueueWorkerHandler, QueueWorkerOptions, ReadDeadLettersOptions, RedisPendingEntry, RedisStreamEntry, RedisStreamQueueLike, RedisStreamQueueOptions, RedisStreamReadResult, } from './types.js';

package/dist/index.js

@@ -0,0 +1,3 @@
+export { QueueError, QueueSchemaError, QueueSerializationError, } from './errors.js';
+export { defineQueue, } from './define.js';
+export { RedisStreamQueue, RedisStreamQueueWorker, } from './queue.js';

package/dist/queue.d.ts

@@ -0,0 +1,47 @@
+import type { QueueAddOptions, QueueAddResult, QueueDeadLetter, QueueDefinition, QueueWorkerHandler, QueueWorkerOptions, ReadDeadLettersOptions, RedisStreamQueueLike, RedisStreamQueueOptions } from './types';
+export declare class RedisStreamQueue {
+    private readonly redis;
+    private readonly prefix;
+    private readonly now;
+    constructor(redis: RedisStreamQueueLike, options?: RedisStreamQueueOptions);
+    add<TData>(definition: QueueDefinition<TData>, payload: TData, options?: QueueAddOptions): Promise<QueueAddResult>;
+    worker<TData>(definition: QueueDefinition<TData>, handler: QueueWorkerHandler<TData>, options?: QueueWorkerOptions): RedisStreamQueueWorker<TData>;
+    readDeadLetters<TData>(definition: QueueDefinition<TData>, options?: ReadDeadLettersOptions): Promise<Array<QueueDeadLetter<TData>>>;
+    runWorkerOnce<TData>(definition: QueueDefinition<TData>, handler: QueueWorkerHandler<TData>, options: RequiredWorkerOptions): Promise<number>;
+    private processEntry;
+    private handleFailure;
+    private createJob;
+    private promoteDelayed;
+    private claimStale;
+    private readNew;
+    private ensureGroup;
+    private decodeEntry;
+    private encodeJob;
+    private streamKey;
+    private delayedKey;
+    private deadLetterKey;
+    private dedupeKey;
+}
+export declare class RedisStreamQueueWorker<TData = unknown> {
+    private readonly queue;
+    private readonly definition;
+    private readonly handler;
+    private readonly options;
+    private running;
+    private loop?;
+    constructor(queue: RedisStreamQueue, definition: QueueDefinition<TData>, handler: QueueWorkerHandler<TData>, options: QueueWorkerOptions);
+    runOnce(): Promise<number>;
+    start(): this;
+    stop(): Promise<void>;
+    private runLoop;
+}
+type RequiredWorkerOptions = {
+    group: string;
+    consumer: string;
+    concurrency: number;
+    block: number;
+    pollInterval: number;
+    claimIdle: number;
+    claimCount: number;
+};
+export {};

package/dist/queue.js

@@ -0,0 +1,348 @@
+import { randomUUID } from 'node:crypto';
+import { isContextData, runWithContext, snapshotContext, } from '@hile/context';
+import { QueueSerializationError } from './errors.js';
+import { parsePayload } from './define.js';
+const DEFAULT_MAX_ATTEMPTS = 1;
+const DEFAULT_CONCURRENCY = 1;
+const DEFAULT_CLAIM_IDLE = 60_000;
+const DEFAULT_POLL_INTERVAL = 1_000;
+const DEFAULT_READ_BLOCK = 0;
+export class RedisStreamQueue {
+    redis;
+    prefix;
+    now;
+    constructor(redis, options = {}) {
+        this.redis = redis;
+        this.prefix = options.prefix ?? '';
+        this.now = options.now ?? Date.now;
+    }
+    async add(definition, payload, options = {}) {
+        const data = parsePayload(definition, payload);
+        const now = this.now();
+        const delay = assertNonNegativeInteger(options.delay ?? 0, 'delay');
+        const maxAttempts = assertPositiveInteger(options.maxAttempts ?? DEFAULT_MAX_ATTEMPTS, 'maxAttempts');
+        const runAt = now + delay;
+        const id = randomUUID();
+        const jobId = options.jobId;
+        const context = snapshotContext();
+        const job = {
+            v: 1,
+            id,
+            queue: definition.name,
+            data,
+            createdAt: now,
+            runAt,
+            attempts: 0,
+            maxAttempts,
+            backoff: normalizeBackoff(options.backoff),
+            ...(jobId ? { jobId } : {}),
+            ...(Object.keys(context).length > 0 ? { context } : {}),
+        };
+        const encoded = this.encodeJob(definition, job);
+        let reservedDedupe = false;
+        if (jobId) {
+            const result = await this.redis.set(this.dedupeKey(definition, jobId), id, 'NX');
+            if (result !== 'OK') {
+                const existing = await this.redis.get(this.dedupeKey(definition, jobId));
+                return {
+                    accepted: false,
+                    duplicate: true,
+                    id: existing ?? id,
+                    jobId,
+                    runAt,
+                };
+            }
+            reservedDedupe = true;
+        }
+        try {
+            if (runAt > now) {
+                await this.redis.zadd(this.delayedKey(definition), runAt, encoded);
+                return {
+                    accepted: true,
+                    duplicate: false,
+                    id,
+                    ...(jobId ? { jobId } : {}),
+                    runAt,
+                };
+            }
+            const streamId = await this.redis.xadd(this.streamKey(definition), '*', 'job', encoded);
+            return {
+                accepted: true,
+                duplicate: false,
+                id,
+                ...(jobId ? { jobId } : {}),
+                streamId,
+                runAt,
+            };
+        }
+        catch (err) {
+            if (reservedDedupe && jobId) {
+                await this.redis.del(this.dedupeKey(definition, jobId)).catch(() => undefined);
+            }
+            throw err;
+        }
+    }
+    worker(definition, handler, options = {}) {
+        return new RedisStreamQueueWorker(this, definition, handler, options);
+    }
+    async readDeadLetters(definition, options = {}) {
+        const entries = await this.redis.xrange(this.deadLetterKey(definition), '-', '+', 'COUNT', options.count ?? 100);
+        return entries.map(([streamId, fields]) => {
+            const stored = this.decodeEntry(definition, fields);
+            const data = parsePayload(definition, stored.data);
+            return {
+                id: stored.id,
+                queue: stored.queue,
+                data,
+                attempt: stored.attempts,
+                maxAttempts: stored.maxAttempts,
+                streamId,
+                ...(stored.jobId ? { jobId: stored.jobId } : {}),
+                createdAt: stored.createdAt,
+                runAt: stored.runAt,
+                ...(stored.context ? { context: stored.context } : {}),
+                ...(stored.firstFailureReason ? { firstFailureReason: stored.firstFailureReason } : {}),
+                ...(stored.lastFailureReason ? { lastFailureReason: stored.lastFailureReason } : {}),
+            };
+        });
+    }
+    async runWorkerOnce(definition, handler, options) {
+        await this.ensureGroup(definition, options.group);
+        await this.promoteDelayed(definition, options.concurrency);
+        const claimed = await this.claimStale(definition, options);
+        const entries = claimed.length > 0 ? claimed : await this.readNew(definition, options);
+        if (entries.length === 0)
+            return 0;
+        await Promise.all(entries.map(entry => this.processEntry(definition, handler, options, entry)));
+        return entries.length;
+    }
+    async processEntry(definition, handler, options, [streamId, fields]) {
+        const stored = this.decodeEntry(definition, fields);
+        const attempt = stored.attempts + 1;
+        try {
+            const job = this.createJob(definition, stored, streamId, attempt);
+            const run = () => Promise.resolve(handler(job));
+            if (stored.context && isContextData(stored.context)) {
+                await runWithContext(stored.context, run);
+            }
+            else {
+                await run();
+            }
+            await this.redis.xack(this.streamKey(definition), options.group, streamId);
+        }
+        catch (err) {
+            await this.handleFailure(definition, stored, streamId, attempt, options.group, err);
+        }
+    }
+    async handleFailure(definition, stored, streamId, attempt, group, err) {
+        const reason = errorReason(err);
+        const failed = {
+            ...stored,
+            attempts: attempt,
+            firstFailureReason: stored.firstFailureReason ?? reason,
+            lastFailureReason: reason,
+        };
+        if (attempt < stored.maxAttempts) {
+            const delay = backoffDelay(stored.backoff, attempt);
+            failed.runAt = this.now() + delay;
+            await this.redis.zadd(this.delayedKey(definition), failed.runAt, this.encodeJob(definition, failed));
+        }
+        else {
+            await this.redis.xadd(this.deadLetterKey(definition), '*', 'job', this.encodeJob(definition, failed));
+        }
+        await this.redis.xack(this.streamKey(definition), group, streamId);
+    }
+    createJob(definition, stored, streamId, attempt) {
+        return {
+            id: stored.id,
+            queue: stored.queue,
+            data: parsePayload(definition, stored.data),
+            attempt,
+            maxAttempts: stored.maxAttempts,
+            streamId,
+            ...(stored.jobId ? { jobId: stored.jobId } : {}),
+            createdAt: stored.createdAt,
+            runAt: stored.runAt,
+            ...(stored.context ? { context: stored.context } : {}),
+        };
+    }
+    async promoteDelayed(definition, limit) {
+        const members = await this.redis.zrangebyscore(this.delayedKey(definition), '-inf', this.now(), 'LIMIT', 0, limit);
+        let promoted = 0;
+        for (const member of members) {
+            const removed = await this.redis.zrem(this.delayedKey(definition), member);
+            if (removed === 0)
+                continue;
+            await this.redis.xadd(this.streamKey(definition), '*', 'job', member);
+            promoted++;
+        }
+        return promoted;
+    }
+    async claimStale(definition, options) {
+        const pending = await this.redis.xpending(this.streamKey(definition), options.group, '-', '+', options.claimCount);
+        const ids = pending
+            .filter(([, , idle]) => idle >= options.claimIdle)
+            .slice(0, options.concurrency)
+            .map(([id]) => id);
+        if (ids.length === 0)
+            return [];
+        return await this.redis.xclaim(this.streamKey(definition), options.group, options.consumer, options.claimIdle, ...ids);
+    }
+    async readNew(definition, options) {
+        const args = [
+            'GROUP',
+            options.group,
+            options.consumer,
+            'COUNT',
+            options.concurrency,
+        ];
+        if (options.block > 0) {
+            args.push('BLOCK', options.block);
+        }
+        args.push('STREAMS', this.streamKey(definition), '>');
+        const result = await this.redis.xreadgroup(...args);
+        return result?.[0]?.[1] ?? [];
+    }
+    async ensureGroup(definition, group) {
+        try {
+            await this.redis.xgroup('CREATE', this.streamKey(definition), group, '0', 'MKSTREAM');
+        }
+        catch (err) {
+            if (!String(err.message ?? err).includes('BUSYGROUP'))
+                throw err;
+        }
+    }
+    decodeEntry(definition, fields) {
+        const value = fieldsToObject(fields)['job'];
+        if (!value)
+            throw new QueueSerializationError(definition.name, new Error('missing job field'));
+        try {
+            return JSON.parse(value);
+        }
+        catch (err) {
+            throw new QueueSerializationError(definition.name, err);
+        }
+    }
+    encodeJob(definition, job) {
+        try {
+            const encoded = JSON.stringify(job);
+            if (encoded === undefined)
+                throw new TypeError('JSON.stringify returned undefined');
+            return encoded;
+        }
+        catch (err) {
+            throw new QueueSerializationError(definition.name, err);
+        }
+    }
+    streamKey(definition) {
+        return `${this.prefix}queue:${definition.name}:stream`;
+    }
+    delayedKey(definition) {
+        return `${this.prefix}queue:${definition.name}:delayed`;
+    }
+    deadLetterKey(definition) {
+        return `${this.prefix}queue:${definition.name}:dlq`;
+    }
+    dedupeKey(definition, jobId) {
+        return `${this.prefix}queue:${definition.name}:job:${jobId}`;
+    }
+}
+export class RedisStreamQueueWorker {
+    queue;
+    definition;
+    handler;
+    options;
+    running = false;
+    loop;
+    constructor(queue, definition, handler, options) {
+        this.queue = queue;
+        this.definition = definition;
+        this.handler = handler;
+        this.options = normalizeWorkerOptions(definition, options);
+    }
+    runOnce() {
+        return this.queue.runWorkerOnce(this.definition, this.handler, this.options);
+    }
+    start() {
+        if (this.running)
+            return this;
+        this.running = true;
+        this.loop = this.runLoop();
+        return this;
+    }
+    async stop() {
+        this.running = false;
+        await this.loop;
+    }
+    async runLoop() {
+        while (this.running) {
+            const processed = await this.runOnce();
+            if (processed === 0) {
+                await sleep(this.options.pollInterval);
+            }
+        }
+    }
+}
+function normalizeWorkerOptions(definition, options) {
+    const concurrency = assertPositiveInteger(options.concurrency ?? DEFAULT_CONCURRENCY, 'concurrency');
+    return {
+        group: options.group ?? `${definition.name}-workers`,
+        consumer: options.consumer ?? `${process.pid}-${randomUUID()}`,
+        concurrency,
+        block: assertNonNegativeInteger(options.block ?? DEFAULT_READ_BLOCK, 'block'),
+        pollInterval: assertNonNegativeInteger(options.pollInterval ?? DEFAULT_POLL_INTERVAL, 'pollInterval'),
+        claimIdle: assertNonNegativeInteger(options.claimIdle ?? DEFAULT_CLAIM_IDLE, 'claimIdle'),
+        claimCount: assertPositiveInteger(options.claimCount ?? concurrency, 'claimCount'),
+    };
+}
+function normalizeBackoff(backoff) {
+    if (backoff === undefined)
+        return { type: 'fixed', delay: 0 };
+    if (typeof backoff === 'number') {
+        return { type: 'fixed', delay: assertNonNegativeInteger(backoff, 'backoff') };
+    }
+    if (backoff.type === 'fixed') {
+        return { type: 'fixed', delay: assertNonNegativeInteger(backoff.delay, 'backoff.delay') };
+    }
+    return {
+        type: 'exponential',
+        baseMs: assertNonNegativeInteger(backoff.baseMs, 'backoff.baseMs'),
+        ...(backoff.maxMs !== undefined ? { maxMs: assertNonNegativeInteger(backoff.maxMs, 'backoff.maxMs') } : {}),
+    };
+}
+function backoffDelay(backoff, attempt) {
+    if (backoff.type === 'fixed')
+        return backoff.delay;
+    const delay = backoff.baseMs * 2 ** Math.max(0, attempt - 1);
+    return backoff.maxMs === undefined ? delay : Math.min(delay, backoff.maxMs);
+}
+function fieldsToObject(fields) {
+    const result = {};
+    for (let i = 0; i < fields.length; i += 2) {
+        const key = fields[i];
+        const value = fields[i + 1];
+        if (key !== undefined && value !== undefined)
+            result[key] = value;
+    }
+    return result;
+}
+function assertPositiveInteger(value, name) {
+    if (!Number.isFinite(value) || value <= 0 || Math.trunc(value) !== value) {
+        throw new TypeError(`${name} must be a positive integer`);
+    }
+    return value;
+}
+function assertNonNegativeInteger(value, name) {
+    if (!Number.isFinite(value) || value < 0 || Math.trunc(value) !== value) {
+        throw new TypeError(`${name} must be a non-negative integer`);
+    }
+    return value;
+}
+function errorReason(err) {
+    if (err instanceof Error)
+        return err.message;
+    return String(err);
+}
+function sleep(ms) {
+    return new Promise(resolve => setTimeout(resolve, ms));
+}

package/dist/types.d.ts

@@ -0,0 +1,119 @@
+import type { ContextData, ContextInput } from '@hile/context';
+export type QueueSafeParseResult<T> = {
+    success: true;
+    data: T;
+} | {
+    success: false;
+    error: unknown;
+};
+export type QueueSchema<T> = {
+    parse(value: unknown): T;
+} | {
+    safeParse(value: unknown): QueueSafeParseResult<T>;
+};
+export type InferQueueSchema<TSchema> = TSchema extends {
+    parse(value: unknown): infer T;
+} ? T : TSchema extends {
+    safeParse(value: unknown): QueueSafeParseResult<infer T>;
+} ? T : never;
+export type QueueDefinition<TData = unknown, TName extends string = string> = {
+    readonly name: TName;
+    readonly schema?: QueueSchema<TData>;
+};
+export type QueueBackoff = number | {
+    type: 'fixed';
+    delay: number;
+} | {
+    type: 'exponential';
+    baseMs: number;
+    maxMs?: number;
+};
+export type QueueAddOptions = {
+    jobId?: string;
+    delay?: number;
+    maxAttempts?: number;
+    backoff?: QueueBackoff;
+};
+export type QueueAddResult = {
+    accepted: boolean;
+    duplicate: boolean;
+    id: string;
+    jobId?: string;
+    streamId?: string;
+    runAt: number;
+};
+export type QueueJob<TData = unknown> = {
+    id: string;
+    queue: string;
+    data: TData;
+    attempt: number;
+    maxAttempts: number;
+    streamId: string;
+    jobId?: string;
+    createdAt: number;
+    runAt: number;
+    context?: ContextInput<ContextData>;
+};
+export type QueueDeadLetter<TData = unknown> = Omit<QueueJob<TData>, 'streamId'> & {
+    streamId: string;
+    firstFailureReason?: string;
+    lastFailureReason?: string;
+};
+export type QueueWorkerHandler<TData = unknown> = (job: QueueJob<TData>) => void | Promise<void>;
+export type QueueWorkerOptions = {
+    group?: string;
+    consumer?: string;
+    concurrency?: number;
+    block?: number;
+    pollInterval?: number;
+    claimIdle?: number;
+    claimCount?: number;
+};
+export type RedisStreamEntry = [id: string, fields: string[]];
+export type RedisStreamReadResult = Array<[stream: string, entries: RedisStreamEntry[]]> | null;
+export type RedisPendingEntry = [id: string, consumer: string, idle: number, deliveries: number];
+export interface RedisStreamQueueLike {
+    xgroup(...args: any[]): Promise<any>;
+    xadd(...args: any[]): Promise<any>;
+    xreadgroup(...args: any[]): Promise<any>;
+    xpending(...args: any[]): Promise<any>;
+    xclaim(...args: any[]): Promise<any>;
+    xack(...args: any[]): Promise<any>;
+    xrange(...args: any[]): Promise<any>;
+    zadd(...args: any[]): Promise<any>;
+    zrangebyscore(...args: any[]): Promise<any>;
+    zrem(...args: any[]): Promise<any>;
+    set(...args: any[]): Promise<any>;
+    get(...args: any[]): Promise<any>;
+    del(...args: any[]): Promise<any>;
+}
+export type RedisStreamQueueOptions = {
+    prefix?: string;
+    now?: () => number;
+};
+export type ReadDeadLettersOptions = {
+    count?: number;
+};
+export type StoredQueueBackoff = {
+    type: 'fixed';
+    delay: number;
+} | {
+    type: 'exponential';
+    baseMs: number;
+    maxMs?: number;
+};
+export type StoredQueueJob = {
+    v: 1;
+    id: string;
+    queue: string;
+    data: unknown;
+    createdAt: number;
+    runAt: number;
+    attempts: number;
+    maxAttempts: number;
+    backoff: StoredQueueBackoff;
+    jobId?: string;
+    context?: ContextInput<ContextData>;
+    firstFailureReason?: string;
+    lastFailureReason?: string;
+};

package/dist/types.js

@@ -0,0 +1 @@
+export {};

package/package.json

@@ -0,0 +1,30 @@
+{
+  "name": "@hile/redis-stream-queue",
+  "version": "3.0.1",
+  "description": "Redis Streams backed durable job queue primitives for Hile applications",
+  "type": "module",
+  "main": "./dist/index.js",
+  "scripts": {
+    "build": "tsc -b && fix-esm-import-path --preserve-import-type ./dist",
+    "dev": "tsc -b --watch",
+    "test": "vitest run && tsc -p tsconfig.typecheck.json"
+  },
+  "files": [
+    "dist",
+    "README.md",
+    "SKILL.md"
+  ],
+  "license": "MIT",
+  "publishConfig": {
+    "access": "public"
+  },
+  "devDependencies": {
+    "fix-esm-import-path": "^1.10.3",
+    "ioredis": "^5.11.0",
+    "vitest": "^4.0.18"
+  },
+  "dependencies": {
+    "@hile/context": "^3.0.1"
+  },
+  "gitHead": "762f9476b1b53f91899b985fe5bf437b618bb224"
+}