babelqueue 1.5.0__tar.gz → 1.6.0__tar.gz

{babelqueue-1.5.0 → babelqueue-1.6.0}/CHANGELOG.md

@@ -7,6 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 The envelope wire format is versioned separately by `meta.schema_version`
 (currently **1**) — see the contract at [babelqueue.com](https://babelqueue.com).
 
+## [Unreleased]
+
+## [1.6.0] - 2026-06-14
+
+### Added
+- **Redis/Laravel reservation parity** — the Redis transport can now consume a
+  **shared** Laravel BabelQueue Redis queue using Laravel's reserved-set / reliable-queue
+  semantics, not just a Python-owned queue. Enable it with the `laravel=1` URL flag
+  (`redis://host:6379/0?laravel=1`) or `RedisTransport(..., laravel_compat=True)`. In this
+  mode the key layout is Laravel's stock Redis queue ([§1 of the broker-bindings
+  contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)): a `queues:<name>`
+  ready list, a `queues:<name>:reserved` **sorted set** scored by a `retry_after` deadline
+  (default 60s), a `queues:<name>:delayed` set, and a `queues:<name>:notify` wake-up list.
+  Reserve, ack and release run the **byte-for-byte same Lua scripts** Laravel uses, so the
+  reserved member a Python worker writes is identical to a Laravel worker's — either side can
+  ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share one
+  Redis queue without losing or double-processing messages. Before each pop, expired
+  reserved/delayed jobs migrate back to the ready list, so a crashed worker's in-flight job
+  is re-reserved exactly as Laravel does. Tunable via `?prefix=` / `?retry_after=` (or the
+  `key_prefix` / `retry_after` constructor args). The default Python-owned reliable-queue
+  mode (`BLMOVE` + `<queue>:processing` list) is unchanged and stays the default, so existing
+  callers are unaffected. The reservation logic is fully unit-tested with an injected
+  in-memory Redis double (no `redis` package, no broker); a live cross-runtime PHP↔Python
+  shared-queue round-trip is covered by the integration suite where a real Redis is present.
+  The envelope is unchanged (`schema_version: 1`); this is purely additive. Ships as a MINOR.
+
 ## [1.5.0] - 2026-06-13
 
 ### Added
@@ -173,7 +199,8 @@ reference at [babelqueue.com](https://babelqueue.com).
 - Pre-1.0: the public API may change before the `1.0.0` tag.
 - The core has **zero runtime dependencies** (standard library only); Python `>=3.9`.
 
-[Unreleased]: https://github.com/BabelQueue/babelqueue-python/compare/v1.2.0...HEAD
+[Unreleased]: https://github.com/BabelQueue/babelqueue-python/compare/v1.6.0...HEAD
+[1.6.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.5.0...v1.6.0
 [1.2.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.0.0...v1.1.0
 [1.0.0]: https://github.com/BabelQueue/babelqueue-python/compare/v0.5.0...v1.0.0

{babelqueue-1.5.0 → babelqueue-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: babelqueue
-Version: 1.5.0
+Version: 1.6.0
 Summary: Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers.
 Project-URL: Homepage, https://babelqueue.com
 Project-URL: Source, https://github.com/BabelQueue/babelqueue-python
@@ -164,9 +164,35 @@ app.run()                               # consume forever (Ctrl-C to stop)
 - **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
   envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
   messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
-- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
-  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
-  tests/local). Bring your own by passing `transport=...`.
+- **Transports:** `redis://` (reliable-queue pattern; add `?laravel=1` to share a
+  Redis queue with a Laravel BabelQueue worker using its reserved-set semantics —
+  see below), `amqp://` (RabbitMQ via `pika`, with the contract AMQP properties)
+  and `memory://` (in-process, great for tests/local). Bring your own by passing
+  `transport=...`.
+
+### Sharing a Redis queue with Laravel
+
+By default the Redis transport owns its queue end-to-end (`RPUSH` to produce;
+`BLMOVE` into a `<queue>:processing` list to reserve; `LREM` on ack). To consume a
+**shared** Laravel BabelQueue Redis queue instead, enable Laravel-compatible mode:
+
+```python
+app = BabelQueue("redis://localhost:6379/0?laravel=1", queue="orders")
+# or: RedisTransport("redis://localhost:6379/0", laravel_compat=True)
+```
+
+This replicates Laravel's stock Redis reservation exactly ([§1 of the
+broker-bindings contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)):
+the ready list is `queues:<name>`, reservations move into the `queues:<name>:reserved`
+**sorted set** scored by a `retry_after` deadline (default 60s), with a
+`queues:<name>:delayed` set and a `queues:<name>:notify` wake-up list. Reserve, ack
+and release run the **byte-for-byte same Lua scripts** Laravel uses, so the reserved
+member a Python worker writes is identical to a Laravel worker's — either side can
+ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share
+one queue without losing or double-processing messages. Before each pop, expired
+reserved/delayed jobs migrate back to the ready list, so a crashed worker's
+in-flight job is re-reserved. Options: `?prefix=queues:` and `?retry_after=60`
+(or the `key_prefix` / `retry_after` constructor args).
 
 ## Framework adapters — Celery & Django
 

{babelqueue-1.5.0 → babelqueue-1.6.0}/README.md

@@ -114,9 +114,35 @@ app.run()                               # consume forever (Ctrl-C to stop)
 - **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
   envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
   messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
-- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
-  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
-  tests/local). Bring your own by passing `transport=...`.
+- **Transports:** `redis://` (reliable-queue pattern; add `?laravel=1` to share a
+  Redis queue with a Laravel BabelQueue worker using its reserved-set semantics —
+  see below), `amqp://` (RabbitMQ via `pika`, with the contract AMQP properties)
+  and `memory://` (in-process, great for tests/local). Bring your own by passing
+  `transport=...`.
+
+### Sharing a Redis queue with Laravel
+
+By default the Redis transport owns its queue end-to-end (`RPUSH` to produce;
+`BLMOVE` into a `<queue>:processing` list to reserve; `LREM` on ack). To consume a
+**shared** Laravel BabelQueue Redis queue instead, enable Laravel-compatible mode:
+
+```python
+app = BabelQueue("redis://localhost:6379/0?laravel=1", queue="orders")
+# or: RedisTransport("redis://localhost:6379/0", laravel_compat=True)
+```
+
+This replicates Laravel's stock Redis reservation exactly ([§1 of the
+broker-bindings contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)):
+the ready list is `queues:<name>`, reservations move into the `queues:<name>:reserved`
+**sorted set** scored by a `retry_after` deadline (default 60s), with a
+`queues:<name>:delayed` set and a `queues:<name>:notify` wake-up list. Reserve, ack
+and release run the **byte-for-byte same Lua scripts** Laravel uses, so the reserved
+member a Python worker writes is identical to a Laravel worker's — either side can
+ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share
+one queue without losing or double-processing messages. Before each pop, expired
+reserved/delayed jobs migrate back to the ready list, so a crashed worker's
+in-flight job is re-reserved. Options: `?prefix=queues:` and `?retry_after=60`
+(or the `key_prefix` / `retry_after` constructor args).
 
 ## Framework adapters — Celery & Django
 

{babelqueue-1.5.0 → babelqueue-1.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "babelqueue"
-version = "1.5.0"
+version = "1.6.0"
 description = "Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers."
 readme = "README.md"
 requires-python = ">=3.9"

{babelqueue-1.5.0 → babelqueue-1.6.0}/src/babelqueue/__init__.py

@@ -19,7 +19,7 @@ from .exceptions import BabelQueueError, UnknownUrnError
 from .routing import UnknownUrnStrategy
 from .transport import InMemoryTransport, ReceivedMessage, Transport
 
-__version__ = "1.2.0"
+__version__ = "1.6.0"
 
 __all__ = [
     "BabelQueue",

babelqueue-1.6.0/src/babelqueue/redis_transport.py

@@ -0,0 +1,254 @@
+"""Redis transport. Requires the ``redis`` extra:
+
+    pip install "babelqueue[redis]"
+
+Two reservation modes share one class, selected by ``laravel_compat``:
+
+**Python-owned (default).** Producing is ``RPUSH queue body``; consuming atomically
+moves the head to a per-queue processing list (``BLMOVE``) so an in-flight message
+survives a worker crash, and ``ack`` removes it from that processing list. Simple,
+self-contained, and the right choice for a queue this runtime owns end-to-end.
+
+**Laravel-compatible (``laravel_compat=True``).** Replicates Laravel's stock Redis
+queue reservation so a Python worker can share one Redis queue with a PHP/Laravel
+worker without losing or double-processing messages (§1 of the broker-bindings
+contract). The key layout is Laravel's: a ``queues:<name>`` ready list, a
+``queues:<name>:reserved`` sorted set scored by a retry-after deadline, a
+``queues:<name>:delayed`` sorted set, and a ``queues:<name>:notify`` wake-up list.
+Reserve/ack/release run the **byte-for-byte same Lua scripts** Laravel uses, so the
+reserved-set member a Python worker writes is identical to what a Laravel worker
+writes — either side can therefore ack (``ZREM``) the other's reservation. Before
+each pop the transport migrates expired reserved/delayed jobs back to the ready
+list, so a crashed worker's in-flight job is re-reserved exactly as Laravel does.
+
+The envelope is unchanged (``schema_version`` stays 1); Redis is purely additive.
+
+URL form: ``redis://host:port/db[?laravel=1&prefix=queues:&retry_after=60]``.
+For richer setups, build the transport directly and pass ``BabelQueue(transport=...)``.
+"""
+
+from __future__ import annotations
+
+import time
+from typing import Any, Optional
+from urllib.parse import parse_qs, urlsplit
+
+from .transport import ReceivedMessage, Transport
+
+# Laravel's stock Redis queue Lua scripts (Illuminate\Queue\LuaScripts). Replicated
+# verbatim so the reserved-set member byte form matches Laravel's — that exact match
+# is what lets a Python and a Laravel worker ack each other's reservations.
+
+# KEYS[1]=ready list, KEYS[2]=notify list; ARGV[1]=payload.
+_PUSH = """
+redis.call('rpush', KEYS[1], ARGV[1])
+redis.call('rpush', KEYS[2], 1)
+"""
+
+# KEYS[1]=ready, KEYS[2]=reserved, KEYS[3]=notify; ARGV[1]=reserved-until score.
+# Returns {original_job, reserved_member}; the reserved member is the attempts+1
+# re-encoding stored in the reserved set and is the ack handle.
+_POP = """
+local job = redis.call('lpop', KEYS[1])
+local reserved = false
+
+if(job ~= false) then
+    reserved = cjson.decode(job)
+    reserved['attempts'] = reserved['attempts'] + 1
+    reserved = cjson.encode(reserved)
+    redis.call('zadd', KEYS[2], ARGV[1], reserved)
+    redis.call('lpop', KEYS[3])
+end
+
+return {job, reserved}
+"""
+
+# KEYS[1]=delayed, KEYS[2]=reserved; ARGV[1]=reserved member, ARGV[2]=available-at.
+_RELEASE = """
+redis.call('zrem', KEYS[2], ARGV[1])
+redis.call('zadd', KEYS[1], ARGV[2], ARGV[1])
+return true
+"""
+
+# KEYS[1]=from (reserved/delayed), KEYS[2]=ready, KEYS[3]=notify; ARGV[1]=now,
+# ARGV[2]=batch size (-1 = all). Moves jobs whose score has expired back to ready.
+_MIGRATE = """
+local val = redis.call('zrangebyscore', KEYS[1], '-inf', ARGV[1], 'limit', 0, ARGV[2])
+
+if(next(val) ~= nil) then
+    redis.call('zremrangebyrank', KEYS[1], 0, #val - 1)
+
+    for i = 1, #val, 100 do
+        redis.call('rpush', KEYS[2], unpack(val, i, math.min(i+99, #val)))
+        for j = i, math.min(i+99, #val) do
+            redis.call('rpush', KEYS[3], 1)
+        end
+    end
+end
+
+return val
+"""
+
+
+class RedisTransport(Transport):
+    def __init__(
+        self,
+        url: str = "redis://localhost:6379/0",
+        *,
+        client: Any = None,
+        processing_suffix: str = ":processing",
+        laravel_compat: bool = False,
+        key_prefix: str = "queues:",
+        retry_after: int = 60,
+        block_for: Optional[float] = None,
+    ) -> None:
+        parts = urlsplit(url) if url else urlsplit("redis://")
+        q = parse_qs(parts.query)
+
+        self._processing_suffix = processing_suffix
+        self._laravel_compat = laravel_compat or _qbool(q, "laravel") or _qbool(q, "laravel_compat")
+        self._key_prefix = key_prefix if "prefix" not in q else (_q1(q, "prefix") or key_prefix)
+        self._retry_after = retry_after if "retry_after" not in q else _qint(q, "retry_after", retry_after)
+        self._block_for = block_for
+
+        if client is not None:
+            self._redis = client
+            return
+        try:
+            import redis
+        except ImportError as exc:  # pragma: no cover - import guard
+            raise ImportError(
+                "RedisTransport requires the 'redis' package. Install with "
+                'pip install "babelqueue[redis]".'
+            ) from exc
+        self._redis = redis.Redis.from_url(url, decode_responses=True)
+
+    # -- key helpers --------------------------------------------------------
+
+    def _processing(self, queue: str) -> str:
+        return f"{queue}{self._processing_suffix}"
+
+    def _ready(self, queue: str) -> str:
+        """The Laravel ready-list key for a logical queue name (``queues:<name>``)."""
+        return f"{self._key_prefix}{queue}"
+
+    # -- Transport ----------------------------------------------------------
+
+    def publish(self, queue: str, body: str) -> None:
+        if not self._laravel_compat:
+            self._redis.rpush(queue, body)
+            return
+        ready = self._ready(queue)
+        self._redis.eval(_PUSH, 2, ready, f"{ready}:notify", body)
+
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        if not self._laravel_compat:
+            return self._pop_owned(queue, timeout)
+        return self._pop_laravel(queue, timeout)
+
+    def ack(self, message: ReceivedMessage) -> None:
+        if not self._laravel_compat:
+            self._redis.lrem(self._processing(message.queue), 1, message.handle)
+            return
+        if not message.handle:
+            return
+        self._redis.zrem(f"{self._ready(message.queue)}:reserved", message.handle)
+
+    def release(self, message: ReceivedMessage, delay: int = 0) -> None:
+        """Return a reserved Laravel job to the (delayed) ready set for retry.
+
+        No-op in Python-owned mode (the runtime re-publishes instead). In
+        Laravel-compatible mode this is ``ZREM :reserved`` + ``ZADD :delayed`` —
+        the exact bookkeeping Laravel's release performs.
+        """
+        if not self._laravel_compat or not message.handle:
+            return
+        ready = self._ready(message.queue)
+        self._redis.eval(
+            _RELEASE, 2, f"{ready}:delayed", f"{ready}:reserved",
+            message.handle, int(time.time()) + int(delay),
+        )
+
+    def close(self) -> None:  # pragma: no cover
+        self._redis.close()
+
+    # -- Python-owned reliable queue ----------------------------------------
+
+    def _pop_owned(self, queue: str, timeout: float) -> Optional[ReceivedMessage]:
+        # redis-py types the BLMOVE timeout as int, but Redis accepts a float
+        # (sub-second) timeout; passing it through is correct at runtime.
+        body = self._redis.blmove(queue, self._processing(queue), timeout, "LEFT", "RIGHT")  # type: ignore[arg-type]
+        if body is None:
+            return None
+        text = _as_text(body)
+        return ReceivedMessage(body=text, queue=queue, handle=text)
+
+    # -- Laravel-compatible reservation -------------------------------------
+
+    def _pop_laravel(self, queue: str, timeout: float) -> Optional[ReceivedMessage]:
+        ready = self._ready(queue)
+        self._migrate(ready)
+
+        result = self._redis.eval(
+            _POP, 3, ready, f"{ready}:reserved", f"{ready}:notify",
+            int(time.time()) + self._retry_after,
+        )
+        job, reserved = _pop_result(result)
+        if job is None or reserved is None:
+            # Nothing ready right now; optionally block on the notify list so a
+            # concurrently-pushed job wakes us, then retry once (mirrors Laravel).
+            block = self._block_for if self._block_for is not None else timeout
+            if block and block > 0 and self._redis.blpop([f"{ready}:notify"], block):
+                result = self._redis.eval(
+                    _POP, 3, ready, f"{ready}:reserved", f"{ready}:notify",
+                    int(time.time()) + self._retry_after,
+                )
+                job, reserved = _pop_result(result)
+            if job is None or reserved is None:
+                return None
+
+        # The reserved member (attempts already incremented) is both the consumed
+        # body and the ack handle, so ZREM removes exactly this reservation.
+        return ReceivedMessage(body=_as_text(reserved), queue=queue, handle=_as_text(reserved))
+
+    def _migrate(self, ready: str) -> None:
+        now = int(time.time())
+        # Delayed jobs that are due, then reserved jobs whose retry-after lapsed
+        # (a crashed worker's in-flight job) — both move back to the ready list.
+        self._redis.eval(_MIGRATE, 3, f"{ready}:delayed", ready, f"{ready}:notify", now, -1)
+        self._redis.eval(_MIGRATE, 3, f"{ready}:reserved", ready, f"{ready}:notify", now, -1)
+
+
+def _as_text(value: Any) -> str:
+    """Normalise a Redis reply to ``str`` (bytes when ``decode_responses`` is off)."""
+    return value if isinstance(value, str) else value.decode()
+
+
+def _pop_result(result: Any) -> tuple[Optional[Any], Optional[Any]]:
+    """Unpack the ``{job, reserved}`` reply; ``false`` Lua values arrive as ``None``."""
+    if not result or len(result) < 2:
+        return None, None
+    job, reserved = result[0], result[1]
+    if not job or not reserved:
+        return None, None
+    return job, reserved
+
+
+def _q1(q: dict, key: str) -> Optional[str]:
+    values = q.get(key)
+    return values[0] if values else None
+
+
+def _qint(q: dict, key: str, default: int) -> int:
+    v = _q1(q, key)
+    if v is None:
+        return default
+    try:
+        return int(v)
+    except ValueError:  # pragma: no cover - defensive
+        return default
+
+
+def _qbool(q: dict, key: str) -> bool:
+    v = _q1(q, key)
+    return v is not None and v.lower() in ("1", "true", "yes", "on")

{babelqueue-1.5.0 → babelqueue-1.6.0}/tests/conformance/manifest.json

@@ -200,5 +200,31 @@
         { "name": "native-equals-body",        "body_attempts": 3, "delivery_count": 3, "expected_attempts": 3 }
       ]
     }
+  },
+  "redis": {
+    "description": "Redis binding conformance (broker-bindings.md §1 — the reliable-queue list pattern). Redis lists carry no native metadata, so there is NO property projection: the queue element IS the canonical envelope JSON, byte-for-byte, with NO wrapping (unlike BullMQ's job structure) and NO added fields. The reserve/ack reliability (Laravel's reserved-sorted-set vs the App runtimes' processing-list) and attempts handling are SDK-runtime-specific and proven by the live interop round-trip, not a static fixture. The one cross-SDK invariant locked here is payload identity.",
+    "payload_identity": {
+      "description": "On produce, the transport MUST store the envelope JSON byte-for-byte as the queue element (RPUSH / equivalent); on reserve it MUST return that element unchanged — no wrapping, no envelope mutation, no added metadata. A §1 SDK round-trips fixtures/order-created.json through produce→reserve and gets back the identical bytes. Applies to every Redis-producing SDK.",
+      "envelope_file": "fixtures/order-created.json"
+    }
+  },
+  "rabbitmq": {
+    "description": "RabbitMQ binding conformance (broker-bindings.md §2 — AMQP 0-9-1). The envelope body stays byte-identical (the 'cases' above); this locks the native AMQP-0-9-1 property projection every producer emits. NOTE: consume-side attempts handling is NOT in scope here because it is runtime-specific — the Laravel driver reads the authoritative x-attempts header (§2.5), while the App-runtime SDKs (Go/Python) keep attempts in the body and republish with attempts+1 (the x-attempts header is then a redundant produce-side mirror). Both stay consistent because Laravel increments the body's attempts too. Per-message values reuse fixtures/order-created.json.",
+    "property_projection": {
+      "description": "On produce, the transport MUST project these native AMQP 0-9-1 properties + headers from the envelope. Header values are NATIVE-typed (AMQP 0-9-1 field-tables carry typed values — integers stay integers, NOT stringified like Kafka/Pulsar): type = job (the URN, route on it without reading the body), correlation_id = trace_id, message_id = meta.id, app_id = 'babelqueue', content_type = 'application/json', header x-schema-version = meta.schema_version (number), x-source-lang = meta.lang, x-attempts = attempts (number). The body is the byte-identical envelope. Applies to every RabbitMQ-producing SDK.",
+      "envelope_file": "fixtures/order-created.json",
+      "properties": {
+        "type":           "urn:babel:orders:created",
+        "correlation_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
+        "message_id":     "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
+        "app_id":         "babelqueue",
+        "content_type":   "application/json"
+      },
+      "headers": {
+        "x-schema-version": 1,
+        "x-source-lang":    "php",
+        "x-attempts":       0
+      }
+    }
   }
 }

babelqueue-1.6.0/tests/test_redis_transport.py

@@ -0,0 +1,396 @@
+"""Tests for the Redis transport.
+
+The unit tests inject a tiny in-memory Redis double, so they run without the
+``redis`` package and without a broker. The double emulates exactly the commands
+the transport uses (``rpush``/``lpop``/``blmove``/``lrem``/``zadd``/``zrem``/
+``blpop``) plus the three Laravel Lua scripts the Laravel-compatible mode runs,
+so the reserved-set / reliable-queue semantics are exercised end-to-end.
+
+A separate integration test round-trips against a real Redis and is skipped unless
+the ``redis`` package and a reachable broker are present (CI runs it).
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import unittest
+import uuid
+
+from babelqueue import BabelQueue, EnvelopeCodec
+from babelqueue.redis_transport import RedisTransport, _pop_result, _qint
+from babelqueue.transport import ReceivedMessage
+
+
+class FakeRedis:
+    """In-memory stand-in for the subset of Redis the transport uses.
+
+    Lists are Python lists; sorted sets are ``{member: score}`` dicts. ``eval``
+    recognises the three Laravel Lua scripts (push / pop / migrate / release) by
+    keyword and emulates their effect — including the ``cjson`` re-encode of the
+    reserved member, which is what makes Python's and Laravel's reserved members
+    byte-identical in production.
+    """
+
+    def __init__(self) -> None:
+        self.lists: dict[str, list[str]] = {}
+        self.zsets: dict[str, dict[str, float]] = {}
+
+    # -- plain commands (Python-owned mode) ---------------------------------
+
+    def rpush(self, key, value):  # noqa: ANN001
+        self.lists.setdefault(key, []).append(value)
+        return len(self.lists[key])
+
+    def lpop(self, key):  # noqa: ANN001
+        lst = self.lists.get(key)
+        return lst.pop(0) if lst else None
+
+    def llen(self, key):  # noqa: ANN001
+        return len(self.lists.get(key, []))
+
+    def lrange(self, key, start, stop):  # noqa: ANN001
+        lst = self.lists.get(key, [])
+        end = len(lst) if stop == -1 else stop + 1
+        return lst[start:end]
+
+    def blmove(self, src, dst, timeout, src_dir, dst_dir):  # noqa: ANN001, ARG002
+        lst = self.lists.get(src)
+        if not lst:
+            return None
+        value = lst.pop(0)
+        self.lists.setdefault(dst, []).append(value)
+        return value
+
+    def lrem(self, key, count, value):  # noqa: ANN001, ARG002
+        lst = self.lists.get(key, [])
+        if value in lst:
+            lst.remove(value)
+            return 1
+        return 0
+
+    def blpop(self, keys, timeout):  # noqa: ANN001, ARG002
+        for key in keys:
+            lst = self.lists.get(key)
+            if lst:
+                return (key, lst.pop(0))
+        return None
+
+    def zadd(self, key, mapping):  # noqa: ANN001
+        self.zsets.setdefault(key, {}).update(mapping)
+
+    def zrem(self, key, member):  # noqa: ANN001
+        z = self.zsets.get(key, {})
+        return 1 if z.pop(member, None) is not None else 0
+
+    def zcard(self, key):  # noqa: ANN001
+        return len(self.zsets.get(key, {}))
+
+    def delete(self, *keys):  # noqa: ANN001
+        for key in keys:
+            self.lists.pop(key, None)
+            self.zsets.pop(key, None)
+
+    def close(self):
+        pass
+
+    # -- Lua scripts (Laravel-compatible mode) ------------------------------
+
+    def eval(self, script, numkeys, *args):  # noqa: ANN001
+        keys = list(args[:numkeys])
+        argv = list(args[numkeys:])
+        if "rpush" in script and "lpop" not in script and "zrange" not in script:
+            return self._push(keys, argv)
+        if "'lpop'" in script and "zadd" in script:
+            return self._pop(keys, argv)
+        if "zrangebyscore" in script:
+            return self._migrate(keys, argv)
+        if "'zrem'" in script and "zadd" in script:
+            return self._release(keys, argv)
+        raise AssertionError("unrecognised Lua script")
+
+    def _push(self, keys, argv):  # noqa: ANN001
+        ready, notify = keys
+        self.rpush(ready, argv[0])
+        self.rpush(notify, "1")
+
+    def _pop(self, keys, argv):  # noqa: ANN001
+        ready, reserved_key, notify = keys
+        job = self.lpop(ready)
+        if job is None:
+            return [None, None]
+        decoded = json.loads(job)
+        decoded["attempts"] = decoded.get("attempts", 0) + 1
+        reserved = json.dumps(decoded, separators=(",", ":"))
+        self.zadd(reserved_key, {reserved: float(argv[0])})
+        self.lpop(notify)
+        return [job, reserved]
+
+    def _migrate(self, keys, argv):  # noqa: ANN001
+        src, dst, notify = keys
+        now = float(argv[0])
+        z = self.zsets.get(src, {})
+        due = [m for m, score in z.items() if score <= now]
+        for member in due:
+            z.pop(member, None)
+            self.rpush(dst, member)
+            self.rpush(notify, "1")
+        return due
+
+    def _release(self, keys, argv):  # noqa: ANN001
+        delayed, reserved_key = keys
+        member, score = argv[0], float(argv[1])
+        self.zrem(reserved_key, member)
+        self.zadd(delayed, {member: score})
+        return True
+
+
+class RedisOwnedModeTest(unittest.TestCase):
+    """The default Python-owned reliable-queue mode (BLMOVE + processing list)."""
+
+    def _tr(self):
+        fake = FakeRedis()
+        return RedisTransport("redis://", client=fake), fake
+
+    def test_publish_uses_raw_queue_key(self):
+        tr, fake = self._tr()
+        tr.publish("orders", "body-1")
+        self.assertEqual(fake.lists["orders"], ["body-1"])
+
+    def test_pop_moves_to_processing_and_ack_removes(self):
+        tr, fake = self._tr()
+        tr.publish("orders", "body-1")
+        msg = tr.pop("orders", timeout=0)
+        self.assertEqual(msg.body, "body-1")
+        self.assertEqual(fake.lists["orders:processing"], ["body-1"])
+        self.assertEqual(fake.lists["orders"], [])
+        tr.ack(msg)
+        self.assertEqual(fake.lists["orders:processing"], [])
+
+    def test_pop_empty_returns_none(self):
+        tr, _ = self._tr()
+        self.assertIsNone(tr.pop("orders", timeout=0))
+
+
+class RedisLaravelCompatTest(unittest.TestCase):
+    """Laravel reserved-set parity: shared queue, reserve / ack / release / migrate."""
+
+    def _tr(self, **kw):
+        fake = FakeRedis()
+        return RedisTransport("redis://", client=fake, laravel_compat=True, **kw), fake
+
+    def test_publish_uses_laravel_keys_and_notify(self):
+        tr, fake = self._tr()
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        tr.publish("orders", EnvelopeCodec.encode(env))
+        # Laravel prefixes logical names with "queues:" and pushes a notify token.
+        self.assertEqual(fake.llen("queues:orders"), 1)
+        self.assertEqual(fake.llen("queues:orders:notify"), 1)
+
+    def test_pop_reserves_into_sorted_set_and_increments_attempts(self):
+        tr, fake = self._tr()
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        tr.publish("orders", EnvelopeCodec.encode(env))
+
+        msg = tr.pop("orders", timeout=0)
+        self.assertIsNotNone(msg)
+        # ready list drained, reserved set holds exactly the in-flight job
+        self.assertEqual(fake.llen("queues:orders"), 0)
+        self.assertEqual(fake.zcard("queues:orders:reserved"), 1)
+        # attempts incremented on reserve, exactly like Laravel's pop Lua
+        self.assertEqual(EnvelopeCodec.decode(msg.body)["attempts"], 1)
+        # the ack handle IS the reserved member (so ZREM matches byte-for-byte)
+        self.assertIn(msg.handle, fake.zsets["queues:orders:reserved"])
+
+    def test_ack_removes_from_reserved_set(self):
+        tr, fake = self._tr()
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        tr.publish("orders", EnvelopeCodec.encode(env))
+        msg = tr.pop("orders", timeout=0)
+        tr.ack(msg)
+        self.assertEqual(fake.zcard("queues:orders:reserved"), 0)
+
+    def test_ack_noop_on_empty_handle(self):
+        tr, fake = self._tr()
+        tr.ack(ReceivedMessage(body="", queue="orders", handle=None))
+        self.assertEqual(fake.zsets, {})
+
+    def test_crashed_worker_reservation_is_re_reserved_on_next_pop(self):
+        """A reserved job whose retry-after has lapsed migrates back and is re-served
+        — the reliable-queue guarantee that prevents message loss on worker crash."""
+        tr, fake = self._tr(retry_after=-1)  # reserve already expired the instant it lands
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        tr.publish("orders", EnvelopeCodec.encode(env))
+
+        first = tr.pop("orders", timeout=0)  # reserves (expired-at-now deadline)
+        self.assertIsNotNone(first)
+        # worker "crashes" without ack; next pop migrates the lapsed reservation back
+        second = tr.pop("orders", timeout=0)
+        self.assertIsNotNone(second)
+        # attempts climbed again on the second reservation
+        self.assertEqual(EnvelopeCodec.decode(second.body)["attempts"], 2)
+
+    def test_release_returns_job_to_delayed_set(self):
+        tr, fake = self._tr()
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        tr.publish("orders", EnvelopeCodec.encode(env))
+        msg = tr.pop("orders", timeout=0)
+        tr.release(msg, delay=0)
+        # left the reserved set, parked on the delayed set for retry
+        self.assertEqual(fake.zcard("queues:orders:reserved"), 0)
+        self.assertEqual(fake.zcard("queues:orders:delayed"), 1)
+
+    def test_release_noop_in_owned_mode(self):
+        tr, fake = self._tr()
+        # owned-mode transport: release is a no-op (handle present but not compat)
+        owned = RedisTransport("redis://", client=fake)
+        owned.release(ReceivedMessage(body="b", queue="orders", handle="b"))
+        self.assertEqual(fake.zsets, {})
+
+    def test_pop_empty_returns_none(self):
+        tr, _ = self._tr()
+        self.assertIsNone(tr.pop("orders", timeout=0))
+
+    def test_pop_block_window_drains_to_none(self):
+        """Empty ready list with a stale notify token: the block path consumes the
+        token (blpop wakeup), re-checks, finds nothing and returns None — the
+        timeout-drain case a consume loop relies on to make progress."""
+        tr, fake = self._tr(block_for=1)
+        fake.rpush("queues:orders:notify", "1")  # token but no actual job
+        self.assertIsNone(tr.pop("orders", timeout=1))
+        # the stale token was consumed by the blpop wakeup
+        self.assertEqual(fake.llen("queues:orders:notify"), 0)
+
+    def test_pop_retrieves_when_job_already_ready(self):
+        """A job already on the ready list is reserved on the first eval (no block)."""
+        tr, fake = self._tr()
+        env = EnvelopeCodec.make("urn:babel:orders:created", {"x": 1}, queue="orders")
+        fake.rpush("queues:orders", EnvelopeCodec.encode(env))
+        fake.rpush("queues:orders:notify", "1")
+        msg = tr.pop("orders", timeout=1)
+        self.assertIsNotNone(msg)
+        self.assertEqual(EnvelopeCodec.decode(msg.body)["attempts"], 1)
+
+    def test_url_enables_compat_and_overrides(self):
+        fake = FakeRedis()
+        tr = RedisTransport("redis://h:6379/0?laravel=1&prefix=q:&retry_after=90", client=fake)
+        self.assertTrue(tr._laravel_compat)
+        self.assertEqual(tr._key_prefix, "q:")
+        self.assertEqual(tr._retry_after, 90)
+
+    def test_pop_result_handles_short_and_false_replies(self):
+        # empty / too-short reply, and Lua `false` (None) job or reserved member
+        self.assertEqual(_pop_result([]), (None, None))
+        self.assertEqual(_pop_result([None, None]), (None, None))
+        self.assertEqual(_pop_result(["job", "reserved"]), ("job", "reserved"))
+
+    def test_qint_falls_back_on_missing_and_non_numeric(self):
+        self.assertEqual(_qint({}, "retry_after", 60), 60)  # absent -> default
+        self.assertEqual(_qint({"retry_after": ["nope"]}, "retry_after", 60), 60)  # non-int
+        self.assertEqual(_qint({"retry_after": ["90"]}, "retry_after", 60), 90)  # parsed
+
+    def test_round_trip_through_app(self):
+        fake = FakeRedis()
+        tr = RedisTransport("redis://", client=fake, laravel_compat=True)
+        app = BabelQueue(transport=tr, queue="orders")
+        seen: dict = {}
+
+        @app.handler("urn:babel:orders:created")
+        def _on(data, meta):  # noqa: ANN001
+            seen.update({"data": data, "meta": meta})
+
+        msg_id = app.publish("urn:babel:orders:created", {"order_id": 7})
+        processed = app.consume("orders", max_messages=1, timeout=0)
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen["data"]["order_id"], 7)
+        self.assertEqual(seen["meta"]["id"], msg_id)
+        # acked: reserved set empty
+        self.assertEqual(fake.zcard("queues:orders:reserved"), 0)
+
+
+# ---------------------------------------------------------------------------
+# Integration: skipped unless a real Redis is reachable (CI runs it).
+# ---------------------------------------------------------------------------
+
+try:
+    import redis as _redis
+except ImportError:  # pragma: no cover
+    _redis = None
+
+REDIS_URL = os.environ.get("BABELQUEUE_TEST_REDIS", "redis://localhost:6379/0")
+
+
+def _redis_available() -> bool:
+    if _redis is None:
+        return False
+    try:
+        _redis.Redis.from_url(REDIS_URL, decode_responses=True).ping()
+        return True
+    except Exception:  # pragma: no cover - connection failure
+        return False
+
+
+@unittest.skipUnless(_redis_available(), f"no reachable Redis at {REDIS_URL}")
+class RedisTransportIntegrationTest(unittest.TestCase):
+    def setUp(self) -> None:
+        self.queue = f"bqtest:{uuid.uuid4().hex}"
+        self.client = _redis.Redis.from_url(REDIS_URL, decode_responses=True)
+
+    def tearDown(self) -> None:
+        self.client.delete(self.queue, f"{self.queue}:processing", f"{self.queue}.dlq")
+        self.client.delete(
+            f"queues:{self.queue}", f"queues:{self.queue}:reserved",
+            f"queues:{self.queue}:delayed", f"queues:{self.queue}:notify",
+        )
+
+    def test_publish_consume_round_trip_and_ack(self) -> None:
+        app = BabelQueue(REDIS_URL, queue=self.queue)
+        seen = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            seen.update(data)
+
+        app.publish("urn:babel:orders:created", {"order_id": 42})
+        processed = app.consume(max_messages=1, timeout=2)
+
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen, {"order_id": 42})
+        self.assertEqual(self.client.llen(self.queue), 0)
+        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
+
+    def test_failure_dead_letters_to_redis(self) -> None:
+        app = BabelQueue(REDIS_URL, queue=self.queue, max_attempts=1, dead_letter=True)
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            raise RuntimeError("boom")
+
+        app.publish("urn:babel:orders:created", {"order_id": 1})
+        app.consume(max_messages=2, timeout=2)
+
+        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
+        dlq = self.client.lrange(f"{self.queue}.dlq", 0, -1)
+        self.assertEqual(len(dlq), 1)
+        env = EnvelopeCodec.decode(dlq[0])
+        self.assertEqual(env["dead_letter"]["reason"], "failed")
+
+    def test_laravel_compat_reserved_set_round_trip(self) -> None:
+        """Against a real Redis: a Laravel-compatible worker reserves into the
+        ``queues:<name>:reserved`` sorted set and acks via ZREM."""
+        tr = RedisTransport(REDIS_URL, laravel_compat=True)
+        app = BabelQueue(transport=tr, queue=self.queue)
+        seen = {}
+        app.register("urn:babel:orders:created", lambda data, meta: seen.update(data))
+
+        app.publish("urn:babel:orders:created", {"order_id": 99})
+        processed = app.consume(self.queue, max_messages=1, timeout=2)
+
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen, {"order_id": 99})
+        self.assertEqual(self.client.zcard(f"queues:{self.queue}:reserved"), 0)
+        self.assertEqual(self.client.llen(f"queues:{self.queue}"), 0)
+
+
+if __name__ == "__main__":
+    unittest.main()

babelqueue-1.5.0/src/babelqueue/redis_transport.py

@@ -1,53 +0,0 @@
-"""Redis transport (reliable-queue pattern). Requires the ``redis`` extra:
-
-    pip install "babelqueue[redis]"
-
-Producing is ``RPUSH queue body``; consuming atomically moves the head to a
-per-queue processing list (``BLMOVE``) so an in-flight message survives a worker
-crash, and ``ack`` removes it from that processing list. This is a Python-owned
-reliable queue; full parity with Laravel's reserved-set reservation on a *shared*
-Redis queue is a separate conformance task (see the roadmap).
-"""
-
-from __future__ import annotations
-
-from typing import Optional
-
-from .transport import ReceivedMessage, Transport
-
-
-class RedisTransport(Transport):
-    def __init__(self, url: str, *, processing_suffix: str = ":processing") -> None:
-        try:
-            import redis  # noqa: F401  (lazy: only needed for this transport)
-        except ImportError as exc:  # pragma: no cover - import guard
-            raise ImportError(
-                "RedisTransport requires the 'redis' package. Install with "
-                "pip install \"babelqueue[redis]\"."
-            ) from exc
-
-        self._redis = redis.Redis.from_url(url, decode_responses=True)
-        self._processing_suffix = processing_suffix
-
-    def _processing(self, queue: str) -> str:
-        return f"{queue}{self._processing_suffix}"
-
-    def publish(self, queue: str, body: str) -> None:
-        self._redis.rpush(queue, body)
-
-    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
-        # redis-py types the BLMOVE timeout as int, but Redis accepts a float
-        # (sub-second) timeout; passing it through is correct at runtime.
-        body = self._redis.blmove(queue, self._processing(queue), timeout, "LEFT", "RIGHT")  # type: ignore[arg-type]
-        if body is None:
-            return None
-        # decode_responses=True yields str; the guard satisfies the type checker
-        # (and is a harmless safety net otherwise).
-        text = body if isinstance(body, str) else body.decode()
-        return ReceivedMessage(body=text, queue=queue, handle=text)
-
-    def ack(self, message: ReceivedMessage) -> None:
-        self._redis.lrem(self._processing(message.queue), 1, message.handle)
-
-    def close(self) -> None:  # pragma: no cover
-        self._redis.close()

babelqueue-1.5.0/tests/test_redis_transport.py

@@ -1,78 +0,0 @@
-"""Integration tests for the Redis transport.
-
-Skipped unless a Redis server is reachable (the `redis` package installed and a
-broker at ``BABELQUEUE_TEST_REDIS`` / localhost). The CI ``integration`` job runs
-these against a Redis service; locally they skip cleanly.
-"""
-
-from __future__ import annotations
-
-import os
-import unittest
-import uuid
-
-try:
-    import redis as _redis
-except ImportError:  # pragma: no cover
-    _redis = None
-
-from babelqueue import BabelQueue, EnvelopeCodec
-
-REDIS_URL = os.environ.get("BABELQUEUE_TEST_REDIS", "redis://localhost:6379/0")
-
-
-def _redis_available() -> bool:
-    if _redis is None:
-        return False
-    try:
-        _redis.Redis.from_url(REDIS_URL, decode_responses=True).ping()
-        return True
-    except Exception:  # pragma: no cover - connection failure
-        return False
-
-
-@unittest.skipUnless(_redis_available(), f"no reachable Redis at {REDIS_URL}")
-class RedisTransportTest(unittest.TestCase):
-    def setUp(self) -> None:
-        self.queue = f"bqtest:{uuid.uuid4().hex}"
-        self.client = _redis.Redis.from_url(REDIS_URL, decode_responses=True)
-
-    def tearDown(self) -> None:
-        self.client.delete(self.queue, f"{self.queue}:processing", f"{self.queue}.dlq")
-
-    def test_publish_consume_round_trip_and_ack(self) -> None:
-        app = BabelQueue(REDIS_URL, queue=self.queue)
-        seen = {}
-
-        @app.handler("urn:babel:orders:created")
-        def handle(data, meta):  # noqa: ANN001
-            seen.update(data)
-
-        app.publish("urn:babel:orders:created", {"order_id": 42})
-        processed = app.consume(max_messages=1, timeout=2)
-
-        self.assertEqual(processed, 1)
-        self.assertEqual(seen, {"order_id": 42})
-        # acked: nothing left on the queue or the processing list
-        self.assertEqual(self.client.llen(self.queue), 0)
-        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
-
-    def test_failure_dead_letters_to_redis(self) -> None:
-        app = BabelQueue(REDIS_URL, queue=self.queue, max_attempts=1, dead_letter=True)
-
-        @app.handler("urn:babel:orders:created")
-        def handle(data, meta):  # noqa: ANN001
-            raise RuntimeError("boom")
-
-        app.publish("urn:babel:orders:created", {"order_id": 1})
-        app.consume(max_messages=2, timeout=2)
-
-        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
-        dlq = self.client.lrange(f"{self.queue}.dlq", 0, -1)
-        self.assertEqual(len(dlq), 1)
-        env = EnvelopeCodec.decode(dlq[0])
-        self.assertEqual(env["dead_letter"]["reason"], "failed")
-
-
-if __name__ == "__main__":
-    unittest.main()