babelqueue 1.4.0__tar.gz → 1.6.0__tar.gz

{babelqueue-1.4.0 → babelqueue-1.6.0}/.github/workflows/ci.yml

@@ -44,7 +44,7 @@ jobs:
       - name: Install (dev + all adapters for type context)
         run: |
           python -m pip install --upgrade pip
-          pip install -e ".[dev,celery,django,redis,amqp,sqs,azureservicebus]"
+          pip install -e ".[dev,celery,django,redis,amqp,sqs,azureservicebus,artemis]"
       - name: Ruff
         run: ruff check src tests
       - name: Mypy
@@ -89,7 +89,7 @@ jobs:
       - name: Install (all adapters — full coverage with brokers)
         run: |
           python -m pip install --upgrade pip
-          pip install -e ".[redis,amqp,sqs,azureservicebus,celery,django,dev]"
+          pip install -e ".[redis,amqp,sqs,azureservicebus,artemis,celery,django,dev]"
 
       - name: Wait for ElasticMQ
         run: |

{babelqueue-1.4.0 → babelqueue-1.6.0}/CHANGELOG.md

@@ -7,6 +7,55 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 The envelope wire format is versioned separately by `meta.schema_version`
 (currently **1**) — see the contract at [babelqueue.com](https://babelqueue.com).
 
+## [Unreleased]
+
+## [1.6.0] - 2026-06-14
+
+### Added
+- **Redis/Laravel reservation parity** — the Redis transport can now consume a
+  **shared** Laravel BabelQueue Redis queue using Laravel's reserved-set / reliable-queue
+  semantics, not just a Python-owned queue. Enable it with the `laravel=1` URL flag
+  (`redis://host:6379/0?laravel=1`) or `RedisTransport(..., laravel_compat=True)`. In this
+  mode the key layout is Laravel's stock Redis queue ([§1 of the broker-bindings
+  contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)): a `queues:<name>`
+  ready list, a `queues:<name>:reserved` **sorted set** scored by a `retry_after` deadline
+  (default 60s), a `queues:<name>:delayed` set, and a `queues:<name>:notify` wake-up list.
+  Reserve, ack and release run the **byte-for-byte same Lua scripts** Laravel uses, so the
+  reserved member a Python worker writes is identical to a Laravel worker's — either side can
+  ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share one
+  Redis queue without losing or double-processing messages. Before each pop, expired
+  reserved/delayed jobs migrate back to the ready list, so a crashed worker's in-flight job
+  is re-reserved exactly as Laravel does. Tunable via `?prefix=` / `?retry_after=` (or the
+  `key_prefix` / `retry_after` constructor args). The default Python-owned reliable-queue
+  mode (`BLMOVE` + `<queue>:processing` list) is unchanged and stays the default, so existing
+  callers are unaffected. The reservation logic is fully unit-tested with an injected
+  in-memory Redis double (no `redis` package, no broker); a live cross-runtime PHP↔Python
+  shared-queue round-trip is covered by the integration suite where a real Redis is present.
+  The envelope is unchanged (`schema_version: 1`); this is purely additive. Ships as a MINOR.
+
+## [1.5.0] - 2026-06-13
+
+### Added
+- **Apache ActiveMQ Artemis transport** (`babelqueue[artemis]`, `python-qpid-proton`) —
+  `ArtemisTransport`, selected by the `artemis://` (or `artemis+ssl://`) URL scheme (e.g.
+  `artemis://localhost:5672`; or pass an injected `connection`). Artemis speaks **AMQP 1.0**
+  (not RabbitMQ's 0-9-1), so the transport uses the `python-qpid-proton` blocking client.
+  Implements [§7 of the broker-bindings
+  contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#apache-activemq-artemis): the
+  canonical envelope is the message body, projected onto the AMQP fields a JMS peer reads —
+  `correlation-id` = `trace_id` (JMSCorrelationID), `creation-time` = `meta.created_at`
+  (JMSTimestamp), the `x-opt-jms-type` annotation = URN (JMSType, the AMQP-JMS mapping), plus
+  the `bq-schema-version`/`bq-source-lang`/`bq-attempts`/`bq-app-id` application properties.
+  Consume reserves one message at a time (`receive` → process → `accept`); `attempts` is
+  reconciled to `max(body, delivery_count)` — the AMQP delivery-count header is 0-based, so it
+  maps directly with no −1 (the Java JMS binding reads the 1-based `JMSXDeliveryCount` and
+  subtracts 1, arriving at the same 0-based `attempts`), and the `max` never lowers a higher
+  body count carried by a republish-driven retry. The projection + reconciliation + pop/ack
+  flow are unit-tested with no broker and no `python-qpid-proton` (the proton import is lazy;
+  the transport talks to an injected connection fake); the publish flow that builds a real
+  proton `Message` is exercised wherever proton is installed. The envelope is unchanged
+  (`schema_version: 1`); Apache ActiveMQ Artemis is purely additive. Ships as a MINOR.
+
 ## [1.4.0] - 2026-06-13
 
 ### Added
@@ -150,7 +199,8 @@ reference at [babelqueue.com](https://babelqueue.com).
 - Pre-1.0: the public API may change before the `1.0.0` tag.
 - The core has **zero runtime dependencies** (standard library only); Python `>=3.9`.
 
-[Unreleased]: https://github.com/BabelQueue/babelqueue-python/compare/v1.2.0...HEAD
+[Unreleased]: https://github.com/BabelQueue/babelqueue-python/compare/v1.6.0...HEAD
+[1.6.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.5.0...v1.6.0
 [1.2.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.1.0...v1.2.0
 [1.1.0]: https://github.com/BabelQueue/babelqueue-python/compare/v1.0.0...v1.1.0
 [1.0.0]: https://github.com/BabelQueue/babelqueue-python/compare/v0.5.0...v1.0.0

{babelqueue-1.4.0 → babelqueue-1.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: babelqueue
-Version: 1.4.0
+Version: 1.6.0
 Summary: Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers.
 Project-URL: Homepage, https://babelqueue.com
 Project-URL: Source, https://github.com/BabelQueue/babelqueue-python
@@ -24,6 +24,8 @@ Classifier: Typing :: Typed
 Requires-Python: >=3.9
 Provides-Extra: amqp
 Requires-Dist: pika>=1.3; extra == 'amqp'
+Provides-Extra: artemis
+Requires-Dist: python-qpid-proton>=0.39; extra == 'artemis'
 Provides-Extra: azureservicebus
 Requires-Dist: azure-identity>=1.15; extra == 'azureservicebus'
 Requires-Dist: azure-servicebus>=7.11; extra == 'azureservicebus'
@@ -162,9 +164,35 @@ app.run()                               # consume forever (Ctrl-C to stop)
 - **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
   envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
   messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
-- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
-  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
-  tests/local). Bring your own by passing `transport=...`.
+- **Transports:** `redis://` (reliable-queue pattern; add `?laravel=1` to share a
+  Redis queue with a Laravel BabelQueue worker using its reserved-set semantics —
+  see below), `amqp://` (RabbitMQ via `pika`, with the contract AMQP properties)
+  and `memory://` (in-process, great for tests/local). Bring your own by passing
+  `transport=...`.
+
+### Sharing a Redis queue with Laravel
+
+By default the Redis transport owns its queue end-to-end (`RPUSH` to produce;
+`BLMOVE` into a `<queue>:processing` list to reserve; `LREM` on ack). To consume a
+**shared** Laravel BabelQueue Redis queue instead, enable Laravel-compatible mode:
+
+```python
+app = BabelQueue("redis://localhost:6379/0?laravel=1", queue="orders")
+# or: RedisTransport("redis://localhost:6379/0", laravel_compat=True)
+```
+
+This replicates Laravel's stock Redis reservation exactly ([§1 of the
+broker-bindings contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)):
+the ready list is `queues:<name>`, reservations move into the `queues:<name>:reserved`
+**sorted set** scored by a `retry_after` deadline (default 60s), with a
+`queues:<name>:delayed` set and a `queues:<name>:notify` wake-up list. Reserve, ack
+and release run the **byte-for-byte same Lua scripts** Laravel uses, so the reserved
+member a Python worker writes is identical to a Laravel worker's — either side can
+ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share
+one queue without losing or double-processing messages. Before each pop, expired
+reserved/delayed jobs migrate back to the ready list, so a crashed worker's
+in-flight job is re-reserved. Options: `?prefix=queues:` and `?retry_after=60`
+(or the `key_prefix` / `retry_after` constructor args).
 
 ## Framework adapters — Celery & Django
 

{babelqueue-1.4.0 → babelqueue-1.6.0}/README.md

@@ -114,9 +114,35 @@ app.run()                               # consume forever (Ctrl-C to stop)
 - **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
   envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
   messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
-- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
-  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
-  tests/local). Bring your own by passing `transport=...`.
+- **Transports:** `redis://` (reliable-queue pattern; add `?laravel=1` to share a
+  Redis queue with a Laravel BabelQueue worker using its reserved-set semantics —
+  see below), `amqp://` (RabbitMQ via `pika`, with the contract AMQP properties)
+  and `memory://` (in-process, great for tests/local). Bring your own by passing
+  `transport=...`.
+
+### Sharing a Redis queue with Laravel
+
+By default the Redis transport owns its queue end-to-end (`RPUSH` to produce;
+`BLMOVE` into a `<queue>:processing` list to reserve; `LREM` on ack). To consume a
+**shared** Laravel BabelQueue Redis queue instead, enable Laravel-compatible mode:
+
+```python
+app = BabelQueue("redis://localhost:6379/0?laravel=1", queue="orders")
+# or: RedisTransport("redis://localhost:6379/0", laravel_compat=True)
+```
+
+This replicates Laravel's stock Redis reservation exactly ([§1 of the
+broker-bindings contract](https://babelqueue.com/docs/spec/1.x/broker-bindings#redis)):
+the ready list is `queues:<name>`, reservations move into the `queues:<name>:reserved`
+**sorted set** scored by a `retry_after` deadline (default 60s), with a
+`queues:<name>:delayed` set and a `queues:<name>:notify` wake-up list. Reserve, ack
+and release run the **byte-for-byte same Lua scripts** Laravel uses, so the reserved
+member a Python worker writes is identical to a Laravel worker's — either side can
+ack (`ZREM`) the other's reservation, so a Python worker and a Laravel worker share
+one queue without losing or double-processing messages. Before each pop, expired
+reserved/delayed jobs migrate back to the ready list, so a crashed worker's
+in-flight job is re-reserved. Options: `?prefix=queues:` and `?retry_after=60`
+(or the `key_prefix` / `retry_after` constructor args).
 
 ## Framework adapters — Celery & Django
 

{babelqueue-1.4.0 → babelqueue-1.6.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "babelqueue"
-version = "1.4.0"
+version = "1.6.0"
 description = "Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -35,6 +35,7 @@ sqs = ["boto3>=1.26"]
 azureservicebus = ["azure-servicebus>=7.11", "azure-identity>=1.15"]
 pulsar = ["pulsar-client>=3.4"]
 kafka = ["confluent-kafka>=2.3"]
+artemis = ["python-qpid-proton>=0.39"]
 celery = ["celery>=5"]
 django = ["django>=4.2"]
 dev = ["pytest>=7", "pytest-cov>=4", "mypy>=1.8", "ruff>=0.5"]

{babelqueue-1.4.0 → babelqueue-1.6.0}/src/babelqueue/__init__.py

@@ -19,7 +19,7 @@ from .exceptions import BabelQueueError, UnknownUrnError
 from .routing import UnknownUrnStrategy
 from .transport import InMemoryTransport, ReceivedMessage, Transport
 
-__version__ = "1.2.0"
+__version__ = "1.6.0"
 
 __all__ = [
     "BabelQueue",

babelqueue-1.6.0/src/babelqueue/artemis_transport.py

@@ -0,0 +1,227 @@
+"""Apache ActiveMQ Artemis transport over AMQP 1.0. Requires the ``artemis`` extra:
+
+    pip install "babelqueue[artemis]"
+
+Artemis speaks AMQP 1.0 (not the 0-9-1 of RabbitMQ), so this transport uses the
+``python-qpid-proton`` blocking client rather than ``pika``. Producing sends the canonical
+envelope as the message body and projects the contract envelope fields onto the AMQP
+properties a JMS peer reads: ``correlation-id`` = trace_id (JMSCorrelationID), ``creation-time``
+= meta.created_at (JMSTimestamp), the ``x-opt-jms-type`` message annotation = URN (JMSType, the
+AMQP-JMS mapping), plus the ``bq_`` application properties (``bq_schema_version`` /
+``bq_source_lang`` / ``bq_attempts`` / ``bq_app_id``; underscores, since JMS property names must
+be valid Java identifiers) — so a Java (JMS) or .NET/Node/... peer
+routes and correlates without parsing the body. The URN in the body's ``job`` field stays
+authoritative.
+
+Consuming reserves one message at a time (``receive`` -> process -> ``accept``); the
+authoritative attempt count is the envelope's ``attempts`` (the body), reconciled against the
+broker's native AMQP ``delivery-count`` as ``attempts = max(body, delivery_count)`` — no -1,
+because the AMQP header counter is 0-based (0 on first delivery), counting prior failed
+deliveries. (The Java JMS binding reads ``JMSXDeliveryCount`` which is 1-based and subtracts 1,
+arriving at the same 0-based ``attempts``.) A message left un-accepted is redelivered
+(at-least-once).
+
+This implements §7 of the broker-bindings contract. The envelope is unchanged
+(``schema_version`` stays 1); Apache ActiveMQ Artemis is purely additive.
+
+URL form: ``artemis://host:5672`` (or ``artemis+ssl://`` for TLS) — translated to the
+``amqp://`` / ``amqps://`` proton speaks. For credentials or a custom connection, build the
+transport directly and pass it via ``BabelQueue(transport=...)`` or
+``ArtemisTransport(connection=...)``.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Dict, Optional
+
+from .codec import EnvelopeCodec
+from .transport import ReceivedMessage, Transport
+
+JMS_TYPE_ANNOTATION = "x-opt-jms-type"
+APP_ID = "babelqueue"
+
+
+class ArtemisTransport(Transport):
+    def __init__(
+        self,
+        url: str = "artemis://localhost:5672",
+        *,
+        connection: Any = None,
+        credit: int = 1,
+        receive_timeout_millis: int = 1000,
+        **connect_options: Any,
+    ) -> None:
+        self._url = url or "artemis://localhost:5672"
+        self._credit = credit
+        self._receive_timeout_millis = receive_timeout_millis
+        self._connect_options = connect_options
+        self._senders: Dict[str, Any] = {}
+        self._receivers: Dict[str, Any] = {}
+
+        if connection is not None:
+            self._connection = connection
+            return
+        self._connection = self._build_connection()  # pragma: no cover - needs Artemis / network
+
+    def _build_connection(self) -> Any:  # pragma: no cover - needs Artemis / network
+        try:
+            from proton.utils import BlockingConnection
+        except ImportError as exc:
+            raise ImportError(
+                "ArtemisTransport requires the 'python-qpid-proton' package. Install with "
+                'pip install "babelqueue[artemis]".'
+            ) from exc
+        return BlockingConnection(self._to_amqp_url(self._url), **self._connect_options)
+
+    # -- helpers ------------------------------------------------------------
+
+    @staticmethod
+    def _to_amqp_url(url: str) -> str:
+        """``artemis://`` -> ``amqp://`` and ``artemis+ssl://`` -> ``amqps://`` (proton's schemes)."""
+        if url.startswith("artemis+ssl://"):
+            return "amqps://" + url[len("artemis+ssl://"):]
+        if url.startswith("artemis://"):
+            return "amqp://" + url[len("artemis://"):]
+        return url
+
+    def _sender(self, queue: str) -> Any:
+        sender = self._senders.get(queue)
+        if sender is None:
+            sender = self._connection.create_sender(queue)
+            self._senders[queue] = sender
+        return sender
+
+    def _receiver(self, queue: str) -> Any:
+        receiver = self._receivers.get(queue)
+        if receiver is None:
+            receiver = self._connection.create_receiver(queue, credit=self._credit)
+            self._receivers[queue] = receiver
+        return receiver
+
+    @staticmethod
+    def _projection(body: str) -> Dict[str, str]:
+        """AMQP application properties (string->string) — a redundant, routable view of the
+        body: bq_schema_version/bq_source_lang/bq_attempts/bq_app_id. §7.2 (the URN is carried
+        by the x-opt-jms-type annotation, trace_id by correlation-id). The names use UNDERSCORES,
+        not hyphens: a JMS property name must be a valid Java identifier, and every Artemis SDK
+        uses the same JMS-legal form for cross-protocol parity."""
+        env = EnvelopeCodec.decode(body)
+        if not env:
+            return {}
+        meta = env.get("meta") or {}
+
+        props: Dict[str, str] = {}
+        if meta.get("schema_version") is not None:
+            props["bq_schema_version"] = str(meta["schema_version"])
+        if meta.get("lang"):
+            props["bq_source_lang"] = str(meta["lang"])
+        props["bq_attempts"] = str(int(env.get("attempts", 0) or 0))
+        props["bq_app_id"] = APP_ID
+        return props
+
+    @staticmethod
+    def _jms_type(body: str) -> str:
+        env = EnvelopeCodec.decode(body)
+        return str(env["job"]) if env and env.get("job") else ""
+
+    @staticmethod
+    def _correlation_id(body: str) -> str:
+        env = EnvelopeCodec.decode(body)
+        return str(env["trace_id"]) if env and env.get("trace_id") else ""
+
+    @staticmethod
+    def _creation_seconds(body: str) -> Optional[float]:
+        """proton's creation_time is float seconds; the contract's created_at is epoch ms."""
+        env = EnvelopeCodec.decode(body)
+        meta = (env or {}).get("meta") or {}
+        created_at = meta.get("created_at")
+        if created_at is None:
+            return None
+        try:
+            return int(created_at) / 1000.0
+        except (TypeError, ValueError):  # pragma: no cover - defensive
+            return None
+
+    def _message(self, body: str) -> Any:
+        """Build the proton Message projecting the §7 JMS-readable metadata."""
+        from proton import Message, symbol
+
+        message: Any = Message(body=body)
+        props = self._projection(body)
+        if props:
+            message.properties = props
+        correlation_id = self._correlation_id(body)
+        if correlation_id:
+            message.correlation_id = correlation_id
+        creation = self._creation_seconds(body)
+        if creation is not None:
+            message.creation_time = creation
+        jms_type = self._jms_type(body)
+        if jms_type:
+            message.annotations = {symbol(JMS_TYPE_ANNOTATION): jms_type}
+        return message
+
+    @staticmethod
+    def _reconcile(body: str, delivery_count: Any) -> str:
+        """Set attempts to max(current, delivery_count). The AMQP delivery-count header is
+        0-based (0 on first delivery), so it maps directly to attempts with no -1. The runtime
+        retries by republishing with attempts+1 in the body (delivery-count back to 0), so a
+        republished message must not have its higher body count lowered."""
+        try:
+            dc = int(delivery_count)
+        except (ValueError, TypeError):
+            return body
+        if dc <= 0:
+            return body
+        env = EnvelopeCodec.decode(body)
+        if not env or dc <= int(env.get("attempts", 0) or 0):
+            return body
+        env["attempts"] = dc
+        return EnvelopeCodec.encode(env)
+
+    @staticmethod
+    def _delivery_count(message: Any) -> int:
+        value = getattr(message, "delivery_count", 0)
+        try:
+            return int(value)
+        except (TypeError, ValueError):  # pragma: no cover - defensive
+            return 0
+
+    @staticmethod
+    def _payload(message: Any) -> str:
+        body = getattr(message, "body", None)
+        if isinstance(body, bytes):
+            return body.decode("utf-8")
+        return str(body) if body is not None else ""
+
+    # -- Transport ----------------------------------------------------------
+
+    def publish(self, queue: str, body: str) -> None:
+        self._sender(queue).send(self._message(body))
+
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        wait = self._receive_timeout_millis / 1000.0
+        if timeout and timeout > 0:
+            wait = timeout
+        receiver = self._receiver(queue)
+        try:
+            message = receiver.receive(timeout=wait)
+        except Exception as exc:  # noqa: BLE001 - proton raises proton.Timeout on no message
+            if type(exc).__name__ == "Timeout":
+                return None
+            raise
+        if message is None:  # pragma: no cover - defensive (real client raises instead)
+            return None
+        body = self._reconcile(self._payload(message), self._delivery_count(message))
+        return ReceivedMessage(body=body, queue=queue, handle=receiver)
+
+    def ack(self, message: ReceivedMessage) -> None:
+        if message.handle is None:
+            return
+        message.handle.accept()
+
+    def close(self) -> None:  # pragma: no cover - resource cleanup
+        try:
+            self._connection.close()
+        except Exception:  # noqa: BLE001 - best-effort cleanup
+            pass