babelqueue 0.1.0__tar.gz → 0.3.0__tar.gz

babelqueue-0.3.0/.github/workflows/ci.yml

@@ -0,0 +1,74 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    name: Python ${{ matrix.python }}
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python: ['3.9', '3.10', '3.11', '3.12', '3.13']
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python }}
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[dev]"
+
+      - name: Run tests
+        run: pytest
+
+  integration:
+    name: Redis integration
+    runs-on: ubuntu-latest
+    services:
+      redis:
+        image: redis:7
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 5s
+          --health-timeout 3s
+          --health-retries 10
+      rabbitmq:
+        image: rabbitmq:3
+        ports:
+          - 5672:5672
+        options: >-
+          --health-cmd "rabbitmq-diagnostics -q ping"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 15
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: '3.12'
+
+      - name: Install (with redis + amqp extras)
+        run: |
+          python -m pip install --upgrade pip
+          pip install -e ".[redis,amqp,dev]"
+
+      - name: Run tests (Redis + RabbitMQ transports included)
+        env:
+          BABELQUEUE_TEST_REDIS: redis://localhost:6379/0
+          BABELQUEUE_TEST_AMQP: amqp://guest:guest@localhost:5672/
+        run: pytest

babelqueue-0.3.0/CHANGELOG.md

@@ -0,0 +1,49 @@
+# Changelog
+
+All notable changes to `babelqueue` (Python) are documented here.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+The envelope wire format is versioned separately by `meta.schema_version`
+(currently **1**) — see the contract at [babelqueue.com](https://babelqueue.com).
+
+## [Unreleased]
+
+### Added
+- **RabbitMQ transport** (`PikaTransport`, `amqp://`): durable queue, persistent
+  delivery, `basic_get` + manual ack, and the contract AMQP properties (`type`=URN,
+  `correlation_id`=trace_id, `x-schema-version`/`x-source-lang`/`x-attempts`).
+  Optional `[amqp]` extra (lazy `pika` import) — the core stays zero-dep.
+
+## [0.2.0] - 2026-06-06
+
+### Added
+- **Runtime** — `BabelQueue(broker_url=...)` app with a `@app.handler("urn:...")`
+  decorator, `publish()`, and a `consume()` / `run()` loop. Routes by URN over the
+  canonical envelope; `attempts`-based retry → opt-in dead-letter queue;
+  `on_unknown_urn` strategies (`fail`/`delete`/`release`/`dead_letter`).
+- **Transports** — a pluggable `Transport` abstraction with `InMemoryTransport`
+  (`memory://`, for tests/local) and `RedisTransport` (`redis://`, reliable-queue
+  pattern via `BLMOVE` + a processing list). Redis client is an optional `[redis]`
+  extra, imported lazily — the core stays zero-dep.
+
+## [0.1.0] - 2026-06-06
+
+### Added
+- `EnvelopeCodec` — builds (`make`, `from_message`), encodes and decodes the
+  canonical `{job, trace_id, data, meta, attempts}` envelope (`schema_version` 1).
+  The single Python implementation of the wire format.
+- Contracts `PolyglotMessage` / `HasTraceId` (typed `Protocol`s).
+- `dead_letter.annotate()` — additive `dead_letter` block builder.
+- `UnknownUrnStrategy` — `fail` / `delete` / `release` / `dead_letter`.
+- `BabelQueueError` / `UnknownUrnError`.
+- Golden conformance fixtures under `tests/fixtures/` (shared cross-SDK set).
+- `py.typed` — ships inline type hints (PEP 561).
+
+### Notes
+- Pre-1.0: the public API may change before the `1.0.0` tag.
+- The core has **zero runtime dependencies** (standard library only); Python `>=3.9`.
+
+[Unreleased]: https://github.com/BabelQueue/babelqueue-python/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/BabelQueue/babelqueue-python/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/BabelQueue/babelqueue-python/releases/tag/v0.1.0

{babelqueue-0.1.0 → babelqueue-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: babelqueue
-Version: 0.1.0
+Version: 0.3.0
 Summary: Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers.
 Project-URL: Homepage, https://babelqueue.com
 Project-URL: Source, https://github.com/BabelQueue/babelqueue-python
@@ -112,13 +112,51 @@ dlq = dead_letter.annotate(envelope, "failed", "orders", attempts=3, error="boom
 # publish `EnvelopeCodec.encode(dlq)` to the "orders.dlq" queue
 ```
 
-## What's here vs. coming
+## Runtime — produce & consume
 
-- **Now (this package):** the codec, contracts, dead-letter and unknown-URN
-  helpers, plus the shared conformance fixtures. Bring your own broker client.
-- **Next (planned):** a built-in runtime — `BabelQueue(broker_url=...)` with an
-  `@app.handler("urn:…")` decorator over `redis`/`pika` — and **Celery** / **Django**
-  adapters. Install via extras (`babelqueue[redis]`, `babelqueue[celery]`, …).
+For an end-to-end app, use `BabelQueue` with a broker. Broker clients come via
+extras:
+
+```bash
+pip install "babelqueue[redis]"   # redis://
+pip install "babelqueue[amqp]"    # amqp:// (RabbitMQ)
+```
+
+```python
+from babelqueue import BabelQueue
+
+app = BabelQueue("redis://localhost:6379/0", queue="orders")
+# or: BabelQueue("amqp://guest:guest@localhost:5672/", queue="orders")
+
+@app.handler("urn:babel:orders:created")
+def on_order_created(data, meta):       # AI/ML, data processing, anything
+    print("order", data["order_id"])
+
+# producer (any service, any language) …
+app.publish("urn:babel:orders:created", {"order_id": 1042})
+
+# worker
+app.run()                               # consume forever (Ctrl-C to stop)
+```
+
+- **Routing** is by URN; the wire format is the canonical envelope, so this
+  consumes messages produced by *any* BabelQueue SDK.
+- **Handlers** receive `(data, meta)`, or `(data, meta, message)` to get the full
+  envelope (incl. `trace_id`).
+- **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
+  envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
+  messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
+- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
+  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
+  tests/local). Bring your own by passing `transport=...`.
+
+> **Celery** / **Django** adapters are the next iterations.
+
+## What's here
+
+The codec/contracts/dead-letter (zero-dep core) **and** the `BabelQueue` runtime
+above (in-memory built in; Redis via `[redis]`, RabbitMQ via `[amqp]`). For
+framework integration, the Celery and Django adapters are planned.
 
 ## Testing
 

{babelqueue-0.1.0 → babelqueue-0.3.0}/README.md

@@ -80,13 +80,51 @@ dlq = dead_letter.annotate(envelope, "failed", "orders", attempts=3, error="boom
 # publish `EnvelopeCodec.encode(dlq)` to the "orders.dlq" queue
 ```
 
-## What's here vs. coming
+## Runtime — produce & consume
 
-- **Now (this package):** the codec, contracts, dead-letter and unknown-URN
-  helpers, plus the shared conformance fixtures. Bring your own broker client.
-- **Next (planned):** a built-in runtime — `BabelQueue(broker_url=...)` with an
-  `@app.handler("urn:…")` decorator over `redis`/`pika` — and **Celery** / **Django**
-  adapters. Install via extras (`babelqueue[redis]`, `babelqueue[celery]`, …).
+For an end-to-end app, use `BabelQueue` with a broker. Broker clients come via
+extras:
+
+```bash
+pip install "babelqueue[redis]"   # redis://
+pip install "babelqueue[amqp]"    # amqp:// (RabbitMQ)
+```
+
+```python
+from babelqueue import BabelQueue
+
+app = BabelQueue("redis://localhost:6379/0", queue="orders")
+# or: BabelQueue("amqp://guest:guest@localhost:5672/", queue="orders")
+
+@app.handler("urn:babel:orders:created")
+def on_order_created(data, meta):       # AI/ML, data processing, anything
+    print("order", data["order_id"])
+
+# producer (any service, any language) …
+app.publish("urn:babel:orders:created", {"order_id": 1042})
+
+# worker
+app.run()                               # consume forever (Ctrl-C to stop)
+```
+
+- **Routing** is by URN; the wire format is the canonical envelope, so this
+  consumes messages produced by *any* BabelQueue SDK.
+- **Handlers** receive `(data, meta)`, or `(data, meta, message)` to get the full
+  envelope (incl. `trace_id`).
+- **Retry & dead-letter:** failures are retried up to `max_attempts` (bumping the
+  envelope's `attempts`); enable `dead_letter=True` to quarantine exhausted
+  messages on `<queue>.dlq`. `on_unknown_urn` = `fail` | `delete` | `release` | `dead_letter`.
+- **Transports:** `redis://` (reliable-queue pattern), `amqp://` (RabbitMQ via
+  `pika`, with the contract AMQP properties) and `memory://` (in-process, great for
+  tests/local). Bring your own by passing `transport=...`.
+
+> **Celery** / **Django** adapters are the next iterations.
+
+## What's here
+
+The codec/contracts/dead-letter (zero-dep core) **and** the `BabelQueue` runtime
+above (in-memory built in; Redis via `[redis]`, RabbitMQ via `[amqp]`). For
+framework integration, the Celery and Django adapters are planned.
 
 ## Testing
 

{babelqueue-0.1.0 → babelqueue-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "babelqueue"
-version = "0.1.0"
+version = "0.3.0"
 description = "Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers."
 readme = "README.md"
 requires-python = ">=3.9"

{babelqueue-0.1.0 → babelqueue-0.3.0}/src/babelqueue/__init__.py

@@ -12,20 +12,26 @@ and dead-letter helpers. Framework adapters (Celery, Django, ...) build on this.
 from __future__ import annotations
 
 from . import dead_letter
+from .app import BabelQueue
 from .codec import SCHEMA_VERSION, SOURCE_LANG, EnvelopeCodec
 from .contracts import HasTraceId, PolyglotMessage
 from .exceptions import BabelQueueError, UnknownUrnError
 from .routing import UnknownUrnStrategy
+from .transport import InMemoryTransport, ReceivedMessage, Transport
 
-__version__ = "0.1.0"
+__version__ = "0.3.0"
 
 __all__ = [
+    "BabelQueue",
     "EnvelopeCodec",
     "SCHEMA_VERSION",
     "SOURCE_LANG",
     "PolyglotMessage",
     "HasTraceId",
     "UnknownUrnStrategy",
+    "Transport",
+    "InMemoryTransport",
+    "ReceivedMessage",
     "BabelQueueError",
     "UnknownUrnError",
     "dead_letter",

babelqueue-0.3.0/src/babelqueue/app.py

@@ -0,0 +1,210 @@
+"""The BabelQueue runtime: produce and consume polyglot messages.
+
+    from babelqueue import BabelQueue
+
+    app = BabelQueue("redis://localhost:6379/0", queue="orders")
+
+    @app.handler("urn:babel:orders:created")
+    def on_order_created(data, meta):
+        ...                       # AI/ML, data processing, anything
+
+    app.publish("urn:babel:orders:created", {"order_id": 1042})
+    app.run()                     # consume forever
+
+Routing is by URN; the wire format is the canonical envelope (shared core codec),
+so this interoperates with the PHP/Laravel, Symfony, Go, ... SDKs. Retry uses the
+top-level ``attempts`` counter; failures past ``max_attempts`` go to a dead-letter
+queue when enabled.
+"""
+
+from __future__ import annotations
+
+import inspect
+from typing import Any, Callable, Dict, Mapping, Optional
+
+from . import dead_letter
+from .codec import EnvelopeCodec
+from .exceptions import UnknownUrnError
+from .routing import UnknownUrnStrategy
+from .transport import ReceivedMessage, Transport, make_transport
+
+Handler = Callable[..., None]
+
+
+class BabelQueue:
+    def __init__(
+        self,
+        broker_url: str = "memory://",
+        *,
+        transport: Optional[Transport] = None,
+        queue: str = "default",
+        on_unknown_urn: str = UnknownUrnStrategy.FAIL,
+        max_attempts: int = 3,
+        dead_letter: bool = False,
+        dead_letter_queue: Optional[str] = None,
+        dead_letter_suffix: str = ".dlq",
+    ) -> None:
+        self.transport = transport if transport is not None else make_transport(broker_url)
+        self.queue = queue
+        self.on_unknown_urn = on_unknown_urn
+        self.max_attempts = max_attempts
+        self.dead_letter_enabled = bool(dead_letter)
+        self.dead_letter_queue = dead_letter_queue
+        self.dead_letter_suffix = dead_letter_suffix
+        self._handlers: Dict[str, Handler] = {}
+
+    # -- Produce ------------------------------------------------------------
+
+    def publish(
+        self,
+        urn: str,
+        data: Mapping[str, Any],
+        *,
+        queue: Optional[str] = None,
+        trace_id: Optional[str] = None,
+    ) -> str:
+        """Publish a message; returns its id (``meta.id``)."""
+        target = queue or self.queue
+        envelope = EnvelopeCodec.make(urn, data, queue=target, trace_id=trace_id)
+        self.transport.publish(target, EnvelopeCodec.encode(envelope))
+        return envelope["meta"]["id"]
+
+    # -- Register handlers --------------------------------------------------
+
+    def handler(self, urn: str) -> Callable[[Handler], Handler]:
+        """Decorator: register ``fn`` as the handler for ``urn``."""
+
+        def decorator(fn: Handler) -> Handler:
+            self._handlers[urn] = fn
+            return fn
+
+        return decorator
+
+    def register(self, urn: str, fn: Handler) -> None:
+        self._handlers[urn] = fn
+
+    # -- Consume ------------------------------------------------------------
+
+    def consume(
+        self,
+        queue: Optional[str] = None,
+        *,
+        max_messages: Optional[int] = None,
+        timeout: float = 1.0,
+    ) -> int:
+        """Consume messages until interrupted (or ``max_messages`` processed).
+
+        Returns the number of messages processed. With ``max_messages`` set, the
+        loop stops once that many are handled or the queue drains within ``timeout``.
+        """
+        target = queue or self.queue
+        processed = 0
+        try:
+            while max_messages is None or processed < max_messages:
+                received = self.transport.pop(target, timeout=timeout)
+                if received is None:
+                    if max_messages is not None:
+                        break
+                    continue
+                self.dispatch(received)
+                processed += 1
+        except KeyboardInterrupt:  # pragma: no cover - graceful Ctrl-C
+            pass
+        return processed
+
+    run = consume
+
+    def dispatch(self, received: ReceivedMessage) -> None:
+        """Route one reserved message to its handler and acknowledge it."""
+        envelope = EnvelopeCodec.decode(received.body)
+        urn = str(envelope.get("job") or envelope.get("urn") or "")
+        handler = self._handlers.get(urn) if urn else None
+
+        try:
+            if handler is None:
+                self._route_unknown(urn, received, envelope)
+                return
+            self._invoke(handler, envelope)
+            self.transport.ack(received)
+        except Exception as exc:  # noqa: BLE001 - one bad message must not kill the loop
+            self._retry_or_dead_letter(received, envelope, exc)
+
+    # -- Internals ----------------------------------------------------------
+
+    def _invoke(self, handler: Handler, envelope: Mapping[str, Any]) -> None:
+        data = dict(envelope.get("data") or {})
+        meta = dict(envelope.get("meta") or {})
+        if _handler_wants_envelope(handler):
+            handler(data, meta, dict(envelope))
+        else:
+            handler(data, meta)
+
+    def _route_unknown(self, urn: str, received: ReceivedMessage, envelope: Mapping[str, Any]) -> None:
+        strategy = self.on_unknown_urn
+        if strategy == UnknownUrnStrategy.DELETE:
+            self.transport.ack(received)
+            return
+        if strategy == UnknownUrnStrategy.RELEASE:
+            self.transport.publish(received.queue, received.body)
+            self.transport.ack(received)
+            return
+        if strategy == UnknownUrnStrategy.DEAD_LETTER:
+            self._dead_letter(received, dict(envelope), "unknown_urn", None)
+            return
+        # FAIL — surfaced through the retry/dead-letter path (never kills the loop).
+        raise UnknownUrnError(
+            f"No handler mapped for URN [{urn or '(empty)'}]."
+        )
+
+    def _retry_or_dead_letter(
+        self, received: ReceivedMessage, envelope: Dict[str, Any], exc: BaseException
+    ) -> None:
+        attempts = int(envelope.get("attempts", 0)) + 1
+        envelope["attempts"] = attempts
+
+        if attempts < self.max_attempts:
+            self.transport.publish(received.queue, EnvelopeCodec.encode(envelope))
+            self.transport.ack(received)
+            return
+
+        if self.dead_letter_enabled:
+            reason = "unknown_urn" if isinstance(exc, UnknownUrnError) else "failed"
+            self._dead_letter(received, envelope, reason, exc)
+            return
+
+        # Retries exhausted, no DLQ configured — drop it (ack so it leaves the queue).
+        self.transport.ack(received)
+
+    def _dead_letter(
+        self,
+        received: ReceivedMessage,
+        envelope: Dict[str, Any],
+        reason: str,
+        exc: Optional[BaseException],
+    ) -> None:
+        original_queue = str((envelope.get("meta") or {}).get("queue") or received.queue)
+        annotated = dead_letter.annotate(
+            envelope,
+            reason,
+            original_queue,
+            int(envelope.get("attempts", 0)),
+            error=(str(exc) if exc is not None else None),
+            exception=(type(exc).__name__ if exc is not None else None),
+        )
+        target = self.dead_letter_queue or (received.queue + self.dead_letter_suffix)
+        self.transport.publish(target, EnvelopeCodec.encode(annotated))
+        self.transport.ack(received)
+
+
+def _handler_wants_envelope(fn: Handler) -> bool:
+    """True if the handler takes a 3rd positional arg (the full envelope)."""
+    try:
+        params = list(inspect.signature(fn).parameters.values())
+    except (TypeError, ValueError):  # pragma: no cover - builtins/C callables
+        return False
+    positional = [
+        p for p in params
+        if p.kind in (p.POSITIONAL_ONLY, p.POSITIONAL_OR_KEYWORD)
+    ]
+    has_varargs = any(p.kind == p.VAR_POSITIONAL for p in params)
+    return has_varargs or len(positional) >= 3

babelqueue-0.3.0/src/babelqueue/pika_transport.py

@@ -0,0 +1,109 @@
+"""RabbitMQ transport over AMQP 0-9-1. Requires the ``amqp`` extra:
+
+    pip install "babelqueue[amqp]"
+
+Producing publishes the envelope to a durable queue with persistent delivery and
+the AMQP properties that are part of the cross-language contract (``type`` = URN,
+``correlation_id`` = trace_id, ``message_id`` = meta.id, ``x-schema-version`` /
+``x-source-lang`` / ``x-attempts`` headers) — so a Go/PHP consumer can route on
+``properties.type`` without parsing the body. Consuming uses ``basic_get`` + manual
+ack (at-least-once), matching the PHP RabbitMQ driver.
+
+Connection is lazy; it (re)connects on first use and after a drop.
+"""
+
+from __future__ import annotations
+
+import json
+from typing import Any, Dict, Optional
+
+from .transport import ReceivedMessage, Transport
+
+
+class PikaTransport(Transport):
+    def __init__(self, url: str) -> None:
+        try:
+            import pika
+        except ImportError as exc:  # pragma: no cover - import guard
+            raise ImportError(
+                "PikaTransport requires the 'pika' package. Install with "
+                'pip install "babelqueue[amqp]".'
+            ) from exc
+
+        self._pika = pika
+        self._url = url
+        self._connection: Any = None
+        self._channel: Any = None
+        self._declared: set[str] = set()
+
+    # -- connection / topology ---------------------------------------------
+
+    def _chan(self) -> Any:
+        if self._connection is None or self._connection.is_closed:
+            self._connection = self._pika.BlockingConnection(self._pika.URLParameters(self._url))
+            self._channel = None
+            self._declared.clear()
+        if self._channel is None or self._channel.is_closed:
+            self._channel = self._connection.channel()
+        return self._channel
+
+    def _declare(self, queue: str) -> None:
+        if queue not in self._declared:
+            self._chan().queue_declare(queue=queue, durable=True)
+            self._declared.add(queue)
+
+    def _properties(self, body: str) -> Any:
+        """AMQP properties derived from the envelope (part of the wire contract)."""
+        try:
+            envelope: Dict[str, Any] = json.loads(body)
+        except (ValueError, TypeError):
+            return self._pika.BasicProperties(content_type="application/json", delivery_mode=2)
+
+        meta = envelope.get("meta") or {}
+        headers = {
+            "x-schema-version": meta.get("schema_version"),
+            "x-source-lang": meta.get("lang"),
+            "x-attempts": envelope.get("attempts", 0),
+        }
+        return self._pika.BasicProperties(
+            content_type="application/json",
+            content_encoding="utf-8",
+            delivery_mode=2,  # persistent
+            message_id=meta.get("id"),
+            correlation_id=envelope.get("trace_id"),
+            type=envelope.get("job"),
+            app_id="babelqueue",
+            headers={k: v for k, v in headers.items() if v is not None},
+        )
+
+    # -- Transport ----------------------------------------------------------
+
+    def publish(self, queue: str, body: str) -> None:
+        self._declare(queue)
+        self._chan().basic_publish(
+            exchange="",
+            routing_key=queue,
+            body=body.encode("utf-8"),
+            properties=self._properties(body),
+        )
+
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        self._declare(queue)
+        method, _props, body = self._chan().basic_get(queue=queue, auto_ack=False)
+        if method is None:
+            # Nothing ready — sleep (heartbeat-safe) so the caller doesn't busy-loop.
+            if timeout and timeout > 0:
+                self._connection.sleep(timeout)
+            return None
+        text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) else str(body)
+        return ReceivedMessage(body=text, queue=queue, handle=method.delivery_tag)
+
+    def ack(self, message: ReceivedMessage) -> None:
+        self._chan().basic_ack(delivery_tag=message.handle)
+
+    def close(self) -> None:  # pragma: no cover
+        try:
+            if self._connection is not None and self._connection.is_open:
+                self._connection.close()
+        except Exception:
+            pass

babelqueue-0.3.0/src/babelqueue/redis_transport.py

@@ -0,0 +1,48 @@
+"""Redis transport (reliable-queue pattern). Requires the ``redis`` extra:
+
+    pip install "babelqueue[redis]"
+
+Producing is ``RPUSH queue body``; consuming atomically moves the head to a
+per-queue processing list (``BLMOVE``) so an in-flight message survives a worker
+crash, and ``ack`` removes it from that processing list. This is a Python-owned
+reliable queue; full parity with Laravel's reserved-set reservation on a *shared*
+Redis queue is a separate conformance task (see the roadmap).
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from .transport import ReceivedMessage, Transport
+
+
+class RedisTransport(Transport):
+    def __init__(self, url: str, *, processing_suffix: str = ":processing") -> None:
+        try:
+            import redis  # noqa: F401  (lazy: only needed for this transport)
+        except ImportError as exc:  # pragma: no cover - import guard
+            raise ImportError(
+                "RedisTransport requires the 'redis' package. Install with "
+                "pip install \"babelqueue[redis]\"."
+            ) from exc
+
+        self._redis = redis.Redis.from_url(url, decode_responses=True)
+        self._processing_suffix = processing_suffix
+
+    def _processing(self, queue: str) -> str:
+        return f"{queue}{self._processing_suffix}"
+
+    def publish(self, queue: str, body: str) -> None:
+        self._redis.rpush(queue, body)
+
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        body = self._redis.blmove(queue, self._processing(queue), timeout, "LEFT", "RIGHT")
+        if body is None:
+            return None
+        return ReceivedMessage(body=body, queue=queue, handle=body)
+
+    def ack(self, message: ReceivedMessage) -> None:
+        self._redis.lrem(self._processing(message.queue), 1, message.handle)
+
+    def close(self) -> None:  # pragma: no cover
+        self._redis.close()

babelqueue-0.3.0/src/babelqueue/transport.py

@@ -0,0 +1,86 @@
+"""Broker transport abstraction for the runtime.
+
+The runtime talks to a broker only through :class:`Transport`, so the routing /
+retry logic is broker-agnostic and unit-testable with :class:`InMemoryTransport`.
+"""
+
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from collections import defaultdict, deque
+from dataclasses import dataclass, field
+from typing import Any, Deque, Dict, Optional
+
+from .exceptions import BabelQueueError
+
+
+@dataclass
+class ReceivedMessage:
+    """A message popped from a queue, plus a transport-internal ack handle."""
+
+    body: str
+    queue: str
+    handle: Any = None
+
+
+class Transport(ABC):
+    """Minimal broker contract: publish a raw body, pop one, acknowledge it."""
+
+    @abstractmethod
+    def publish(self, queue: str, body: str) -> None:
+        """Append an already-encoded envelope to ``queue``."""
+
+    @abstractmethod
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        """Reserve the next message from ``queue``, or ``None`` if none arrives."""
+
+    @abstractmethod
+    def ack(self, message: ReceivedMessage) -> None:
+        """Acknowledge (remove) a reserved message."""
+
+    def close(self) -> None:  # pragma: no cover - optional
+        """Release any resources (override if needed)."""
+
+
+class InMemoryTransport(Transport):
+    """In-process transport for tests and broker-free local runs (``memory://``)."""
+
+    def __init__(self) -> None:
+        self._queues: Dict[str, Deque[str]] = defaultdict(deque)
+
+    def publish(self, queue: str, body: str) -> None:
+        self._queues[queue].append(body)
+
+    def pop(self, queue: str, timeout: float = 1.0) -> Optional[ReceivedMessage]:
+        dq = self._queues.get(queue)
+        if not dq:
+            return None
+        return ReceivedMessage(body=dq.popleft(), queue=queue)
+
+    def ack(self, message: ReceivedMessage) -> None:
+        # Already removed on pop; nothing to do.
+        return None
+
+    def size(self, queue: str) -> int:
+        return len(self._queues.get(queue, ()))
+
+
+def make_transport(broker_url: str) -> Transport:
+    """Build a transport from a broker URL scheme (``memory://``, ``redis://``)."""
+    scheme = broker_url.split("://", 1)[0] if "://" in broker_url else broker_url
+
+    if scheme in ("", "memory"):
+        return InMemoryTransport()
+    if scheme in ("redis", "rediss"):
+        from .redis_transport import RedisTransport
+
+        return RedisTransport(broker_url)
+    if scheme in ("amqp", "amqps"):
+        from .pika_transport import PikaTransport
+
+        return PikaTransport(broker_url)
+
+    raise BabelQueueError(
+        f"Unsupported broker scheme {scheme!r}. Use 'memory://', 'redis://' or "
+        "'amqp://', or pass your own Transport via BabelQueue(transport=...)."
+    )

babelqueue-0.3.0/tests/test_app.py

@@ -0,0 +1,125 @@
+"""The runtime: publish, @handler routing, retry and dead-lettering.
+
+All tests use the in-memory transport — no broker required.
+"""
+
+from __future__ import annotations
+
+import unittest
+
+from babelqueue import BabelQueue, EnvelopeCodec, UnknownUrnStrategy
+
+
+class AppTest(unittest.TestCase):
+    def test_publish_then_consume_invokes_handler_and_acks(self) -> None:
+        app = BabelQueue("memory://", queue="orders")
+        seen = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            seen["data"] = data
+            seen["meta"] = meta
+
+        msg_id = app.publish("urn:babel:orders:created", {"order_id": 7})
+        processed = app.consume(max_messages=1)
+
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen["data"], {"order_id": 7})
+        self.assertEqual(seen["meta"]["lang"], "python")
+        self.assertEqual(seen["meta"]["id"], msg_id)
+        self.assertEqual(app.transport.size("orders"), 0)  # acked / drained
+
+    def test_three_arg_handler_receives_full_envelope(self) -> None:
+        app = BabelQueue("memory://")
+        seen = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta, message):  # noqa: ANN001
+            seen["trace"] = message["trace_id"]
+            seen["job"] = message["job"]
+
+        app.publish("urn:babel:orders:created", {}, trace_id="trace-1")
+        app.consume(max_messages=1)
+
+        self.assertEqual(seen["trace"], "trace-1")
+        self.assertEqual(seen["job"], "urn:babel:orders:created")
+
+    def test_round_trips_with_the_canonical_codec(self) -> None:
+        app = BabelQueue("memory://", queue="orders")
+        captured = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            captured.update(data)
+
+        app.publish("urn:babel:orders:created", {"order_id": 9, "amount": "9.90"})
+        # the raw body on the queue is a canonical envelope decodable by any SDK
+        raw = app.transport._queues["orders"][0]  # noqa: SLF001 - test introspection
+        env = EnvelopeCodec.decode(raw)
+        self.assertEqual(list(env.keys()), ["job", "trace_id", "data", "meta", "attempts"])
+
+        app.consume(max_messages=1)
+        self.assertEqual(captured, {"order_id": 9, "amount": "9.90"})
+
+    def test_unknown_urn_delete_drops_message(self) -> None:
+        app = BabelQueue("memory://", queue="q", on_unknown_urn=UnknownUrnStrategy.DELETE)
+        app.publish("urn:babel:nobody", {})
+        app.consume(max_messages=1)
+        self.assertEqual(app.transport.size("q"), 0)
+
+    def test_unknown_urn_dead_letter_quarantines(self) -> None:
+        app = BabelQueue(
+            "memory://", queue="q",
+            on_unknown_urn=UnknownUrnStrategy.DEAD_LETTER, dead_letter=True,
+        )
+        app.publish("urn:babel:nobody", {"x": 1})
+        app.consume(max_messages=1)
+
+        self.assertEqual(app.transport.size("q"), 0)
+        dlq_raw = app.transport._queues["q.dlq"][0]  # noqa: SLF001
+        env = EnvelopeCodec.decode(dlq_raw)
+        self.assertEqual(env["dead_letter"]["reason"], "unknown_urn")
+        self.assertEqual(env["dead_letter"]["original_queue"], "q")
+
+    def test_handler_failure_retries_then_dead_letters(self) -> None:
+        app = BabelQueue("memory://", queue="orders", max_attempts=2, dead_letter=True)
+        calls = {"n": 0}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            calls["n"] += 1
+            raise RuntimeError("boom")
+
+        app.publish("urn:babel:orders:created", {"order_id": 1})
+        # attempt 1 -> requeue, attempt 2 -> dead-letter
+        app.consume(max_messages=5)
+
+        self.assertEqual(calls["n"], 2)
+        self.assertEqual(app.transport.size("orders"), 0)
+        dlq_raw = app.transport._queues["orders.dlq"][0]  # noqa: SLF001
+        env = EnvelopeCodec.decode(dlq_raw)
+        self.assertEqual(env["dead_letter"]["reason"], "failed")
+        self.assertEqual(env["dead_letter"]["error"], "boom")
+        self.assertEqual(env["dead_letter"]["attempts"], 2)
+
+    def test_failure_without_dlq_drops_after_max_attempts(self) -> None:
+        app = BabelQueue("memory://", queue="orders", max_attempts=1, dead_letter=False)
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            raise RuntimeError("nope")
+
+        app.publish("urn:babel:orders:created", {})
+        app.consume(max_messages=3)
+        self.assertEqual(app.transport.size("orders"), 0)
+        self.assertEqual(app.transport.size("orders.dlq"), 0)
+
+    def test_unsupported_broker_scheme_raises(self) -> None:
+        from babelqueue import BabelQueueError
+
+        with self.assertRaises(BabelQueueError):
+            BabelQueue("kafka://localhost:9092")
+
+
+if __name__ == "__main__":
+    unittest.main()

babelqueue-0.3.0/tests/test_pika_transport.py

@@ -0,0 +1,98 @@
+"""Integration tests for the RabbitMQ (pika) transport.
+
+Skipped unless a broker is reachable (the `pika` package installed and a broker at
+``BABELQUEUE_TEST_AMQP`` / localhost). The CI ``integration`` job runs these
+against a RabbitMQ service; locally they skip cleanly.
+"""
+
+from __future__ import annotations
+
+import os
+import unittest
+import uuid
+
+try:
+    import pika as _pika
+except ImportError:  # pragma: no cover
+    _pika = None
+
+from babelqueue import BabelQueue, EnvelopeCodec
+
+AMQP_URL = os.environ.get("BABELQUEUE_TEST_AMQP", "amqp://guest:guest@localhost:5672/")
+
+
+def _amqp_available() -> bool:
+    if _pika is None:
+        return False
+    try:
+        conn = _pika.BlockingConnection(_pika.URLParameters(AMQP_URL))
+        conn.close()
+        return True
+    except Exception:  # pragma: no cover - connection failure
+        return False
+
+
+@unittest.skipUnless(_amqp_available(), f"no reachable RabbitMQ at {AMQP_URL}")
+class PikaTransportTest(unittest.TestCase):
+    def setUp(self) -> None:
+        self.queue = f"bqtest-{uuid.uuid4().hex}"
+        self.conn = _pika.BlockingConnection(_pika.URLParameters(AMQP_URL))
+        self.ctl = self.conn.channel()
+
+    def tearDown(self) -> None:
+        for q in (self.queue, f"{self.queue}.dlq"):
+            try:
+                self.ctl.queue_delete(queue=q)
+            except Exception:
+                pass
+        self.conn.close()
+
+    def _depth(self, queue: str) -> int:
+        method = self.ctl.queue_declare(queue=queue, durable=True, passive=True)
+        return method.method.message_count
+
+    def test_publish_consume_round_trip_and_ack(self) -> None:
+        app = BabelQueue(AMQP_URL, queue=self.queue)
+        seen = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            seen.update(data)
+
+        app.publish("urn:babel:orders:created", {"order_id": 42})
+        processed = app.consume(max_messages=1, timeout=3)
+
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen, {"order_id": 42})
+        self.assertEqual(self._depth(self.queue), 0)  # acked
+
+    def test_publish_sets_contract_amqp_properties(self) -> None:
+        app = BabelQueue(AMQP_URL, queue=self.queue)
+        app.publish("urn:babel:orders:created", {"order_id": 1}, trace_id="trace-amqp")
+
+        method, props, body = self.ctl.basic_get(queue=self.queue, auto_ack=True)
+        self.assertIsNotNone(method)
+        self.assertEqual(props.type, "urn:babel:orders:created")     # route on properties.type
+        self.assertEqual(props.correlation_id, "trace-amqp")         # trace_id
+        self.assertEqual(props.content_type, "application/json")
+        self.assertEqual(props.delivery_mode, 2)                     # persistent
+        self.assertEqual(props.app_id, "babelqueue")
+
+    def test_failure_dead_letters(self) -> None:
+        app = BabelQueue(AMQP_URL, queue=self.queue, max_attempts=1, dead_letter=True)
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            raise RuntimeError("boom")
+
+        app.publish("urn:babel:orders:created", {"order_id": 1})
+        app.consume(max_messages=2, timeout=3)
+
+        self.assertEqual(self._depth(f"{self.queue}.dlq"), 1)
+        _m, _p, body = self.ctl.basic_get(queue=f"{self.queue}.dlq", auto_ack=True)
+        env = EnvelopeCodec.decode(body.decode("utf-8"))
+        self.assertEqual(env["dead_letter"]["reason"], "failed")
+
+
+if __name__ == "__main__":
+    unittest.main()

babelqueue-0.3.0/tests/test_redis_transport.py

@@ -0,0 +1,78 @@
+"""Integration tests for the Redis transport.
+
+Skipped unless a Redis server is reachable (the `redis` package installed and a
+broker at ``BABELQUEUE_TEST_REDIS`` / localhost). The CI ``integration`` job runs
+these against a Redis service; locally they skip cleanly.
+"""
+
+from __future__ import annotations
+
+import os
+import unittest
+import uuid
+
+try:
+    import redis as _redis
+except ImportError:  # pragma: no cover
+    _redis = None
+
+from babelqueue import BabelQueue, EnvelopeCodec
+
+REDIS_URL = os.environ.get("BABELQUEUE_TEST_REDIS", "redis://localhost:6379/0")
+
+
+def _redis_available() -> bool:
+    if _redis is None:
+        return False
+    try:
+        _redis.Redis.from_url(REDIS_URL, decode_responses=True).ping()
+        return True
+    except Exception:  # pragma: no cover - connection failure
+        return False
+
+
+@unittest.skipUnless(_redis_available(), f"no reachable Redis at {REDIS_URL}")
+class RedisTransportTest(unittest.TestCase):
+    def setUp(self) -> None:
+        self.queue = f"bqtest:{uuid.uuid4().hex}"
+        self.client = _redis.Redis.from_url(REDIS_URL, decode_responses=True)
+
+    def tearDown(self) -> None:
+        self.client.delete(self.queue, f"{self.queue}:processing", f"{self.queue}.dlq")
+
+    def test_publish_consume_round_trip_and_ack(self) -> None:
+        app = BabelQueue(REDIS_URL, queue=self.queue)
+        seen = {}
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            seen.update(data)
+
+        app.publish("urn:babel:orders:created", {"order_id": 42})
+        processed = app.consume(max_messages=1, timeout=2)
+
+        self.assertEqual(processed, 1)
+        self.assertEqual(seen, {"order_id": 42})
+        # acked: nothing left on the queue or the processing list
+        self.assertEqual(self.client.llen(self.queue), 0)
+        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
+
+    def test_failure_dead_letters_to_redis(self) -> None:
+        app = BabelQueue(REDIS_URL, queue=self.queue, max_attempts=1, dead_letter=True)
+
+        @app.handler("urn:babel:orders:created")
+        def handle(data, meta):  # noqa: ANN001
+            raise RuntimeError("boom")
+
+        app.publish("urn:babel:orders:created", {"order_id": 1})
+        app.consume(max_messages=2, timeout=2)
+
+        self.assertEqual(self.client.llen(f"{self.queue}:processing"), 0)
+        dlq = self.client.lrange(f"{self.queue}.dlq", 0, -1)
+        self.assertEqual(len(dlq), 1)
+        env = EnvelopeCodec.decode(dlq[0])
+        self.assertEqual(env["dead_letter"]["reason"], "failed")
+
+
+if __name__ == "__main__":
+    unittest.main()

babelqueue-0.1.0/.github/workflows/ci.yml

@@ -1,33 +0,0 @@
-name: CI
-
-on:
-  push:
-    branches: [main]
-  pull_request:
-
-permissions:
-  contents: read
-
-jobs:
-  test:
-    name: Python ${{ matrix.python }}
-    runs-on: ubuntu-latest
-    strategy:
-      fail-fast: false
-      matrix:
-        python: ['3.9', '3.10', '3.11', '3.12', '3.13']
-    steps:
-      - uses: actions/checkout@v4
-
-      - name: Setup Python
-        uses: actions/setup-python@v5
-        with:
-          python-version: ${{ matrix.python }}
-
-      - name: Install
-        run: |
-          python -m pip install --upgrade pip
-          pip install -e ".[dev]"
-
-      - name: Run tests
-        run: pytest

babelqueue-0.1.0/CHANGELOG.md

@@ -1,30 +0,0 @@
-# Changelog
-
-All notable changes to `babelqueue` (Python) are documented here.
-
-The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
-and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
-The envelope wire format is versioned separately by `meta.schema_version`
-(currently **1**) — see the contract at [babelqueue.com](https://babelqueue.com).
-
-## [Unreleased]
-
-### Added
-- `EnvelopeCodec` — builds (`make`, `from_message`), encodes and decodes the
-  canonical `{job, trace_id, data, meta, attempts}` envelope (`schema_version` 1).
-  The single Python implementation of the wire format.
-- Contracts `PolyglotMessage` / `HasTraceId` (typed `Protocol`s).
-- `dead_letter.annotate()` — additive `dead_letter` block builder.
-- `UnknownUrnStrategy` — `fail` / `delete` / `release` / `dead_letter`.
-- `BabelQueueError` / `UnknownUrnError`.
-- Golden conformance fixtures under `tests/fixtures/` (shared cross-SDK set).
-- `py.typed` — ships inline type hints (PEP 561).
-
-### Notes
-- Pre-1.0: the public API may change before the `1.0.0` tag.
-- **Zero runtime dependencies** (standard library only). Requires Python `>=3.9`.
-- This is the framework-agnostic **core**. The broker runtime
-  (`BabelQueue(broker_url=...)` + `@app.handler`, over `redis`/`pika`) and the
-  Celery/Django adapters are planned next iterations, built on this core.
-
-[Unreleased]: https://github.com/BabelQueue/babelqueue-python/commits/main