babelqueue 1.22.0__tar.gz → 1.23.0__tar.gz

{babelqueue-1.22.0 → babelqueue-1.23.0}/.github/workflows/ci.yml

@@ -177,7 +177,7 @@ jobs:
           --health-retries 10
     env:
       # The pinned conformance release (ADR-0032). Bump together with tests/conformance/.
-      BQ_CONFORMANCE_VERSION: 1.6.0
+      BQ_CONFORMANCE_VERSION: 1.7.0
       # sha256 of that release's fetch.sh: the script runs on the runner, so it is pinned rather
       # than trusted from a mutable tag. Bump together with BQ_CONFORMANCE_VERSION.
       BQ_FETCH_SH_SHA256: 8bb6ba567b288fb0ac71a70fd17041b2503df119e01998d76fdad372570bf849

{babelqueue-1.22.0 → babelqueue-1.23.0}/.github/workflows/release.yml

@@ -14,7 +14,7 @@ jobs:
     # Recommended for PyPI Trusted Publishing — configure a "pypi" environment +
     # a trusted publisher on PyPI (repo BabelQueue/babelqueue-python, this workflow).
     environment: pypi
-    # redis_layout (conformance 1.6.0) drives the real Lua scripts: a real redis:7 is mandatory,
+    # redis_layout (conformance 1.7.0) drives the real Lua scripts: a real redis:7 is mandatory,
     # exactly as in ci.yml's `conformance` job — without it the cases are reported as failures.
     services:
       redis:
@@ -28,7 +28,7 @@ jobs:
           --health-retries 10
     env:
       # The pinned conformance release (ADR-0032). Keep identical to ci.yml.
-      BQ_CONFORMANCE_VERSION: 1.6.0
+      BQ_CONFORMANCE_VERSION: 1.7.0
       BQ_FETCH_SH_SHA256: 8bb6ba567b288fb0ac71a70fd17041b2503df119e01998d76fdad372570bf849
     permissions:
       id-token: write   # PyPI Trusted Publishing (OIDC) — no API token needed

{babelqueue-1.22.0 → babelqueue-1.23.0}/CHANGELOG.md

@@ -9,6 +9,76 @@ The envelope wire format is versioned separately by `meta.schema_version`
 
 ## [Unreleased]
 
+## [1.23.0] - 2026-10-25
+
+Implements ADR-0036 (post-R1 parity) decisions 2, 4, 5 and 6. A MINOR: every change is a
+consumer-side settle decision, so the envelope stays at `schema_version: 1`. The vendored
+conformance suite moves to v1.7.0 (new sections `identity_verdict` and `pulsar_exhaustion`, both
+passing; `conformance-known-gaps.json` stays empty).
+
+### Changed (behaviour)
+- **A body that does not decode, or has no identity, is poison on every transport (ADR-0036
+  decisions 4 and 6).** A JSON object that has neither a valid string `job` nor a valid string
+  `urn` (ADR-0032 section 6) used to be an *unknown URN*, and on SQS, AMQP, Pulsar, Azure Service
+  Bus, Artemis and the memory transport a body that did not decode was decoded leniently into an
+  empty envelope and retried as an unknown URN, finally dead-lettered as `reason: "unknown_urn"`
+  without the original body. Both are now **poison** and take the poison route of Go's
+  `routePoison`: they are never handed to a handler and never take the unknown-URN strategies
+  (those remain for a valid envelope whose URN has no handler). With the DLQ on (`dead_letter=True`
+  or `on_unknown_urn=DEAD_LETTER`) the message is written to the DLQ at once, with no retry, as the
+  ADR-0035 record (`dead_letter.reason: "poison"`, the original body preserved: a strict JSON
+  object spliced byte for byte, anything else as `raw`, base64 plus `raw_encoding` when it is not
+  valid UTF-8) and acknowledged after the write; `on_unknown_urn=DELETE` drops it (logged); with
+  the DLQ off `RELEASE` is replaced by the bounded `FAIL` path (re-published with `attempts + 1`
+  up to `max_attempts`, then dropped and logged). An operator who relied on a poison body being
+  retried through the unknown-URN strategy, or on a `reason: "unknown_urn"` record for it, sees the
+  new behaviour. The classification is `EnvelopeCodec.poison_cause(raw)` (new, pure): the typed
+  decode of the Go reference plus the identity rule; `decode` / `decode_lenient` are unchanged.
+- **Pulsar poison records carry `reason: "poison"`** (they were `unknown_urn`). The Pulsar
+  transport also reports the reconciled `bq-attempts` / redelivery count to the runtime
+  (`ReceivedMessage.native_attempts`, new), as does SQS (`ApproximateReceiveCount - 1`), so a poison
+  body, which cannot carry `attempts`, is still bounded by `max_attempts`.
+- **Redis: an identity-less body is dead-lettered unconditionally (ADR-0036 decision 4,
+  broker-bindings section 1.1 "Identity-less body").** In both layouts (legacy `:processing`
+  and Laravel-compatible zset) a body that decodes but names no identity (including a bare `null`,
+  which used to follow the bounded `FAIL` path) is handed over as transport poison: it is written
+  to `<queue>.dlq` as the ADR-0035 record with `reason: "poison"` in every configuration
+  (`on_unknown_urn=DELETE` and dead-lettering off change nothing), never handed to a handler and
+  never retried. Nothing else in the Redis transport changes.
+- **Kafka without retry tiers: a poison record no longer blocks the partition (ADR-0036 decision
+  5, broker-bindings section 6.5 degrade-to-delete).** With no `retry_tiers_ms`, dead-lettering off
+  and `on_unknown_urn` not `DELETE`, the bounded retry of a poison record (including an
+  identity-less body) needs a tier and raised, leaving the record uncommitted with `bq-attempts`
+  never advancing. It is now committed, dropped and logged, and the partition advances. A failing
+  handler and `RELEASE` of a valid envelope keep the section 6.5 rule (they still raise
+  `BabelQueueError` without tiers). The start-up warning for a tierless transport now says so; enable
+  `dead_letter` to keep such records in `<topic>.dlq`.
+- A message dropped after its retries are exhausted with no DLQ configured is now logged
+  (`babelqueue.app`, WARNING) instead of being dropped silently.
+
+- **Pulsar: a failed DLQ write or retry publish is negatively acknowledged (Go `Rewind`).**
+  `PulsarTransport` is now a `Rewinder` (`negative_acknowledge`), so the unsettled original is
+  redelivered by the broker instead of staying un-acknowledged for the life of the consumer. A
+  graceful-shutdown release of an in-flight Pulsar message is now a nack as well rather than a
+  re-publish (on a `Shared` subscription the native redelivery count advances; on `Exclusive` /
+  `Failover` it does not, which is harmless: `bq-attempts` carries the attempt limit).
+
+### Fixed
+- **Bytes that are not valid UTF-8 no longer crash the AMQP, Pulsar and Artemis consumers.** They
+  were decoded strictly in `pop`; they are now kept losslessly (`surrogateescape`). Invalid UTF-8
+  is poison wherever it sits, **also inside a JSON string** (`EnvelopeCodec.poison_cause`,
+  error-handling section 7): it is never handed to a handler and is dead-lettered as the ADR-0035
+  base64 record, like on Redis and Kafka.
+
+### Added
+- `EnvelopeCodec.poison_cause(raw, *, require_identity=True)`; `Transport.classifies_poison`
+  (Kafka and Redis classify poison themselves and opt out of the runtime's classification);
+  `ReceivedMessage.native_attempts`.
+- Conformance runners for `identity_verdict` (the core verdict on all 97 bodies, plus the fate on
+  the memory transport) and `pulsar_exhaustion` (all 29 rows, driven through the real
+  `PulsarTransport` and runtime over a duck-typed client), and per-transport poison tests.
+- `ci.yml` / `release.yml` pin conformance `1.7.0` (the `fetch.sh` SHA-256 is unchanged).
+
 ## [1.22.0] - 2026-10-24
 
 Implements ADR-0033 (Redis) and ADR-0034 (Kafka tiers, backoff, poison). The envelope stays at

{babelqueue-1.22.0 → babelqueue-1.23.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: babelqueue
-Version: 1.22.0
+Version: 1.23.0
 Summary: Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers.
 Project-URL: Homepage, https://babelqueue.com
 Project-URL: Source, https://github.com/BabelQueue/babelqueue-python
@@ -313,9 +313,31 @@ the DLQ on (`dead_letter` or `on_unknown_urn=DEAD_LETTER`) it is dead-lettered o
 written to `<topic>.dlq` (which must exist) and its offset is committed once that write
 is confirmed; a failed write leaves it uncommitted and rewound. With the DLQ off nothing is written:
 `on_unknown_urn=DELETE` drops it, any other strategy retries it through the retry tiers up to
-`max_attempts` and then drops it (without tiers the record is redelivered in place, as above). On
-the Redis legacy (`<queue>:processing`) layout a body that Go's typed decode rejects (neither a JSON
-object nor `null`, or wrong member types) is dead-lettered raw on its first
+`max_attempts` and then drops it. **Without retry tiers** that bounded retry has no tier to use, so
+(with the DLQ off and `on_unknown_urn` not `DELETE`) the poison record is committed, dropped and logged
+instead of blocking the partition (section 6.5 degrade-to-delete, ADR-0036 decision 5); enable
+`dead_letter` to keep it in `<topic>.dlq`. A failing handler without tiers still cannot be retried, as above.
+
+### Poison messages on every other transport
+
+A message body that does not decode into an envelope (not a JSON object, malformed, a wrongly typed
+member, or bytes that are not valid UTF-8, even inside a JSON string) or that has neither a valid
+string `job` nor a valid string `urn` is **poison** on every transport (ADR-0036 decisions 2, 4 and
+6; conformance `identity_verdict`). It is never handed to a handler and never takes the
+unknown-URN strategies, which are for a valid envelope whose URN has no handler:
+
+- DLQ on (`dead_letter` or `on_unknown_urn=DEAD_LETTER`): written to `<queue>.dlq` at once, with no
+  retry, as the ADR-0035 record (`dead_letter.reason: "poison"`, the original body preserved,
+  base64 plus `raw_encoding` when it is not UTF-8), then acknowledged. A failed write leaves it
+  unacknowledged.
+- `on_unknown_urn=DELETE`: acknowledged and dropped (logged).
+- DLQ off: the bounded `FAIL` path (`RELEASE` is replaced by it): retried up to `max_attempts`, then
+  dropped and logged. The count comes from the broker where it has one (SQS receive count, Pulsar
+  `bq-attempts` / redelivery count), otherwise from a bare `{"attempts": n}` body the retry leaves.
+
+On Redis (both layouts) an identity-less body is dead-lettered as poison in **every** configuration,
+including `on_unknown_urn=DELETE` and dead-lettering off (broker-bindings section 1.1). On the legacy
+(`<queue>:processing`) layout a body that Go's typed decode rejects is dead-lettered raw on its first
 delivery and removed from `:processing`.
 
 ### Sharing a Redis queue with Laravel

{babelqueue-1.22.0 → babelqueue-1.23.0}/README.md

@@ -259,9 +259,31 @@ the DLQ on (`dead_letter` or `on_unknown_urn=DEAD_LETTER`) it is dead-lettered o
 written to `<topic>.dlq` (which must exist) and its offset is committed once that write
 is confirmed; a failed write leaves it uncommitted and rewound. With the DLQ off nothing is written:
 `on_unknown_urn=DELETE` drops it, any other strategy retries it through the retry tiers up to
-`max_attempts` and then drops it (without tiers the record is redelivered in place, as above). On
-the Redis legacy (`<queue>:processing`) layout a body that Go's typed decode rejects (neither a JSON
-object nor `null`, or wrong member types) is dead-lettered raw on its first
+`max_attempts` and then drops it. **Without retry tiers** that bounded retry has no tier to use, so
+(with the DLQ off and `on_unknown_urn` not `DELETE`) the poison record is committed, dropped and logged
+instead of blocking the partition (section 6.5 degrade-to-delete, ADR-0036 decision 5); enable
+`dead_letter` to keep it in `<topic>.dlq`. A failing handler without tiers still cannot be retried, as above.
+
+### Poison messages on every other transport
+
+A message body that does not decode into an envelope (not a JSON object, malformed, a wrongly typed
+member, or bytes that are not valid UTF-8, even inside a JSON string) or that has neither a valid
+string `job` nor a valid string `urn` is **poison** on every transport (ADR-0036 decisions 2, 4 and
+6; conformance `identity_verdict`). It is never handed to a handler and never takes the
+unknown-URN strategies, which are for a valid envelope whose URN has no handler:
+
+- DLQ on (`dead_letter` or `on_unknown_urn=DEAD_LETTER`): written to `<queue>.dlq` at once, with no
+  retry, as the ADR-0035 record (`dead_letter.reason: "poison"`, the original body preserved,
+  base64 plus `raw_encoding` when it is not UTF-8), then acknowledged. A failed write leaves it
+  unacknowledged.
+- `on_unknown_urn=DELETE`: acknowledged and dropped (logged).
+- DLQ off: the bounded `FAIL` path (`RELEASE` is replaced by it): retried up to `max_attempts`, then
+  dropped and logged. The count comes from the broker where it has one (SQS receive count, Pulsar
+  `bq-attempts` / redelivery count), otherwise from a bare `{"attempts": n}` body the retry leaves.
+
+On Redis (both layouts) an identity-less body is dead-lettered as poison in **every** configuration,
+including `on_unknown_urn=DELETE` and dead-lettering off (broker-bindings section 1.1). On the legacy
+(`<queue>:processing`) layout a body that Go's typed decode rejects is dead-lettered raw on its first
 delivery and removed from `:processing`.
 
 ### Sharing a Redis queue with Laravel

{babelqueue-1.22.0 → babelqueue-1.23.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "babelqueue"
-version = "1.22.0"
+version = "1.23.0"
 description = "Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers."
 readme = "README.md"
 requires-python = ">=3.9"

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/__init__.py

@@ -42,7 +42,7 @@ from .transport import (
     Transport,
 )
 
-__version__ = "1.22.0"
+__version__ = "1.23.0"
 
 __all__ = [
     "BabelQueue",

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/_envelope_shape.py

@@ -15,7 +15,7 @@ Go reference's ``babelqueue.Decode``, except that member names are case-sensitiv
   decoder read them) and every occurrence of a duplicated key is checked; any other member (a
   differently-cased name or a non-string ``urn`` alias included) is only checked for syntax.
   A non-string ``job`` is **not** poison either: the identity resolver ignores it (ADR-0032 section 6),
-  so the body is readable and simply carries no identity (conformance ``kafka_poison``, v1.6.0).
+  so the body is readable and simply carries no identity (conformance ``kafka_poison``, v1.6.0, and ``identity_verdict``, v1.7.0).
 
 A well-typed but non-conformant envelope is not poison: Go's consume path never calls ``Accepts``.
 """

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/app.py

@@ -326,15 +326,37 @@ class BabelQueue:
             # §3.7/§4.7/§5.7/§6.7/§7.7): the raw body goes to the DLQ byte-identical, so there is
             # no envelope to read here.
             gated = received.route_urn.startswith(_UNSUPPORTED_SCHEMA_ROUTE_PREFIX)
-            envelope = {} if gated else EnvelopeCodec.decode_lenient(received.body)
+            # A body that does not decode, or has no identity, is poison (ADR-0036 decisions 4 and
+            # 6): never decoded leniently into an empty envelope, never handed to a handler and
+            # never routed as an unknown URN. A transport that already hands its poison over under
+            # a poison route (Kafka, Redis) is not classified twice. A delivery that carries an
+            # out-of-band routing URN is exempt from the identity half of the rule only.
+            poison_cause = (
+                None
+                if gated or getattr(self.transport, "classifies_poison", False)
+                else EnvelopeCodec.poison_cause(
+                    received.body, require_identity=not received.route_urn
+                )
+            )
+            envelope = (
+                {}
+                if gated or poison_cause is not None
+                else EnvelopeCodec.decode_lenient(received.body)
+            )
             urn = str(envelope.get("job") or envelope.get("urn") or "")
             if received.route_urn and envelope:
                 # Transport-supplied routing override; body untouched. Only for a body that
                 # decodes: an undecodable one (decode() yields {}) stays on the poison path.
                 urn = received.route_urn
-            handler = self._handlers.get(urn) if urn else None
+            handler = (
+                self._handlers.get(urn) if urn and poison_cause is None else None
+            )
 
             try:
+                if poison_cause is not None:
+                    self._phase = _SETTLING
+                    self._route_body_poison(received, poison_cause)
+                    return
                 if handler is None:
                     self._phase = _SETTLING
                     self._route_unknown(urn, received, envelope)
@@ -344,15 +366,19 @@ class BabelQueue:
                 self._phase = _SETTLING
             except Exception as exc:  # noqa: BLE001 - one bad message must not kill the loop
                 self._phase = _SETTLING
-                if received.route_urn.startswith(_UNSUPPORTED_SCHEMA_ROUTE_PREFIX):
-                    # A version-gated delivery never takes the generic retry path, even when its
-                    # dead-letter write fails: a retry re-publish may rebuild the version header
-                    # from the (valid) body and walk the message past the gate into a handler.
-                    # It is left unacked; the broker (or a rewind, for cumulative-offset brokers
-                    # such as Kafka) brings it back and the gate rejects it again.
+                if poison_cause is not None or received.route_urn.startswith(
+                    _UNSUPPORTED_SCHEMA_ROUTE_PREFIX
+                ):
+                    # A version-gated or poison delivery never takes the generic retry path, even
+                    # when its dead-letter write fails: a retry re-publish may rebuild the version
+                    # header from the (valid) body and walk the message past the gate into a
+                    # handler, and a poison body has no envelope to retry. It is left unacked; the
+                    # broker (or a rewind, for cumulative-offset brokers such as Kafka) brings it
+                    # back and the same decision is taken again.
                     logger.error(
-                        "Dead-lettering a version-gated message (route %r) on queue %r failed; it was NOT "
-                        "retried or acknowledged; it will come back and the dead-letter write will be retried.",
+                        "Dead-lettering a version-gated or poison message (route %r) on queue %r "
+                        "failed; it was NOT retried or acknowledged; it will come back and the "
+                        "dead-letter write will be retried.",
                         received.route_urn,
                         received.queue,
                         exc_info=exc,
@@ -579,13 +605,16 @@ class BabelQueue:
         through a tier. That only happens once a handler fails, so it is a warning (not a
         start-up error).
         """
-        tiers = getattr(self.transport, "retry_tiers_ms", None)
-        if isinstance(tiers, list) and not tiers and self.max_attempts > 1 and isinstance(self.transport, Redeliverer):
+        if self._retry_tiers_unavailable() and self.max_attempts > 1:
             logger.warning(
                 "The Kafka transport has no retry tiers but max_attempts=%d: a failing handler "
                 "cannot be retried -- the record is left uncommitted and redelivered after a "
                 "pause, so it keeps failing in place. Configure retry_tiers_ms (or set "
-                "max_attempts=1) to retry through the tiers.",
+                "max_attempts=1) to retry through the tiers. A poison record (a body that does "
+                "not decode, or has no identity) is different: with dead-lettering off and "
+                "on_unknown_urn not 'delete' it is committed and dropped with a logged warning "
+                "(broker-bindings section 6.5, degrade-to-delete), so it never blocks the "
+                "partition; enable dead_letter to keep it in <topic>.dlq instead.",
                 self.max_attempts,
             )
 
@@ -657,6 +686,19 @@ class BabelQueue:
         if self.dead_letter_enabled or self.on_unknown_urn == UnknownUrnStrategy.DEAD_LETTER:
             self._settle_version_gated(received, always=True)
             return
+        if self._retry_tiers_unavailable():
+            # broker-bindings §6.5 / ADR-0036 decision 5: no retry tier, no dead-lettering and not
+            # ``delete`` degrades to ``delete``. The bounded retry of a poison record needs a tier
+            # and would raise, leaving the record uncommitted with ``bq-attempts`` never advancing,
+            # so the partition would block. Commit, drop, and say so.
+            logger.warning(
+                "Dropping a poison record on queue %r (route %r): the Kafka transport has no retry "
+                "tiers and dead-lettering is off, so it can be neither retried nor dead-lettered "
+                "(broker-bindings section 6.5, degrade-to-delete). The offset was committed.",
+                received.queue, str(route),
+            )
+            self._ack_settled(received)
+            return
         attempts = int(getattr(route, "attempts", 0) or 0)
         self._retry_or_dead_letter(
             received,
@@ -664,8 +706,60 @@ class BabelQueue:
             UnknownUrnError("The record is poison: it carries no decodable envelope."),
         )
 
-    def _settle_transport_poison(self, received: ReceivedMessage) -> None:
-        """Dead-letter a Redis poison reservation (broker-bindings §1.1 / §1.3, ADR-0033, ADR-0035).
+    def _route_body_poison(self, received: ReceivedMessage, cause: str) -> None:
+        """Settle a body that does not decode into an envelope, or has no identity, on SQS, AMQP,
+        Pulsar, Azure Service Bus, Artemis, the memory transport and any other transport that does
+        not classify poison itself (ADR-0036 decisions 2, 4 and 6; Go ``routePoison``).
+
+        The body is never handed to a handler and never takes the unknown-URN strategies:
+
+        * ``on_unknown_urn=DELETE`` drops it (acknowledged and logged);
+        * with the DLQ on (``dead_letter`` or the DEAD_LETTER strategy) it is written to the DLQ as
+          the ADR-0035 record with ``reason: "poison"`` (original body preserved) and acknowledged
+          after the write, with no retry;
+        * with the DLQ off it takes the bounded FAIL path (``RELEASE`` is replaced by it, a poison
+          body would otherwise be released forever): re-published with ``attempts + 1`` up to
+          ``max_attempts``, then dropped. The count is the larger of the transport's own delivery
+          count and the ``attempts`` of the bare body a previous retry left behind.
+
+        A failed DLQ write propagates: :meth:`dispatch` leaves the delivery unacknowledged."""
+        if self.on_unknown_urn == UnknownUrnStrategy.DELETE:
+            logger.warning(
+                "Dropping a poison message on queue %r (on_unknown_urn=delete): %s.",
+                received.queue, cause,
+            )
+            self._ack_settled(received)
+            return
+        attempts = self._poison_attempts(received)
+        if self.dead_letter_enabled or self.on_unknown_urn == UnknownUrnStrategy.DEAD_LETTER:
+            self._settle_transport_poison(received, cause, attempts)
+            return
+        self._retry_or_dead_letter(
+            received,
+            {"attempts": attempts},
+            UnknownUrnError("The message is poison: " + cause),
+        )
+
+    @staticmethod
+    def _poison_attempts(received: ReceivedMessage) -> int:
+        """Deliveries a poison body has already had: the larger of the transport's own count and
+        the integer ``attempts`` of the body (a bare ``{"attempts": N}`` left by a retry)."""
+        body_attempts = parse_envelope(received.body).get("attempts")
+        if isinstance(body_attempts, bool) or not isinstance(body_attempts, int):
+            body_attempts = 0
+        return max(int(received.native_attempts or 0), body_attempts, 0)
+
+    def _retry_tiers_unavailable(self) -> bool:
+        """Whether the transport is a Kafka transport with no retry tiers configured."""
+        tiers = getattr(self.transport, "retry_tiers_ms", None)
+        return isinstance(tiers, list) and not tiers and isinstance(self.transport, Redeliverer)
+
+    def _settle_transport_poison(
+        self, received: ReceivedMessage, cause: Optional[str] = None, attempts: int = 0
+    ) -> None:
+        """Dead-letter a poison delivery (broker-bindings §1.1 / §1.3, ADR-0033, ADR-0035): a Redis
+        poison reservation (unconditional) or, via :meth:`_route_body_poison`, a poison body of any
+        other transport (``cause`` and ``attempts`` are then known).
 
         The DLQ write is an unconditional MUST, so it ignores ``dead_letter_enabled``. The record
         follows ADR-0035 like every other poison record: a strict JSON object body is spliced byte
@@ -676,8 +770,8 @@ class BabelQueue:
             {},
             "poison",
             received.queue,
-            0,
-            error="Undecodable BabelQueue body",
+            attempts,
+            error="Undecodable BabelQueue body" + (": " + cause if cause else ""),
             exception=BabelQueueError.__name__,
         )["dead_letter"]
         self._write_dead_letter(
@@ -730,7 +824,12 @@ class BabelQueue:
             self._dead_letter(received, envelope, reason, exc)
             return
 
-        # Retries exhausted, no DLQ configured — drop it (ack so it leaves the queue).
+        # Retries exhausted, no DLQ configured — drop it (ack so it leaves the queue), loudly.
+        logger.warning(
+            "Dropping a message on queue %r after %d attempts: retries are exhausted and no "
+            "dead-letter queue is configured.",
+            received.queue, attempts,
+        )
         self._ack_settled(received)
 
     def _dead_letter(

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/artemis_transport.py

@@ -233,7 +233,8 @@ class ArtemisTransport(Transport):
     def _payload(message: Any) -> str:
         body = getattr(message, "body", None)
         if isinstance(body, bytes):
-            return body.decode("utf-8")
+            # Lossless: see ``PulsarTransport._payload`` (ADR-0035 poison record, no crash).
+            return body.decode("utf-8", "surrogateescape")
         return str(body) if body is not None else ""
 
     # -- Transport ----------------------------------------------------------

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/codec.py

@@ -16,7 +16,7 @@ import time
 import uuid
 from typing import Any, Dict, List, Mapping, Optional, Tuple
 
-from ._envelope_shape import CORE_META_KEYS
+from ._envelope_shape import CORE_META_KEYS, envelope_violation
 from .exceptions import BabelQueueError, EnvelopeKeyError
 
 SCHEMA_VERSION = 1
@@ -89,6 +89,15 @@ def has_forbidden_keys(envelope: Mapping[str, Any]) -> bool:
     return _has_forbidden_keys(envelope)
 
 
+def has_invalid_utf8(text: str) -> bool:
+    """Whether ``text`` carries a byte that was not valid UTF-8 when the transport read it.
+
+    Transports read a body losslessly (``errors="surrogateescape"``), so every invalid byte
+    appears as a lone surrogate in ``U+DC80..U+DCFF``. A ``\\udcff`` *escape* in the JSON text is
+    plain ASCII and is not flagged."""
+    return any("\udc80" <= ch <= "\udcff" for ch in text)
+
+
 def _is_valid_urn(value: Any) -> bool:
     """A usable identity: a string that is not blank under the ASCII set (ADR-0032 §5/§6).
 
@@ -299,6 +308,42 @@ class EnvelopeCodec:
         parsed = _parse_object(raw)
         return {} if parsed is None else _fold_identity(parsed)
 
+    @staticmethod
+    def poison_cause(raw: str, *, require_identity: bool = True) -> Optional[str]:
+        """Why ``raw`` is a **poison** body, or ``None`` when it decodes into an envelope with an
+        identity (ADR-0036 decision 4, conformance ``identity_verdict``).
+
+        A body is poison when it is not exactly one JSON object (malformed, BOM, trailing tokens,
+        ``NaN`` / ``Infinity``, a bare ``null``, an array, a scalar), when a known member has the
+        wrong type (the trigger set of :func:`~babelqueue._envelope_shape.envelope_violation`, the
+        Go reference's typed ``Decode``), when the core parser cannot read it, or, with
+        ``require_identity``, when it has neither a valid string ``job`` nor a valid string
+        ``urn`` (ADR-0032 section 6: a ``job`` that is not a string is ignored, a blank one falls
+        back to ``urn``, a ``urn`` that is not a non-blank string is ignored; the last value of a
+        repeated member wins). A poison body is never handed to a handler and never takes the
+        unknown-URN strategies, which are for a valid envelope whose URN has no handler.
+
+        ``require_identity=False`` keeps only the structural checks, for a delivery that a
+        transport routes on an out-of-band identity property (the Pulsar ``bq-job``), which is
+        outside the identity rule. Pure: no warning is logged and nothing is mutated."""
+        if has_invalid_utf8(raw):
+            # error-handling.md section 7: a body that is not valid UTF-8 is poison even when its
+            # invalid bytes sit inside a JSON string, where ``json.loads`` would accept the lone
+            # surrogate they were escaped to. It must never reach a handler or be re-encoded.
+            return "the body is not valid UTF-8"
+        try:
+            violation = envelope_violation(raw)
+            if violation is not None:
+                return violation
+            parsed = json.loads(raw)
+        except (ValueError, RecursionError, TypeError) as exc:
+            return "the core decoder cannot parse the body: " + str(exc)
+        if not isinstance(parsed, dict):
+            return "the JSON root is not an object"
+        if require_identity and not (_is_valid_urn(parsed.get("job")) or _is_valid_urn(parsed.get("urn"))):
+            return "the body carries no valid 'job' or 'urn' identity (ADR-0032 section 6)"
+        return None
+
     @staticmethod
     def urn(envelope: Mapping[str, Any]) -> str:
         """The message URN: canonical ``job``, with ``urn`` accepted as an alias.

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/kafka_transport.py

@@ -28,8 +28,10 @@ spliced byte for byte, anything else is carried as ``{"raw": "<body>"}`` (base64
 ``raw_encoding: "base64"`` when it is not valid UTF-8), and a ``dead_letter`` block with
 ``reason: "poison"`` is added (the key is preserved; the DLQ is the configured one or ``<queue>.dlq``, which must exist). With
 the DLQ off it is NOT written to a DLQ (Go ``routePoison``): ``on_unknown_urn=DELETE`` drops it, any other
-strategy takes the bounded FAIL path (retried through the tiers up to ``max_attempts``, then dropped).
-The record is handed over raw under a
+strategy takes the bounded FAIL path (retried through the tiers up to ``max_attempts``, then dropped); with
+no retry tiers configured as well, that path has no tier to retry through, so the record is committed, dropped
+and logged (the §6.5 degrade-to-delete rule, ADR-0036 decision 5) instead of blocking the partition. The record is
+handed over raw under a
 ``PoisonRoute`` and wrapped by ``publish_settle`` only when it is written to a queue other than the one
 it was read from, so a ``redrive`` restore into the DLQ it popped it from (any DLQ name) is byte for
 byte and never double-wraps. The version gate (§6.7) still
@@ -161,6 +163,10 @@ def _brokers_from_url(url: str) -> str:
 
 
 class KafkaTransport(Transport):
+    #: Classifies poison itself (``_poll_one``, ADR-0034 section 10): the runtime does not classify
+    #: the body a second time.
+    classifies_poison = True
+
     def __init__(
         self,
         url: str = "kafka://localhost:9092",

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/pika_transport.py

@@ -115,7 +115,9 @@ class PikaTransport(Transport):
             if timeout and timeout > 0:
                 self._connection.sleep(timeout)
             return None
-        text = body.decode("utf-8") if isinstance(body, (bytes, bytearray)) else str(body)
+        # Lossless: bytes that are not UTF-8 stay recoverable (``surrogateescape``) so a poison
+        # body is dead-lettered as the ADR-0035 base64 record instead of crashing the consumer.
+        text = body.decode("utf-8", "surrogateescape") if isinstance(body, (bytes, bytearray)) else str(body)
         headers = _delivery_headers(props)
         return ReceivedMessage(body=text, queue=queue, handle=method.delivery_tag, headers=headers)
 

{babelqueue-1.22.0 → babelqueue-1.23.0}/src/babelqueue/pulsar_transport.py

@@ -230,6 +230,17 @@ class PulsarTransport(Transport):
         value = int(raw)
         return value if value <= _MAX_ATTEMPTS else None
 
+    @staticmethod
+    def _native_attempts(redelivery_count: Any, properties: Optional[Dict[str, str]] = None) -> int:
+        """``max(bq-attempts, RedeliveryCount)`` (§5.5; 0-based): the reconciled count of a delivery
+        that has no decodable body attempts. An invalid or absent ``bq-attempts`` counts as 0."""
+        from_property = PulsarTransport._parse_attempts((properties or {}).get("bq-attempts")) or 0
+        try:
+            native = int(redelivery_count)
+        except (TypeError, ValueError):
+            native = 0
+        return max(from_property, native, 0)
+
     @staticmethod
     def _reconcile(
         body: str,
@@ -320,7 +331,9 @@ class PulsarTransport(Transport):
         getter = getattr(message, "data", None)
         data = getter() if callable(getter) else getter
         if isinstance(data, bytes):
-            return data.decode("utf-8")
+            # Lossless: bytes that are not UTF-8 stay recoverable (``surrogateescape``) so a poison
+            # body is dead-lettered as the ADR-0035 base64 record instead of crashing the consumer.
+            return data.decode("utf-8", "surrogateescape")
         return str(data) if data is not None else ""
 
     # -- Transport ----------------------------------------------------------
@@ -370,13 +383,33 @@ class PulsarTransport(Transport):
             properties,
             self._on_routing_mismatch,
         )
-        return ReceivedMessage(body=body, queue=queue, handle=message, route_urn=route_urn)
+        return ReceivedMessage(
+            body=body,
+            queue=queue,
+            handle=message,
+            route_urn=route_urn,
+            # The §5.5 reconciled count without a body: what a poison body (no envelope to carry
+            # ``attempts``) is counted by (ADR-0036 decision 2).
+            native_attempts=self._native_attempts(self._redelivery_count(message), properties),
+        )
 
     def ack(self, message: ReceivedMessage) -> None:
         if message.handle is None:
             return
         self._consumer(message.queue).acknowledge(message.handle)
 
+    def rewind(self, message: ReceivedMessage) -> None:
+        """:class:`~babelqueue.transport.Rewinder`: negatively acknowledge ``message`` so the broker
+        redelivers it (Go ``Rewind`` = ``Nack``; ADR-0036 decision 1: a failed DLQ write or retry
+        publish leaves the original un-acknowledged for redelivery instead of leaving it unsettled
+        for the life of the consumer). On a ``Shared`` subscription the native redelivery count
+        advances, which the §5.5 reconciliation reads back; on ``Exclusive`` / ``Failover`` it does
+        not, which is harmless: the attempt limit is carried by the ``bq-attempts`` property and
+        the body, never by the nack."""
+        if message.handle is None:
+            return
+        self._consumer(message.queue).negative_acknowledge(message.handle)
+
     def close(self) -> None:  # pragma: no cover - resource cleanup
         for resource in (*self._producers.values(), *self._consumers.values()):
             try: