babelqueue 1.18.0__tar.gz → 1.20.0__tar.gz

{babelqueue-1.18.0 → babelqueue-1.20.0}/.github/workflows/ci.yml

@@ -166,7 +166,7 @@ jobs:
       - uses: actions/checkout@v7
       - name: Verify vendored conformance matches the canonical suite
         run: |
-          git clone --depth 1 --branch v1.2.0 https://github.com/BabelQueue/conformance.git "$RUNNER_TEMP/conformance"
+          git clone --depth 1 --branch v1.4.0 https://github.com/BabelQueue/conformance.git "$RUNNER_TEMP/conformance"
           diff -u "$RUNNER_TEMP/conformance/CONFORMANCE_VERSION" "tests/conformance/CONFORMANCE_VERSION"
           diff -ru "$RUNNER_TEMP/conformance/manifest.json" "tests/conformance/manifest.json"
           diff -ru "$RUNNER_TEMP/conformance/fixtures"      "tests/conformance/fixtures"

{babelqueue-1.18.0 → babelqueue-1.20.0}/CHANGELOG.md

@@ -9,6 +9,87 @@ The envelope wire format is versioned separately by `meta.schema_version`
 
 ## [Unreleased]
 
+## [1.20.0] - 2026-10-17
+
+Conformance suite 1.4.0, [ADR-0031](https://babelqueue.com). The envelope is unchanged
+(`schema_version: 1`); this is a MINOR with one deliberate behaviour change (encode rejects).
+
+### Added
+- `EnvelopeCodec.make(urn, data, *, queue, trace_id, meta=None)`: a producer `meta` write API. The entries
+  land in `meta` next to the core keys (recommended shape `meta.ext.<vendor>`). A collision with a core
+  meta key or a forbidden key raises the new `EnvelopeKeyError` (subclass of `BabelQueueError`, with a
+  `pointer` attribute). `_envelope_shape.CORE_META_KEYS` is the single copy of the core key list.
+- `decode` folds the `urn` alias into `job` (ADR-0031 (c)): a `job` that is missing, empty or only ASCII
+  whitespace (`" \t\n\v\f\r"`; a U+2003 `job` is NOT blank) falls back to `urn` silently; when both carry
+  a value, `job` wins and `urn` is dropped with a warning naming `/urn`. `accepts()` / `urn()` follow the
+  same rule. The decoded envelope never keeps a `urn` key.
+- Tests: the runner now implements the 1.4.0 fields (`expect_preserved` on `forbidden_keys`;
+  `expect_absent_after_reencode`, `apply_dead_letter`, `expect_dead_letter` on `roundtrip`, the latter via
+  `dead_letter.annotate`) plus the `extension_write` and `encode_reject` sections, with a runner
+  self-test proving violations are reported.
+
+### Changed
+- **Behaviour change: `EnvelopeCodec.encode` raises `EnvelopeKeyError`** for an envelope carrying a
+  forbidden key (message-envelope.md section 10) or a top-level `urn`, instead of dropping it with a
+  warning (policy K-15 (a)). Nothing is written. `decode` still warns and drops, so re-encoding a decoded
+  envelope is safe. Code that built an envelope from a raw legacy body and relied on encode to clean it
+  must decode it first (`EnvelopeCodec.encode(EnvelopeCodec.decode(raw))`).
+- `test_all_five_contract_keys_are_covered` is a subset check (the suite adds `/urn` cases).
+- Docs only: the note in the previous release that the Go reference's case-insensitive envelope-key read "is tightened separately" is now closed — Go core 1.14.0 reads envelope keys exactly (case-sensitive, last duplicate wins), so every SDK now decides poison on the same exact-case keys.
+
+### Fixed
+- Raw-body re-encode paths now decode before they encode, so the new `encode` rejection cannot strand a
+  legacy body: the runtime's raw-body RELEASE re-publish (`app.py`), and the SQS and Pulsar attempts
+  reconciliation (Kafka, Azure Service Bus and Artemis already decoded first). A body with a legacy
+  `timestamp` or only a `urn` alias is re-emitted normalised (forbidden key dropped and warned, `urn`
+  folded into `job`); unknown top-level and `meta` keys, including `meta.ext`, are preserved.
+
+## [1.19.0] - 2026-10-12
+
+### Changed
+- Tests only: the vendored conformance suite is now 1.3.0 (exact-case envelope keys, last-wins duplicates: new `invalid-wrong-case-job`, `wrong-case-keys-roundtrip` and `duplicate-keys-last-wins` fixtures). No production change; the Python core already behaved this way.
+- **Kafka: a poison record is dead-lettered wrapped (`dead_letter.reason: "poison"`), like the Go
+  reference (§6.5, ADR-0009).** A value that is not valid UTF-8, not valid JSON, not a JSON object, or has
+  a member of the wrong type (the exact set is listed in the next entry) used to reach `<topic>.dlq` as its raw
+  bytes. It is now written as a body that keeps its keys when it is exactly one JSON object, otherwise
+  `{"raw": "<body>"}` (bytes that are not UTF-8 are stored with U+FFFD substitutions), plus a
+  `dead_letter` block (`reason: "poison"`, `original_queue`, `attempts` from the `bq-attempts` header, `lang`);
+  the record key is preserved. A poison record is still never retried and is dropped when no DLQ is
+  configured (the `on_unknown_urn` strategy, `DELETE` included, no longer matters: see below). The version-gate rejection (`bq-schema-version`, §6.7) still
+  dead-letters its record raw. The wrap is applied when the runtime writes the DLQ record (not when the
+  record is read), so a poison record that `redrive` restores into the DLQ it popped it from stays byte
+  for byte, whatever that DLQ is named. `original_queue` is the `bq-original-topic` header when present
+  (else the consumed topic), like the Java, .NET and Node consumers; a body whose numbers cannot be
+  written back as valid JSON (e.g. `1e400`) is carried as `{"raw": ...}` instead of producing an invalid DLQ body.
+  A body that decodes but names no URN is not poison (unknown-URN path, as before). Consumers of the DLQ
+  that expected the raw bytes of a malformed record must now read the wrapped body.
+- **Kafka: the poison trigger set now equals what the Go reference's typed `Decode` rejects.** Behaviour
+  change; the old Python-only rules (a lenient `attempts` read, `Accepts`-style conformance checks) are gone.
+  A record is poison when it is not valid UTF-8, not valid JSON (an empty or whitespace-only body, a BOM,
+  trailing data, leading zeros, `NaN` / `Infinity`, single quotes), or not a JSON object (a bare `null` is
+  not poison: it is an empty envelope and takes the unknown-URN path), or when a member has the wrong type:
+  `job`, `trace_id`, `meta.id|queue|lang` and `dead_letter.reason|error|exception|original_queue|lang`
+  must be a string or null; `meta.schema_version`, `meta.created_at`, `attempts` and
+  `dead_letter.failed_at|attempts` must be an int64 integer literal or null (a string such as `"3"` or
+  `"2026"`, a fraction or exponent such as `3.0` / `1e2`, a bool, an array and an object are poison);
+  `data`, `meta` and `dead_letter` must be an object or null; every number inside `data` must be a finite
+  double. Member names are matched exactly as spelled (case-sensitive, as the contract schema and the core decoder read them; a `JOB` or `Meta` member is an unknown member, not a typed one) and every duplicate key is checked. A record Go decodes
+  is no longer poison, even when it is not a conformant envelope: a non-string `urn` alias, an absent
+  `data`, `"attempts": null` (read as 0 and written back as 0) and `meta.schema_version: 2` are routed by URN
+  like any other record (Go's consume path never calls `Accepts`). The Kafka transport no longer
+  delivers records under the shared non-numeric-`attempts` route; they are plain poison now.
+  `redrive` restores a poison-routed record into the DLQ unchanged instead of replaying or re-wrapping it,
+  even when its `job` still reads as a URN. The only Python-specific limit is JSON nested deeper than
+  the interpreter's recursion limit (about 1000 levels, Go allows 10000), which is treated as poison.
+- Known, intentional differences from the Go reference for a poison record: `dead_letter.attempts` is the `bq-attempts` header (Go writes 0) and `original_queue` is the `bq-original-topic` header when present (Go writes the consumed topic); a float literal such as `1.0` / `1e2` is re-formatted when the body is re-encoded. The wire envelope and the `dead_letter` shape are unchanged.
+- **Kafka: any decode failure is poison; the partition is never left to wedge.** Member names are matched case-sensitively, exactly as the core decoder reads them (no `encoding/json`-style folding, no `str.casefold()`); the Go reference's case-insensitive read is a known difference that contradicts the contract schema and is tightened separately. Whatever `EnvelopeCodec.decode` raises after the shape scan (e.g. `RecursionError` at the interpreter's limit) is now routed as poison instead of escaping `pop` (which left the offset uncommitted and the partition redelivered forever); the poison wrapper falls back to `{"raw": ...}` on `RecursionError` or a body that cannot be written as UTF-8. A lone surrogate escape (`"\ud800"`) that survives into a retry/DLQ write is substituted by U+FFFD, as Go does, instead of raising `UnicodeEncodeError`. Invalid UTF-8 remains poison (Go substitutes U+FFFD and accepts it). The wire envelope and the `dead_letter` shape are unchanged.
+- **Kafka: a body the core JSON parser refuses is poison, not an empty envelope.** The core decoder turns a `ValueError` from `json.loads` into an empty envelope; where the shape scan accepts a body that plain `json.loads` rejects (Python 3.11+: an unknown member holding an integer of more than 4300 digits), the record would have reached a handler with its content dropped. It is now dead-lettered as poison. No change to the wire envelope or the `dead_letter` shape.
+- **Kafka: a poison record is dead-lettered whatever `on_unknown_urn` is.** The runtime checked the `DELETE` strategy before it recognised a poison record, so with `on_unknown_urn="delete"` a poison record was acknowledged and silently dropped even with a DLQ configured, where the Go reference writes it to `<topic>.dlq` with `dead_letter.reason: "poison"` and commits. A poison delivery (`PoisonRoute`) is now settled before the strategy is consulted: written wrapped to the DLQ and committed under `fail`, `delete`, `dead_letter` and `release`, and dropped only when no DLQ is configured (unchanged). The version-gate path and every other transport keep their behaviour. No change to the wire envelope or the `dead_letter` shape.
+
+### Fixed
+- Kafka: a wrongly-typed `meta` or `meta.created_at` in a wrapped poison body no longer fails the write
+  that settles it (header projection and record timestamp tolerate it).
+
 ## [1.18.0] - 2026-10-11
 
 ### Added

{babelqueue-1.18.0 → babelqueue-1.20.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: babelqueue
-Version: 1.18.0
+Version: 1.20.0
 Summary: Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers.
 Project-URL: Homepage, https://babelqueue.com
 Project-URL: Source, https://github.com/BabelQueue/babelqueue-python
@@ -106,6 +106,29 @@ The envelope is identical to every other SDK's:
 }
 ```
 
+### Extension metadata, forbidden keys and the `urn` alias (1.20.0, ADR-0031)
+
+```python
+# Producer meta: extra keys next to the core meta keys (recommended shape: meta.ext.<vendor>).
+envelope = EnvelopeCodec.make(
+    "urn:babel:orders:created", {"order_id": 1042},
+    queue="orders", meta={"ext": {"acme": {"tier": "gold"}}},
+)
+```
+
+- `meta=` is keyword-only. A key that collides with a core meta key (`id`, `queue`, `lang`,
+  `schema_version`, `created_at`) or with a forbidden key (`max_retries`, `attempts`, `source`, `ts`)
+  raises `EnvelopeKeyError` (a `BabelQueueError`; its `pointer` names the key, e.g. `/meta/ts`).
+- Unknown top-level and `meta` keys of an inbound message survive a decode -> `attempts + 1` -> encode
+  hop (retry, redrive, dead-letter), so a vendor `meta.ext` block is never lost.
+- `EnvelopeCodec.encode` **rejects** an envelope that carries a forbidden key (top-level `timestamp`;
+  `meta.max_retries`, `meta.attempts`, `meta.source`, `meta.ts`) or the inbound-only top-level `urn`
+  alias: it raises `EnvelopeKeyError` instead of dropping the key with a warning. `decode` is unchanged
+  (it still warns and drops), so a decoded envelope can always be re-encoded.
+- `decode` folds `urn` into `job`: a `job` that is missing, empty or only ASCII whitespace (space, `\t`,
+  `\n`, `\v`, `\f`, `\r`) falls back to `urn`; when both carry a value, `job` wins and `urn` is dropped
+  with a warning naming `/urn`. The decoded envelope never keeps a `urn` key.
+
 ### Typed messages (optional)
 
 ```python

{babelqueue-1.18.0 → babelqueue-1.20.0}/README.md

@@ -52,6 +52,29 @@ The envelope is identical to every other SDK's:
 }
 ```
 
+### Extension metadata, forbidden keys and the `urn` alias (1.20.0, ADR-0031)
+
+```python
+# Producer meta: extra keys next to the core meta keys (recommended shape: meta.ext.<vendor>).
+envelope = EnvelopeCodec.make(
+    "urn:babel:orders:created", {"order_id": 1042},
+    queue="orders", meta={"ext": {"acme": {"tier": "gold"}}},
+)
+```
+
+- `meta=` is keyword-only. A key that collides with a core meta key (`id`, `queue`, `lang`,
+  `schema_version`, `created_at`) or with a forbidden key (`max_retries`, `attempts`, `source`, `ts`)
+  raises `EnvelopeKeyError` (a `BabelQueueError`; its `pointer` names the key, e.g. `/meta/ts`).
+- Unknown top-level and `meta` keys of an inbound message survive a decode -> `attempts + 1` -> encode
+  hop (retry, redrive, dead-letter), so a vendor `meta.ext` block is never lost.
+- `EnvelopeCodec.encode` **rejects** an envelope that carries a forbidden key (top-level `timestamp`;
+  `meta.max_retries`, `meta.attempts`, `meta.source`, `meta.ts`) or the inbound-only top-level `urn`
+  alias: it raises `EnvelopeKeyError` instead of dropping the key with a warning. `decode` is unchanged
+  (it still warns and drops), so a decoded envelope can always be re-encoded.
+- `decode` folds `urn` into `job`: a `job` that is missing, empty or only ASCII whitespace (space, `\t`,
+  `\n`, `\v`, `\f`, `\r`) falls back to `urn`; when both carry a value, `job` wins and `urn` is dropped
+  with a warning naming `/urn`. The decoded envelope never keeps a `urn` key.
+
 ### Typed messages (optional)
 
 ```python

{babelqueue-1.18.0 → babelqueue-1.20.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "babelqueue"
-version = "1.18.0"
+version = "1.20.0"
 description = "Polyglot Queues, Simplified — the Python core: the canonical BabelQueue wire-envelope codec, contracts and dead-letter helpers."
 readme = "README.md"
 requires-python = ">=3.9"

{babelqueue-1.18.0 → babelqueue-1.20.0}/src/babelqueue/__init__.py

@@ -41,7 +41,7 @@ from .transport import (
     Transport,
 )
 
-__version__ = "1.18.0"
+__version__ = "1.20.0"
 
 __all__ = [
     "BabelQueue",

babelqueue-1.20.0/src/babelqueue/_envelope_shape.py

@@ -0,0 +1,156 @@
+"""The poison trigger set of the Kafka transport (§6.5).
+
+A body is undecodable when a strict, typed decode into ``Envelope`` / ``Meta`` / ``DeadLetter`` (the
+Go reference's ``babelqueue.Decode``, except that member names are case-sensitive here) rejects it:
+
+* the body must be exactly one JSON value (no BOM, trailing tokens, ``NaN`` / ``Infinity``, malformed
+  numbers); a bare ``null`` decodes to an empty envelope, any other non-object root is rejected;
+* ``job``, ``trace_id``, ``meta.id|queue|lang`` and ``dead_letter.reason|error|exception|original_queue|lang``
+  are string-or-null;
+* ``attempts``, ``meta.schema_version|created_at`` and ``dead_letter.failed_at|attempts`` are
+  integer-or-null: an integer literal (no fraction or exponent) that fits ``int64``;
+* ``data``, ``meta`` and ``dead_letter`` are object-or-null, and a number inside ``data`` must be a
+  finite double;
+* member names are matched exactly as spelled (case-sensitive, as the contract schema and the core
+  decoder read them) and every occurrence of a duplicated key is checked; any other member (a
+  differently-cased name or a non-string ``urn`` alias included) is only checked for syntax.
+
+A well-typed but non-conformant envelope is not poison: Go's consume path never calls ``Accepts``.
+"""
+
+import json
+import math
+from typing import Any, Dict, List, Optional, Tuple
+
+_INT64_MIN = -(2**63)
+_INT64_MAX = 2**63 - 1
+#: Beyond this many digits an integer literal cannot fit an int64 or a double; it is kept as a
+#: sentinel so ``json`` never trips the interpreter's integer-string-length limit.
+_MAX_INT_DIGITS = 4000
+
+
+class _Pairs(List[Tuple[str, Any]]):
+    """A decoded JSON object as its ``(key, value)`` list, so duplicate keys survive."""
+
+
+class _HugeInt:
+    """An integer literal too large for any int64 or double."""
+
+
+_HUGE = _HugeInt()
+
+_STRING = "string"
+_INTEGER = "integer"
+_OBJECT = "object"
+_DATA = "data"
+
+_META = {
+    "id": _STRING,
+    "queue": _STRING,
+    "lang": _STRING,
+    "schema_version": _INTEGER,
+    "created_at": _INTEGER,
+}
+#: The core ``meta`` keys of message-envelope.md section 4 (the single copy; the producer ``meta``
+#: write API of the codec rejects a collision with any of them, ADR-0031 (b)).
+CORE_META_KEYS = frozenset(_META)
+_DEAD_LETTER = {
+    "reason": _STRING,
+    "error": _STRING,
+    "exception": _STRING,
+    "original_queue": _STRING,
+    "lang": _STRING,
+    "failed_at": _INTEGER,
+    "attempts": _INTEGER,
+}
+_TOP = {
+    "job": _STRING,
+    "trace_id": _STRING,
+    "attempts": _INTEGER,
+    "data": _DATA,
+    "meta": _OBJECT,
+    "dead_letter": _OBJECT,
+}
+
+
+def _reject_constant(name: str) -> Any:
+    raise ValueError("invalid JSON constant " + name)
+
+
+def _parse_int(literal: str) -> Any:
+    return int(literal) if len(literal) <= _MAX_INT_DIGITS else _HUGE
+
+
+def _is_finite_number(value: Any) -> bool:
+    if value is _HUGE:
+        return False
+    if isinstance(value, float):
+        return math.isfinite(value)
+    if isinstance(value, int) and not isinstance(value, bool):
+        try:
+            return math.isfinite(float(value))
+        except OverflowError:
+            return False
+    return True
+
+
+def _all_numbers_finite(root: Any) -> bool:
+    stack = [root]
+    while stack:
+        value = stack.pop()
+        if isinstance(value, _Pairs):
+            stack.extend(item for _, item in value)
+        elif isinstance(value, list):
+            stack.extend(value)
+        elif not _is_finite_number(value):
+            return False
+    return True
+
+
+def _check_member(kind: str, name: str, value: Any) -> Optional[str]:
+    if value is None:
+        return None
+    if kind == _STRING:
+        return None if isinstance(value, str) else "member %r is not a string" % name
+    if kind == _INTEGER:
+        if type(value) is int and _INT64_MIN <= value <= _INT64_MAX:
+            return None
+        return "member %r is not an int64 integer" % name
+    if not isinstance(value, _Pairs):
+        return "member %r is not an object" % name
+    if kind == _DATA and not _all_numbers_finite(value):
+        return "member %r holds a number that is not a finite double" % name
+    return None
+
+
+def _check_object(pairs: _Pairs, kinds: Dict[str, str], prefix: str) -> Optional[str]:
+    for key, value in pairs:
+        kind = kinds.get(key)
+        if kind is None:
+            continue
+        failure = _check_member(kind, prefix + key, value)
+        if failure is not None:
+            return failure
+        if kind == _OBJECT and value is not None:
+            nested = _check_object(value, _META if key == "meta" else _DEAD_LETTER, key + ".")
+            if nested is not None:
+                return nested
+    return None
+
+
+def envelope_violation(text: str) -> Optional[str]:
+    """Why ``text`` is not a decodable BabelQueue body, or ``None`` when Go's typed decode accepts it."""
+    try:
+        parsed = json.loads(
+            text,
+            object_pairs_hook=_Pairs,
+            parse_int=_parse_int,
+            parse_constant=_reject_constant,
+        )
+    except (ValueError, RecursionError) as exc:
+        return "invalid JSON: " + str(exc)
+    if parsed is None:
+        return None  # a bare ``null`` is an empty envelope, not poison
+    if not isinstance(parsed, _Pairs):
+        return "the JSON root is not an object"
+    return _check_object(parsed, _TOP, "")

{babelqueue-1.18.0 → babelqueue-1.20.0}/src/babelqueue/app.py

@@ -41,6 +41,7 @@ from .replay import HEADER_REPLAY_BYPASS, _replay_scope
 from .routing import UnknownUrnStrategy
 from .transport import (
     HeaderPublisher,
+    PoisonRoute,
     ReceivedMessage,
     Redeliverer,
     Rewinder,
@@ -457,8 +458,9 @@ class BabelQueue:
         self._stop.wait(self._rewind_pause)
 
     def _release_unprocessed(self, received: ReceivedMessage) -> None:
-        """Release a message whose handler never completed (shutdown): unchanged body, no
-        backoff, and the runtime adds no ``attempts`` increment. A broker that counts
+        """Release a message whose handler never completed (shutdown): unchanged body
+        (normalised only if it carries a forbidden key or a ``urn`` alias), no backoff, and the
+        runtime adds no ``attempts`` increment. A broker that counts
         deliveries still counts it: on SQS the release bumps ``ApproximateReceiveCount``, so the
         redelivered envelope reads one more attempt (and a ``RedrivePolicy`` sees it too)."""
         self._release(received, received.body, 0.0)
@@ -476,6 +478,14 @@ class BabelQueue:
 
     def _route_unknown(self, urn: str, received: ReceivedMessage, envelope: Mapping[str, Any]) -> None:
         strategy = self.on_unknown_urn
+        # A Kafka poison record (§6.5: not valid UTF-8 / not a decodable body) is not an unknown URN: it
+        # never decoded into an envelope, so ``on_unknown_urn`` has no say. It is checked before the
+        # DELETE strategy and settled like the Go reference does -- dead-lettered (wrapped, reason
+        # ``poison``) and committed, or dropped only when no DLQ is configured. Other transports'
+        # version-gated deliveries keep their existing order below.
+        if isinstance(received.route_urn, PoisonRoute):
+            self._settle_version_gated(received)
+            return
         if strategy == UnknownUrnStrategy.DELETE:
             self._ack_settled(received)
             return
@@ -569,12 +579,15 @@ _UNSUPPORTED_SCHEMA_ROUTE_PREFIX = "babelqueue:unsupported-schema-version:"
 
 def _without_forbidden_keys(body: str) -> str:
     """``body`` unchanged, unless it carries a forbidden envelope key (message-envelope.md §10,
-    e.g. from a legacy producer) — then it is re-encoded without it (warned by the codec), so a
+    e.g. from a legacy producer) or a ``urn`` alias — then it is re-encoded without it (warned by the codec), so a
     raw-body re-publish never re-emits a key the decode boundary reported as dropped."""
     parsed = parse_envelope(body)
-    if not parsed or not has_forbidden_keys(parsed):
+    # ``urn`` is not a §10 key but ``encode`` rejects it and ADR-0031 (a)/(c) forbid re-emitting it.
+    if not parsed or not (has_forbidden_keys(parsed) or "urn" in parsed):
         return body
-    return EnvelopeCodec.encode(parsed)
+    # ``decode`` (not the raw parse) so the dropped keys — and a top-level ``urn`` alias, folded
+    # into ``job`` — are gone before ``encode``, which rejects them (ADR-0031 (c)).
+    return EnvelopeCodec.encode(EnvelopeCodec.decode(body))
 
 
 def _handler_wants_envelope(fn: Handler) -> bool:

{babelqueue-1.18.0 → babelqueue-1.20.0}/src/babelqueue/codec.py

@@ -16,15 +16,17 @@ import time
 import uuid
 from typing import Any, Dict, List, Mapping, Optional, Tuple
 
-from .exceptions import BabelQueueError
+from ._envelope_shape import CORE_META_KEYS
+from .exceptions import BabelQueueError, EnvelopeKeyError
 
 SCHEMA_VERSION = 1
 SOURCE_LANG = "python"
 
 #: Non-canonical keys (message-envelope.md §10), as ``(container, key)`` pairs where the
 #: container is ``None`` for the top level or ``"meta"``. Policy K-15: decode WARNS and drops
-#: them (old producers keep working); encode NEVER emits them. Encode currently drops them with
-#: a warning; K-15 (a) / R1-E0 will turn that into a rejection (an error) in a future MINOR.
+#: them (old producers keep working); encode NEVER emits them and REJECTS an envelope that
+#: carries one (ADR-0031 (c)). The top-level ``urn`` alias is not in this table: it is an inbound
+#: alias that decode folds into ``job`` (see :meth:`EnvelopeCodec.decode`), but encode rejects it.
 FORBIDDEN_KEYS: Tuple[Tuple[Optional[str], str], ...] = (
     (None, "timestamp"),
     ("meta", "max_retries"),
@@ -33,6 +35,10 @@ FORBIDDEN_KEYS: Tuple[Tuple[Optional[str], str], ...] = (
     ("meta", "ts"),
 )
 
+#: The closed ASCII whitespace set that makes a ``job`` "blank" (cross-SDK parity, GR-5): never a
+#: bare ``str.strip()``, which also strips Unicode spaces such as U+2003.
+_ASCII_WHITESPACE = " \t\n\v\f\r"
+
 #: Logger every forbidden-key warning is reported through (one record per dropped key).
 logger = logging.getLogger("babelqueue.codec")
 
@@ -52,11 +58,7 @@ def strip_forbidden_keys(envelope: Dict[str, Any]) -> List[str]:
         del target[key]
         pointer = "/" + key if container is None else f"/{container}/{key}"
         dropped.append(pointer)
-        logger.warning(
-            "Dropped forbidden envelope key %s from the envelope "
-            "(non-canonical, message-envelope.md §10).",
-            pointer,
-        )
+        _warn_dropped(pointer)
     return dropped
 
 
@@ -76,10 +78,42 @@ def parse_envelope(raw: Any) -> Dict[str, Any]:
 
 
 def has_forbidden_keys(envelope: Mapping[str, Any]) -> bool:
-    """Whether ``envelope`` carries any of the non-canonical keys of message-envelope.md §10."""
+    """Whether ``envelope`` carries any of the non-canonical keys of message-envelope.md §10.
+
+    The top-level ``urn`` alias is NOT included (it is not a §10 key: decode folds it into ``job``),
+    although ``encode`` also rejects it — a caller deciding whether a raw body needs normalising
+    before re-emit must check ``"urn" in envelope`` as well.
+    """
     return _has_forbidden_keys(envelope)
 
 
+def _is_blank(value: Any) -> bool:
+    """Missing, ``None``, empty or only ASCII whitespace; a falsy non-string (``0``, ``false``,
+    ``[]``, ``{}``) is blank too, as in ``job or urn``. A truthy non-string is not blank."""
+    if isinstance(value, str):
+        return value.strip(_ASCII_WHITESPACE) == ""
+    return not value
+
+
+def _warn_dropped(pointer: str) -> None:
+    logger.warning(
+        "Dropped forbidden envelope key %s from the envelope "
+        "(non-canonical, message-envelope.md §10).",
+        pointer,
+    )
+
+
+def _first_encode_violation(envelope: Mapping[str, Any]) -> Optional[str]:
+    """RFC 6901 pointer of the first key encode refuses to write, or ``None``."""
+    for container, key in FORBIDDEN_KEYS:
+        target = envelope if container is None else envelope.get(container)
+        if isinstance(target, Mapping) and key in target:
+            return "/" + key if container is None else f"/{container}/{key}"
+    if "urn" in envelope:
+        return "/urn"
+    return None
+
+
 def _has_forbidden_keys(envelope: Mapping[str, Any]) -> bool:
     for container, key in FORBIDDEN_KEYS:
         target = envelope if container is None else envelope.get(container)
@@ -101,12 +135,18 @@ class EnvelopeCodec:
         *,
         queue: str = "default",
         trace_id: Optional[str] = None,
+        meta: Optional[Mapping[str, Any]] = None,
     ) -> Dict[str, Any]:
         """Build the canonical envelope for a ``(urn, data)`` pair.
 
         ``trace_id`` is reused when given (trace continuation), otherwise a fresh
         UUID is minted. ``attempts`` starts at 0 and is the top-level transport
         counter (kept out of the immutable ``meta`` block).
+
+        ``meta`` (ADR-0031 (b)) adds producer-supplied entries (e.g. ``{"ext": {"acme": 1}}``) to
+        the ``meta`` block next to the core keys. An entry whose key is a core ``meta`` key
+        (``id``, ``queue``, ``lang``, ``schema_version``, ``created_at``) or a forbidden ``meta``
+        key (message-envelope.md §10) raises :class:`~babelqueue.exceptions.EnvelopeKeyError`.
         """
         urn = (urn or "").strip()
         if not urn:
@@ -117,19 +157,36 @@ class EnvelopeCodec:
 
         trace_id = (trace_id or "").strip() or str(uuid.uuid4())
 
-        return {
+        extra: Dict[str, Any] = {}
+        if meta is not None:
+            if not isinstance(meta, Mapping):
+                raise BabelQueueError("The producer meta must be a mapping of string keys to values.")
+            forbidden_meta = {key for container, key in FORBIDDEN_KEYS if container == "meta"}
+            for key, value in meta.items():
+                if key in CORE_META_KEYS or key in forbidden_meta:
+                    raise EnvelopeKeyError(
+                        f"/meta/{key}",
+                        f"The producer meta key /meta/{key} is reserved or forbidden "
+                        "(core meta keys and message-envelope.md §10 keys cannot be set).",
+                    )
+                extra[key] = value
+
+        meta_block: Dict[str, Any] = {
+            "id": str(uuid.uuid4()),
+            "queue": queue,
+            "lang": SOURCE_LANG,
+            "schema_version": SCHEMA_VERSION,
+            "created_at": int(time.time() * 1000),
+        }
+        meta_block.update(extra)
+        envelope: Dict[str, Any] = {
             "job": urn,
             "trace_id": trace_id,
             "data": dict(data),
-            "meta": {
-                "id": str(uuid.uuid4()),
-                "queue": queue,
-                "lang": SOURCE_LANG,
-                "schema_version": SCHEMA_VERSION,
-                "created_at": int(time.time() * 1000),
-            },
+            "meta": meta_block,
             "attempts": 0,
         }
+        return envelope
 
     @staticmethod
     def from_message(message: Any, queue: str = "default") -> Dict[str, Any]:
@@ -152,20 +209,20 @@ class EnvelopeCodec:
     def encode(envelope: Mapping[str, Any]) -> str:
         """Encode the envelope as compact UTF-8 JSON (unescaped unicode/slashes).
 
-        Forbidden keys (message-envelope.md §10) are never emitted: if the mapping carries any,
-        they are dropped (with a warning) from a copy — the caller's mapping is not mutated.
-
-        Transitional: K-15 (a) asks encode to *reject* forbidden keys. Dropping is the interim
-        behaviour until R1-E0, when this call will raise instead — do not rely on encode to
-        clean an envelope; strip it yourself (:func:`strip_forbidden_keys`) if you build one
-        from a legacy source.
+        Forbidden keys (message-envelope.md §10) are never emitted: an envelope that carries one
+        (top-level ``timestamp``; ``meta.max_retries`` / ``meta.attempts`` / ``meta.source`` /
+        ``meta.ts``) or the inbound-only top-level ``urn`` alias raises
+        :class:`~babelqueue.exceptions.EnvelopeKeyError` (ADR-0031 (c)) and nothing is written.
+        Envelopes produced by :meth:`decode` are always safe to re-encode: decode already drops
+        those keys. Unknown, non-forbidden keys are kept as-is.
         """
-        if _has_forbidden_keys(envelope):
-            copy = dict(envelope)
-            if isinstance(copy.get("meta"), Mapping):
-                copy["meta"] = dict(copy["meta"])
-            strip_forbidden_keys(copy)
-            envelope = copy
+        pointer = _first_encode_violation(envelope)
+        if pointer is not None:
+            raise EnvelopeKeyError(
+                pointer,
+                f"Refusing to encode the forbidden envelope key {pointer} "
+                "(non-canonical, message-envelope.md §10).",
+            )
         return json.dumps(envelope, ensure_ascii=False, separators=(",", ":"))
 
     @staticmethod
@@ -175,15 +232,33 @@ class EnvelopeCodec:
         Unknown keys are kept as-is (forward compatibility). Forbidden keys
         (message-envelope.md §10) are dropped and reported as warnings on :data:`logger`
         (policy K-15: warn, never reject), so re-encoding never re-emits them.
+
+        The ``urn`` alias is folded into ``job`` (ADR-0031 (c)): when ``job`` is missing, empty or
+        only ASCII whitespace, the ``urn`` value becomes ``job`` (silently); when both carry a
+        value, ``job`` wins and ``urn`` is dropped with a warning naming ``/urn``. The decoded
+        envelope never keeps a ``urn`` key.
         """
         decoded = parse_envelope(raw)
         strip_forbidden_keys(decoded)
+        if "urn" in decoded:
+            if _is_blank(decoded.get("job")):
+                decoded["job"] = decoded.pop("urn")
+            else:
+                del decoded["urn"]
+                _warn_dropped("/urn")
         return decoded
 
     @staticmethod
     def urn(envelope: Mapping[str, Any]) -> str:
-        """The message URN: canonical ``job``, with ``urn`` accepted as an alias."""
-        return str(envelope.get("job") or envelope.get("urn") or "")
+        """The message URN: canonical ``job``, with ``urn`` accepted as an alias.
+
+        A blank ``job`` (missing, empty or only ASCII whitespace) falls back to ``urn``.
+        """
+        job = envelope.get("job")
+        if isinstance(job, str) and not _is_blank(job):
+            return job
+        alias = envelope.get("urn")
+        return alias if isinstance(alias, str) and not _is_blank(alias) else ""
 
     @staticmethod
     def accepts(envelope: Mapping[str, Any]) -> bool:

{babelqueue-1.18.0 → babelqueue-1.20.0}/src/babelqueue/exceptions.py

@@ -7,6 +7,21 @@ class BabelQueueError(Exception):
     """Base for every recoverable BabelQueue error (bad config, empty URN, ...)."""
 
 
+class EnvelopeKeyError(BabelQueueError):
+    """An envelope key the codec refuses to write (ADR-0031 (b), (c)).
+
+    Raised by :meth:`~babelqueue.codec.EnvelopeCodec.make` when the ``meta`` entries collide with a
+    core ``meta`` key (``id``, ``queue``, ``lang``, ``schema_version``, ``created_at``) or carry a
+    forbidden key, and by :meth:`~babelqueue.codec.EnvelopeCodec.encode` when the envelope holds a
+    forbidden key of message-envelope.md section 10 (including the top-level ``urn`` alias).
+    ``pointer`` is the RFC 6901 pointer of the first offending key (e.g. ``"/meta/ts"``).
+    """
+
+    def __init__(self, pointer: str, message: str) -> None:
+        super().__init__(message)
+        self.pointer = pointer
+
+
 class UnknownUrnError(BabelQueueError):
     """A consumed message carries a URN with no mapped handler (strategy "fail")."""