redis-streams-fxmacrodata 0.1.0__tar.gz

redis_streams_fxmacrodata-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Robert Tidball
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

redis_streams_fxmacrodata-0.1.0/MANIFEST.in

@@ -0,0 +1,11 @@
+include LICENSE
+include README.md
+include pyproject.toml
+include PUBLIC_FILES.json
+include MANIFEST.in
+include src/fxmacrodata_redis_streams/__init__.py
+include src/fxmacrodata_redis_streams/py.typed
+include tests/test_publisher.py
+include examples/consumer_group_dedupe.py
+global-exclude *.pyc *.pyo
+global-exclude __pycache__/*

redis_streams_fxmacrodata-0.1.0/PKG-INFO

@@ -0,0 +1,124 @@
+Metadata-Version: 2.4
+Name: redis-streams-fxmacrodata
+Version: 0.1.0
+Summary: Publish public FXMacroData responses to Redis Streams
+License-Expression: MIT
+Project-URL: Documentation, https://fxmacrodata.com/documentation/reference?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=package
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fxmacrodata-platform-core==0.1.0
+Requires-Dist: redis<9,>=8.1.0
+Provides-Extra: test
+Requires-Dist: pytest<9,>=8; extra == "test"
+Dynamic: license-file
+
+# FXMacroData for Redis Streams
+
+Append FXMacroData public responses to a Redis Stream with the official
+`redis` Python client. Feed macro announcements to your own consumer groups
+while retaining the source payload and revision metadata.
+
+[FXMacroData documentation](https://fxmacrodata.com/documentation/reference?utm_source=github&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=readme)
+
+## Install and connect
+
+Requires Python 3.10 or later. Install this package and its adjacent public
+client dependencies from the distribution supplied to you:
+
+```sh
+python -m pip install /path/to/fxmacrodata-public-client /path/to/fxmacrodata-platform-core /path/to/redis-streams-fxmacrodata
+```
+
+Configure `REDIS_URL` (a `rediss://` URL with the credentials your
+administrator provides) in your runtime secret store. Then append to a
+customer-owned stream:
+
+```python
+import os
+import redis
+from fxmacrodata_platform_core import read
+from fxmacrodata_redis_streams import publish
+
+client = redis.Redis.from_url(os.environ["REDIS_URL"])
+try:
+    batch = read("indicator_history", {"currency": "usd", "indicator": "policy_rate"})
+    result = publish(batch, client, "fxmacrodata:usd", maxlen=100_000)
+    print(result.messages, "entries in", result.batches, "pipelines")
+finally:
+    client.close()
+```
+
+The destination is the stream key (letters, digits, `_`, `.`, `:`, `/` and
+`-`, up to 256 characters); `XADD` creates it on first use. URLs and other
+values are rejected before any command runs.
+
+Every operation exposed by `fxmacrodata_platform_core.operations()` is
+available, including REST and MCP operations. `publish_operation` reads one
+operation and publishes it in a single call:
+
+```python
+from fxmacrodata_redis_streams import publish_operation
+
+publish_operation("release_calendar", {"currency": "usd"}, client, "fxmacrodata:calendar",
+                  api_key_env="FXMD_API_KEY")
+```
+
+## Entry format
+
+Each envelope row becomes one stream entry whose fields are the envelope
+columns as strings: `operation`, `source_url`, `record_id`, `currency`,
+`indicator`, `date`, `announcement_datetime`, `value`, `value_text`,
+`record_json`, `response_json` and the publisher's `content_hash` (SHA-256 of
+the ten source columns). A null column is omitted from the entry; `value` is
+written as a Python float repr, such as `4.25`, and `value_text` keeps the
+source's own text. Dates and announcement times remain source strings. No
+publication time is inferred, and no missing value is filled. `record_json`
+retains the original record; `response_json` retains the complete response,
+including pagination and metadata. An empty response produces one entry with
+`record_json` equal to `null`, so absence of data and the server's metadata
+remain inspectable. It is not an observation.
+
+The `key` field is `operation:record_id:content_hash`; the same source
+record with the same content always produces the same key, and changed data
+or metadata produces a different one. Redis assigns the entry ID itself; no
+timestamp is derived from the data.
+
+## Delivery guarantee
+
+Delivery is at-least-once. Entries are appended in blocks of `batch_size`
+rows (1 to 10,000, 1,000 by default), one non-transactional pipeline per
+block. A failed block raises `PublishError` without the server's own
+diagnostic text and is not retried; earlier blocks may already be stored, and
+a partially executed pipeline may have appended some of its entries.
+Consumers should deduplicate on the `key` field. `maxlen` applies an
+approximate length bound (`MAXLEN ~`) on every `XADD` so the stream does not
+grow without limit; by default no bound is applied.
+
+`max_message_bytes` (16 MiB by default) rejects an individual envelope larger
+than the budget before anything is appended; narrow the source date range to
+reduce the response size. The source response is materialized before
+publishing; choose a source date range suitable for memory.
+
+`publish_operation` accepts `api_key_env`, `complete_history`, `max_pages`
+and `client_factory` for the public source reader. Supply source credentials
+through your runtime secret store, exposing only the environment variable
+name; omit `api_key_env` for anonymous USD access. Destination authentication
+belongs to the official client you pass in. The adapter neither saves nor
+prints credentials and never places one in an entry.
+
+The package licence covers this adapter code. FXMacroData data access and
+onward redistribution remain subject to your subscription and applicable
+licence. Appending data to a stream does not grant permission to publish or
+resell it.
+
+## Development
+
+Tests run offline against an in-memory fake client and pipeline:
+
+```sh
+python -m pytest tests -q
+```
+
+Official SDK reference:
+[redis-py XADD](https://redis.readthedocs.io/en/stable/commands.html?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=upstream-reference#redis.commands.core.CoreCommands.xadd).

redis_streams_fxmacrodata-0.1.0/PUBLIC_FILES.json

@@ -0,0 +1,11 @@
+[
+  "LICENSE",
+  "README.md",
+  "pyproject.toml",
+  "PUBLIC_FILES.json",
+  "MANIFEST.in",
+  "src/fxmacrodata_redis_streams/__init__.py",
+  "src/fxmacrodata_redis_streams/py.typed",
+  "tests/test_publisher.py",
+  "examples/consumer_group_dedupe.py"
+]

redis_streams_fxmacrodata-0.1.0/README.md

@@ -0,0 +1,109 @@
+# FXMacroData for Redis Streams
+
+Append FXMacroData public responses to a Redis Stream with the official
+`redis` Python client. Feed macro announcements to your own consumer groups
+while retaining the source payload and revision metadata.
+
+[FXMacroData documentation](https://fxmacrodata.com/documentation/reference?utm_source=github&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=readme)
+
+## Install and connect
+
+Requires Python 3.10 or later. Install this package and its adjacent public
+client dependencies from the distribution supplied to you:
+
+```sh
+python -m pip install /path/to/fxmacrodata-public-client /path/to/fxmacrodata-platform-core /path/to/redis-streams-fxmacrodata
+```
+
+Configure `REDIS_URL` (a `rediss://` URL with the credentials your
+administrator provides) in your runtime secret store. Then append to a
+customer-owned stream:
+
+```python
+import os
+import redis
+from fxmacrodata_platform_core import read
+from fxmacrodata_redis_streams import publish
+
+client = redis.Redis.from_url(os.environ["REDIS_URL"])
+try:
+    batch = read("indicator_history", {"currency": "usd", "indicator": "policy_rate"})
+    result = publish(batch, client, "fxmacrodata:usd", maxlen=100_000)
+    print(result.messages, "entries in", result.batches, "pipelines")
+finally:
+    client.close()
+```
+
+The destination is the stream key (letters, digits, `_`, `.`, `:`, `/` and
+`-`, up to 256 characters); `XADD` creates it on first use. URLs and other
+values are rejected before any command runs.
+
+Every operation exposed by `fxmacrodata_platform_core.operations()` is
+available, including REST and MCP operations. `publish_operation` reads one
+operation and publishes it in a single call:
+
+```python
+from fxmacrodata_redis_streams import publish_operation
+
+publish_operation("release_calendar", {"currency": "usd"}, client, "fxmacrodata:calendar",
+                  api_key_env="FXMD_API_KEY")
+```
+
+## Entry format
+
+Each envelope row becomes one stream entry whose fields are the envelope
+columns as strings: `operation`, `source_url`, `record_id`, `currency`,
+`indicator`, `date`, `announcement_datetime`, `value`, `value_text`,
+`record_json`, `response_json` and the publisher's `content_hash` (SHA-256 of
+the ten source columns). A null column is omitted from the entry; `value` is
+written as a Python float repr, such as `4.25`, and `value_text` keeps the
+source's own text. Dates and announcement times remain source strings. No
+publication time is inferred, and no missing value is filled. `record_json`
+retains the original record; `response_json` retains the complete response,
+including pagination and metadata. An empty response produces one entry with
+`record_json` equal to `null`, so absence of data and the server's metadata
+remain inspectable. It is not an observation.
+
+The `key` field is `operation:record_id:content_hash`; the same source
+record with the same content always produces the same key, and changed data
+or metadata produces a different one. Redis assigns the entry ID itself; no
+timestamp is derived from the data.
+
+## Delivery guarantee
+
+Delivery is at-least-once. Entries are appended in blocks of `batch_size`
+rows (1 to 10,000, 1,000 by default), one non-transactional pipeline per
+block. A failed block raises `PublishError` without the server's own
+diagnostic text and is not retried; earlier blocks may already be stored, and
+a partially executed pipeline may have appended some of its entries.
+Consumers should deduplicate on the `key` field. `maxlen` applies an
+approximate length bound (`MAXLEN ~`) on every `XADD` so the stream does not
+grow without limit; by default no bound is applied.
+
+`max_message_bytes` (16 MiB by default) rejects an individual envelope larger
+than the budget before anything is appended; narrow the source date range to
+reduce the response size. The source response is materialized before
+publishing; choose a source date range suitable for memory.
+
+`publish_operation` accepts `api_key_env`, `complete_history`, `max_pages`
+and `client_factory` for the public source reader. Supply source credentials
+through your runtime secret store, exposing only the environment variable
+name; omit `api_key_env` for anonymous USD access. Destination authentication
+belongs to the official client you pass in. The adapter neither saves nor
+prints credentials and never places one in an entry.
+
+The package licence covers this adapter code. FXMacroData data access and
+onward redistribution remain subject to your subscription and applicable
+licence. Appending data to a stream does not grant permission to publish or
+resell it.
+
+## Development
+
+Tests run offline against an in-memory fake client and pipeline:
+
+```sh
+python -m pytest tests -q
+```
+
+Official SDK reference:
+[redis-py XADD](https://redis.readthedocs.io/en/stable/commands.html?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=upstream-reference#redis.commands.core.CoreCommands.xadd).

redis_streams_fxmacrodata-0.1.0/examples/consumer_group_dedupe.py

@@ -0,0 +1,24 @@
+"""Read FXMacroData envelopes with a consumer group, dropping redeliveries by the key field."""
+import os
+
+import redis
+
+STREAM, GROUP, CONSUMER = "fxmacrodata:usd", "fxmacrodata-readers", "reader-1"
+client = redis.Redis.from_url(os.environ["REDIS_URL"], decode_responses=True)
+seen: set[str] = set()
+
+try:
+    client.xgroup_create(STREAM, GROUP, id="0", mkstream=True)
+except redis.ResponseError as error:
+    if "BUSYGROUP" not in str(error):
+        raise
+
+while True:
+    for _, entries in client.xreadgroup(GROUP, CONSUMER, {STREAM: ">"}, count=100, block=5000) or []:
+        for entry_id, fields in entries:
+            if fields["key"] not in seen:
+                seen.add(fields["key"])
+                if fields["record_json"] != "null":
+                    print(fields["operation"], fields.get("currency"), fields.get("indicator"),
+                          fields.get("date"), fields.get("value"))
+            client.xack(STREAM, GROUP, entry_id)

redis_streams_fxmacrodata-0.1.0/pyproject.toml

@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "redis-streams-fxmacrodata"
+version = "0.1.0"
+description = "Publish public FXMacroData responses to Redis Streams"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+dependencies = ["fxmacrodata-platform-core==0.1.0", "redis>=8.1.0,<9"]
+
+[project.urls]
+Documentation = "https://fxmacrodata.com/documentation/reference?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=package"
+
+[project.optional-dependencies]
+test = ["pytest>=8,<9"]
+
+[tool.setuptools]
+packages = ["fxmacrodata_redis_streams"]
+package-dir = {"" = "src"}
+include-package-data = false
+
+[tool.setuptools.package-data]
+fxmacrodata_redis_streams = ["py.typed"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

redis_streams_fxmacrodata-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

redis_streams_fxmacrodata-0.1.0/src/fxmacrodata_redis_streams/__init__.py

@@ -0,0 +1,158 @@
+"""Publish public FXMacroData responses to a customer-owned Redis Stream."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+import hashlib
+import json
+import math
+import re
+from typing import Any
+
+from fxmacrodata_platform_core import read
+
+__version__ = "0.1.0"
+
+COLUMNS = (
+    "operation", "source_url", "record_id", "currency", "indicator", "date",
+    "announcement_datetime", "value", "value_text", "record_json", "response_json",
+    "content_hash",
+)
+KEY_FIELD = "key"
+MAX_MESSAGE_BYTES = 16 * 1024 * 1024
+_STREAM = re.compile(r"[A-Za-z0-9_.:/-]{1,256}\Z")
+
+
+class PublishError(RuntimeError):
+    """A destination publish failed; inspect the destination before retrying."""
+
+
+@dataclass(frozen=True)
+class PublishResult:
+    destination: str
+    messages: int
+    batches: int
+
+
+def _stream(stream: str) -> str:
+    if not isinstance(stream, str) or not _STREAM.fullmatch(stream) or "://" in stream:
+        raise ValueError("destination must be a stream key of letters, digits, '_', '.', ':', '/' or '-', not a URL")
+    return stream
+
+
+def _bounds(batch_size: int, max_message_bytes: int, maxlen: int | None) -> None:
+    if isinstance(batch_size, bool) or not isinstance(batch_size, int) or not 1 <= batch_size <= 10000:
+        raise ValueError("batch_size must be an integer between 1 and 10000")
+    if isinstance(max_message_bytes, bool) or not isinstance(max_message_bytes, int) or not 1 <= max_message_bytes <= MAX_MESSAGE_BYTES:
+        raise ValueError(f"max_message_bytes must be an integer between 1 and {MAX_MESSAGE_BYTES}")
+    if maxlen is not None and (isinstance(maxlen, bool) or not isinstance(maxlen, int) or maxlen < 1):
+        raise ValueError("maxlen must be a positive integer or None")
+
+
+def _rows(batch: Any) -> list[dict[str, Any]]:
+    rows = []
+    for source in batch.rows():
+        row = {name: source.get(name) for name in COLUMNS[:-1]}
+        if not isinstance(row["operation"], str) or not isinstance(row["record_id"], str):
+            raise ValueError("every row requires string operation and record_id")
+        for name, value in row.items():
+            if name == "value":
+                if value is not None and (
+                    isinstance(value, bool) or not isinstance(value, (int, float)) or
+                    not math.isfinite(value)
+                ):
+                    raise ValueError("value must be a finite number or null")
+            elif value is not None and not isinstance(value, str):
+                raise ValueError(f"{name} must be a string or null")
+        encoded = json.dumps(row, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
+        row["content_hash"] = hashlib.sha256(encoded.encode("utf-8")).hexdigest()
+        rows.append(row)
+    return rows
+
+
+def message_key(row: dict[str, Any]) -> str:
+    """Return the idempotency key stored in the `key` field of every entry."""
+    return f"{row['operation']}:{row['record_id']}:{row['content_hash']}"
+
+
+def encode_message(row: dict[str, Any]) -> bytes:
+    """Return the canonical UTF-8 JSON of one envelope row (used for the byte budget)."""
+    return json.dumps(row, sort_keys=True, separators=(",", ":"), ensure_ascii=False,
+                      allow_nan=False).encode("utf-8")
+
+
+def entry_fields(row: dict[str, Any]) -> dict[str, str]:
+    """Return the stream entry fields: every non-null column as a string, plus `key`."""
+    fields = {}
+    for name in COLUMNS:
+        value = row.get(name)
+        if value is None:
+            continue
+        fields[name] = repr(float(value)) if name == "value" else value
+    fields[KEY_FIELD] = message_key(row)
+    return fields
+
+
+def _blocks(rows: list[dict[str, Any]], batch_size: int,
+            max_message_bytes: int) -> list[list[dict[str, str]]]:
+    output: list[list[dict[str, str]]] = []
+    block: list[dict[str, str]] = []
+    for row in rows:
+        if len(encode_message(row)) > max_message_bytes:
+            raise ValueError("one envelope exceeds max_message_bytes; narrow the source date range")
+        if len(block) >= batch_size:
+            output.append(block)
+            block = []
+        block.append(entry_fields(row))
+    if block:
+        output.append(block)
+    return output
+
+
+def publish(batch: Any, client: Any, destination: str, *, batch_size: int = 1000,
+            max_message_bytes: int = MAX_MESSAGE_BYTES, maxlen: int | None = None) -> PublishResult:
+    """Append one stream entry per envelope row with XADD, one pipeline per block.
+
+    Entry fields are the envelope columns as strings (null columns omitted,
+    `value` as a float repr) plus `key`, equal to
+    `operation:record_id:content_hash`. `maxlen` applies an approximate
+    length bound (`MAXLEN ~`) on every XADD. Delivery is at-least-once; a
+    failed block raises PublishError and is not retried.
+    """
+    stream = _stream(destination)
+    _bounds(batch_size, max_message_bytes, maxlen)
+    blocks = _blocks(_rows(batch), batch_size, max_message_bytes)
+    count, messages = 0, 0
+    for block in blocks:
+        try:
+            pipeline = client.pipeline(transaction=False)
+            for fields in block:
+                pipeline.xadd(stream, fields, maxlen=maxlen, approximate=True)
+            pipeline.execute()
+        except Exception:
+            raise PublishError(
+                f"Redis XADD block {count + 1} failed; earlier blocks may be stored. "
+                "Inspect the destination and deduplicate by the key field on retry."
+            ) from None
+        count += 1
+        messages += len(block)
+    return PublishResult(stream, messages, count)
+
+
+def publish_operation(operation: str, args: dict[str, Any] | None, client: Any, destination: str,
+                      *, api_key_env: str | None = None, complete_history: bool = True,
+                      max_pages: int = 10000, client_factory: Any = None,
+                      batch_size: int = 1000, max_message_bytes: int = MAX_MESSAGE_BYTES,
+                      maxlen: int | None = None) -> PublishResult:
+    """Read one public operation and publish its complete response envelope."""
+    _stream(destination)
+    _bounds(batch_size, max_message_bytes, maxlen)
+    batch = read(operation, args, api_key_env=api_key_env,
+                 complete_history=complete_history, max_pages=max_pages,
+                 client_factory=client_factory)
+    return publish(batch, client, destination, batch_size=batch_size,
+                   max_message_bytes=max_message_bytes, maxlen=maxlen)
+
+
+__all__ = ["COLUMNS", "KEY_FIELD", "MAX_MESSAGE_BYTES", "PublishError", "PublishResult",
+           "encode_message", "entry_fields", "message_key", "publish", "publish_operation"]

redis_streams_fxmacrodata-0.1.0/src/redis_streams_fxmacrodata.egg-info/PKG-INFO

@@ -0,0 +1,124 @@
+Metadata-Version: 2.4
+Name: redis-streams-fxmacrodata
+Version: 0.1.0
+Summary: Publish public FXMacroData responses to Redis Streams
+License-Expression: MIT
+Project-URL: Documentation, https://fxmacrodata.com/documentation/reference?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=package
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fxmacrodata-platform-core==0.1.0
+Requires-Dist: redis<9,>=8.1.0
+Provides-Extra: test
+Requires-Dist: pytest<9,>=8; extra == "test"
+Dynamic: license-file
+
+# FXMacroData for Redis Streams
+
+Append FXMacroData public responses to a Redis Stream with the official
+`redis` Python client. Feed macro announcements to your own consumer groups
+while retaining the source payload and revision metadata.
+
+[FXMacroData documentation](https://fxmacrodata.com/documentation/reference?utm_source=github&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=readme)
+
+## Install and connect
+
+Requires Python 3.10 or later. Install this package and its adjacent public
+client dependencies from the distribution supplied to you:
+
+```sh
+python -m pip install /path/to/fxmacrodata-public-client /path/to/fxmacrodata-platform-core /path/to/redis-streams-fxmacrodata
+```
+
+Configure `REDIS_URL` (a `rediss://` URL with the credentials your
+administrator provides) in your runtime secret store. Then append to a
+customer-owned stream:
+
+```python
+import os
+import redis
+from fxmacrodata_platform_core import read
+from fxmacrodata_redis_streams import publish
+
+client = redis.Redis.from_url(os.environ["REDIS_URL"])
+try:
+    batch = read("indicator_history", {"currency": "usd", "indicator": "policy_rate"})
+    result = publish(batch, client, "fxmacrodata:usd", maxlen=100_000)
+    print(result.messages, "entries in", result.batches, "pipelines")
+finally:
+    client.close()
+```
+
+The destination is the stream key (letters, digits, `_`, `.`, `:`, `/` and
+`-`, up to 256 characters); `XADD` creates it on first use. URLs and other
+values are rejected before any command runs.
+
+Every operation exposed by `fxmacrodata_platform_core.operations()` is
+available, including REST and MCP operations. `publish_operation` reads one
+operation and publishes it in a single call:
+
+```python
+from fxmacrodata_redis_streams import publish_operation
+
+publish_operation("release_calendar", {"currency": "usd"}, client, "fxmacrodata:calendar",
+                  api_key_env="FXMD_API_KEY")
+```
+
+## Entry format
+
+Each envelope row becomes one stream entry whose fields are the envelope
+columns as strings: `operation`, `source_url`, `record_id`, `currency`,
+`indicator`, `date`, `announcement_datetime`, `value`, `value_text`,
+`record_json`, `response_json` and the publisher's `content_hash` (SHA-256 of
+the ten source columns). A null column is omitted from the entry; `value` is
+written as a Python float repr, such as `4.25`, and `value_text` keeps the
+source's own text. Dates and announcement times remain source strings. No
+publication time is inferred, and no missing value is filled. `record_json`
+retains the original record; `response_json` retains the complete response,
+including pagination and metadata. An empty response produces one entry with
+`record_json` equal to `null`, so absence of data and the server's metadata
+remain inspectable. It is not an observation.
+
+The `key` field is `operation:record_id:content_hash`; the same source
+record with the same content always produces the same key, and changed data
+or metadata produces a different one. Redis assigns the entry ID itself; no
+timestamp is derived from the data.
+
+## Delivery guarantee
+
+Delivery is at-least-once. Entries are appended in blocks of `batch_size`
+rows (1 to 10,000, 1,000 by default), one non-transactional pipeline per
+block. A failed block raises `PublishError` without the server's own
+diagnostic text and is not retried; earlier blocks may already be stored, and
+a partially executed pipeline may have appended some of its entries.
+Consumers should deduplicate on the `key` field. `maxlen` applies an
+approximate length bound (`MAXLEN ~`) on every `XADD` so the stream does not
+grow without limit; by default no bound is applied.
+
+`max_message_bytes` (16 MiB by default) rejects an individual envelope larger
+than the budget before anything is appended; narrow the source date range to
+reduce the response size. The source response is materialized before
+publishing; choose a source date range suitable for memory.
+
+`publish_operation` accepts `api_key_env`, `complete_history`, `max_pages`
+and `client_factory` for the public source reader. Supply source credentials
+through your runtime secret store, exposing only the environment variable
+name; omit `api_key_env` for anonymous USD access. Destination authentication
+belongs to the official client you pass in. The adapter neither saves nor
+prints credentials and never places one in an entry.
+
+The package licence covers this adapter code. FXMacroData data access and
+onward redistribution remain subject to your subscription and applicable
+licence. Appending data to a stream does not grant permission to publish or
+resell it.
+
+## Development
+
+Tests run offline against an in-memory fake client and pipeline:
+
+```sh
+python -m pytest tests -q
+```
+
+Official SDK reference:
+[redis-py XADD](https://redis.readthedocs.io/en/stable/commands.html?utm_source=pypi&utm_medium=referral&utm_campaign=redis-streams-fxmacrodata&utm_content=upstream-reference#redis.commands.core.CoreCommands.xadd).

redis_streams_fxmacrodata-0.1.0/src/redis_streams_fxmacrodata.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+LICENSE
+MANIFEST.in
+PUBLIC_FILES.json
+README.md
+pyproject.toml
+examples/consumer_group_dedupe.py
+src/fxmacrodata_redis_streams/__init__.py
+src/fxmacrodata_redis_streams/py.typed
+src/redis_streams_fxmacrodata.egg-info/PKG-INFO
+src/redis_streams_fxmacrodata.egg-info/SOURCES.txt
+src/redis_streams_fxmacrodata.egg-info/dependency_links.txt
+src/redis_streams_fxmacrodata.egg-info/requires.txt
+src/redis_streams_fxmacrodata.egg-info/top_level.txt
+tests/test_publisher.py

redis_streams_fxmacrodata-0.1.0/src/redis_streams_fxmacrodata.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

redis_streams_fxmacrodata-0.1.0/src/redis_streams_fxmacrodata.egg-info/requires.txt

@@ -0,0 +1,5 @@
+fxmacrodata-platform-core==0.1.0
+redis<9,>=8.1.0
+
+[test]
+pytest<9,>=8

redis_streams_fxmacrodata-0.1.0/src/redis_streams_fxmacrodata.egg-info/top_level.txt

@@ -0,0 +1 @@
+fxmacrodata_redis_streams

redis_streams_fxmacrodata-0.1.0/tests/test_publisher.py

@@ -0,0 +1,159 @@
+import json
+
+import pytest
+
+from fxmacrodata_platform_core import Batch, IntegrationError
+import fxmacrodata_redis_streams as adapter
+
+
+class Pipeline:
+    def __init__(self, client):
+        self.client = client
+        self.queued = []
+
+    def xadd(self, name, fields, id="*", maxlen=None, approximate=True, **kwargs):
+        self.queued.append((name, fields, maxlen, approximate))
+
+    def execute(self):
+        self.client.executed.append(self.queued)
+        if len(self.client.executed) == self.client.fail_block:
+            raise RuntimeError("private destination diagnostic")
+        self.client.entries.extend(self.queued)
+        return [b"0-%d" % i for i in range(len(self.queued))]
+
+
+class Client:
+    def __init__(self, fail_block=None):
+        self.executed = []
+        self.entries = []
+        self.fail_block = fail_block
+        self.transactions = []
+
+    def pipeline(self, transaction=True, shard_hint=None):
+        self.transactions.append(transaction)
+        return Pipeline(self)
+
+
+STREAM = "fxmacrodata:usd"
+
+
+def batch(count=3, metadata=None):
+    return Batch("indicator_history", {
+        "data": [{"currency": "USD", "indicator": "policy_rate", "value": str(i),
+                  "date": "2026-10-01", "publication_time_status": "unknown"}
+                 for i in range(count)],
+        "metadata": metadata or {"revision": 1},
+    })
+
+
+def test_entries_preserve_full_payload_as_string_fields_with_dedup_key():
+    source, client = batch(), Client()
+    result = adapter.publish(source, client, STREAM, batch_size=2)
+    assert result == adapter.PublishResult(STREAM, 3, 2)
+    assert [len(block) for block in client.executed] == [2, 1]
+    assert client.transactions == [False, False]
+    name, fields, maxlen, approximate = client.entries[0]
+    assert name == STREAM and maxlen is None and approximate is True
+    assert all(isinstance(v, str) for v in fields.values())
+    assert "announcement_datetime" not in fields
+    assert fields["date"] == "2026-10-01"
+    assert fields["value"] == "0.0" and fields["value_text"] == "0"
+    assert json.loads(fields["record_json"])["publication_time_status"] == "unknown"
+    assert json.loads(fields["response_json"]) == source.payload
+    assert fields["key"] == f"indicator_history:{fields['record_id']}:{fields['content_hash']}"
+    assert set(fields) == {name for name in adapter.COLUMNS if name != "announcement_datetime"} | {"key"}
+
+
+def test_empty_response_is_a_metadata_envelope_not_an_observation():
+    source, client = batch(0), Client()
+    result = adapter.publish(source, client, STREAM)
+    fields = client.entries[0][1]
+    assert result.messages == 1 and fields["record_json"] == "null"
+    assert "value" not in fields and "date" not in fields
+    assert json.loads(fields["response_json"]) == source.payload
+
+
+def test_retries_have_stable_keys_and_changed_metadata_is_distinct():
+    first, retry, changed = Client(), Client(), Client()
+    adapter.publish(batch(1), first, STREAM)
+    adapter.publish(batch(1), retry, STREAM)
+    adapter.publish(batch(1, {"revision": 2}), changed, STREAM)
+    assert first.entries == retry.entries
+    assert first.entries[0][1]["record_id"] == changed.entries[0][1]["record_id"]
+    assert first.entries[0][1]["key"] != changed.entries[0][1]["key"]
+
+
+@pytest.mark.parametrize("stream", ["fx macro", "redis://host/stream", "", None, "x" * 257, "a\nb"])
+def test_bad_streams_never_publish(stream):
+    client = Client()
+    with pytest.raises(ValueError):
+        adapter.publish(batch(), client, stream)
+    assert client.executed == []
+
+
+@pytest.mark.parametrize("options", [
+    {"batch_size": 0}, {"batch_size": -1}, {"batch_size": 10001}, {"batch_size": True},
+    {"batch_size": 2.5}, {"max_message_bytes": 0}, {"max_message_bytes": 16 * 1024 * 1024 + 1},
+    {"maxlen": 0}, {"maxlen": True}, {"maxlen": "1000"},
+])
+def test_batch_bounds(options):
+    client = Client()
+    with pytest.raises(ValueError):
+        adapter.publish(batch(), client, STREAM, **options)
+    assert client.executed == []
+
+
+def test_failed_block_is_not_retried_or_exposed():
+    client = Client(fail_block=2)
+    with pytest.raises(adapter.PublishError) as error:
+        adapter.publish(batch(), client, STREAM, batch_size=1)
+    assert "block 2" in str(error.value)
+    assert "private destination diagnostic" not in str(error.value)
+    assert len(client.executed) == 2 and len(client.entries) == 1
+
+
+def test_oversized_envelope_fails_before_any_publish():
+    client = Client()
+    with pytest.raises(ValueError, match="narrow the source date range"):
+        adapter.publish(batch(), client, STREAM, max_message_bytes=1)
+    assert client.executed == []
+
+
+def test_maxlen_is_applied_approximately_to_every_xadd():
+    client = Client()
+    adapter.publish(batch(2), client, STREAM, maxlen=5000)
+    assert [(m, a) for _, _, m, a in client.entries] == [(5000, True), (5000, True)]
+
+
+def test_operation_forwards_public_reader_options(monkeypatch):
+    calls = []
+    monkeypatch.setattr(adapter, "read", lambda *args, **kw: calls.append((args, kw)) or batch(1))
+    factory = object()
+    adapter.publish_operation("indicator_history", {"currency": "usd", "indicator": "policy_rate"},
+                              Client(), STREAM, api_key_env="FXMD_API_KEY", max_pages=12,
+                              complete_history=False, client_factory=factory)
+    assert calls[0][0] == ("indicator_history", {"currency": "usd", "indicator": "policy_rate"})
+    assert calls[0][1] == dict(api_key_env="FXMD_API_KEY", complete_history=False,
+                                max_pages=12, client_factory=factory)
+
+
+def test_invalid_configuration_never_reads_source(monkeypatch):
+    def unexpected(*args, **kwargs):
+        pytest.fail("source must not be called for invalid publisher configuration")
+    monkeypatch.setattr(adapter, "read", unexpected)
+    for options in ({"batch_size": 0}, {"max_message_bytes": 0}, {"maxlen": 0}):
+        with pytest.raises(ValueError):
+            adapter.publish_operation("release_calendar", {"currency": "usd"}, Client(), STREAM, **options)
+    with pytest.raises(ValueError):
+        adapter.publish_operation("release_calendar", {"currency": "usd"}, Client(), "bad stream")
+
+
+def test_credential_arguments_are_rejected_before_publishing():
+    client = Client()
+    with pytest.raises(IntegrationError):
+        adapter.publish_operation("indicator_history", {"currency": "usd", "indicator": "policy_rate",
+                                                        "api_key": "secret"}, client, STREAM)
+    with pytest.raises(IntegrationError):
+        adapter.publish_operation("release_calendar", {"currency": "usd"}, client, STREAM,
+                                  api_key_env="not-an-env-name")
+    assert client.executed == []