okama-macro 0.1.0__tar.gz

okama_macro-0.1.0/PKG-INFO

@@ -0,0 +1,151 @@
+Metadata-Version: 2.4
+Name: okama-macro
+Version: 0.1.0
+Summary: Macro-economic data-source clients (CPI, central-bank rates) for the okama project
+License: MIT
+Author: mbk-dev
+Requires-Python: >=3.11,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: lxml
+Requires-Dist: pandas
+Requires-Dist: requests
+Requires-Dist: urllib3
+Project-URL: Homepage, https://github.com/mbk-dev/okama-macro
+Project-URL: Repository, https://github.com/mbk-dev/okama-macro
+Description-Content-Type: text/markdown
+
+# okama-macro
+
+Macro-economic data-source clients — CPI inflation and central-bank / policy
+rates — for the [okama](https://github.com/mbk-dev/okama) project.
+
+One package, one HTTP/DataFrame layer, one dependency. It consolidates the macro
+source clients that used to live as separate per-source micro-repos and as
+in-repo modules of okama-API, behind a single unified series API. Rationale and
+migration history: [mbk-dev/okama-API#41](https://github.com/mbk-dev/okama-API/issues/41)
+and its ADR 0001.
+
+## Install
+
+```bash
+pip install okama-macro
+```
+
+Requires **Python ≥ 3.11**. Runs on both **pandas 2.x and 3.x**.
+
+## Quick start
+
+```python
+import okama_macro
+
+okama_macro.list_series()
+# ['CHN_LPR1.RATE', 'CHN_LPR5.RATE', 'CNY.INFL', 'EU_DFR.RATE',
+#  'EU_MLR.RATE', 'EU_MRO.RATE', 'GBP.INFL', 'HKD.INFL', 'HK_BR.RATE',
+#  'ILS.INFL', 'IND_RBI.RATE', 'INR.INFL', 'ISR_IR.RATE', 'UK_BR.RATE',
+#  'USD.INFL', 'US_EFFR.RATE']
+
+# A rate as a decimal fraction (3.62% -> 0.0362), observations only:
+okama_macro.get('US_EFFR.RATE', first_date='2020-01-01')
+
+# Monthly m/m inflation as a decimal fraction:
+okama_macro.get('USD.INFL')
+```
+
+## The contract
+
+Every series returned by `get()` obeys the same contract, so callers never
+special-case a source:
+
+- **Decimal fractions** — m/m inflation `0.0042`, a rate `0.0525` (never percent,
+  never an index level).
+- **CPI series** are monthly, stamped on the **first of the month**, derived from
+  the source's index via `pct_change()` (base-invariant).
+- **Rate series** normally carry **observations only — no padding**. Forward-fill
+  to a daily grid on the consumer side if you need one. `UK_BR.RATE` is the
+  documented exception: the Bank of England source publishes change dates, and
+  the client safely forward-fills them into a daily series.
+- **Ascending `DatetimeIndex`**, `float` dtype, and `Series.name == key`.
+
+`get()` raises `ValueError` for an unknown key (listing the known ones);
+`list_series()` returns the available keys, sorted.
+
+## Available series
+
+| Key | Series | Source module |
+|-----|--------|---------------|
+| `USD.INFL` | US CPI, m/m | `fred` (FRED `CPIAUCNS`) |
+| `HKD.INFL` | Hong Kong Composite CPI, m/m | `censtatd` (HK C&SD) |
+| `INR.INFL` | India General CPI, m/m | `mospi` (MOSPI) |
+| `CNY.INFL` | China CPI, m/m | `nbsc` (NBS China) |
+| `GBP.INFL` | UK CPIH, m/m | `ons` (UK ONS) |
+| `ILS.INFL` | Israel CPI, m/m | `boi` (Bank of Israel) |
+| `US_EFFR.RATE` | US Federal Funds rate | `fred` (FRED `DFF`) |
+| `HK_BR.RATE` | HKMA Discount Window Base Rate | `hkma` |
+| `IND_RBI.RATE` | RBI policy repo rate | `bis` (history) + `rbi` (same-day tail) |
+| `CHN_LPR1.RATE` | China one-year Loan Prime Rate | `cfets` |
+| `CHN_LPR5.RATE` | China five-year Loan Prime Rate | `cfets` |
+| `ISR_IR.RATE` | Bank of Israel policy rate | `boi` |
+| `EU_MRO.RATE` | ECB main refinancing operations rate | `ecb` |
+| `EU_MLR.RATE` | ECB marginal lending facility rate | `ecb` |
+| `EU_DFR.RATE` | ECB deposit facility rate | `ecb` |
+| `UK_BR.RATE` | Bank of England Bank Rate | `boe` |
+
+## Raw per-source clients
+
+Each source also exposes its raw client under `okama_macro.sources.*`, returning
+data **as the agency publishes it** (CPI index levels, rates in percent) — use
+these only if you need the unnormalised series; prefer `get()` otherwise.
+
+```python
+from okama_macro.sources import (
+    bis, boe, boi, censtatd, cfets, ecb, fred, hkma, mospi, nbsc, ons, rbi
+)
+
+hkma.get_base_rate()          # percent, daily
+censtatd.get_composite_cpi()  # CPI index level, monthly
+```
+
+## Configuration (environment)
+
+| Variable | Needed for | Notes |
+|----------|-----------|-------|
+| `FRED_API_KEY` | `USD.INFL`, `US_EFFR.RATE` | Free key from FRED; kept out of logs. |
+| `PROXY_HOST`, `PROXY_PORT` | `bis`, `mospi`, `rbi` | Optional outbound HTTP proxy. |
+| `PROXY_USER`, `PROXY_PASS` | — | Optional proxy credentials. |
+
+## Architecture
+
+```
+okama_macro/
+├── __init__.py        # public API: get(), list_series()
+├── registry.py        # key -> normalised Series (the contract)
+├── _http.py           # shared Session: retry/back-off, proxy, browser UA,
+│                       #   legacy-TLS, secret redaction
+├── _frame.py          # DataFrame/Series shaping helpers
+└── sources/           # one client per source (bis, boe, boi, censtatd, cfets,
+                        #   ecb, fred, hkma, mospi, nbsc, ons, rbi)
+```
+
+Two layers: thin **sources** (data as published, on the shared `_http`) and a
+**registry** that normalises each key to the contract above.
+
+## Development
+
+```bash
+poetry install
+poetry run pytest        # test suite
+poetry run ruff check .  # lint (rules C, E, F, W, B)
+```
+
+CI runs the suite and lint on Python 3.11, 3.12, 3.13 and 3.14. `poetry.lock` is
+intentionally untracked.
+
+## License
+
+MIT.
+

okama_macro-0.1.0/README.md

@@ -0,0 +1,129 @@
+# okama-macro
+
+Macro-economic data-source clients — CPI inflation and central-bank / policy
+rates — for the [okama](https://github.com/mbk-dev/okama) project.
+
+One package, one HTTP/DataFrame layer, one dependency. It consolidates the macro
+source clients that used to live as separate per-source micro-repos and as
+in-repo modules of okama-API, behind a single unified series API. Rationale and
+migration history: [mbk-dev/okama-API#41](https://github.com/mbk-dev/okama-API/issues/41)
+and its ADR 0001.
+
+## Install
+
+```bash
+pip install okama-macro
+```
+
+Requires **Python ≥ 3.11**. Runs on both **pandas 2.x and 3.x**.
+
+## Quick start
+
+```python
+import okama_macro
+
+okama_macro.list_series()
+# ['CHN_LPR1.RATE', 'CHN_LPR5.RATE', 'CNY.INFL', 'EU_DFR.RATE',
+#  'EU_MLR.RATE', 'EU_MRO.RATE', 'GBP.INFL', 'HKD.INFL', 'HK_BR.RATE',
+#  'ILS.INFL', 'IND_RBI.RATE', 'INR.INFL', 'ISR_IR.RATE', 'UK_BR.RATE',
+#  'USD.INFL', 'US_EFFR.RATE']
+
+# A rate as a decimal fraction (3.62% -> 0.0362), observations only:
+okama_macro.get('US_EFFR.RATE', first_date='2020-01-01')
+
+# Monthly m/m inflation as a decimal fraction:
+okama_macro.get('USD.INFL')
+```
+
+## The contract
+
+Every series returned by `get()` obeys the same contract, so callers never
+special-case a source:
+
+- **Decimal fractions** — m/m inflation `0.0042`, a rate `0.0525` (never percent,
+  never an index level).
+- **CPI series** are monthly, stamped on the **first of the month**, derived from
+  the source's index via `pct_change()` (base-invariant).
+- **Rate series** normally carry **observations only — no padding**. Forward-fill
+  to a daily grid on the consumer side if you need one. `UK_BR.RATE` is the
+  documented exception: the Bank of England source publishes change dates, and
+  the client safely forward-fills them into a daily series.
+- **Ascending `DatetimeIndex`**, `float` dtype, and `Series.name == key`.
+
+`get()` raises `ValueError` for an unknown key (listing the known ones);
+`list_series()` returns the available keys, sorted.
+
+## Available series
+
+| Key | Series | Source module |
+|-----|--------|---------------|
+| `USD.INFL` | US CPI, m/m | `fred` (FRED `CPIAUCNS`) |
+| `HKD.INFL` | Hong Kong Composite CPI, m/m | `censtatd` (HK C&SD) |
+| `INR.INFL` | India General CPI, m/m | `mospi` (MOSPI) |
+| `CNY.INFL` | China CPI, m/m | `nbsc` (NBS China) |
+| `GBP.INFL` | UK CPIH, m/m | `ons` (UK ONS) |
+| `ILS.INFL` | Israel CPI, m/m | `boi` (Bank of Israel) |
+| `US_EFFR.RATE` | US Federal Funds rate | `fred` (FRED `DFF`) |
+| `HK_BR.RATE` | HKMA Discount Window Base Rate | `hkma` |
+| `IND_RBI.RATE` | RBI policy repo rate | `bis` (history) + `rbi` (same-day tail) |
+| `CHN_LPR1.RATE` | China one-year Loan Prime Rate | `cfets` |
+| `CHN_LPR5.RATE` | China five-year Loan Prime Rate | `cfets` |
+| `ISR_IR.RATE` | Bank of Israel policy rate | `boi` |
+| `EU_MRO.RATE` | ECB main refinancing operations rate | `ecb` |
+| `EU_MLR.RATE` | ECB marginal lending facility rate | `ecb` |
+| `EU_DFR.RATE` | ECB deposit facility rate | `ecb` |
+| `UK_BR.RATE` | Bank of England Bank Rate | `boe` |
+
+## Raw per-source clients
+
+Each source also exposes its raw client under `okama_macro.sources.*`, returning
+data **as the agency publishes it** (CPI index levels, rates in percent) — use
+these only if you need the unnormalised series; prefer `get()` otherwise.
+
+```python
+from okama_macro.sources import (
+    bis, boe, boi, censtatd, cfets, ecb, fred, hkma, mospi, nbsc, ons, rbi
+)
+
+hkma.get_base_rate()          # percent, daily
+censtatd.get_composite_cpi()  # CPI index level, monthly
+```
+
+## Configuration (environment)
+
+| Variable | Needed for | Notes |
+|----------|-----------|-------|
+| `FRED_API_KEY` | `USD.INFL`, `US_EFFR.RATE` | Free key from FRED; kept out of logs. |
+| `PROXY_HOST`, `PROXY_PORT` | `bis`, `mospi`, `rbi` | Optional outbound HTTP proxy. |
+| `PROXY_USER`, `PROXY_PASS` | — | Optional proxy credentials. |
+
+## Architecture
+
+```
+okama_macro/
+├── __init__.py        # public API: get(), list_series()
+├── registry.py        # key -> normalised Series (the contract)
+├── _http.py           # shared Session: retry/back-off, proxy, browser UA,
+│                       #   legacy-TLS, secret redaction
+├── _frame.py          # DataFrame/Series shaping helpers
+└── sources/           # one client per source (bis, boe, boi, censtatd, cfets,
+                        #   ecb, fred, hkma, mospi, nbsc, ons, rbi)
+```
+
+Two layers: thin **sources** (data as published, on the shared `_http`) and a
+**registry** that normalises each key to the contract above.
+
+## Development
+
+```bash
+poetry install
+poetry run pytest        # test suite
+poetry run ruff check .  # lint (rules C, E, F, W, B)
+```
+
+CI runs the suite and lint on Python 3.11, 3.12, 3.13 and 3.14. `poetry.lock` is
+intentionally untracked.
+
+## License
+
+MIT.

okama_macro-0.1.0/okama_macro/__init__.py

@@ -0,0 +1,11 @@
+"""okama-macro: consolidated macro-economic data-source clients for okama.
+
+Public API: ``get(key, first_date=None, last_date=None)`` returns a normalised
+``pd.Series`` (decimal fractions, ascending DatetimeIndex, observations only);
+``list_series()`` lists the available keys. Per-source raw clients live in
+``okama_macro.sources``.
+"""
+
+from okama_macro.registry import get, list_series
+
+__all__ = ['get', 'list_series']

okama_macro-0.1.0/okama_macro/_frame.py

@@ -0,0 +1,14 @@
+"""Shared DataFrame/Series shaping helpers."""
+
+import pandas as pd
+
+
+def clip_window(obj: pd.Series | pd.DataFrame,
+                first_date: object = None,
+                last_date: object = None) -> pd.Series | pd.DataFrame:
+    """Clip a DatetimeIndex-ed object to the inclusive [first_date, last_date]."""
+    if first_date is not None:
+        obj = obj[obj.index >= pd.Timestamp(first_date)]
+    if last_date is not None:
+        obj = obj[obj.index <= pd.Timestamp(last_date)]
+    return obj

okama_macro-0.1.0/okama_macro/_http.py

@@ -0,0 +1,127 @@
+"""Shared HTTP plumbing for all okama-macro source clients.
+
+``get()`` is a plain GET with a browser-like User-Agent (several statistical
+agencies reject the default python-requests UA), an optional outbound proxy
+taken from the ``PROXY_*`` environment variables, and linear-back-off retries
+on transient upstream 5xx responses. 4xx responses and hard connection errors
+fail fast. On failure it raises ``RuntimeError`` with the ``redact`` strings
+masked so API keys never leak into logs. ``legacy_tls_session()`` (Task 3)
+serves endpoints that need OpenSSL legacy server renegotiation.
+"""
+
+import logging
+import os
+import ssl
+import time
+
+import requests
+import urllib3
+from requests.adapters import HTTPAdapter
+from urllib3 import PoolManager
+
+info_logger = logging.getLogger('okama_macro.http')
+
+DEFAULT_TIMEOUT = 60  # seconds
+USER_AGENT = 'Mozilla/5.0 (okama-data pipeline)'
+
+
+def proxies_from_env() -> dict[str, str] | None:
+    """Build the requests proxies dict from PROXY_* env vars, or None if unset.
+
+    Foreign sources go through the local HAProxy on the production server;
+    without the env vars (e.g. in tests) requests go direct.
+    """
+    host, port = os.getenv('PROXY_HOST'), os.getenv('PROXY_PORT')
+    if not (host and port):
+        return None
+    user, password = os.getenv('PROXY_USER'), os.getenv('PROXY_PASS')
+    auth = f'{user}:{password}@' if user and password else ''
+    url = f'http://{auth}{host}:{port}'
+    return {'http': url, 'https': url}
+
+
+class _LegacyRenegotiationAdapter(HTTPAdapter):
+    """HTTPAdapter enabling OpenSSL legacy server connect (api.mospi.gov.in)."""
+
+    def __init__(self, ssl_context: ssl.SSLContext, **kwargs):
+        self._ssl_context = ssl_context
+        super().__init__(**kwargs)
+
+    def init_poolmanager(self, connections, maxsize, block=False, **pool_kwargs):
+        pool_kwargs['ssl_context'] = self._ssl_context
+        self.poolmanager = PoolManager(
+            num_pools=connections, maxsize=maxsize, block=block, **pool_kwargs,
+        )
+
+    def proxy_manager_for(self, proxy, **proxy_kwargs):
+        # Proxied HTTPS uses a separate ProxyManager that does NOT inherit the
+        # context from init_poolmanager, so inject it here too. On the
+        # production server MOSPI is fetched through the local HAProxy; without
+        # this the tunneled TLS handshake loses OP_LEGACY_SERVER_CONNECT and
+        # fails with UNSAFE_LEGACY_RENEGOTIATION_DISABLED.
+        proxy_kwargs['ssl_context'] = self._ssl_context
+        return super().proxy_manager_for(proxy, **proxy_kwargs)
+
+
+def get(url: str,
+        *,
+        params: dict | None = None,
+        headers: dict | None = None,
+        timeout: int = DEFAULT_TIMEOUT,
+        max_attempts: int = 3,
+        backoff: float = 1.0,
+        use_proxy: bool = False,
+        redact: tuple[str, ...] = (),
+        label: str = 'request') -> requests.Response:
+    """GET ``url`` with UA/proxy defaults and linear-back-off retries on 5xx.
+
+    Caller-supplied ``headers`` are merged over the defaults, so an explicit
+    ``User-Agent`` from the caller wins.
+    """
+    merged_headers = {'User-Agent': USER_AGENT} | (headers or {})
+    proxies = proxies_from_env() if use_proxy else None
+    for attempt in range(max_attempts):
+        try:
+            response = requests.get(url, params=params, headers=merged_headers,
+                                    timeout=timeout, proxies=proxies)
+            response.raise_for_status()
+            return response
+        except requests.RequestException as error:
+            resp = getattr(error, 'response', None)
+            transient = resp is not None and resp.status_code >= 500
+            if transient and attempt < max_attempts - 1:
+                info_logger.warning(
+                    f'{label}: HTTP {resp.status_code}; '
+                    f'retry {attempt + 1}/{max_attempts - 1}'
+                )
+                time.sleep(backoff * (attempt + 1))
+                continue
+            message = str(error)
+            for secret in redact:
+                message = message.replace(secret, '***')
+            raise RuntimeError(f'{label} failed: {message}') from None
+    msg = f'{label} failed after {max_attempts} attempts'
+    raise RuntimeError(msg)  # pragma: no cover
+
+
+def legacy_tls_session() -> requests.Session:
+    """A Session for endpoints needing legacy TLS renegotiation (MOSPI).
+
+    Verification is disabled (the endpoint presents a chain OpenSSL rejects) —
+    the same workaround as MOSPI's own reference client. UA and env proxies
+    are preset.
+    """
+    urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
+    ssl_context = ssl.create_default_context()
+    ssl_context.minimum_version = ssl.TLSVersion.TLSv1_2
+    ssl_context.options |= getattr(ssl, 'OP_LEGACY_SERVER_CONNECT', 0x4)
+    ssl_context.check_hostname = False
+    ssl_context.verify_mode = ssl.CERT_NONE
+    session = requests.Session()
+    session.headers.update({'User-Agent': USER_AGENT})
+    session.verify = False
+    session.mount('https://', _LegacyRenegotiationAdapter(ssl_context))
+    proxies = proxies_from_env()
+    if proxies:
+        session.proxies = proxies
+    return session